If you love gemstones then you have to take time out to visit Scratch Patch, a place where you can sit scratching through piles of beautiful gemstones and selecting the ones that take your fancy. Scratch Patch have branches at the v&A Waterfront and in Simon's Town, so if you are visiting, and especially if you have kids, then consider placing this outing on your todo list.
If you'd like to buy gems in bulk, then visit Topstones, manufacturers and distributors of the Scratch Patch stones. I've never been, but I'm guessing that the stones will be somewhat cheaper than at Scratch Patch. :)