The great thing about the Naschmarkt, at least on Saturdays, is the way it progresses from tidy little Austrian huts at one end to a thoroughly oriental flea market at the other end. It’s like walking from one world to another: the best illustration of Metternich's comment that the Balkans begin at Kärtnerstrasse. Mind your handbag, but have fun! If you are staying in a place where you can do your own cooking, this is the place to stock up. And if your feet get tired, you could have a coffee in the sumptuously mirrored and gay-friendly Savoy Cafe.
Naschmarkt, between the 'Wiener Zielen'. Closest underground: Kettenbruckengasse U4
Updates
Metternich did not say the Balkans began at Kärntnerstrasse, but at the Rennweg (The road that leads out of town to the north east of the Belvedere gardens). This is ironic in that he himself lived at No. 27.
Is this your business?