I should note that I'm interested in various types of music, none of them really intersecting with mainstream chart/pop/radio much, so the following guide is coming from that viewpoint :
HMV : Large store in Bourke St Mall. Two levels, which are often rearranged (e.g. dance music was downstairs, now up, classical was upstairs in a closed-off area, now it's downstairs in a partly-closed-off area.) Range of music : Classical range is reasonable. Metal range is pretty thin, not much beyond the standard groups. Dance range is reasonable (after all, it has its own closed-off area), but for some reason I keep finding myself buying 5-year old CDs of dodgy techno there. This store also carries DVDs now (after the last rearrangement), with a modest range. They do allow you to listen to CDs, at busy times only two per session.
Sanity : Nearer the West end of Bourke St Mall, a smallish store over two levels. Dance music range was reasonable, but the top level is now closed off - and I suspect the store is moving or closing soon (there are Sanity stores in many other locations). The metal and classical ranges there are token efforts, really. Do allow CD listening, also have (had) listening stations with a selection of 6 discs.
Myers : Once had a decent dance music section, oddly enough, with seemingly knowledgeable dance-music-only staff present. Now just a standard range of music.
Central Station Records : Downstairs from Swanston St. Specialists in dance music, but I can never seem to find much to inspire me there.
Gaslight Records : Near the East end of Bourke St, stocking mostly "normal" music, a reasonable range of jazz, and a little bit of dance, metal & classical.
JB Hi-Fi : Jam-packed store on Elizabeth St, just north of Lonsdale St. Quite crowded, but a good range of many things, and lots of categories which are hard to find elsewhere (e.g. ballroom dancing music, which I only found a few discs of in one other store). Not much metal or dance, but lots of classical.
Basement discs: Downstairs from a small arcade off Lt Collins St - maybe The Block arcade (I know how to get there :-) ). Mainly specializes in jazz and similar areas, also a reasonable selection of world music.
Heartland Records : On Peel St, opposite the Queen Victoria Markets. Specialists in Goth & Industrial (and various types of "dark" music). Also a modest range of metal. Lots of music here that can't be found in any other store, but limited listening facility - no players with headphones, you ask a staff member and they can play the CD over the stereo system there. Not ideal if you're like me, and want to skip through the tracks quickly to see if anything leaps out and either inspires or repels me.
Metal Mayhem : Specialists in (obviously) metal. There could well be more categories of metal there than most places have CDs. Black metal, doom metal, death metal, progressive metal, operatic metal ... you get the idea. Similar listening system to Heartland, above, but friendlier about it to customers.
Discurio : On Hardware Lane, just North of Lonsdale St. Specialists in jazz and classical, also with a fairly good range of world music. Very good listening service - a number of stations with comfortable stools to perch on while listening to CDs of your choice.
Whew. That went on for a while, and I've almost certainly forgotten a few. Will try to sneak in an update sometime.