I have tinnitus, so I have to have music going all the time to drowned out the ringing in my ears, so having game music just clashes with that anyway, but generally the only time I want music in a game is when it fits the logic of the world. Like if I turn on my radio, or if I walk into a bar, have a bard in the party, something that makes sense, otherwise it is immersion-breaking. Most games have pretty bad/boring music anyway, and it just distracts from the the important sounds, like someone sneaking up behind you, or some other audio cue.
