I still haven't ran into that problem, but I don't think I've played for spurts of more than 30 minutes anyways.
Honestly, main thing for me is Audio Stream. Disc swap only affects a few games (and someone in another thread already mentioned using Dolphin to solve that problem), but some of my favorites (Eternal Darkness, SFA [don't hate]) have the whole game affected because of this. I know there's a whole lot of tech reasons why this is so much trouble (even mod chips are flaky with it, which baffles me), but I just never would've expected certain portions of audio to be missing from a game. Just sounds silly when you think about it.