Umm.. yeah, it DOES matter.
I've got a 512MB stick, and I can't fit the GITS:SAC game on it. For... testing purposes, of course... (hey, I've bought all my PSP games...)
Especially if you're doing any Neo Geo CD gaming, those take up a ton of space, 80-200MB PER GAME, possibly more for some...