Move the start/select to under the screen, put another analog stick below the buttons, and if possible, make the whole device a bit taller and a bit more longer.
Then this'd be the absolute best way to play ROMs. At least on the go. I'd pay up to $120 for one.
Edit: Also, I didn't notice one, but this has to have a headphone jack output. I wouldn't even consider buying one if it didn't.
Another thing, please make the volume control (at least) buttons instead of a slider wheel. The power doesn't bother me as much, although I'd like for that to include a button instead, as well. But it'd make a huge difference for me if they were buttons and not.. A wheel.