If you're looking for visual novels on the go, the PSP is probably going to be your best bet as far as budget options since many windows VNs were ported over either officially or by fans.
You could probably also try one of the many handheld android emulation machines out there but they probably won't do any good for playing the windows native ones unless there are compatible VN runtimes/interpreters for them out there.