Menu exploits or game exploits? Also do you have any self imposed limits for this one -- you could sign the resulting code if you wanted that might dodge the "was it a crash or was it an exploit" aspect or indeed remove some protections and work up to an exploit with an otherwise signed firmware (think the equivalent of full updated PC with anti virus and proper user vs old PC with adobe flash/pdf reader and no AV).
Generally you will want to look for either a developer mode/fix this thing mode or something that reads external data (save games, pictures, network maybe, audio streams in for code purposes...), preferably in a higher level mode (such as the menu) and work in from there, possibly with a stop to any changelogs if it is an external library responsible for things as was the case.
New vs old versions has many possibilities and considerations.
Older stuff is more likely to be thrown together and have since revealed issues with libraries and whatnot, as well as most protections being reactionary rather than thought out from the top (don't spend time, money and CPU cycles when you could be having flashy animations or getting it "working), though at the same time chances are whatever dev was tapped to do the last updates (don't know if the PSP got one for the EU privacy thing that saw several other previously dead consoles get updates) probably considered it a lost cause, might have been an intern and likely was not concerned with the full battery of tests that might be run.
I don't think there was much on the PSP as time went on other than eboot encryption -- it is usually left to later consoles entirely to change hardware access levels, security philosophies and add in new protections even if theoretically software could be used to boost things.
