Debug kits contain 128MB of RAM compared to the regular 64MB in retail units, making them useful for debugging purposes and playing Chihiro games (since these games run on Xbox hardware, but require more than 64MB of memory).
Obviously, debug kits also have the XDK launcher instead of the normal Xbox Dashboard on bootup so developers can manage their developed content. If you set up the Xbox SDK on a PC (or a VM) with Windows XP, you can transfer content over an ethernet cable connection via the Xbox Neighbourhood. There are guides online that walk you through how to set it up.
IIRC, there are also tools that allow you to sign retail XBE executables to make them run on a devkit, but I don't know much about it since I've never owned a debug kit myself. There's also a homebrew you can use that allows you to temporarily reboot into retail mode to be able to play regular retail discs, if you have any.
As for betas, they are quite valuable if they've not already been shared online. Some people try to sell them off for more money, but others often just share it on sites dedicated to obscure/beta content for preservation purposes.
