Depends. For Catholics a definite yes they go to hell as they where born with the stain of the original sin of Adam.
EDIT: I believe for Muslims their infants definitely got to heaven. There seems to be three schools of thought on the others, they go to hell with their parents, go to heaven, or...