An allergy is possible, I know it seems mad but it does happen. If it is hay or straw they generally arent too bad unless in a smaller space with the hay/straw. had a goat who was allergic to straw, and it flared up worst when she being transported (smaller space, more enclosed with the straw) I also had a billy goat once who I swore used to get hay fever. He would snuffle in the summer months- especially after a night out in the field.
I think I would less worried with a low temp than a high one, but it does seem a bit on the low side. Are his ears cold to the touch, or if you put your finger inside his mouth is the inside of his cheek feel cold? If yes to either then he is too cold, and there is something wrong.
Beth