Deffo pop him under a heat lamp or try putting him back with mother goose.
I always get another egg and give them the yolk if poorly, as this will have all the nutrients in it he will need first.
He will need water, just use a pipette in the corners of his beak. I wouldn't give milk or sugar, I have never heard of that before!
I do hope he makes it, let us know x