He just doesn't know you yet Emmett, so he doesn't know if you're going to hurt him or not.
It took quite a while of sitting with my rescued pygmy wethers, not stroking them, just drinking my cuppa and talking, having raisins in my hand......eventually they came and ate, but it was a lot longer before they wanted touching.
When you start petting him, don't start with his head. Just stroke the side of his back leg a bit. Eventually you can stroke his back, then scritch his chest. Some animals never like their heads being touched.
I'd have the vet castrate him if he's just going to be a pet. Less smelly, and if he's currently still a youngster, less bolshy too once the testosterone levels have dropped.
And yes, he'll need a friend :-) 'Course if you're thinking of breeding from him (is he good enough - and if he is, why didn't your friend sell him for that?) that's a different business.