Well, I've done it and not had any problems. I wouldn't do it a lot, but when it's the odd ewe - his mum, one or two half sisters - if he's a good specimen and the basic flock isn't inbred, you should be okay, I'd have thought. I wouldn't keep on any daughters where the ewe was related, of course, nor indeed any sons.
The dangers are in reinforcing any negatives, so you do need to be extremely robust in checking him over for faults, and his mum too, I guess.
BH used to castrate a lot less than he does now, with the result that we often got an occasional early inbred lamb or few. They were always good lambs, so much so that we used to joke that we needn't be spending hundreds on bought-in tups when our homebreds were clearly just as good!
What breed is he?