This is my understanding. I would be very glad to have my understanding improved by others who know more about it!
All clover fixes nitrogen, which is good
White clover signals ground of lower fertility, but the clover nitrogen-fixing helps to maintain sufficient fertility for your pasture grass to grow. I don't think it makes enough fertility for a thick sward of rye-grass, though. I don't think there are any downsides with white clover?
Red clover grows on and sustains ground of a higher fertility than white clover. Farmers are increasingly sowing clover as an alternative to using artificial fertiliser; it will grow happily along with, and will help the growth of, a temporary sward (usually high in rye-grass and hence feed value.)
Red clover is high in oestrogen. You therefore need to be a little bit cautious about what stock you put on it and when. In theory it can act like a contraceptive pill! (Though I am not sure if this really happens.) But it certainly stimulates milk production, (I've seen it put bags on gimmers in sunny Somerset) so you have to be careful with ewes you are wanting to dry off, who are typically going to be going to the tup soon.
I expect there are good and bad times to put cattle on it too - again, you wouldn't put a cow you were wanting to dry off on it.
We don't have so much red clover here, even when we sow it, for me to know much more about it than that.