In all honesty, I cannot say that there is no difference is taste. And that's because I don't like the taste of cows milk, but I do like the taste of goats milk. I do agree, do not consider that your own home produced milk will taste like shop bought goats milk, it won't.
However, 2 years ago, my sister moved onto her own small holding, and was able to move her goats in with her for the first time. She has 4 daughters, then they were 14, 12, 10 and 10. 2.5 years on, and she still buys cows milk. 2 of them are fine, the other two taste the difference, even in tea! They do use a lot less bought milk, and goats milk is used in all the cooking (cheese /white sauces, baking etc).
Beth