Personally I got everything (except teat wipes - that's usually mole valley farmers or wherever else I am buying other equipment etc from when I need a top up, they last ages!) from the local bargain/pound shop.
I have a couple of plastic buckets to milk into, about 5 litres, anything bigger is hard to fit under a goat! So I can alternate which one is being used, and usually use the other one to fill with Milton sterilising liquid - the one for babies bottles. Then I use Terry cloth/muslin and a funnel to strain the milk into a plastic container with a lid - flip or screw top - I have a few different ones now, depending on what the shop has at the time.
Then I rinse everything thoroughly with hot water, squeeze out the straining cloth and soak it all in the Milton.
I drink it raw as I'm happy that I'm milking healthy goats and don't feel the need to pasteurise.