Roxy and little blue have made some great points. Although we don't keep our goats in a back garden, I have known plenty of people who do or have. Its also something that I think I will have to do in the future- I probably can't afford a place with enough land on my own- at the moment we share a house with my mum.
Something else to consider, is what you going to do with their manure? Unless you can use it all on your garden and veg plots, you will have to find something else to do with it. You can get underground tanks, which store it into a kind of slurry, and the company will come and empty it out for you, but I think it is quite expensive. Or you have to find other arrangements for it. A friend of mine has her goats in the back garden, and they put all the muckings out onto a small flatbed trailer, and take it to a local farmer's muckheap. Where he then obviously spreads the whole lot onto his fields. But this is a labour intensive way of doing things, as you have to lift the dung into the trailer, and then back out when you get to the muckheap. Also, they go to the muckheap virtually every weekend (although this is because they "clean" the pens twice a day by taking out the fresh droppings). But first you have to befriend a local farmer who will let you do this.
I would recommend having an area yarded off for the goats so that you can let them out for exercise, even if you have to pave this if it gets muddy. Ideally a small area of grass would be better, but it depends on how much room you have. My friend has a goat shed that is about 20ft by 20 ft, but this is bigger than you might actually need. They have 4 goat pens in one half of the shed, and the other half which is divided off by a wall and a gate, lets them store hay and straw, and their goat food and chicken food.
You can keep goats in lots of different buildings- when we first moved here we kept the goats in our garage (single and a half garage sized) and a wooden shed which was about the same size. But then we had hay in a different shed, and straw in yet another one. That's time consuming to go and get and cart to the goats, and then you also have to visit wherever you get hay/straw from more often as well.
The garage was good because it has a concrete floor, and when the time came for them to move out into our new shed, we could just clean it out and then power wash it clean. Plus if it got too warm, the big up and over door could be opened to let in more air.
If they can't browse naturally or graze, then I would say you need to provide extra browsing for them. It could be any number of things- raiding hedgerows on the way home every night for some branches. Or growing stuff in your veggie patch for them- ie kale, carrots, the leaves off of your cauliflowers.
For dimensions for how much room you need, this page from the british goat society has some info on it.
http://www.allgoats.com/housing.htm