I try to clear away any volunteer potato plants, for the reason Floyd gives, disease. Sometimes there's no problem but other times there is, not just with blight. As we live in Scotland where many seed potatoes are produced I would rather not be a source of disease problems for the producers. Volunteer tomatoes are even more of a problem, quickly spreading disease to new plants - I know this because I couldn't resist one year letting all those tomato plants which appeared in my polytunnel so nice and early grow, and ended up with a crop disaster.