Thats not a nice situation at all, if you are alone, and this man I assume knows it?
If there is something you could specifically report to the police, perhaps him seeing a police car outside your home, would be enough to make him at least think, and back off you.