For the purposes of this conversation I will recall my previous life. Afetr this, please go back to treating me like a smallholder type person.
Anyway, for 20 years or so I was a mortgage adviser. In an estate agency.
I'm pretty certain the builder will know it has a movement issue.
His intention will be either to fix it (good guy) or conceal it (bad guy).
Anyway - step one, is to ascertain if he is a good guy or a bad guy. Make friends with him and make your judgement.
That is, treat him like a new neighbour, welcome him etc and get an idea. Innocent until proven guilty and all that. The chances are, he's a good guy, most of them are. You don't hear about good guys, there's nothing exciting about them.
Next question - are you claiming on your buildings insurance or can you, or do you have legal cover on your insurance and can get some legal advice (on the party wall act).
Or - seeing as you know the guy is moving on - say 'are you doing anything about the subsidence - better upsetting him than your upsetting your next neighbour who is going to live there for years.
Good luck ... putting my smallholder uniform back on now.