My friend had an ex racing standardbred with shot up legs and she managed to keep his quality of life good on bute for a good many years. He was her life, she polished every shaving in his stable. But tho we thought she wouldnt ever recognise when 'the time' had come, she did, when she felt, ignoring the stiffness and all the physical stuff, that he was not as happy as previously, not as interested in life, not trying to drag her or trying to bite her when she groomed him. That was his time.
Based on what you have said about your lovely neddie, I would be looking more at his mood and happiness level than the physical, unless that becomes overwhelming ie maybe ok if he cant roll, but not ok if he lies down and struggles to get up/gets cast in the stable etc, that would be a signal to me. Some peeps would call a halt earlier, before anything like that could happen, and some would leave it later, but for me chronic physical degeneration or a downhill slide in happiness would be 'it'....
I really sympathise.