I think "the committee" have the right to turf you off if they want. My friend had a plot, she was going through a marriage break up, and her plot got overgrown, so I can understand why they made her leave. But it sounds like yours is tidy, do completely different. Maybe there is a rule somewhere saying if you have not planted within a certain time, then you cannot continue.
Not sure if appealing would help, but by all means try. Really you need an explanantion as to exactly why they are not happy.