We'll my old Honda big red 4x4 is only 300cc (trx300) and seems to be reasonably powerful, but then I don't really tow anything particularly heavy, so I probably wouldn't go less than that.
you don't say if your dealing with steep hills or flat, or planning on using in winter mud or on hard surfaces; either hills or mud will need more power.