Oh dear...
You can get collars with magnetic thingies and a matching magnetic thingy for your cat flap, so that only your cats can go through - but I know you don't put collars on them, so that's probably not an option for you.
I accidentally discovered that an opening-outwards-only cat flap was a good choice - your own cats pretty soon learn to hook the flap up and towards them with a claw but marauders won't know that trick.
However, the choice I always made was to have a dog my side of the cat flap. Never got marauders!
(And my old cat, Mix, used to sometimes come running home and call for me to let the dog out to see off the neighbourhood Cat Brotherhood who were chasing him with evil intent!
)