We have a stable door with a hand aperture so you can slide the inside bolt from outside. The swallows fly straight out through that, without slowing down. If you're really lucky you get a face full of swallow. The other stable door at the other end stays open throughout the swallow season.
Our hen houses (converted garden sheds), where we have had swallows nesting ever since we first put them up, have an extended ventilation hole above the door - about 6" wide by about 3 - 4" tall. That's high enough for the swallows to fly straight in, or to sit in and contemplate the world. Originally we intended to make a flap as you describe but we never have as the hens need the air. However, if it wasn't for the swallows we would cover the hole with small mesh to make sure the stoats don't get in - they haven't climbed that high yet.
We have bats flying at night, but they don't roost here. If they did there would be no problem with access to any of our sheds or the barn, as none of the doors fit so tightly that a bat couldn't shuffle through.
What you describe sounds perfect.