I would leave it rowed up so the air can get through it, and there is the minimal amount in contact with the ground. I wouldn't heap more than one windrow together though. It will stand like that fine for a couple of days, especially if it doesn't rain. When you get back, turn each windrow upside down into the neighbouring gap, so the ground which has been covered for the two days has a chance to dry, and the bottom hay to catch up with the rest. Then carry on turning and spreading as normal - it won't come to any harm, so don't worry about it.