Well as a male I don't knit (and don't natter very well either).
Due to the number of us living in this area of south Wales I think a meet up has been 'mooted' before but never materialised (and I'm not sure Spring/Summer is the best time).
As for venues that would depend on numbers. If literally only 2 or 3 for a knit and natter surely someone's house would be best? If a larger gathering (and here I'm thinking more along the lines of a mini small holders association group) well the Angel in Llandeilo has a good upstairs room (with an interesting ceiling!) they allow to be used for free (if prebooked) and not being used for anything else.
If it was only for say about 5 people to get together for a cuppa and a natter you could ask Wrights Food Emporium if you could take over a little section for an hour or two. They may well be supportive of smallholders and would be worth approaching as an outlet for produce anyway.
http://www.wrightsfood.co.ukThey are well worth a visit in any event and have excellent food (although not inexpensive). They are also literally just around the corner from me.
Personally I'm up for trying to get a more general meeting organised later in the year as we move into winter. If we could get an ongoing group organised there is the prospect of mutual support, networking (sorry to use that word) and even visits to each others holdings
Just my (not very useful) 2p