is an Aaronic Priesthood commemoration is it not? It should be father's & sons IMO. 

I WAS in the same position as the OP. I had only daughters for years and There were married guys with no kids in the ward. One of them and I would take a boy who had no father. It was great. Got to know these fatherless boys and they had fun, felt included. 
Eventually Lifi Jr. came along and started taking him when he was two. 
every ward I've been in has had fathers & sons in May and a family camp out in the fall. 
I just think Fathers and Sons outing should be that. And always take the boys who have no father.