Seems like this is a case of letting kids be kids. They were fine, not coming over saying they needed food or help. They just wanted to come play at your house. I think it is good that the mom let them play but said they should let her know where they ended up. Definitely don't see this as pawning off child care. It isn't like she sent them to you or that you couldn't say no.
Seems to me like the way kids are supposed to be raised.