It is a lot of driving for little kids. Expect to see a lot of bison up close, and maybe some Elk. The boiling mud pots are cool, old faithful is fun, some of the big waterfalls are incredible. There are lots of places to get out and walk around. Pack food with you because stuff in there is pretty pricey.
The problem I see is that your kids will probably be interested in each of those for a total of 10-15 minutes (cumulative) and from Rexburg to there and the drive around the park, it will probably be at least 8 hours of driving...lots of traffic in the park, etc.
To be honest, you kids will have more fun at that age at the local zoo and a trip to a good swimming pool. I took my kids up there a few times and they just don't want to be in the car that long...and we started out an hour closer than Rexburg...one hour each way.
It is a beautiful park and as an adult I enjoy it and have been multiple times, but I know a lot of adults consider it a one and done...including most of my wife's siblings.
Personally, I would do a more age appropriate trip now (and a lot less $$) and wait for Yellowstone until your little guys can really hike on their own. If you want a really cool trip, wait until your kids are teenagers and rent snowmobiles in the Max Inn area and go during the winter. That should definitely be on your bucket list. That being said, my wife and I took drove our kids through when they were little...mostly due to the wife.