Uncle Mike, one of the worst things to have happen is to disturb accidentally a nest of black face bumblebees. They usually are under the edges of buildings or stacks of wood or old lumber piles. They can be very vicious when in attack mode
There is an old tale that yellow face bumble bees won't sting. I know that is not true, because as you say of your kids, etc., I used to play with them as a child until I got stung by a yellowface.
My reason for whacking them is the structural damage they do to buildings and sheds. Had some 16 ft 2X6's cut and stored under a shed and later cut some up for a project and the bees had traveled and bored the entire length of the boards and not just one hole, some had 3 and 4 holes. The boards were ruined because they had to support a large amount of weight.
Tell the kids to be careful, cause they might pick up the wrong kind one of these days. Pete