Uncle Mike, one of the worst things to have happen is to disturb accidentally a nest of black face bumblebees. They usually are under the edges of buildings or stacks of wood or old lumber piles. They can be very vicious when in attack mode

.
There is an old tale that yellow face bumble bees won't sting. I know that is not true, because as you say of your kids, etc., I used to play with them as a child until I got stung by a yellowface.
My reason for whacking them is the structural damage they do to buildings and sheds. Had some 16 ft 2X6's cut and stored under a shed and later cut some up for a project and the bees had traveled and bored the entire length of the boards and not just one hole, some had 3 and 4 holes. The boards were ruined because they had to support a large amount of weight.
Tell the kids to be careful, cause they might pick up the wrong kind one of these days. Pete