As a responsible parent, you are within your rights in this country to teach your children about firearms and shooting. Safety in storage, handling, and use is the first and highest priority in any such training. This is true of anyone who is learning to shoot, but especially of children.
There are in fact guns designed especially for children, the famous Stevens Favorite .22 rifle being, well, a real favorite. This is a single-shot rifle with a specific and slow loading procedure and logical safety features. It is a perfect teaching tool, though still a lethal weapon. It is also small and fairly light, but heavy enough to emphasize that it is not a toy.