This wasted time is a big pet peeve for me, too. Fortunately, if I'm working the game I have the chance to do something about it!
Most rule sets say that the pitcher has one minute to deliver her five warm-up pitches between innings. That one minute begins from the instant that the third out is recorded in the previous half-inning.
If the catcher isn't ready, too bad. Get another player to catch the warm-up pitches. If a catcher is late getting out, then the pitcher just might have her warm-up throws reduced or eliminated when we hit the one minute mark.
I'm not suggesting that an umpire should be fanatical about this, timing off the one minute with a stop watch, but keeping the game moving along and enforcing the between innings drill is all a part of good game management duties. The simpliest ways to handle this is to encourage the players to hustle in and out, encourage the coaches to have their catchers ready to go and for the umpire to actually count the warm-up pitches between innings- like you're supposed to!- allowing no more than five.
If we are in a non-timed game, I might be a little more relaxed about this, as long as the teams are generally quick about changing sides. Sometimes I'll loose track of the warm-up pitches while a coach discusses substitutions or I grab a quick drink of water on a 90-degree day. But I still don't allow prolonged wasted time between innings.
In a time limit game, I consider enforcing this rule to be more critical. The wasted time between innings can be the difference between playing the entire game or having to cheat kids out of the last couple of innings.
A few years ago I took a closer look at this for one of the local rec leagues where I worked. By enforcing the one minute between innings, most of my games went the full seven innings. In contrast, games by other umpires that let the between innings routine drag on for several minutes were only getting in about 4-5 innings, on average.
Coaches can set the tone by hustling their players in and out, having the catcher ready to go and making their subs as quickly as possible. While that might help, if the between inning downtime is dragging out, I still put most of the blame on the umpire. He is responsible for controlling the pace of the game and should be enforcing (within reason) the one minute limit between innings.