Which brings us back to weighting results more toward work ethic than height. But I'll add in my two-cents.
Another obvious factor is genetics, which can be cruel - or it can be a gift - depending on how you look at it. Greg Oden (OSU Basketball) is touted as possibly one of the best "big men" in 20 years, but I'll bet he'd never get into the navy's Top Guns flight academy. And don't forget Michael Jordan's short lived baseball career.
The type of muscle fibers you inherit, for instance, has a LOT to do with your plateu pitch speed point and jumping ability. A person with a lot of fast-twitch muscle fibers will ultimately be able to generate more explosive power than someone who does not. That's just a fact of life.
IMO, athleticism, another genetically inherited trait, is way more important than physical stature. Pitching SPEED is about arm speed, leg drive, timing and endurance - all related to athletic ability. BALL MOVEMENT (spin), LOCATION and ACCURACY are more about correct mechanics, which can be learned by any pitcher, regardless of athletic ability. Thus, the "power" pitcher vs. the "junk" pitcher. Both are effective in their own right.
Inherited physical traits (which you have no control over) and work ethic (which you CAN control) are the ultimate determining factors.