Yes, there really is that great of an advantage. If you have a fixed pitch prop, it is really only efficient in a narrow band of airspeed. You may have noticed that when you were flying a single engine plane, that you could not get full rated rpm in the climb, and that once you were leveled out in cruise, it was necessary to throttle back to keep from exceeding the redline on the tach.
The reason for this is that most airplanes with a fixed pitch prop use a compromise between a propeller pitch that will give best climb performance or best cruise performance. If the fixed pitch prop had a fine pitch it would increase climb performance by allowing the engine to reach full rated rpm while flying at the relatively low airspeeds that give best climb. The penalty is that once in cruise, you have to throttle back quite a bit to keep from overspeeding the engine. A climb prop could easily cost you 10 knots in cruise airspeed. If you opt for best cruise performance you will have more of a coarse pitch, and will be able to cruise at higher power settings, but the engine will not be able to climb at full power. A cruise prop may cost you several hundred fpm in climb.
To mitigate some of the undesirable effects of having either a pure climb or cruise prop, most airplanes are fitted with a compromise. Since the majority of an aircrafts time is spent in cruise flight, the manufacturer will try to fit a prop that allows as high a cruise speed as possible while giving acceptable climb performance.
Or you can spend more money and get an aircraft with a constant speed prop. This allows full rated power in the climb for best climb performance and allows the airplane to cruise at high power settings.
One other advantage that a constant speed prop has over a fixed pitch prop for multi-engine flying is that you can usually feather them. If you loose an engine and cannot feather the prop, you are not going anywhere but down. A windmilling prop creates a huge amount of drag and will murder your climb performance on one engine, in most cases you will not even be able to maintain altitude.