That scenario is possible, but not common. It doesn't break all the laws of physics, but it would probably be a tough one to get into accidently. And it certainly doesn't occur during a normal descent.
The best example I can think of where it DOES occur would be during an outside loop, especially at the top of the outside loop. Imagine flying an aerobatic aircraft. You start the manuever at the bottom by rolling inverted first and then once upside down, you push negative 3-4 g. The airplane has a negative angle of attack, and it begins climbing upside down. By the time you get to the top of the inverted loop, you are right-side up, slowed down, with the power still set high. You'd have bled off most of the airspeed (and you wouldn't have as much g available), but you'd still be pushing forward on the yoke or stick (high negative AOA), still high power setting, and low speed. At this point, you'd have a high p-factor and in the opposite direction of the one that you commonly experience.
The key to popaviator's question is this. P-factor is opposite during negative angles of attack, but descents don't necessarily (or even typically) equate to a negative angle of attack. In fact, most maneuvers in flying don't involve a high negative angle of attack unless you start talking about some aerobatic maneuvers.