FAA study finds flaws in pilot training for automation

[braunpilot]

http://www.flightglobal.com/article...ous-flaws-in-pilot-training-for-handling.html

FAA study finds serious flaws in pilot training for handling automation

A significant emerging study suggests that flightcrew have never been properly trained for operating highly automated aircraft, and that for many of the problems they have to deal with there are no checklists, leaving the pilots to manage using ingenuity and airmanship.

Inadequate crew knowledge of automated systems was a factor in more than 40% of accidents and 30% of serious incidents between 2001 and 2009, delegates at the 2-5 November Flight Safety Foundation International Aviation Safety Seminar in Milan, Italy, were told.

The US Federal Aviation Administration's 1996 landmark report on "interfaces between flightcrews and modern flight deck systems is in the process of being dramatically updated, potentially with far greater implications for change than the original.

Presenting progress in her research toward the new report, FAA human factors specialist Dr Kathy Abbott catalogued the evidence of disharmony between crews and their highly automated aircraft. The study is based on accident and incident data and line operation safety audits over the period 2001- 2009. Abbott adds the caveat that she is presenting raw data at this point, and there is much more work yet to do to understand it fully.

Among the recurring handling problems pilots demonstrate, Abbott's findings include: lack of recognition of autopilot or autothrottle disconnect; lack of monitoring and failure to maintain energy/speed; incorrect upset recovery; inappropriate control inputs and dual sidestick inputs.

Regarding flight management system use, she found that pilots frequently focused on programming the FMS to the detriment of monitoring the flight path.
There are many failures with which pilots have to deal with little or no help from checklists or training of any kind, observes Abbott. These include failures or malfunctions of air data computers, computer or software failures, electrical failures, and uncommanded autopilot disconnects or pitchup for which the reason is not known. Of all these problems pilots face Abbot delivers the judgement: "Failure assessment is difficult, failure recovery is difficult, and the failure modes were not anticipated by the designers."

Despite the sometimes fickle nature of the automation, she observes, pilots frequently abdicate too much responsibility to automated systems. The reasons for this, she found, include: a perceived lack of trust in pilot performance by the airline; policies that encourage use of automated systems rather than manual operations; and insufficient training, experience or judgment, the result of which is that "pilots may not be prepared to handle non-routine situations".

Abbott has discovered particular vulnerabilities in automated systems and their man/machine interfaces. These include: mode confusion, and a pilot tendency to use information from automated systems instead of raw data. Another problem she identified is that much of the information supplied to pilots is, itself, automated - what she calls "information automation". Abbot also found that there was no consistency among operators in their policies for the use of automated systems.
Pilot knowledge was found seriously lacking in many areas relating to automated systems, including: understanding of flight director, autopilot, autothrottle/autothrust, and flight management system/computer systems and their limitations; operating procedures, mode transitions and behaviour; and unusual attitude recognition and recovery.

Although the report is far from finished, Abbott says that probable recommendations are likely to include these: focus training and standard operating procedures on flight path management; distinguish between guidance and control; encourage flight crews to tell air traffic "unable to comply" when appropriate; ensure that SOPs are tailored to the operator's needs

The industry as a whole, says Abbott, needs to review practice, regulatory guidance, and requirements for training in numerous areas including: flight path and energy management; recovery from off-path circumstances; use of alternative modes to meet air traffic clearances/requirements; operators' operational policies; and managing malfunctions.

Abbott emphasises that these recommendations are not final until reviewed and approved by the US Performance-Based Operations Aviation Rulemaking Committee and the Commercial Aviation Safety Team.

 

[2BEER]

"ingenuity and airmanship"

Oh the horror!!!

 

[braunpilot]

"ingenuity and airmanship"

Oh the horror!!!

What's that?

 

[Autothrust Blue]

http://www.flightglobal.com/article...ous-flaws-in-pilot-training-for-handling.html

Surprise? This starts the same place most training deficiencies start...

The industry as a whole, says Abbott, needs to review practice, regulatory guidance, and requirements for training in numerous areas including: flight path and energy management; recovery from off-path circumstances; use of alternative modes to meet air traffic clearances/requirements; operators' operational policies; and managing malfunctions.

Got a great mode to use for changes

 

[Orange Anchor]

Stage 1) What is it doing now?
Stage 2) I've never seen THAT before!!
State 3) Yep. It does that sometimes.

 

[Dugie8]

Stage 1) What is it doing now?
Stage 2) I've never seen THAT before!!
State 3) Yep. It does that sometimes.

The sad fact is, almost all "professional" pilots see that as the norm and accept faulty system performance as something a team player deals with and doesn't worry about getting fixed.

 

[nosehair]

The sad fact is, almost all "professional" pilots see that as the norm and accept faulty system performance as something a team player deals with and doesn't worry about getting fixed.

S'matter, kid? (spits between teeth) Cain'cha take it?

 

[braunpilot]

Surprise?....

Actually, not surprised at all, just posting something I found interesting to the forum. Honestly, that is one reason I fly freight. Not many operators actually have an A/P.

The sad fact is, almost all "professional" pilots see that as the norm and accept faulty system performance as something a team player deals with and doesn't worry about getting fixed.

Agreed, it's sad actually when we don't totally understand what is going on with the automation. I remember doing a flight from the back seat of a C172 where the pilot thought he knew more than me. I had to ask him three different ways about why the autopilot was doing something before he realized why the autopilot need to be "bumped" 100 feet higher rather than pushing the "UP' button five times. And he had had the training class and I had just read the autopilot manual.

 

[Orange Anchor]

The sad fact is, almost all "professional" pilots see that as the norm and accept faulty system performance as something a team player deals with and doesn't worry about getting fixed.

Not sure I agree with that. What can the line pilot do about the software that is installed? Most of the time crews develop 'work arounds' and ways to defeat the system's lock-outs. What I found as a check airman was often crews did not fully realize what mode they were in and what would happen next with the automation. Often the answer to 'what is it doing now' was 'it is doing what you programmed it to do." And too, training often failed to tell the crews what the defaults were, especially in reference to VNAV.

On early Boeing FMCs, if you had a crossing and speed restriction, sometimes the FMC would begin to slow the airplane before TOD which often caused confusion. And it would default out of VNAV to FLT CHG if it saw it could not make the restriction. So, many guys would add 5-10 miles to the crossing fix just to ensure they didn't have to fill out a NASA.

But I am interested in your solutions to faulty system performance by the line pilot

 

[Autothrust Blue]

Actually, not surprised at all, just posting something I found interesting to the forum. Honestly, that is one reason I fly freight. Not many operators actually have an A/P.

Very Chuck Norris of you, sir. The airplane I fly now is also not autopilot equipped...might have a Century III put in it. Didn't fly an airplane with an autopilot until I was well, well into primary, and even then the instructor would reach over and pull the breaker.

And I was being slightly sarcastic about "surprise". I'm not either. We're building pilots now - not airmen. A pilot is a technician that pushes the right button. Airmanship, on the other hand...

Good airmanship is that indefinable something that separates the superior airman/airwoman from the average: it is not a measure of skill or technique, rather it is a measure of a person's awareness of the aircraft and its environment and of her/his own capabilities and behavioural characteristics, combined with wise decision making and a high sense of self-discipline.

Automation is great - provided that the human operator is still sitting up, paying attention, and exercising that awareness.

 

[dasleben]

Actually, not surprised at all, just posting something I found interesting to the forum. Honestly, that is one reason I fly freight. Not many operators actually have an A/P.

Nothing wrong with automation. When I was flying single-pilot IFR in the 402, I'd always use the autopilot to the maximum extent possible on heavy weather days. It's much easier to manage a flight when you don't have to sit there chasing altitude and airspeed. You actually have time to come up with Plans A through G, accurately keep track of fuel, etc. My hand flying skills were fine; I didn't feel the need to try to impress anyone with them. In fact, of all the pilots I knew at that company, I was on autopilot the most. The thing is, you have to be able to tell when the automation isn't working for whatever application you need, and do something about it. Kick it off, get your head out of the box, whatever.

But for most professional applications, heck yeah, turn the autopilot on. It's there for a reason, and really does a good job and reducing your workload (for the most part).

 

[bizjetter]

Stage 1) What is it doing now?
Stage 2) I've never seen THAT before!!
Stage 3) Yep. It does that sometimes.

Stage 3 sounds all to familiar.

ProLine 21 does not always do a NAV to NAV transfer -> changing from FMS mode to raw data ILS. Usually it does......just not always Even the training folks at Bombardier in Dallas cant explain why.

 

[braunpilot]

Nothing wrong with automation. When I was flying single-pilot IFR in the 402, I'd always use the autopilot to the maximum extent possible on heavy weather days. It's much easier to manage a flight when you don't have to sit there chasing altitude and airspeed. You actually have time to come up with Plans A through G, accurately keep track of fuel, etc. My hand flying skills were fine; I didn't feel the need to try to impress anyone with them. In fact, of all the pilots I knew at that company, I was on autopilot the most. The thing is, you have to be able to tell when the automation isn't working for whatever application you need, and do something about it. Kick it off, get your head out of the box, whatever.

But for most professional applications, heck yeah, turn the autopilot on. It's there for a reason, and really does a good job and reducing your workload (for the most part).

I agree, if their airmenship skills are up to snuff. But I now know that I can take an aircraft, launch into VV001 @ 1/2SM when the OAT is -5C and land to something similar. I'm know that I can do that, that's one reason I like hand flying the aircraft. I gained the experience I wanted. Does that mean that I think those who fly on an autopilot are horrible flyers? No, I just wanted to do it old skool. That's what I wanted, and I gained that experience.

Very Chuck Norris of you, sir. The airplane I fly now is also not autopilot equipped...might have a Century III put in it. Didn't fly an airplane with an autopilot until I was well, well into primary, and even then the instructor would reach over and pull the breaker.

And I was being slightly sarcastic about "surprise". I'm not either. We're building pilots now - not airmen. A pilot is a technician that pushes the right button. Airmanship, on the other hand...

Automation is great - provided that the human operator is still sitting up, paying attention, and exercising that awareness.

:clap::beer:

 

[dasleben]

I agree, if their airmenship skills are up to snuff. But I now know that I can take an aircraft, launch into VV001 @ 1/2SM when the OAT is -5C and land to something similar. I'm know that I can do that, that's one reason I like hand flying the aircraft. I gained the experience I wanted. Does that mean that I think those who fly on an autopilot are horrible flyers? No, I just wanted to do it old skool. That's what I wanted, and I gained that experience.

I agree totally. I wanted that experience too. I'd hand flown a couple hundred raw data approaches before thinking "Huh, I'm making this way more difficult than it should be." By the time I ramped up my autopilot usage I'd already proven that I could do it manually. Wouldn't trade that experience. So far that job resulted in the biggest professional growth for me as a pilot.

That said, I DO think professional pilots shooting for more automated aircraft in the future should get some exposure to autopilots. We had two guys in my most recent new-hire class who were very wary of the automation, and thought they could squeeze through training by turning it all off and going "old skool." Nope, gotta learn the box like everyone else. One guy struggled a ton through several sims and resigned before the inevitable happened. The other guy gave 110% but was sent home for lack of progress.

Old school is great, but learning how to use the automation provided to you is important too.

 

[Orange Anchor]

Stage 3 sounds all to familiar. ProLine 21 does not always do a NAV to NAV transfer -> changing from FMS mode to raw data ILS. Usually it does......just not always Even the training folks at Bombardier in Dallas cant explain why.

And therein is the problem. MOST of the time it works as advertised and as expected. And then there is the one occasion when it doesn't and that is the contradiction also. We Mark 1 Humans are terrible at monitoring, especially systems that are very highly reliable. So when the 'machine' decides to do something different, there is a moment of disbelief followed by trying to problem solve something that was NOT explained or discussed in training.