Here is the
https://www.pilotsourcestudy.org/
Background
In February 2010, the Federal Aviation Administration (FAA) issued an Advance Notice of Proposed Rulemaking (ANPRM) titled “New Pilot Certification Requirements for Air Carrier Operations” that proposed changes to regulations relating to the certification of pilots conducting domestic, flag, and supplemental operations. Industry and educator members commissioned the first Pilot Source Study (PSS) 2010 to answer one ANPRM question, “Are aviation/pilot graduates from accredited aviation university degree programs likely to have a more solid academic knowledge base than other pilots hired for air carrier operations?” The research question was, “What are the background characteristics (education, flight experience, etc.) of pilots and how did their backgrounds (source) influence their success in regional airline training? Background and performance data for 2,156 pilots hired by regional airlines between 2005 and 2009, a convenience sampling of six regional airlines, showed that pilots who experienced fewer extra training events and fewer non-completions were pilots who: (a) graduated from a flight program accredited by the Aviation Accreditation Board International (AABI), (b) had an aviation degree, (c) completed their flight training in a collegiate program, (d) had a CFI certificate, or (e) had between 501-1,000 total flight hours.
In February 2012, the FAA issued a Notice of Proposed Rulemaking (NPRM) that would require first officers in Part 121 operations to hold an ATP certificate and type rating for the aircraft to be flown, with restricted privileges for pilots with an aviation degree or military pilot experience. Industry and educator members commissioned the second Pilot Source Study 2012 with the same research question, expanding the data to include other regional airlines. Background and performance data for these 4,024 pilots hired between 2005 and 2011, a convenience sampling of seven regional airlines not included in the previous study, showed that pilots who experienced fewer extra training events and fewer non-completions were pilots who: (a) graduated from an AABI-accredited flight program, (b) had an aviation degree, (c) completed their flight training in a collegiate program, (d) had a CFI certificate, or (e) had between 1,001-1,500 total flight hours. With the exception of total hours, the results of PSS 2012 were consistent with the results of PSS 2010.
On August 1, 2010, the US Congress passed Public Law 111-216, titled the “Airline Safety and Federal Aviation Administration Extension Act.” On July 15, 2013, the FAA issued the “Pilot Certification and Qualification Requirements for Air Carrier Operations” rule (FOQ Rule) that abruptly changed the pilot hiring situation for US air carriers operating under Part 121. The FOQ Rule required all pilots operating in a Part 121 airline to have an ATP certificate that requires them to be at least 23 years old and have at least 1,500 hours total flight time. Other changes required by the FOQ Rule were that all Part 121 pilots must complete the Airline Transport Pilot Certification Training Program (ATP CTP), have 50 hours of multiengine experience, and possess an aircraft type rating. The FOQ Rule which became effective on August 1, 2013, provided an avenue for alternatively-trained first officers to fly as required crewmembers under Restricted Airline Transport Pilot (R-ATP) privileges at the age of 21 instead of 23 years old if they:
- graduated from an FAA-approved R-ATP Bachelor’s degree program with 60 approved credit hours and had 1,000 hours total time, or
- graduated from an FAA-approved R-ATP Bachelor’s degree program or Associate’s degree program and had 1,250 hours total time, or
- were prior US Military pilots and had 750 hours total time.
PSS 2015 Protocol
In January 2015, industry and educator members at the “Pilot Supply Summit” requested the Pilot Source Study researchers to conduct a new study to answer the question, “What is the effect of PL 111‑216 and the FOQ Rule on pilot hiring and pilot training in US regional airlines?” The new study, Pilot Source Study 2015, was conceived as a replication study, involving the same 13 airlines that provided data for the two previous studies. To accommodate a very condensed timeframe, two research teams were assembled – a Data Collection Team and a Data Analysis Team, both led by co-principal investigators, Dr. Guy M. Smith from Embry-Riddle Aeronautical University and Dr. Elizabeth Bjerke from the University of North Dakota. In May, the PSS Data Collection Team, at the Regional Airline Association (RAA) Annual Convention in Cleveland, OH, briefed the RAA Board of Directors, the Regional Operations Council, and the Flight Training Committee to request permission to come to the airlines to collect data for the study. Acknowledging an urgent need for the study, the senior management of many airlines applauded the research effort, promising to cooperate with the Data Collection Team, including airlines not included in Pilot Source Study 2010 or 2012. Urged by this enthusiastic response, the PSS Data Collection Team redesigned the study into a population study that would include virtually all US regional airlines and all pilots hired by these airlines from August 1, 2013 to the date of data collection. AABI managed a separate Pilot Source Study fund to cover travel costs for the Data Collection Team – a principal investigator (professor), a data collection manager (graduate student), and additional data collectors, as necessary. The data collection manager and additional data collectors received a stipend of $100 per day; the principal investigators did not receive any payment beyond reimbursement of travel expenses. The researchers on the Data Analysis Team did not receive any payment for their work on the Pilot Source Study. The donors to the Pilot Source Study fund were two universities, five major airlines, an association, and a consulting firm. To maintain objectivity, the Pilot Source Study fund did not request or accept any contributions from AABI, RAA, or any regional airline.
Two documents were critical to the study. At the beginning of every visit, all members of the Data Collection Team signed a non-disclosure agreement, asserting that “all data shall remain in the airline's control, except de-identified data specifically released by the airline for the purposes of the Pilot Source Study.” Another document explained the research protocol that would be strictly followed at each airline: the Data Collection Manager would collect identified background data from Human Resources or Pilot Recruiting records; the Principal Investigator would collect identified performance data on Training, Initial Operating Experience (IOE), and Recurrent Training from training or operational records; the Data Collection Manager, after combining the two identified records, would delete all identifying information (name, ID number, age, gender, ethnicity, etc.) and deliver the de-identified dataset to the Pilot Source Study data repository. The Data Collection Team visited 22 US regional airlines from April to October 2015, collecting 7,073 pilot records. These data were combined into two composite spreadsheets for analysis. The data from 19 Part 121 airlines (6,734 records) were analyzed by the Data Analysis Team consisting of six researchers from different universities and one independent research consultant. These Part 121 results are reported below. The data from three Part 135 airlines (339 records) were analyzed separately because the restrictions of the FOQ Rule did not pertain to these airlines. These results are not reported in this document.
PSS 2015 Results
For analysis, the background data was combined into two categories: Educational Background (college degree, AABI-accredited flight degree, aviation degree, and college GPA) and Experience Background (years since graduation, previous employment, CFI certificate, military pilot, ATP certificate, and aeronautical experience). Four indicators of performance (outcomes) were analyzed: non-completions, extra training events, extra IOE, and extra recurrent training. Significant results of the study are summarized in Table A at the end of the document and are described here.
For College Degree (graduate 8%, bachelors 63%, associate 9%, high school 18%, unknown 2%), pilots with a bachelor’s degree had fewer non-completions and less extra training than expected; pilots with an associate degree had more non-completions, more extra training events and more extra IOE than expected; and pilots with no degree (high school) had more non-completions and more extra training events than expected.
In the dataset, 23% of the pilots graduated from AABI-accredited flight programs. These pilots had fewer non-completions, less extra training, less extra IOE, and less extra recurrent training than expected.
The variable, Aviation Degree (48%), included graduates from AABI-accredited flight programs, graduates from other flight programs, and graduates from aviation disciplines other than flight. Pilots with an aviation degree had fewer non-completions, less extra training, and less extra recurrent training than expected. Pilots with a non-aviation degree had more non-completions, more extra training events, and more extra recurrent training than expected.
Only 38% of the pilot records had college GPA information. Pilots whose college GPA was 3.0 or lower had more extra training events, more extra IOE, and more extra recurrent training than expected.
In the dataset, 55% of the records included graduation dates, which was converted to years since graduation. Pilots with fewer than four years since graduation had fewer non-completions, less extra training, and less extra recurrent training than expected. Pilots with more than 10 years since graduation had more non-completions, more extra training events, more extra IOE, and more extra recurrent training than expected.
For Previous Employment (26% Part 121, 16% Part 135, 8% Part 91, 36% flight instructor, and 14% other), pilots who were previously flight instructors had fewer non-completions than expected but they required more extra IOE and more extra recurrent training than expected. Pilots who were previously employed in Part 121 operations had less extra training, less extra IOE, and less extra recurrent training than expected. Pilots who were previously employed in Part 91 operations had more non-completions and more extra training events than expected.
In contrast with Pilot Source Study 2010 and 2012, having a CFI certificate did not show any significant advantage over the expected outcomes for pilots in the 2015 study. However, pilots who did not have a CFI certificate had more non-completions and more extra training events than expected.
In the dataset, 12% were prior military pilots. They had less extra training than expected.
As required by the FOQ Rule, all pilots had an ATP certificate (2% military R-ATP, 15% institutional authority R-ATP, and 83% traditional ATP). Pilots with an institutional authority R-ATP had fewer non-completions, less extra training, and less extra recurrent training than expected.
Total Time was binned into the following categories: 27% with 1,500 hours or fewer; 42% between 1,501 and 3,000 hours; 14% between 3,001 and 4,500 hours; and 17% with more than 4,500 hours. Pilots with 1,500 hours or fewer had fewer non-completions, fewer extra training events, and less extra recurrent training than expected. Pilots with more than 4,500 hours had more non-completions but less extra recurrent training than expected. Pilots with between 1,501 and 3,000 hours had more extra recurrent training than expected.
Pilots with less piloting experience (instrument hours, cross-country hours, pilot-in-command hours, second-in-command hours, multi-engine hours, turbine hours, dual-given hours, and total time) had fewer non-completions and fewer extra training events than expected. Pilots with more experience had less extra IOE and less extra recurrent training than expected.
Comparing the backgrounds of the pilots in PSS 2015 to pilots in the combined 2010 and 2102 datasets, there was no difference in highest degree (bachelor, associate, or no degree). There were significantly fewer pilots with an aviation degree, fewer pilots with an AABI-accredited flight degree, more military pilots, and fewer CFI certificates with more hours of dual-given. By law, none of the 2015 PSS pilots had commercial pilot certificates but 17% had R-ATP certificates. Also by law, the 2015 PSS dataset had significantly fewer pilots with less than 1,500 total flight hours.
Comparing outcomes between the PSS 2015 pilots (Post-Law) and the 2010/2102 pilots (Pre-Law), Post-Law pilots had more non-completions and required more extra training. Having an AABI-accredited flight degree, an aviation degree, or a CFI certificate had a positive effect on the number of extra training events for Post-Law pilots. Post-Law completions were positively affected by having a bachelor’s degree, an AABI-accredited flight degree, an aviation degree, or being a CFI. The additional total hours required by the FOQ Rule was less beneficial to regional airline training for Post-Law pilots; as the number of total hours increased, so did the proportion of non-completions and extra training events. Most importantly, completions decreased from 93.4% in the Pre-Law dataset to 83.6% in the Post-Law dataset, and the Post-Law pilots required significantly more extra training. Using approximate costs of training from seven regional airlines, the Data Analysis Team estimated an airline’s average expenditure per pilot who did not complete training to be $38,464 with zero return-on-investment to the airline.
In Summary, ranked by the magnitude or size of the effect, pilots in Pilot Source Study 2015 with fewer non-completions and fewer extra training events were:
- pilots with less than four years since graduation,
- pilots with 1,500 or fewer total flight hours,
- pilots who graduated from an AABI-accredited flight program,
- pilots with an institutional authority R-ATP,
- pilots with an aviation degree,
- pilots with a bachelor’s degree,
- pilots whose previous employment was in a Part 121 operation, and
- prior military pilots.
