Winter 2014 --
Fact or Fiction?
The Proper Use of Military Flight Operations Quality Assurance (MFOQA) Analysis and Unstable Approach Rates
By SEAN BORDENAVE, AMC A3TO
Whoa, whoa, whoa - back the bus up! Where did you get those grades? Did you give me credit for the winds? As a matter of fact, have any of you pinheads flown into Lajes Airfield?! Obviously not or you would be familiar with those killer winds! Did you factor that into your unstable rates? You try being stable in those conditions!
Let's roll some video to prove my point!
As this intro suggests, MFOQA analysis and the unstable approach rates unfortunately sometimes end up being a parochial argument mired in "point values, scoring, and single events" rather than focusing on trends, methodical analysis, and risk. Let us break down some of these parochial points and separate fact from fiction to discover the proper use of MFOQA analysis and unstable approach rates.
Fiction - Unstable Approach Rates are a Pilot's Report Card!
One of the benefits of MFOQA analysis is the ability to statistically measure SOP compliance in order to identify trends. However, a better description of this particular benefit is the ability to analyze flight data to detect mishap precursors and identify mitigation measures. Unfortunately, we sometimes fixate on oversimplifying the MFOQA analysis in an effort to simplify the problem statement, cause, and fix. This oversimplification is how we arrive at the statistically measured SOP compliance in order to identify trends. We then quickly and erroneously translate an unstable approach rate into a problem statement of aircrew not flying stable approaches and the cause of the problem being non-compliant and unprofessional pilots. The easy solution would be to shoot any pilot flying an unstable approach and the pilot monitoring for not directing the go-around. After executing a few of these non-compliant pilots, Darwinism would reign true, and the rest of the herd would get the picture. The following month, we would see a dramatic decrease in unstable approach rates. DONE! NEXT PROBLEM! Unfortunately, this is how we end up with the false perception that unstable approach rates are a pilot's report card! Obviously, this is not the intent of producing an unstable approach rate. Now that we have identified the incorrect meaning of unstable approach rates, let's look at what the unstable approach rate is really trying to tell us!
Our Lajes Airfield Unstable Approach Example ... With the Proper Context This Time
Since we started with the Lajes examples, we can rewind and do a double take with the unstable approach rate into Lajes Airfield. Since our KC-135 brethren fly into Lajes on Coronet Missions (fighter drags across the pond), we will "peel the onion back" on KC-135 unstable approach rates to demonstrate the proper use of MFOQA.
First Layer of the Onion - the Big Picture
Before we jump into the Lajes unstable approach rate, we need to start with the big picture. The first layer is the overall KC-135 unstable approach rate. The overall unstable approach rate for the entire KC-135 fleet in the past year (May 2013 to April 2014) is 12 percent. This encompasses over 85,000 KC-135 approaches in the MFOQA database, which is a pretty healthy sample size. Additionally, the KC-135 data capture compares the number of KC 135 sorties in the MFOQA database to the actual number of KC 135 sorties flown. It is approximately 85 percent, which further reinforces that the unstable rate is representative of the overall fleet trend. Finally, this overall KC-135 unstable approach rate encompasses all units and all airfields visited in those 85,000 sorties.
A key takeaway from the overall unstable approach rate is that we now have a baseline for measurement and analysis. The overall unstable approach rate now allows us to objectively determine if we are trending upwards (a negative trend of a higher unstable approach rate) or trending downwards (a positive trend of a lower unstable approach rate). More simply, the overall unstable approach rate allows us to objectively measure if we are doing better or worse than average. This baseline measurements tell us what is "normal."
But we don't stop at the overall unstable approach rate in our analysis--we continue drilling down! Now that we know what "normal" looks like, we traditionally look for negative trends or "worse than normal." As a reminder, our primary focus in MFOQA analysis is to detect mishap precursors. Typically, those mishap precursors are worse than normal.
Second Layer of the Onion - "One of These Things is Not Like the Others."
With the baseline established, we can now drill down further into areas of focus by analyzing different flight parameters and data points available in the databases. Armed with the overall unstable rate, we can now employ the Sesame Street analysis technique, "One of These Things is Not Like the Others." Simply put, the overall unstable rate allows us to quickly identify points that are higher than average (and not like the others).
As with any good analysis, one of the first basic questions you want answered in problem identification is "Where is it happening?" One of the data points available in the MFOQA database is location, which allows us to ask "What airfields have a high unstable approach rate?" With that question in mind, we can now hone the analysis to highlight airfields with higher than normal unstable approach rates.
In the chart (right), we can quickly see all those airfields with unstable approach rates greater than 12 percent. Our Lajes example comes into focus with an unstable approach rate of 20.8 percent, highlighting that Lajes is "one of those things that is not like the others." Now that we have established Lajes has a higher than average unstable rate, the next logical question is "Why?"
The Recap of What We Learned from the MFOQA Analysis Thus Far
The MFOQA analysis has yielded a tremendous amount of critical data points in over 85,000 approaches. The key data points are:
Who: KC-135
What: Unstable Approach rate of 12 percent
When: May 2013 to April 2014
Where: Lajes (in our example) with an unstable approach rate of 20.8 percent.
Why: Unknown, but we know that "approach speed high" is the main culprit (factor), as we will soon see.
It is also important to note that we have very quickly and efficiently extracted those key nuggets of data from over 85,000 approaches ... without a mishap!
Third Layer of the Onion - Detecting Why
The MFOQA analysis has led us to an important question: Why is Lajes unstable more than other airfields? We have reached a limitation of our MFOQA analysis. MFOQA does not tell us the "why." MFOQA only gives us what transpired, how the pilot was controlling the aircraft, and the flight parameters at the time of the event. Unfortunately, the aircraft doesn't tell you why it happened and about other factors (such as other traffic, ATC constraints, weather, and terrain) the pilot was trying to manage during the approach.
But let's not be dismissive because MFOQA did not determine why ... yet. MFOQA gives us some specific clues about where to look for this answer. Our MFOQA analysis can tell us what triggered the unstable approach. Getting back to our Lajes Example, the KC-135 MFOQA analysis tells us that 16.7 percent of the unstable approaches triggered for approach speed high. More specifically, the KC-135 approach speed high trigger is set at Vapp + 15 knots (airspeed) for 5 seconds below 500 feet, so we now see that is a pretty significant airspeed differential.
Armed with the approach speed high clue from the MFOQA analysis, we can now look at other credible data sources to determine why. Of course, it does not take a rocket scientist to know that most aircraft T.O.s direct an increase in airspeed for gusty winds (gust factor), so wind is an obvious first choice as a factor causing the speed high trigger in the MFOQA analysis.
In our Lajes example, we can quickly see the evidence that winds are a mostly likely reason for the unstable approach. The historical wind data for Lajes airfields from the 14th Weather Squadron shows how strong the gusts can be at Lajes:
Additionally, the historical wind data shows the winds at Lajes Airfield are greater than 25 knots a high percentage of the time.
Before we declare victory on proving why, we must comply with the high school English teacher's rule of being able to cite at least three separate data sources in our bibliography. For our final source, the Europe, North Africa, and Middle East (FLIP) Supplement states in the remark section for Lajes (below):
... parl to rwy 0.25 NM E to 503'. Winds are extremely hi dur Oct-May. Expect lo level windshear and large hdg corrections on final apch. STRONG CROSSWIND POSSIBLE is incl in the fcst when cond are favorable for development of haz crosswinds. This rmk should alert aircrews to closely mnt LAJES Wx while enrt. If possible, consider maintaining fuel reserves to reach altn destn outside of the Azores wx pat. Rwy not vis dur portions of ...
In our Lajes example, we can now start to see that strong winds are most likely a factor in the unstable approaches for that airfield. With a little more detective work, the MFOQA analyst can link a specific unstable approach with a specific historic METAR to further make the connection between winds and the speed high triggered events.
In previous cases, MFOQA analyses have been able to link unstable approaches at some airfields with external factors such as winds, terrain, airfield restrictions, and instrument approach requirements. So, while the MFOQA analysis did not specifically tell us why, it gave us enough clues to provide linkages between external factors and unstable approaches to provide a logical conclusion about why.
Fiction - Vindicating Pilots
Well, it looks like our Lajes example vindicates pilots for causing the unstable approaches. Wrong! This perception is also fiction! First, MFOQA does not assign blame; it looks for factors that could potentially lead to a mishap.
Those factors could be pilot error, organizational factors, external factors, or combination of some or all of those. We have learned from traditional mishap investigations that when all those contributing factors (hazards) line up in "perfect" succession, we increase our chance of a mishap. MFOQA helps us seek out those hazards and find risk mitigation strategies like our stable approach procedures.
Second, even though MFOQA analysis helped us identify that an external factor is contributing to the approach instability, it does not mean that it is acceptable to fly unstable approach at that location. On the contrary, when these external factors are identified through the MFOQA analysis, it highlights an increased level of complexity that a pilot must still successfully manage. A high unstable approach rate at a particular airfield is telling you that you had better bring your "A" game to that location. Wise old pilots will tell you that for every successful takeoff, you should have a corresponding successful landing. External factors (e.g. weather, terrain, high pressure altitude, etc.) make it more difficult for you to ensure that you have an equal amount of takeoffs and landings in your logbook! Think of it this way: an approach into Lajes may not be that bad ... until you factor in darkness, or IFR conditions, or gusty winds, or strong crosswinds, or heavy weight landing, or all of those conditions!
Third, vindicating or blaming pilots is not a goal of the MFOQA program. Using unstable approach rates to decide which weapon system has the worst pilots or the most undisciplined pilots or using external factors as an "excuse" for not flying a stable approach are all myths that distract us from seeing the true use of the analysis.
Finally, we do not set approach trigger events to make pilots look good, or look bad, or to chase a metric.
Again, unstable approach rates are not a report card. The approach trigger events are set and approved by MAJCOM weapon system subject matter experts to provide effective measurements to detect trends. In our Lajes examples, we learned that winds were mostly driving approach speed high unstable approaches. This example proves that the approach trigger events were working as advertised--as an issue was detected that needed focus and emphasis. Additionally, we can sometimes detect seasonal or environmental trends and see external factors like winds. The triggered events and the subsequent analysis made us aware that winds were an issue in the approach and landing at Lajes!
Fact - Aggregate Trends
Before the advent of MFOQA (and other proactive safety programs), we identified negative trends mostly using number of aircraft mishaps, mishap classification (Class A, B, etc.), number of fatalities, and Safety Investigation Board (SIB) Analysis about what went wrong. MFOQA allows us to measure those same trends without the negative effects of mishaps. The real trends that we seek to identify and measure are the trends that could result in a mishap. We are looking for the "iceberg" and trying to determine if it is the tip of the iceberg (emerging new trend) and whether it is a Titanic-sized iceberg or an ice cube (risk severity).
MFOQA analysis and unstable approach rates are more than just a metric or value. Additionally, we cannot summarize the analysis and unstable approach rate into a single number or percentage, as the analysis and trend starts to lose context and meaning. The intent of the analysis is to determine whether trends are emerging, seasonal, or sustained. The more we drill down into the layers of data, the more we start to see clarity, context, potential factors, and root causes of the trends. This is where we find the true benefit of MFOQA! So don't get sucked into the vortex of the report card mentality or distracted by MFOQA myths. The goal of MFOQA is analyzing flight data to detect mishap precursors and identify mitigation measures.
Our Approach:
· Collect data by all possible means
· Identify trends
· Mitigate risk
· Inform aviators
· Foster a Just Culture!