The Uptime blog

A few weeks ago I wrote about the importance of helping mechanics to fix rare problems faster—accelerating the repair of systems that don’t fail very often. Today’s post follows up on that theme as it relates to the aviation MRO industry. Because aircraft systems are generally quite reliable, when something fails unexpectedly it is called a non-routine event (NR). (It should be noted that aircraft systems have multiple redundancies, so an NR is not necessarily a cause for concern. But it still needs to be fixed.)

Non-routine events are a major source of unscheduled aircraft maintenance. (Other industries may refer to this as a break-fix event because something failed outside the normal maintenance schedule.) The question therefore is, “What to do about NRs?” The idea of planning for unscheduled maintenance seems like an oxymoron. After all, how can you plan for something that you can’t predict? And that is really the point; on any complex machine—and an aircraft is really complex—you know something is going to break, you just don’t know what it will be.

To explain the difficulty of the problem let’s look at some real customer data.  In a fleet of wide body, long-haul aircraft over a two-year period, an airline found that there were over 3,000 non-routine events that could be traced back to almost 300 different systems. 10 systems caused almost half of the NRs. (In fact, 1 system alone caused 20% of the NRs.) But the majority of the NRs (51%) were caused by failures in one of the other 270+ systems.

When this data is plotted on a graph (above) it is easy to see the trend. The top 10 causes of NRs occured much more frequently than the rest of the systems. (The number 1 cause needs unscheduled maintenance almost once a day and number 10 occurs at least every other week.) The rest of the systems in this study (97%) occurred an average of 4 times per year, forming a long tail on our statistical data. However, since the top 10 problems account for less than half of all non-routine events, if an airline wishes to reduce the overall impact of NRs, then it must address the long tail.

Several conclusions can be drawn from these numbers:
• Most of the systems on an aircraft are quite reliable. (Given the complexity of an airplane and the physical stresses it endures, this is a great testament to the aircraft manufacturers.)
• Mechanics can quickly gain experience fixing the top 10 causes of NRs.
• Mechanics do not gain significant experience fixing the other half of the NRs. (They just don’t see them often enough.)

A typical maintenance approach is to focus attention on the most frequent failures—the top 10. Training to fix the top 10 is pretty straight forward however, the long tail consists of many different problems that happen infrequently. (These are individual systems that rarely break but, when added together, account for the majority of the unscheduled maintenance.) The only way to deal with such a wide-ranging set of problems is to improve the mechanic’s ability to respond to the unknown. Better training is not the solution, automating information systems is.

It should be obvious but when time is tight and a repair is urgent it’s best to bring service information to the mechanic, rather than the other way around. This can be accomplished by integrating maintenance information systems with maintenance planning, inventory and other back-office systems to provide a fully integrated scheduling and execution environment. Then, no matter what type of problem the technician uncovers they can quickly obtain the information they need and begin the process of repair.

Along with our partner Oracle, I’ll be co-presenting an Air Transport World magazine webinar on this topic in a few weeks (December 9, to be specific); if you’d like to attend that webinar, sign up here.

The October 2008 issue of Air Transport World includes an article titled “Mechanics Wanted,” regarding the challenges of recruiting enough aviation service technicians to meet the future needs of the aviation industry. 

According to the article, the demand for aircraft maintenance technicians is expected to more than double in the next two decades, to support fleet growth and replace retiring technicians. This echoes what we’ve referenced in a previous blogs about the graying of the workforce and the need to accelerate the education of new service technicians.

Why can’t the industry recruit the number of technicians they need? First of all, it takes a long time to complete maintenance training, typically five years to be licensed to sign-off on repairs. Second, service technician jobs often don’t pay as well as other professions (both inside and out of the aviation industry). The opportunity to work on aircraft is just not appealing to enough people. 

To make matters worse, once recruits are in place it has become harder to train them because there aren’t enough planes on the ground to provide the hands-on experience they need. (This is a good thing for travelers but makes training a challenge.)

“It’s getting more and more difficult to get access to a real maintenance environment for training purposes because the planes are always in use, the maintenance schedule is getting tighter and tighter, ground time is getting shorter and shorter.” —Klaus Schmidt-Klyk, Director of Marketing, Sales, and Customer Service, Lufthansa Technical Training.

Since Enigma is in the business of reducing maintenance delays, we are happy to hear that aircraft uptime is increasing, and customers tell us that our aircraft maintenance solutions are part of that improvement. But we sympathize with the MRO shops and airline operators that are trying to train technicians and believe that technology plays a strong role in achieving further improvements.

I just returned from the 2008 ATA E-Business/ S1000D Forum in Budapest, Hungary. This was the first year that ATA (Air Transport Association) combined the E-Business and S1000D meetings. The resulting event concentrated more on authoring/tech pubs than on the aftermarket service and support focus of past years. Almost 300 people attended, which is larger than either of the two individual events in 2007 but smaller than the combined total from last year. It wasn’t clear to me if this drop-off was because of the location, the topics, or the economy.

This year’s event drew fewer airlines than last year (14) but those that attended were serious about improving maintenance processes and job cards. There were also many OEMs in attendance trying to learn how to implement the S1000D spec but a few wanted to learn how to improve customer support and field service through electronic catalogs. Enigma stood out as one of the only exhibitors solely focused on making service information usable to mechanics and technicians, and a number of airlines sought us out for a demo. Each one expressed an urgent need to reduce maintenance costs. So while the quantity of attendees was down, the quality seemed to be up.

One change from last year was an increase in the amount of competitive eavesdropping. I often noticed Boeing and Airbus people hovering around as I gave product demos. This was particularly amusing from Airbus because when they presented the A380 information system during one of the open sessions it looked a lot like the Enigma solution from 2002. However, during the presentation Airbus mentioned some problems around incremental updates so I guess they haven’t copied everything.

Regarding the event itself, ATA did a fantastic job planning and coordinating the location and the topics. The hotel was first-rate and the city was very pleasant, despite the fact that the Communists were marching to commemorate the failed 1956 Revolt and protesting all things democratic. (Perhaps that’s why my bag arrived home two days after I did.) For me, the whole event was a worthwhile adventure. I learned a few things that could improve our products, and I was able to meet potential customers and partners to discuss the business opportunities in the aviation aftermarket. Kudos to Brad Ballance and the ATA!

MRO Asia 2008 concluded today in Singapore. Here are some of my impressions from this year’s show:
 
1. The prevailing mood was definitely on the somber side. Every speaker mentioned the “financial crisis” or “market turmoil” at some point or other in his/her speech, some more than once. The presentation from the Association of Asia Pacific Airlines (AAPA) included one slide that had several bullet points all saying, in diffferent words: “expect bad times ahead, we have no idea what to expect”. The continuously falling oil price was small consolation in the general feeling of uncertainty.
 
2. The exhibition floor was comparable in size to last year’s show in Shanghai, but some of the players in the MRO IT space were conspicuously absent. While most of the best-of-breed MRO providers were there, Swiss Aviation Software was absent. The two “gorillas” in this space - SAP and Oracle - were also absent, although Axon Global was there, with their new iMRO offering, which basically is to replace SAP MRO. From the content delivery players, OpenConnent was not there. Perhaps the European players do not view Asia as a market they need to invest in.
 
3. As for the attendees, most Asian airlines and many of the non-Asian ones, were represented. However, there seemed to be fewer delegates compared to last year, definitely fewer representatives from Chinese airlines. I know for a fact that one of the airlines planned to send five senior delegates but, with the recent upheavals in the world economy, decided at the last minute to cancel the trip and send two junior delegates instead.

4. Any vendor that attends a major show like MRO Asia carefully monitors the topics that customers and prospects want to discuss. In this regard, Enigma InService Job Card Generator drew a lot of attention from attendees. It is clear that airlines and MROs view the ability to provide intelligent job cards quickly and easily as a key component to accelerating maintenance. Several airlines were seeking even greater automation, and took great interest in Enigma InService MRO. In this regard, it was nice to have important customers, like KLM, at the show that can verify the cost reductions and productivity improvements that are available through advanced technology.
 
Next year’s show is in Hong Kong. See you all there!

A few years ago I met with a customer service executive (Bill) from a well-known office equipment manufacturer. Over the course of several months we had a really good discussion regarding the best ways to improve field service. During this time Bill shared the service statistics for one of his products, which reflected the data I’ve seen in many other industries. What surprised me was the way he interpreted these statistics. Here’s what he found:

1. Over an 8-month period, one product line received 5591 service calls
2. Those calls were the result of 423 different fault codes
3. The top 20 fault codes accounted for 50% of all service calls
4. The other 403 fault codes (95% of total) were responsible for the rest of the service calls
5. Each of those 403 fault codes occurred no more than 51 times. (In fact, 364 fault codes—86%—were tripped 20 times or less.)

Bill told me his company was trying to improve service by focusing on the most common problems—the top 20 fault codes—but he thought this was the wrong approach. He believed the top 20 problems didn’t pose an issue for most technicians because they repaired them so often. It was the other 50% of the calls that made life difficult.

The key to understanding Bill’s perspective can be found in the last two bullets—half of the service calls involved fault codes that occur less than 1% of the time. (In fact, 140 service calls involved fault codes that only occurred once.) Therefore, field technicians got little or no experience repairing these rare problems. Further analysis showed these service calls often took more than two hours to resolve and were at the heart of scheduling delays and blown response times.

Bill felt that the real key to improving customer service was not faster repairs for well-known problems but a more consistent approach to all problems. He said that once he changed his focus from 20 fault codes to 423, he realized the answer was not to streamline call handling, travel, part lookup or closeout but rather it was improving information access in general. If his service teams had an integrated parts and service information system they could address any fault code efficiently and consistently.

Recently, Bill told me his company never did act on his advice and he subsequently left the firm. He realized that a slick new call-center gets more political traction than a system to support field service engineers. After all, a tour of the call center was a great selling tool during customer visits. Unfortunately, at the end of the day it’s those same customers that suffer the results of this decision.

An article in the August issue of Airline Procurement Magazine discusses the challenges of responding to aircraft-on-ground (AOG) events. An AOG is described as any equipment-related event that prevents an aircraft from being dispatched. (This means that bad weather doesn’t count as an AOG.) The article points out that an AOG event results in lost revenue, higher maintenance costs and decreased passenger commitment. Because the impact of an AOG is so great, finding ways to minimize the impact is critical.

If you think the previous paragraph merely states the obvious, I agree. The article fails to discuss any of the means used by airlines to minimize the impact of AOGs. That’s a shame because there is a lot of software out there that can address various aspects of the AOG problem. Whether it is troubleshooting/fault isolation, parts identification, scheduling service, or inventory management, there are many pieces of technology that can help solve this puzzle.

Those of you not in the aviation industry may be wondering if this matters to you. It does. Every industry has some version of an AOG. Whether the crane stops hoisting, the excavator stops digging, the combine stops harvesting or the top drive stops turning, when production comes to a standstill companies will spend whatever it takes to get up-and-running again.

It may never be possible to eliminate AOGs. Sooner or later everything breaks, sometimes without warning. Smart companies however, find ways to guard against such risks and to reduce the cost of emergency maintenance and repairs. Maintenance software that can be fully integrated with other enterprise systems ensures that accurate information flows seamlessly, which is the key ingredient for good maintenance decisions and rapid response.

This Airline Procurement article highlighted several aspects of a serious problem—a problem that affects every industry. Now it’s time to start discussing solutions.

Manufacturing Business Technology recently announced its MBT 2008 Innovation Insight Awards, and among them was an Honorable Mention for Advanced Technology Services (ATS), a company that provides maintenance and repair services for manufacturing plants.

ATS received the kudos because they deployed Knowledgebase—their own proprietary database of processes, procedures and best practices for their field service technicians. What is interesting is that much of the content in this custom application came from the technicians themselves. Knowledgebase helps ATS to fix equipment faster, thereby reducing their customers’ downtime.

Here at Enigma we agree wholeheartedly with capturing and reusing technical expertise, which is why our products such as InService MRO, InService EPC and the Enigma 3C Platform all include a feature called eNotes. (For a mini product demo of this feature, see our August 15 podcast post.) 

While service manuals may contain some previously established best practices or proprietary techniques, eNotes allow mechanics to insert new comments connected to the original content. This lets companies capture feedback on-the-fly and also gives context to the comments because they are linked to the specific task that was being performed. eNotes are also available to other users (based on authorization/distribution rules). This makes it easy for other mechanics to learn from their peers and for OEMs to update documentation so that it reflects real world experience.

But it’s not just eNotes that make Enigma products useful; our products aggregate and deliver all product content in one place, in ways that streamline the entire maintenance execution process. The ability for service technicians to use a single application to pull relevant OEM information, whether parts information or service manuals, from multiple sources, is a tremendous advantage in the field or the service depot. The description of ATS’ Knowledgebase is impressive, but it sounds like an expensive solution. What is more impressive is that, today, any company can provide similar capabilities using Enigma’s out-of-the-box solutions.

I thought of giving this blog post the title, “Yeah, what he said!” That’s how strongly I feel about the comments made by Scott Luckett at the 2008 Aftermarket eForum. Mr. Luckett is the vice president of Technology Standards and Solutions at the Automotive Aftermarket Industry Association, so he is probably well qualified to speak on the topic of electronic data and the automotive aftermarket. His premise is that the lack of accurate electronic data is increasing the cost of doing business. I agree.

The problem is that Mr. Luckett oversimplifies a number of issues. He wants all aftermarket automotive data to be electronic. But what does that mean, what format should it be in? Should it be PDF, XML, Word? He talks as if getting the data into an electronic format and following certain standards is the answer. It’s not that simple. For example, the aerospace industry has strict data standards but that has not helped them achieve the electronic nirvana that Mr. Luckett envisions. Even though every aerospace manufacturer adheres to the same standards, the data still doesn’t integrate well. The problem of data interoperability is even worse in the automotive industry where there are more OEMs, more suppliers and fewer standards. The vast number of brands and trim packages ensures that agreement on standards, and how to interpret them, is a long way off.

The key to interoperability is not the data, but the software that uses that data. Mr. Luckett properly points out that Services Oriented Architecture (SOA) is the key technology for reliable interoperability. This is especially true for electronic parts catalogs (EPC), the starting point for most parts and service activity. Because of the difficulty in establishing usable data standards, the Automotive Aftermarket Industry Association (AAIA) should spend its’ time advocating that OEMs move to electronic data using SOA. This would help OEMs realize the benefits that modern technology brings to the aftermarket, without waiting for standards to be finalized. Such a strategy would simplify the deployment of innovative solutions that improve part selection and increase first time fix rates (FTFR), which benefits the customer, the OEM and the dealer. Furthermore, such a strategy helps preserve one of the OEMs’ chief worries—protecting intellectual property.

I’m looking forward to my first visit to Budapest, Hungary, not only to sample the goulash but to attend the ATA e-Business Forum in October.  If your company has any responsibility for aircraft maintenance, it’s worth sending someone to this event. I’ve been focusing on aftermarket service and support technology for eight years, but last year was my first ATA e-biz show and I have to tell you, the people that attended impressed me. They knew their stuff and were very involved in the technical sessions. What I really liked was that they wouldn’t let the speakers give easy answers to hard questions.

Of course it wouldn’t be an aviation technology event without the inevitable hype regarding S1000D and how it’s changing the world. There were plenty of vendors trying to convince airlines, MROs and OEMs to buy their solution and get a jump on the competition in moving to the new standard. But in between all the hype, there were a lot of people talking about how to improve the use of information they already had, without S1000D. (For the record, Enigma is a strong supporter of S1000D but we think airlines, MROs and OEMs need solutions that work with legacy data as well.) These people understood that at the end of the day, even though the new standard holds a lot of promise, it will take years (decades?) to get all the relevant information converted. After all, we’re talking about Terabytes (Petabytes? Exabytes?) of information and in the meantime the airlines, MROs and OEMs have businesses to run.  

With that in mind, many of the people attending ATA e-biz were more interested in how to improve their business processes today…with the assurance of supporting S1000D tomorrow. These people wanted to accelerate job card processes, to ensure maintenance crews have the right information all the time, every time. They were looking for ways to improve non-routine job card processes, so that aircraft maintenance could stay on schedule, or ahead of schedule. They wanted to improve the efficiency and consistency of one of their largest workforces.

The people I met were realists who understood that, for their current fleets, it will be a long-long time before all the maintenance manuals, repair manuals, engine manuals, component manuals, illustrated parts catalogs, service bulletins, technical revisions, planning documents, equipment lists, schematics and other configuration information is converted to the new format (if ever). All the airline folks agreed that S1000D was gaining acceptance with the 787 and A380 (some, not all of the manuals will be S1000D) but that didn’t help them with the rest of their Boeing, Airbus, Bombardier, Embraer, ATR, Fokker, BAE, GE, Pratt & Whitney, Rolls-Royce, CFM, IAE and Honeywell fleets.

The best thing about ATA e-biz is that a lot of really good issues, that seem to be hidden but have far-reaching consequences, are brought under the microscope, inspected, discussed and in some cases resolved. What I learned from last year’s ATA e-biz was that it’s a bunch of smart people having honest discussions about applying technology to current maintenance issues and future business opportunities. I hope this year’s event is just as good.

In asset-intensive industries like oil & gas, energy & utilities, mining and transportation, capital equipment downtime can cost hundreds of thousands of dollars per day. One of the primary challenges to keeping assets up and running is the daunting complexity and various configurations of the equipment: maintenance planners and service technicians must reference enormous volumes of documentation, some of which is spread across multiple locations; some in paper form, others in online databases.

Many companies in asset-intensive industries have made substantial investments in Enterprise Asset Management (EAM) systems or Computerized Maintenance Management Systems (CMMS). While these systems can have a wide range of capabilities, broadly speaking they try to improve maintenance planning by indicating what to do and when to do it. This often includes job scheduling and assignment, preventive maintenance (PM), inventory control, and other activities associated with asset availability, reliability and operational safety.

Largely beyond the scope of EAM/CMMS applications is direct support for how to perform maintenance activities—particularly unplanned maintenance activities—in the field. While some EAM systems can generate work orders, and perhaps material lists for various maintenance tasks, the primary purpose of these systems is to support the maintenance planner, not the mechanic.

That’s where Enigma’s technology comes into play, because it delivers the how to content as a fully integrated maintenance solution: fault isolation/troubleshooting manuals (how to diagnose), service manuals (how to repair), service bulletins (how to incorporate the latest procedures), parts catalogs (how to find the proper parts) and collaboration (how to share maintenance history/experience). Improving the efficiency of mechanics has a major impact on reducing asset downtime and thus on bottom-line profitability and competitiveness; it’s common sense to increase the value of EAM by integrating Enigma into the equation. In future blog posts we’ll provide specific examples that show companies how to further leverage EAM/CMMS investments.