An analytical guide to the down-select process.

Do you know when your choice of a vacuum pump or pump system is correct before the purchase is made, money exchanged, and the system installed? Judging by the instances of systems that have failed to perform and live up to expectations, many users have not known. How do you verify that the pump type is the optimum selection and appropriate for the process? When purchasing vacuum systems used for industrial processes, there are risks. Many customers choose to mitigate these risks by using a turnkey system from an integrator or an original equipment manufacturer, the furnace manufacturer for instance, hoping its expertise will reduce the chance the installed equipment will underperform.

Generally, this is a smart decision, and it can remove the burden of becoming an expert and scale the risk back for customers. However, eventually the customer must have a significant level of competence with the theory, use, and operation of the equipment for maintenance, troubleshooting, and optimizing the process. The best place to start this learning curve and get up to speed on the technology in the vacuum industry is to create a specification for your system and use it as a discussion tool with your vendor. Process equipment is a large investment, and the time spent specifying parameters, operational, and equipment specific details will pay big dividends once the equipment is installed and the day-to-day production stress affects the new installation. This is particularly important when a vacuum system is used in the process. Understanding the customer’s vacuum requirements is a critical aspect of a specification for any process, so a carefully specified system will benefit cycle time, reliability, and cost for the life of the equipment.

In many instances, the equipment used by an OEM is advertised as the “best” or “ideal” for an application in part because it is what the OEM sells, and the OEM has confidence in that equipment. There is a strong financial incentive for businesses to sell a certain brand particularly where they receive the best wholesale cost. With all the reputable equipment available, this can be a reasonable default for the customer and increases customer risk only marginally.  However, it is prudent to apply the old adage “Trust but Verify” for large equipment purchases and critical installations. Developing a specification and negotiating through it with the OEM can bring the strengths and weaknesses of a proposed system to light and, at the very least, provide a framework for customer understanding of the engineering used to integrate the proposed selection by verifying the process parameters and comparing them to the expected operational performance. This process can benefit customers in other ways, since a methodical analysis identifies omissions in the customer’s list of requirements. A systematic selection process is just good engineering practice.

How can the complexities of a vacuum system or any complex machinery selection be quantified and systematically evaluated reducing the influence of preferential bias? There are both attributes and variables to evaluate, the interrelations of performance characteristics, pros and cons of machinery types, subjective information from experts, personal experience, cost, and a host of other factors that complicate the decision. All of these should be identified, rated, compared, and contrasted. Without a systematic evaluation method, the volume of information can be difficult to manage and might contain errors or omissions, particularly for anyone unfamiliar with vacuum technology. Using a universally applied methodical evaluation method helps organize inputs to the decision making in a process where many stakeholders are involved. In these cases, sometimes the loudest voice gets the most attention, which might result in less than an optimum selection.

In an attempt to move away from subjective methods that often rely on personal bias, an analytical tool that can organize all the pieces and types of information is needed. It must allow characteristics to be prioritized and systematically rated while retaining the important and valid experiential elements from experts and real-world use but incorporate them in a manner that prevents equipment selection from relying on subjective preferences.

There is a tool developed in the mid-1960s in Japan now used in companies with Six Sigma programs that can be adapted to conduct this variable/attribute type of analysis and selection. Quality Function Deployment (QFD) is a structured method used to identify and quantify requirements, then translate them into critical parameters. It can assist in the prioritization of general items or detailed specifics; it is scalable so it can manage small or large amounts of information with relative ease. A key feature of the method is the ability to make analytical comparisons using ratings of criticality and correlation, to evaluate competing and complimentary characteristics, specifications, or attributes in order to systematically organize and prioritize information adding fidelity and resolution to the process.

With some creativity, the QFD House of Quality (HoQ) format lends itself well to a down-select analysis. Learning to populate the matrix with meaningful information and maximize the benefit has a learning curve, but once familiar with the structure and method of the House of Quality, it becomes an expedient tool. Like any analytical tool, the analysis is only as good as the information entered into the process. Detail improves the accuracy of the process but adds time to create and evaluate. It is a good idea to limit the customer requirements or “Whats” to only the important requirements considered essential and that improve performance when they are optimized. Characteristics that are industry standards, regulatory requirements, and other absolutes demanded from all the candidate equipment, will complicate the QFD unnecessarily, providing no value in the down-select process. Leave these out of the analysis, but not out of the specification. Concentrate on what the customer needs for the equipment and how the equipment performs to improve the resolution of the process. This voice of the customer section is the key to making equipment selections result in delivering the equipment that is required and maximizing satisfaction for the customer and ultimately the users of the equipment.

Figure 1: House of Quality (HoQ), Quality Function Deployment diagram.

There are a few areas worth emphasizing that are well suited for the down-selecting process. The diagram in Figure 1 is a partial example of a QFD analysis for optimizing the selection of a roughing pump for a vacuum induction melting process and shown so all the sections can be seen collectively. Some of the details of the HoQ will be described in this article. Note there are references at the end of the article with tutorials that will describe the HoQ in more detail.

The most critical and first information placed into the analysis is what the customer requires. This is the demanded quality or the “What’s.” They need to be carefully selected and prioritized for the numeric values to indicate what is really important to the customer. This is an area where focus and eliminating extraneous detail is essential. List only the critical requirements to simplify the process. Spend as much time as needed to flesh out the voice of the customer in this section. When selecting equipment, the needs of your customer must be identified in these requirements. Note that all the elements listed drive down to the basics of efficiency, reliability, and convenience that result in low cost and on-time products.

Figure 2: House of Quality (HoQ), What’s section, How’s section matrix with legend.

The next section to work through is the quality characteristics, (functional requirements) or “How’s.” These are the technical requirements (How’s) necessary to realize the customer requirements (What’s) itemized on the left side of the HoQ. They generally are engineering drivers for the system. FIgure 2 shows the detail of the What’s and How’s sections and the relationship table associated with the two along with a legend to define some terminology and the values for weighing importance. To demonstrate the relationship between the “What’s” and “How’s,” look at row 1 and identify the customer requirement for fast pump down time. To achieve a fast pump down time the pump must have sufficient speed for moving the gas within the timeframe desired.

Figure 3: House of Quality (HoQ) Functional Requirements Correlation table

CFM speed is listed as a functional requirement and can be associated with item 1 as a strong relationship providing a method or “How” for that requirement. Also, important here, is identifying the direction of improvement in the bar above the functional requirements. These indicators identify whether maximizing, minimizing, or hitting a specific target is desired for each functional requirement. Using the same case above, maximizing the CFM speed will decrease pump down time so the maximize character is selected. The roof area of the HoQ, Figure 3, provides a table to identify correlations between the functional requirements. The functional requirements are compared and correlation values assigned to pairs of requirements in order to establish a hierarchy of how each requirement may influence another. The correlation ratings are described in the legend. No correlation results in a blank cell. Identifying these relationships forces decisions to consider how technologies used to achieve the customer requirements compliment or limit their implementation.

On the left side of the HoQ, a rating system includes a weight tabulated that assigns an importance value for each customer requirement. This places a relative weight to each customer requirement, ranking it in degree of importance. The weight/importance is established by the customer or considerations based on the customer’s requirements. When requirements are assigned the same value, they are equally important. Ratings must be consistently applied for meaningful results, especially when large numbers of requirements complicate the process. The importance ranking is fundamental for increasing the fidelity of the process and providing a sorting mechanism based on the information from the customer.

Figure 4: House of Quality (HoQ) Relationship table

One particularly valuable feature of the HoQ method is the relationship table that indicates how each customer requirement relates to each functional requirement by assigning a value or rating that is incorporated into the overall calculation of weight/importance of the functional requirements. In Figure 4, a part of the relationship table in the example is shown with some of the What’s and How’s. When a relationship exists between the requirements, it is assigned a symbol that corresponds to a numerical value — typically 1, 3, or 9 — to indicate how strong a relationship exists. Because it affects the rating weight, time here is well spent and worth working through carefully. This analysis sifts through each customer requirement and identifies if there is a relationship with each functional requirement, and if there is, indicates how strong a relationship it is. The numeric values identified in the table are incorporated into the weight/importance value assessment in a column on the left side of the customer requirements.

Figure 5: House of Quality (HoQ), Competitive Analysis.

There is a comparison table on the right side of the HoQ that is used to graphically display how various candidate pumps, in our example, compare against the customer requirements. This competitive analysis is based on a scale from 0 equaling the worst to 5 equaling the best when compared to the demanded quality requirements listed on the left side of the HoQ. When populated, the graph clusters the best performance to the right side of the view, each color representing a different candidate pump, shown in Figure 5. The graph visualizes what candidate would perform the best when contrasted against one another using the customer requirements as the performance sorting criteria.

While all the mathematical values are attribute based to some degree when there are opportunities to assign variable data, they can be captured. As an example, when there are numerical values that are targets, maximums or minimums defined for the functional requirements, they will fall into the target or limit value table at the bottom of the HoQ. When there is a numeric value and a specified direction of improvement, it assists in establishing a competitive ranking in this analysis. Figure 6 shows the target/limit value table.

Figure 6: House of Quality (HoQ), Target or Limit Value.

Below the target/limit value table, there is a row to define the difficulty of accomplishing the target based on available knowledge of the performance compared to the criteria for performance. It is an excellent reference for evaluating if a pump meets a requirement or is at the edge of performance and just able to meet a requirement. For process equipment, operating at the maximum capacity is usually trouble for the system eventually. It is prudent to have some cushion in your pumping capacity so adverse factors such as moisture in the process gas or pump wear do not reduce pump capacity prematurely, rendering the system underpowered for the process.

When selecting new equipment, the selection process is limited by available information. Organizing the down-select process and information systematically helps tease out the less obvious details that might otherwise be overlooked. Using the QFD HoQ to organize the information keeps the analysis and selection process focused on the customer requirements that can be overshadowed by engineering details early on. Using the HoQ where the customer requirements relate directly to the functional requirements maintains a direct association between what needs to be done to how it is done. Controlling this close relationship prevents the equipment selection from straying away from the requirements and allows equipment characteristics to be compared to verify they meet requirements.

All of this analysis can be done before any purchase agreements are made. Carried further, it will result in a specification that is straightforward, easy to understand, and supported with a systematic and logical thought process. The example above is just a small snapshot of the potential of the HoQ. For more complex projects there is a five-step house of quality process that captures customer requirements and focuses them all the way through the production process to build the quality control parameters for purchasing and production. For more information consult the references sited at the end of the article.


  1. Quality Function Deployment (QFD) Iowa State University
  2. QFD Online: House of Quality (QFD) Tutorial
  3. QFD Online moving into the House of Quality
  4. Ficalora, Joseph P. Quality Function Deployment and Six Sigma a QFD Handbook Second Edition: Prentice Hall 2009
  5. Terninko, John. Step-by-Step QFD Customer-Driven Product Design: CRC Press LLC 1997.
  6. ASQ
  7. Excel Template source (Template in Excel format identical to what is used in the article).
Previous articleAMS2750E as it applies to Nadcap
Next articleCase Depth Determination
Christopher Estkowski has been working in engineering for 32 years. After few years leading in the Quality Engineering/Assurance disciplines, he moved into engineering management and is the founder and president of a successful product development firm. Seeking additional challenges, as VP of Engineering at Metallurgical High Vacuum Corporation, Estkowski is engaged in all aspects of industrial vacuum science. Metallurgical High Vacuum is a premium manufacturer and rebuilder of high vacuum equipment used in many industries serving customers across the United States.