Example of a Custom Design Construction

Use the Custom Design platform to design an experiment to investigate eight 2-level and one 4-level factors. Your goal is to determine which processing factors affect the taste of a Pinot Noir wine. Before the grapes are processed, you set up your experimental design. Once processed, the wine samples are aged for 12 months, then filtered and bottled.

The response of interest is the wine quality measured on a rating scale of 0 to 20, where 0 is the worst and 20 is the best. Most of your vineyard’s product is sold to five large wine distributors. You arrange for a wine-tasting expert from each distributor to evaluate the wine samples for quality. You want to identify the wine-related factors that maximize the rating. You decide that each expert can realistically rate eight different samples. This means that your design needs to have 40 wine samples, or runs.

A blocking factor is used to account for variation that is not necessarily of direct interest. In your experiment, ratings by one expert are likely to have similar characteristics and to differ from ratings by a different expert. Yet, you are interested in which properties of the wine lead to high ratings by all experts.

Because each rater tastes eight wines, Rater is a blocking factor with eight runs per block. For this experiment, only these five raters are of concern. You are not interested in generalizing to a larger population of raters.

You have identified nine process factors for the study. These include the grape variety, the field on which the grapes were grown, and seven other factors related to processing. You can experiment with any combination of these factors. Also, the factors can be varied at will as part of the experiment. Relative to the experiment, these factors are all “Easy” to change. For information about specifying factor changes, see Changes and Random Blocks.

The factors and their levels appear in Table 4.1. Note that all of these factors are categorical. The factors and their levels are also given in the factor table Wine Factors.jmp in the Design Experiment folder of the Sample Data Folder.

To experiment with all possible combinations of these factors would require a staggering 4 x 28 = 1024 runs. However, in this example, you are able to construct a compelling design in only 40 runs.

Now that the experimental goals, factors, and responses have been defined, you can build your custom design.

Add Responses

For this custom design example add your response, the response Goal, and, if appropriate, the Lower Limit, Upper Limit, and Importance. Here, the response is Rating.

1. Select DOE > Custom Design.

2. Double-click Y under Response Name and type Rating.

Note that the default Goal is Maximize. Because you want to maximize the taste rating, do not change the goal.

3. Click under Lower Limit and type 0.

The least desirable rating is 0.

4. Click under Upper Limit and type 20.

The most desirable rating is 20.

5. Leave the area under Importance blank.

Because there is only one response, that response is given Importance 1 by default.

Figure 4.2 shows the completed Responses section.

Add Factors

For this custom design example, enter factors manually.

Tip: Once you have entered factors, you can save them to a data table using the Save Factors option. You can load factors from a factors table to populate the Factors.

1. First, add the blocking factor, Rater. Click Add Factor > Blocking > 8 runs per block.

2. Type Rater over the default Name of X1.

Note that Role is set to Blocking. Note also that only one setting for Values appears. This is because the number of blocks cannot be determined until the desired number of runs is specified. Once you specify the Number of Runs in the Design Generation section, the number of levels for Rater updates to what is required.

3. Click Add Factor > Categorical > 2 Level.

4. Type Variety over the default Name of X2.

Note that Role is set to Categorical, as requested, and that Changes is set to Easy by default.

5. Click L1 and L2 and change them to Bernard and Dijon.

6. Click Add Factor > Categorical > 4 Level.

7. Type Field over the default Name of X3.

8. Click L1, L2, L3, and L4, and change them to 1, 2, 3, and 4.

9. Click Add Factor > Categorical > 2 Level.

10. Type De-Stem over the default Name of X4.

11. Click L1 and L2 and change them to No and Yes.

12. Type 6 next to Add N Factors, and then click Add Factor > Categorical > 2 Level. This adds six categorical two-level factors to your design.

13. Change the default factor names and values:

– Yeast (Cultured and Wild)

– Temperature (High and Low)

– Press (Hard and Soft)

– Barrel Age (New and 2 Years)

– Barrel Seasoning (Air and Kiln)

– Filtering (No and Yes)

Note: Use the Units column to enter units for factors or responses. Units are saved as a column property.

Figure 4.2 Completed Responses and Factors 

14. Click Continue.

The following sections are added to the Custom Design window:

• Define Factor Constraints (not used in this example)

• Model

• Alias Terms

• Design Generation

Define the Model

The Model section shows all main effects as Necessary, indicating that the design needs to be capable of estimating all main effects. Your assumed model reflects your interest in main effects only. However, if you wanted to estimate other effects, you could add them to the Model section. For more information about models, see Model.

Figure 4.3 Model Section 

Define Alias Terms

The Alias Terms section specifies the effects to be shown in the Alias Matrix, which appears later. Open the Alias Terms section node to verify that all two-factor interactions are listed. For more information about the alias matrix, see Alias Terms.

Figure 4.4 Partial View of the Alias Terms Section 

Set Random Seed to Duplicate Design

The custom design algorithm begins with a random starting design. To duplicate a design such as in this example, or in a teaching setting, set the random seed used to define the staring design.

1. Click the Custom Design red triangle and select Set Random Seed.

2. Type 1234 (the random seed).

3. Click OK.

4. Click the Custom Design red triangle and select Number of Starts.

5. Type 2.

6. Click OK.

Note: Setting the Random Seed and Number of Starts reproduces the exact results shown in this example. In constructing a design on your own, these steps are not necessary.

Generate the Design

In the Design Generation section, you can enter additional details about the structure and size of your custom design. In this example, the default design shows 16 runs. But you have five raters, each of whom can sample eight wines. This means that you want a design with 40 runs. Change the number of design runs:

1. Under Number of Runs, type 40 in the User Specified box.

Because you do not want to replicate runs, leave the Number of Replicate Runs set to 0.

2. Click Make Design.

The Design and Design Evaluation sections are added to the Custom Design window. The Output Options panel also appears.

Verify the Design

The Design section shows the runs in the custom design that you have constructed. Later, you are able to randomize the order under Output Options. For now, verify that this design is appropriate for your experiment. For example, check that each of five Raters evaluates eight wines, that all necessary factors are shown, and that none of the settings represent infeasible combinations.

Figure 4.5 Design for Wine Experiment 

Evaluate the Design

The Design Evaluation section provides different ways to evaluate your custom design. For this example, open the Design Evaluation section, and examine the Color Map on Correlations, the Alias Matrix, and Design Diagnostics. For more information about the Design Evaluation section, see Evaluate Designs.

Figure 4.6 Color Map on Correlations 

The Color Map on Correlations shows the absolute value of the correlation between any two effects that appear in either the Model or the Alias Terms section. The main effects are represented by the 15 terms in the upper left corner of the map. The white corresponding to the correlations of main effects with other main effects indicate correlations of 0. This means that all main effects are orthogonal and can be estimated independently.

The only black in Figure 4.6 is on the main diagonal. Black indicates absolute correlations of one, reflecting that each term is perfectly correlated with itself. It follows that no main effect is completely confounded with any two-way interaction. In fact, the absolute values of the correlations of main effects with two-way interactions are fairly low. This means that estimates of main effects might be only slightly biased by the presence of active two-way interactions.

Tip: Hover over cells in the color map to see the absolute correlations between effects. Right-click below the legend to save the correlations to a data table.

Figure 4.7 Partial View of Alias Matrix 

In the Alias Matrix, model effects for your custom design are listed in the column on the left. For a given model effect, a column entry indicates the degree to which the column effect (if active) biases the estimate of the model effect. For example, consider the model effect Barrel Seasoning. If Variety*Press is active, then the expected value of the estimate for the Barrel Seasoning effect differs from an unbiased estimate of that effect. The amount by which it differs is equal to -0.2 times the effect of Variety*Press. Therefore, what appears to be a significant Barrel Seasoning estimated effect could in reality be a significant Variety*Press effect.

Figure 4.8 Design Diagnostics 

The Design Diagnostics section provides information about the efficiency of your custom design. Efficiency measures compare your design to a theoretically optimal design, which might not exist. The efficiency values are ratios, expressed as percents, of the efficiency of your design to the efficiency of this optimal design. For more information about the efficiency measures, see Estimation Efficiency.

Notice that the D-, G-, and A-efficiency values are all 100%. Because your design is orthogonal for main effects, the design is optimal for the main effects model relative to all three efficiency criteria.

The first line in the Design Diagnostics section indicates that your design was constructed to optimize the D-efficiency criterion. See the Optimality Criterion description in Custom Design Options. In this case, your design has D Efficiency of 100%.

Specify Design Table Options

The final steps of your custom design generation are to specify options for the design table and to make the table. Specify the order of runs in your data table using the Output Options panel. The default selection, Randomize within Blocks, is appropriate for this example. Simply click Make Table.

A Custom Design table is created and opens, similar to the one in Figure 4.9.

Note: Your table might look different because the algorithm that creates it uses a random starting design. To obtain the precise table shown in Figure 4.9, set the random seed and number of starts as described in Set Random Seed to Duplicate Design.

Figure 4.9 Partial Custom Design Table 

Note the following:

• In the Table panel, the Model, Evaluate Design, and DOE Dialog scripts are added during the design creation process. The Model script opens a Fit Model window containing the effects that you specified as Necessary in the Custom Design window. The DOE Dialog script re-creates the window used to generate the design table.

• In the Columns panel, the asterisks to the right of the factors and response indicate column properties that have been saved to the columns in the data table. These column properties are used in the analysis of the data. For more information about column properties, see Factors and Factor Column Properties.

Want more information? Have questions? Get answers in the JMP User Community (community.jmp.com).