An evaluation of multiple comparisons procedures in agricultural experiments

Project Abstract

<p> </p><p>The main objective of this study is to test and evaluate the different Methods<br>(or Procedures) of Multiple Comparisons by determining the conditions<br>under which each of them is suitable especially in terms of protection<br>against errors. Data used for this study were generated by means of<br>computer simulation. The Experiment (Simulation) was repeated 500 times<br>for every set of conditions, so that empirical estimates for the Error Rates<br>and the Correct Decision Rate can be computed for each Comparison<br>Procedure. The result of the study shows that the Methods of Multiple<br>Comparisons can be classified into two. The first group consist of LSD,<br>SNK and Duncan, these differ significantly from the MCPs in the second<br>group and are characterized by high levels of Experimentwise and<br>Comparisonwise type I Error Rates. The second group consists of Tukey’s<br>HSD, Scheffe, Bonferroni, Sidak, Gabriel and Hochberg, characterized by<br>relatively low type 1 error rates.</p><p>&nbsp;</p> <br><p></p>

Project Overview

<p> 1.0 Introduction<br>The object of an agricultural experimenter is generally to measure the<br>effect of varying some factor, for example the level of protein in poultry<br>14<br>diets. It is logical to expect that if different levels of protein are applied to<br>different birds, the variation in the weight gains observed would be due<br>partly to the different levels of feeding and partly to the basic variation<br>between birds fed at the same level. The first problem for the experimenter<br>is to disentangle these two parts of the variation i.e. to carryout an analysis<br>of variance (ANOVA) so as to obtain an estimate of the true difference<br>caused by his treatments, i.e. the feeding levels. A significant F-value from<br>the ANOVA indicates that there are differences in the treatment means.<br>The second problem of the experimenter may be to draw some further<br>conclusions. He may want to decide which pairs of treatments are different,<br>or he may want to contrast one treatment effect with the average of some<br>other treatments.<br>To identify where the differences are, he could do a series of pairwise<br>t-tests. The major set back here is that the significance levels can be<br>misleading. If you have 6 groups for example, there will be 15 pairwise<br>comparisons of means; it has long been recognized, however, that if several<br>t-tests have been performed at 5% level of significance, say, the probability<br>that at least one of these is apparently significant is greater than 0.05<br>(Cochran &amp; Cox 1957). If the t-tests are independent, this probability is<br>0.23 for 5 tests, 0.4 for 10 tests and 0.64 for 20 tests.<br>15<br>Multiple Comparison Procedures (MCPs) give more detailed<br>information about the differences among the treatment means, while<br>controlling the probability of making an incorrect decision. Several multiple<br>comparison procedures are available to researchers. Some notable ones are:<br>1. Fisher’s least significant Difference;<br>2. Duncan’s New Multiple range test;<br>3. The Student-Newman Keuls’ Procedure;<br>4. Tukey’s Honestly Significant Difference;<br>5. Scheffe’s Method.<br>Which procedure should be used depends upon which type of error is more<br>serious (Schirley &amp; Wearden 1985). Where a type I error is not serious, a<br>very powerful test like Fisher’s Least Significant Difference (LSD) could be<br>used, otherwise more conservative tests like Tukey’s or Scheffe’s are<br>preferable.<br>The Fisher’s multiple comparison procedure is based on a t-test. If<br>the treatment groups are all of equal size n, then two sample averages<br>(Ó¯1 and Ó¯2) can be tested for a significant difference by a t-statistic. In order<br>to protect the overall type I error rate for the experiment, Fisher’s procedure<br>requires a prior significant F-test in the analysis of variance. With this<br>condition, the overall error rate (comparison wise error Rate, CER) has been<br>16<br>shown to be approximately the α of the F test (Shirley &amp; Stanley 1985).<br>Duncan (1975) considers the error rate for each pairwise comparison<br>and allows a higher rate for pairs of sample averages that are further apart<br>when ordered by size. This method also is believed to control the C E R.<br>All the other procedures of multiple comparisons also aim at reducing<br>the error rates so that valid conclusions can be drawn at the end of the<br>analysis of variance.<br>1.1 Purpose of the Study<br>The purpose of this study is to come up with concise criteria for<br>choosing a suitable method for multiple comparisons of means in<br>Agricultural experiments. This will go along way towards reducing the<br>problem of subjective choice of method being faced by experimenters.<br>1.2 Objectives of the Theses<br>The objectives of this thesis are:<br>To identify the different Multiple Comparison Procedures;<br>To use simulation methods to generate results of Agricultural Experiments<br>for the purpose of testing and evaluating the different methods of multiple<br>comparison of means and<br>To determine the conditions under which each of the various methods is<br>suitable.<br>17<br>1.3 Significance of the Study<br>With any of the multiple comparison procedures, the observed<br>difference between any two means is compared to the appropriate critical<br>value for that procedure. Since the magnitudes of the critical values vary<br>among procedures, results obtained from the application of one procedure to<br>a given set of data will often differ from those obtained if another procedure<br>is utilized. This has led to disagreement among statisticians concerning the<br>appropriate criteria for choosing a procedure for pairwise multiple<br>comparisons of means.<br>It is our sincere hope that at the end of this study, we will come up<br>with a guideline that will help Agricultural researchers and indeed all other<br>scientific researchers, to choose objectively from among the numerous<br>multiple comparison procedures, so that there will be no basis for doubt<br>about the appropriateness of the MCP adopted by any researcher. The<br>significance of this study therefore, cannot be over emphasized.<br>18 <br></p>