1.3. EDA Techniques
1.3.5. Quantitative Techniques
1.3.5.15. |
Chi-Square Goodness-of-Fit Test |
Test for distributional adequacy
An attractive feature of the chi-square goodness-of-fit test is that it can be applied to any univariate distribution for which you can calculate the cumulative distribution function. The chi-square goodness-of-fit test is applied to binned data (i.e., data put into classes). This is actually not a restriction since for non-binned data you can simply calculate a histogram or frequency table before generating the chi-square test. However, the value of the chi-square test statistic are dependent on how the data is binned. Another disadvantage of the chi-square test is that it requires a sufficient sample size in order for the chi-square approximation to be valid.
The chi-square test is an alternative to the Anderson-Darling and Kolmogorov-Smirnov goodness-of-fit tests. The chi-square goodness-of-fit test can be applied to discrete distributions such as the binomial and the Poisson. The Kolmogorov-Smirnov and Anderson-Darling tests are restricted to continuous distributions.
Additional discussion of the chi-square goodness-of-fit test is contained in the product and process comparisons chapter (chapter 7).
| H0: | The data follow a specified distribution. |
| Ha: | The data do not follow the specified distribution. |
| Test Statistic: |
For the chi-square goodness-of-fit computation, the data
are divided into k bins and the test statistic is
defined as
 is the observed
frequency for bin i and
is the expected frequency for bin i. The expected
frequency is calculated by
This test is sensitive to the choice of bins. There is no optimal choice for the bin width (since the optimal bin width depends on the distribution). Most reasonable choices should produce similar, but not identical, results. Dataplot uses 0.3*s, where s is the sample standard deviation, for the class width. The lower and upper bins are at the sample mean plus and minus 6.0*s, respectively. For the chi-square approximation to be valid, the expected frequency should be at least 5. This test is not valid for small samples, and if some of the counts are less than five, you may need to combine some bins in the tails. |
| Significance Level: |
 .
|
| Critical Region: |
The test statistic follows, approximately, a chi-square
distribution with (k - c) degrees of freedom
where k is the number of non-empty cells and
c = the number of estimated parameters (including
location and scale parameters
and shape parameters) for the
distribution + 1. For example, for a 3-parameter Weibull
distribution, c = 4.
Therefore, the hypothesis that the data are from a population with the specified distribution is rejected if 
 is the chi-square percent
point function with k - c degrees of
freedom and a significance level of
.
In the above formulas for the critical
regions, the Handbook follows the convention that
|
is the
upper critical value from the chi-square distribution
and 
is the
lower critical value from the chi-square distribution.
Note that this is the opposite of what is used in some
texts and software programs. In particular, Dataplot uses
the opposite convention.
The normal random numbers were stored in the variable Y1, the double exponential random numbers were stored in the variable Y2, the t random numbers were stored in the variable Y3, and the lognormal random numbers were stored in the variable Y4.
*************************************************
** normal chi-square goodness of fit test y1 **
*************************************************
CHI-SQUARED GOODNESS-OF-FIT TEST
NULL HYPOTHESIS H0: DISTRIBUTION FITS THE DATA
ALTERNATE HYPOTHESIS HA: DISTRIBUTION DOES NOT FIT THE DATA
DISTRIBUTION: NORMAL
SAMPLE:
NUMBER OF OBSERVATIONS = 1000
NUMBER OF NON-EMPTY CELLS = 24
NUMBER OF PARAMETERS USED = 0
TEST:
CHI-SQUARED TEST STATISTIC = 17.52155
DEGREES OF FREEDOM = 23
CHI-SQUARED CDF VALUE = 0.217101
ALPHA LEVEL CUTOFF CONCLUSION
10% 32.00690 ACCEPT H0
5% 35.17246 ACCEPT H0
1% 41.63840 ACCEPT H0
CELL NUMBER, BIN MIDPOINT, OBSERVED FREQUENCY,
AND EXPECTED FREQUENCY
WRITTEN TO FILE DPST1F.DAT
*************************************************
** normal chi-square goodness of fit test y2 **
*************************************************
CHI-SQUARED GOODNESS-OF-FIT TEST
NULL HYPOTHESIS H0: DISTRIBUTION FITS THE DATA
ALTERNATE HYPOTHESIS HA: DISTRIBUTION DOES NOT FIT THE DATA
DISTRIBUTION: NORMAL
SAMPLE:
NUMBER OF OBSERVATIONS = 1000
NUMBER OF NON-EMPTY CELLS = 26
NUMBER OF PARAMETERS USED = 0
TEST:
CHI-SQUARED TEST STATISTIC = 2030.784
DEGREES OF FREEDOM = 25
CHI-SQUARED CDF VALUE = 1.000000
ALPHA LEVEL CUTOFF CONCLUSION
10% 34.38158 REJECT H0
5% 37.65248 REJECT H0
1% 44.31411 REJECT H0
CELL NUMBER, BIN MIDPOINT, OBSERVED FREQUENCY,
AND EXPECTED FREQUENCY
WRITTEN TO FILE DPST1F.DAT
*************************************************
** normal chi-square goodness of fit test y3 **
*************************************************
CHI-SQUARED GOODNESS-OF-FIT TEST
NULL HYPOTHESIS H0: DISTRIBUTION FITS THE DATA
ALTERNATE HYPOTHESIS HA: DISTRIBUTION DOES NOT FIT THE DATA
DISTRIBUTION: NORMAL
SAMPLE:
NUMBER OF OBSERVATIONS = 1000
NUMBER OF NON-EMPTY CELLS = 25
NUMBER OF PARAMETERS USED = 0
TEST:
CHI-SQUARED TEST STATISTIC = 103165.4
DEGREES OF FREEDOM = 24
CHI-SQUARED CDF VALUE = 1.000000
ALPHA LEVEL CUTOFF CONCLUSION
10% 33.19624 REJECT H0
5% 36.41503 REJECT H0
1% 42.97982 REJECT H0
CELL NUMBER, BIN MIDPOINT, OBSERVED FREQUENCY,
AND EXPECTED FREQUENCY
WRITTEN TO FILE DPST1F.DAT
*************************************************
** normal chi-square goodness of fit test y4 **
*************************************************
CHI-SQUARED GOODNESS-OF-FIT TEST
NULL HYPOTHESIS H0: DISTRIBUTION FITS THE DATA
ALTERNATE HYPOTHESIS HA: DISTRIBUTION DOES NOT FIT THE DATA
DISTRIBUTION: NORMAL
SAMPLE:
NUMBER OF OBSERVATIONS = 1000
NUMBER OF NON-EMPTY CELLS = 10
NUMBER OF PARAMETERS USED = 0
TEST:
CHI-SQUARED TEST STATISTIC = 1162098.
DEGREES OF FREEDOM = 9
CHI-SQUARED CDF VALUE = 1.000000
ALPHA LEVEL CUTOFF CONCLUSION
10% 14.68366 REJECT H0
5% 16.91898 REJECT H0
1% 21.66600 REJECT H0
CELL NUMBER, BIN MIDPOINT, OBSERVED FREQUENCY,
AND EXPECTED FREQUENCY
WRITTEN TO FILE DPST1F.DAT
As we would hope, the chi-square test does not reject the
normality hypothesis for the normal distribution data set and
rejects it for the three non-normal cases.
- Are the data from a normal distribution?
- Are the data from a log-normal distribution?
- Are the data from a Weibull distribution?
- Are the data from an exponential distribution?
- Are the data from a logistic distribution?
- Are the data from a binomial distribution?
There are many non-parametric and robust techniques that are not based on strong distributional assumptions. By non-parametric, we mean a technique, such as the sign test, that is not based on a specific distributional assumption. By robust, we mean a statistical technique that performs well under a wide range of distributional assumptions. However, techniques based on specific distributional assumptions are in general more powerful than these non-parametric and robust techniques. By power, we mean the ability to detect a difference when that difference actually exists. Therefore, if the distributional assumption can be confirmed, the parametric techniques are generally preferred.
If you are using a technique that makes a normality (or some other type of distributional) assumption, it is important to confirm that this assumption is in fact justified. If it is, the more powerful parametric techniques can be used. If the distributional assumption is not justified, a non-parametric or robust technique may be required.
Kolmogorov-Smirnov Test
Shapiro-Wilk Normality Test
Probability Plots
Probability Plot Correlation Coefficient Plot
