The Kolmogorov-Smirnov (K-S) test is based on the empirical distribution function (ECDF). Given N ordered data points Y₁, Y₂, ..., Y_N, the ECDF is defined as

The graph below is a plot of the empirical distribution function with a normal cumulative distribution function for 100 normal random numbers. The K-S test is based on the maximum distance between these two curves.

1. It only applies to continuous distributions.
2. It tends to be more sensitive near the center of the distribution than at the tails.
3. Perhaps the most serious limitation is that the distribution must be fully specified. That is, if location, scale, and shape parameters are estimated from the data, the critical region of the K-S test is no longer valid. It typically must be determined by simulation.

Due to limitations 2 and 3 above, many analysts prefer to use the Anderson-Darling goodness-of-fit test. However, the Anderson-Darling test is only available for a few specific distributions.

The normal random numbers were stored in the variable Y1, the double exponential random numbers were stored in the variable Y2, the t random numbers were stored in the variable Y3, and the lognormal random numbers were stored in the variable Y4.

       *********************************************************
       **  normal Kolmogorov-Smirnov goodness of fit test y1  **
       *********************************************************
  
  
                   KOLMOGOROV-SMIRNOV GOODNESS-OF-FIT TEST
  
 NULL HYPOTHESIS H0:      DISTRIBUTION FITS THE DATA
 ALTERNATE HYPOTHESIS HA: DISTRIBUTION DOES NOT FIT THE DATA
 DISTRIBUTION:            NORMAL
    NUMBER OF OBSERVATIONS      =     1000
  
 TEST:
 KOLMOGOROV-SMIRNOV TEST STATISTIC     =   0.2414924E-01
  
    ALPHA LEVEL         CUTOFF              CONCLUSION
            10%        0.03858               ACCEPT H0
             5%        0.04301               ACCEPT H0
             1%        0.05155               ACCEPT H0
  
       *********************************************************
       **  normal Kolmogorov-Smirnov goodness of fit test y2  **
       *********************************************************
  
  
                   KOLMOGOROV-SMIRNOV GOODNESS-OF-FIT TEST
  
 NULL HYPOTHESIS H0:      DISTRIBUTION FITS THE DATA
 ALTERNATE HYPOTHESIS HA: DISTRIBUTION DOES NOT FIT THE DATA
 DISTRIBUTION:            NORMAL
    NUMBER OF OBSERVATIONS      =     1000
  
 TEST:
 KOLMOGOROV-SMIRNOV TEST STATISTIC     =   0.5140864E-01
  
    ALPHA LEVEL         CUTOFF              CONCLUSION
            10%        0.03858               REJECT H0
             5%        0.04301               REJECT H0
             1%        0.05155               ACCEPT H0
  
       *********************************************************
       **  normal Kolmogorov-Smirnov goodness of fit test y3  **
       *********************************************************
  
  
                   KOLMOGOROV-SMIRNOV GOODNESS-OF-FIT TEST
  
 NULL HYPOTHESIS H0:      DISTRIBUTION FITS THE DATA
 ALTERNATE HYPOTHESIS HA: DISTRIBUTION DOES NOT FIT THE DATA
 DISTRIBUTION:            NORMAL
    NUMBER OF OBSERVATIONS      =     1000
  
 TEST:
 KOLMOGOROV-SMIRNOV TEST STATISTIC     =   0.6119353E-01
  
    ALPHA LEVEL         CUTOFF              CONCLUSION
            10%        0.03858               REJECT H0
             5%        0.04301               REJECT H0
             1%        0.05155               REJECT H0
  
       *********************************************************
       **  normal Kolmogorov-Smirnov goodness of fit test y4  **
       *********************************************************
  
  
                   KOLMOGOROV-SMIRNOV GOODNESS-OF-FIT TEST
  
 NULL HYPOTHESIS H0:      DISTRIBUTION FITS THE DATA
 ALTERNATE HYPOTHESIS HA: DISTRIBUTION DOES NOT FIT THE DATA
 DISTRIBUTION:            NORMAL
    NUMBER OF OBSERVATIONS      =     1000
  
 TEST:
 KOLMOGOROV-SMIRNOV TEST STATISTIC     =   0.5354889
  
    ALPHA LEVEL         CUTOFF              CONCLUSION
            10%        0.03858               REJECT H0
             5%        0.04301               REJECT H0
             1%        0.05155               REJECT H0
      

• Are the data from a normal distribution?
• Are the data from a log-normal distribution?
• Are the data from a Weibull distribution?
• Are the data from an exponential distribution?
• Are the data from a logistic distribution?

There are many non-parametric and robust techniques that are not based on strong distributional assumptions. By non-parametric, we mean a technique, such as the sign test, that is not based on a specific distributional assumption. By robust, we mean a statistical technique that performs well under a wide range of distributional assumptions. However, techniques based on specific distributional assumptions are in general more powerful than these non-parametric and robust techniques. By power, we mean the ability to detect a difference when that difference actually exists. Therefore, if the distributional assumptions can be confirmed, the parametric techniques are generally preferred.

If you are using a technique that makes a normality (or some other type of distributional) assumption, it is important to confirm that this assumption is in fact justified. If it is, the more powerful parametric techniques can be used. If the distributional assumption is not justified, using a non-parametric or robust technique may be required.