This is probably the most basic and widely used of the non-parametric statistical tests. Developed in the 1930s by Andrei Nikolaevich Kolmogorov and Nikolai Vasilyevich Smirnov, test allows the comparison of a frequency distribution to some other known (continuous one-dimensional) distribution, such as a Gaussian normal distribution.

Some people use the K-S test as an alternative to the student T-test. Unfortunately, this use violates basic constraints of the K-S test (i.e. that one distribution must be known beforehand), and is meaningless. (However, this is OK if you're comparing a sample to an entire population).

The test works by comparing cumulative frequency values for the data points against the cumulative frequencies expected for the distribution in question.

Although it's called a 'goodness of fit' test, it's really a 'badness of fit' test, as all it can tell you is whether the data deviate significantly from the distribution. Because of this, we have to formulate our hypotheses in a counterintuituve way: The null hypothesis H0 is that the data do not significantly deviate from the distribution.

Before you can perform the test, you must have an "empirical cumulative distribution function" (ECDF) describing the expected cumulative frequencies. After deciding on a confidence interval you can then order the data, and calculate the actual cumulative frequencies for the points. You then determine a statistic

D = max |ECDF(i) - F(i)|

from these values. If this D value is less than a critical value for the number of points and the confidence interval you chose, you cannot reject your null hypothesis, and must proceed as if the data fit the distribution.

Obviously, if you want the data to fit the distribution, it's tempting to jump to the conclusion that the data do fit the distribution. Unfortunately, this latter fact isn't determined by the test. A rather unsatisfying conclusion, and the only way to achieve a higher comfort level is to use more data points (which lowers the critical values).

For an example, let's test a sample of random numbers. I generated 41 random numbers (in the Data column of the table below) to represent sample data points. We'll pick the confidence level of 0.05 used commonly in the social sciences. The critical value for D is thus 0.3004.

To test the sample distribution against a normal distribution, we first calculate a Z score for each sample point. Then generate the expected cumulative frequency for each Z from a formula which estimates this value for the normal distribution1. F(Z) is the actual cumulative frequency encountered (the point index divided by 41). Finally, calculate the difference between ECDF(Z) and F(Z) for each Z:

```
Point    Data     Z     ECDF(z)  F(z)   |F(z)-ECDF(z)|
----- -------  -------- ------  ------  --------
1      0.7807  -1.5248  0.0608  0.0244  0.0364
2      2.5164  -1.4670  0.0690  0.0488  0.0202
3      7.9011  -1.2875  0.0981  0.0732  0.0249
4      8.0303  -1.2832  0.0989  0.0976  0.0013
5     10.6726  -1.1951  0.1155  0.1220  0.0064
6     12.8190  -1.1235  0.1303  0.1463  0.0161
7     14.6225  -1.0634  0.1436  0.1707  0.0271
8     17.1340  -0.9797  0.1635  0.1951  0.0316
9     20.6383  -0.8629  0.1941  0.2195  0.0254
10    21.2010  -0.8441  0.1993  0.2439  0.0446
11    21.7543  -0.8257  0.2045  0.2683  0.0638
12    23.5697  -0.7652  0.2221  0.2927  0.0706
13    25.2114  -0.7104  0.2387  0.3171  0.0783
14    29.7567  -0.5589  0.2881  0.3415  0.0533
15    30.1014  -0.5474  0.2921  0.3659  0.0738
16    33.3612  -0.4388  0.3304  0.3902  0.0598
17    33.5060  -0.4340  0.3322  0.4146  0.0825
18    35.2678  -0.3752  0.3538  0.4390  0.0853
19    38.5917  -0.2644  0.3957  0.4634  0.0677
20    39.3931  -0.2377  0.4061  0.4878  0.0818
21    39.5253  -0.2333  0.4078  0.5122  0.1044
22    41.9588  -0.1522  0.4395  0.5366  0.0971
23    44.3276  -0.0732  0.4708  0.5610  0.0902
24    45.0259  -0.0500  0.4801  0.5854  0.1053
25    48.1024   0.0526  0.5210  0.6098  0.0888
26    49.3623   0.0946  0.5377  0.6341  0.0965
27    49.7351   0.1070  0.5426  0.6585  0.1159
28    50.6010   0.1359  0.5540  0.6829  0.1289
29    62.2574   0.5244  0.7000  0.7073  0.0073
30    68.9057   0.7460  0.7722  0.7317  0.0405
31    76.8052   1.0094  0.8436  0.7561  0.0875
32    78.6105   1.0695  0.8576  0.7805  0.0771
33    79.5384   1.1005  0.8644  0.8049  0.0596
34    84.1996   1.2559  0.8954  0.8293  0.0661
35    87.4173   1.3631  0.9136  0.8537  0.0599
36    91.1650   1.4880  0.9316  0.8780  0.0536
37    94.2047   1.5894  0.9440  0.9024  0.0416
38    95.7160   1.6397  0.9495  0.9268  0.0226
39    96.4987   1.6658  0.9521  0.9512  0.0009
40    96.8248   1.6767  0.9532  0.9756  0.0224
41    99.8979   1.7791  0.9624  1.0000  0.0376
```

The maximum value in the last column occurs at point 28, and so our D statistic is 0.1289. This is well below our critical value. We cannot reject our null hypothesis, and must proceed as if the points were normally distributed.

The test can also be performed by picking Z values at regular intervals, and generating F(Z) by counting the number of points whose Z values are less than each critical Z:

```  Z   ECDF(Z) pts<z   F(z)   |F(z)-ECDF(z)|
----  ------- -----  ------  ------
-2.0  0.0065      0  0.0000  0.0065
-1.8  0.0276      0  0.0000  0.0276
-1.6  0.0509      0  0.0000  0.0509
-1.4  0.0792      2  0.0488  0.0304
-1.2  0.1145      4  0.0976  0.0170
-1.0  0.1585      7  0.1707  0.0122
-0.8  0.2119     11  0.2683  0.0564
-0.6  0.2743     13  0.3171  0.0428
-0.4  0.3446     17  0.4146  0.0700
-0.2  0.4207     21  0.5122  0.0915
0.0  0.5000     24  0.5854  0.0854
0.2  0.5793     28  0.6829  0.1037
0.4  0.6554     28  0.6829  0.0275
0.6  0.7257     29  0.7073  0.0184
0.8  0.7881     30  0.7317  0.0564
1.0  0.8413     30  0.7317  0.1096
1.2  0.8849     33  0.8049  0.0801
1.4  0.9192     35  0.8537  0.0656
1.6  0.9452     37  0.9024  0.0428
1.8  0.9641     41  1.0000  0.0359
2.0  0.9772     41  1.0000  0.0228
```

(US) National Institute of Standards and Technology, Engineering Statistics Handbook. 1.3.5.16. Kolmogorov-Smirnov Goodness-of-Fit Test http://www.itl.nist.gov/div898/handbook/eda/section3/eda35g.htm

1US Department of Commerce, Handbook of Mathematical Functions, June 1964 26.2 (especially 26.2.18), Normal or Gaussian Probability Function, pp. 931-933

The original references appear to be

Kolmogorov, A. N. (1933) "On the empirical determination of a distribution function," (Italian) Giornale dellâ€™Instituto Italiano degli Attuari, 4, 83-91.

Smirnov, N. V. (1939), "On the estimation of the discrepancy between empirical curves of distribution for two independent samples." (Russian) Bulletin of Moscow University, 2, 3-16.