MerQuR+Statistics Glossary

Statistics Glossary → Mann-Kendall + Sen's Slope

Mann-Kendall + Sen's Slopeanalysis

A non-parametric method that tests whether a time series shows a steady increase or decrease over time (a monotonic trend) without distributional assumptions, and gives the size of the trend as Sen's slope.

Concept

A common question with climate, hydrology and environmental data is 'is this variable rising or falling over the years?'. Linear regression can answer it, but it assumes normality, and a few extreme years can change the slope a great deal. The Mann-Kendall test looks only at the ordering of the values rather than the values themselves; it is therefore robust to skewed distributions and outliers, and it requires the trend only to be in one direction (monotonic), not linear.

The test compares every possible pair of observations: +1 is recorded if the later observation is larger and −1 if it is smaller, and their sum gives the S statistic. A clearly positive S indicates an increasing trend and a negative S a decreasing one; the Z value computed from the variance of S gives the p value. The size of the trend is measured with Sen's slope: the median of the slopes (difference in value / difference in time) over all pairs. Kendall's τ, the rank correlation between time and value, serves as an effect size.

The classical test assumes the observations are independent. If successive years resemble each other (positive autocorrelation), the test tends to declare trends that are not really there; autocorrelation corrections should then be used. The test shows that a trend exists but not why; also, a sudden shift in the middle of the period may be indistinguishable from a slow trend, which is what the change-point tests are for.

When to use it

  • To test for an increasing or decreasing trend in series such as annual rainfall, temperature, river flow or pollutant concentration without distributional assumptions.
  • If seasonality masks the trend in monthly or seasonal data, use the seasonal Kendall test; if successive observations are autocorrelated, use the Hamed-Rao, Yue-Wang or TFPW correction.
  • If the data are normal and a linear rate of change is to be estimated, use linear regression (with Newey-West standard errors if autocorrelation is present); to see a series' trend and seasonal components, STL decomposition is preferred.

An everyday example

Suppose the yearly average water level of a lake next to a village has been measured for 40 years. Because some years are dry and others wet, the level goes up and down, and a few extremely dry years distort the plot. The Mann-Kendall test compares each year with all the later years; if it finds that later years are mostly lower, it reports a significant decreasing trend. If Sen's slope comes out at, say, −3 cm per year, the lake has dropped by roughly 1.2 metres over the 40 years, and this estimate is hardly affected by the few extreme years.

Parameters in MerQur

Date (optional) and Slope unit (Automatic / Per year / Per day / Per observation)

Concept: Sen's slope is a difference in value divided by a difference in time, so its unit depends on how the time axis is measured. Without dates the slope is read 'per observation'; with years or calendar dates it is read 'per year' or 'per day'.

Everyday example: If the lake was measured once a year, the slope per observation and per year are the same: −3 cm. Expressed per day, the same series has a slope of about −0.008 cm per day; the meaning is unchanged, only the unit differs.

What it does: The time axis of Sen's slope. Without a date, the observation index; with a numeric year, the year difference; with a calendar date, day difference/365.25 (year) or days. Automatic: per year if there is a date and the span exceeds 2 years.

Effect on the result: Only Sen's slope and its interval; Mann-Kendall S, Z and p do not depend on the date.

On the validation data: Nile: -2.6 /observation with the index; -2.6 /year with a numeric year; -2.600036 /year with a calendar date (because of leap years); -0.007119 per day.

→ Sen's slope

Autocorrelation correction (None (classical MK) / Hamed-Rao / Yue-Wang / TFPW (pre-whitening))

Concept: With positive autocorrelation, successive observations resemble each other and the series carries less independent information than it seems (the effective sample size shrinks). Without a correction, the variance of S is underestimated and the p value comes out too small; Hamed-Rao and Yue-Wang fix this by inflating the variance, TFPW by prewhitening the series.

Everyday example: If the lake is low one year, it is likely to be low the next, because a lake refills only slowly. The 40-year series therefore does not carry as much information as 40 independent measurements; the correction accounts for this resemblance and tests more honestly whether the trend is truly significant.

What it does: Positive autocorrelation makes the Mann-Kendall p value too small. Hamed and Rao (1998) and Yue and Wang (2004) inflate Var(S) by the effective sample size ratio; TFPW (Yue et al. 2002) removes the trend and applies lag-1 pre-whitening.

Effect on the result: Var(S), Z and p (also S under TFPW).

On the validation data: Nile (r1 = 0.498): classical p = 3.66e-05; Hamed-Rao n/n* = 2.143, p = 0.0048; Yue-Wang p = 3.49e-05; TFPW S = -1515, p = 4.7e-06.

→ Autocorrelation correction and prewhitening (Hamed-Rao, TFPW)

Seasonal Kendall period (0 = off)

Concept: The seasonal Kendall test compares each season (e.g. each month) only within itself across years and then combines the results. The large difference between summer and winter is thus not mistaken for a trend, and a strong seasonal pattern does not mask a weak trend.

Everyday example: If the lake level is measured monthly, it rises every spring and falls every autumn. With period 12 the test compares each April only with the other Aprils and each October with the other Octobers, bringing out the real decline over the years.

What it does: Each season is compared within itself (Hirsch, Slack and Smith 1982) and a seasonal Sen's slope is computed.

Effect on the result: Seasonal S, Var(S), Z, p and seasonal Sen's slope.

On the validation data: nottem, m = 12: S = 224, Var(S) = 11364, Z = 2.0919, p = 0.03645, seasonal Sen 0.05 /year. Non-seasonal Mann-Kendall p = 0.433.

→ Seasonal Kendall test

Hypothesis direction (Two-sided / One-sided: increase / One-sided: decrease)

Concept: States whether the research question asks 'is there a trend?' (two-sided) or is posed in one pre-specified direction ('is it decreasing?'). A one-sided test is more sensitive to a trend in that direction but does not test a trend in the opposite direction at all.

Everyday example: If the villagers complain that the lake is shrinking and the only question is 'is the level falling?', the one-sided decreasing option can be chosen. The direction must be fixed in advance, not after looking at the data.

What it does: Direction of the alternative hypothesis; also applied in the seasonal section.

Effect on the result: The p value and the trend decision.

On the validation data: Nile: two-sided p = 3.66e-05; decrease p = 1.83e-05; increase p ≈ 1 (trend decision "none").

→ One-sided and two-sided tests (hypothesis direction)

Additional output: Change point, Exact p, Save

Concept: Change-point tests (Pettitt, Buishand, SNHT) look for a sudden shift in the series' mean at some moment and show the most likely shift point. The exact p value uses the exact distribution of S instead of the normal approximation in small series without ties.

Everyday example: If a dam was built upstream of the lake in year 20, the level may have dropped all at once rather than gradually. Mann-Kendall also sees this as a 'decreasing trend'; the change-point test shows that the shift happened around year 20 and helps tell the two interpretations apart.

What it does: Change point: Pettitt, Buishand and SNHT tests. Exact p: the exact distribution of S in a tie-free series with n ≤ 60. Save: Sen fit and residual columns.

Effect on the result: Only additional output.

On the validation data: Nile: Pettitt K = 1617 (p = 3.6e-07), Buishand R/√n = 2.952, SNHT T = 43.22; all three point to observation 28 (1898). austa (n = 36) exact p = 3.26e-30.

→ Change-point tests (Pettitt, Buishand, SNHT)

Confidence Level (%90 / %95 / %99 / Other, custom value)

Concept: Sets the confidence of the interval given for Sen's slope and the threshold (α) for the trend decision. The interval is formed by two slopes picked from the ordered list of pairwise slopes according to the variance of S.

Everyday example: If Sen's slope is −3 cm per year with a 95% interval of [−4.1, −1.8], the lake can be said to drop by at least 1.8 and at most 4.1 cm a year. Asking for 99% confidence widens the interval, but as long as it excludes zero the decision of a decline stands.

What it does: Sen's slope interval (C = z_{1-α/2}·√Var(S)), trend decision, r1 warning bound and the Hamed-Rao significant-lag bound.

Effect on the result: Interval limits and the decision.

On the validation data: Nile Sen's slope interval: %90 [-3.4286, -1.6591], %95 [-3.6279, -1.4286], %99 [-4.0333, -1.04].

→ Confidence level and confidence interval

Example on real data

Data: Nile, Durbin and Koopman 2001, R datasets

Variables: value flow (annual Nile flow at Aswan), no date (observation index)

  • Question: Is there a monotonic trend in the annual flow of the Nile (1871-1970, n = 100)?
  • Result: Mann-Kendall S = -1387, Var(S) = 112728.33, Z = -4.13, p < .001; the decreasing trend is significant.
  • Effect size: Kendall τ-b = -0.281; a moderate negative monotonic relationship between time and flow.
  • Slope: Sen's slope = -2.60 units/observation (95% CI [-3.63, -1.43]); the flow decreases by about 2.6 units per year on average.
  • Assumption: Lag-1 autocorrelation r1 = 0.498 (Ljung-Box Q(1) = 25.59, p < .001); the classical MK p value may be too small, and a Hamed-Rao or TFPW correction is recommended.
  • APA: The Mann-Kendall test showed a significant decreasing trend, S = -1387, Z = -4.13, p < .001, τ = -.28; Sen's slope = -2.60 [-3.63, -1.43].

Related concepts

Sen's slopeAutocorrelation correction and prewhitening (Hamed-Rao, TFPW)Seasonal Kendall testChange-point tests (Pettitt, Buishand, SNHT)Seasonality and seasonal periodAutocorrelation (ACF) and partial autocorrelation (PACF) functionsOne-sided and two-sided tests (hypothesis direction)p-value

The numbers in the examples are MerQur's output on independent, publicly available validation data; see the MerQur Technical Note for details.