(***Links to additional versions of the test performance software application will be posted here***)
A statistical meaningfulness test is a method in time seriesanalysis which separates statistically meaningful trends from not statistically meaningful trends. The method can be used for determining whether a variable is increasing or decreasing over time.
The statistical meaningfulness test is a stricter quality criterion than using a confidence level of a least-squares fit. Both methods use confidence levels while the statistical meaningfulness test also accounts for the extent to which data are scattered from the trendline[1].
Illustration of the difference between high statistical significance and statistical meaningfulness of time trends
Definition
The statistical meaningfulness test is based on the probability (p-value) and the coefficient of determination (the R2 value). If the least-squares fit of a time series yields R2 ≥ 0.65 at p ≤ 0.05, then the test result is positive. A positive test result can also be determined by dividing the time series into intervals, if the least-squares fit of interval mean values yields R2 ≥ 0.65 at p ≤ 0.05[1].
Difference compared to least-square fits
The confidence level of a least-square fit of a trend is affected by the number of data in a time series. A standard least-squares fit of a time series with a very large number of data can be statistically significant at the 95% or 99% confidence level even if the data are scattered far away from the trendline. Such a time series is less likely to pass a statistical meaningfulness test since this test is also based on the R2 value which measures the spread of data from the trendline.
Thus, the main two differences between least-square fit and the statistically meaningfulness test are
- least-squares fits are easier to perform
- the statistical meaningfulness test is stricter and disqualifies long time series which have significant trends despite containing many data points far away from the trendline.
Software for performing the test
The statistical meaningfulness test can be performed using Microsoft Excel with an application (add-in) which is freely downloadable from the website of the journal PLoS ONE[1]. The journal also has an instructions manual for installing and using the add-in.
See also
- Trend estimation
- Regression analysis
- Trend analysis
- Least squares
- Probability
- P-value
- Coefficient of determination
- Linear regression
References
^ a b c Bryhn AC, Dimberg PH (2011). “An operational definition of a statistically meaningful trend”. PLoS ONE 6 (4): e19241. http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0019241.
