Regression of real-world data on model output: An appropriate overall test of validity

Abstract

For validation purposes, the appropriateness of regression analysis and the simultaneous F-test of unit slope and zero intercept was challenged in a study by S.R. Harrison (Agricultural Systems, 34 (1990) 183–190). Using a more comprehensive Monte-Carlo experiment, we show that this test performs well, except in situations where the errors are autocorrelated. The F-test is very powerful in rejecting models which are invalid, and rejects only the expected proportions of valid models when the underlying assumptions hold. Because it tests for the degree of relationship between real-world data and model outputs, it is far superior to other statistics based on location measures, which have been suggested for validation.