Real-world recommendations: Do limits to validation constrain model usefulness?

Abstract

This paper describes efforts to validate a model of glyphosate resistance in awnless barnyard grass (Echinochloa colona (L.) Link.) in the sub-tropical northern grains region of Australia. Possible constraints to the perceived usefulness of the model due to hidden or sparsely documented system factors are discussed. The evolution of herbicide resistance in real weed populations becomes apparent only after years or decades of selection. Modelling is therefore particularly useful in investigating and describing the change in the weed population and the relative importance of different factors pushing towards or mitigating against that change. A key role of modelling in farming systems research is to improve understanding of issues such as the evolution of herbicide resistance. However, model results are increasingly important for formulating or adjusting recommendations, including for resistance prevention, to be used by land managers. Importantly, the research projects under which models of agricultural systems are produced are likely to include the development or refinement of recommendations for change as part of their expected outcomes. The comprehensiveness of model validation can be thought to equate generally to the strength of an argument in favour of a model's hypotheses. By extension (and by convention), more comprehensive validation is often given to equate to a stronger argument in favour of practical recommendations that arise more or less directly from the model's outputs. That is, increasing confidence in the recommendations derived from a model is a result of the perceived validation of the model's predictions as much as of its hypotheses. Conversely, validation that is (by necessity or otherwise) partial or piecemeal, and which is structural rather than empirical, may not be seen as providing as secure an argument in favour of any recommendations for farmers that may be made. In the case of glyphosate resistance, hidden weed population variables and insufficiently detailed farming systems data make empirical, operational validation difficult. It is especially difficult to validate these models in time for them to be most useful in formulating practical recommendations for resistance prevention. A model of glyphosate resistance in awnless barnyard grass, a key northern Australian weed has been constructed, and attempts made to validate it in order to encourage trust in the model's predictions and the recommendations to industry that might be made from them. Empirical validation was performed through comparison with a population of the weed that was confirmed to be glyphosate resistant in 2007. Structural and behavioural pattern validation were performed during model development. In the historical dataset used for empirical validation, there are both hidden variables (in particular, the initial proportion of resistance-conferring alleles in the population before selection began) and sparsely documented variables (including the year in which glyphosate was first used, and the number and efficacy of glyphosate applications made to this weed population since then) that create difficulty in making direct comparisons between the real population and the model's predictions. The type of data that is available for use in empirical validation of the model is also a constraint. Herbicide resistance is identified as a point of failure in the agricultural system. Therefore, data from real fields is unlikely to contain information about whether or how the rate of evolution of resistance changes over time in response to the types of system actions that are included in the model. In this paper, potential validation methods and pitfalls are discussed, showing that recommendations can be made using the model's predictions with some confidence, but that herbicide model validation constraints may affect the credibility of the model's predictions particularly for non-scientist users.