Far casting cross-validation

Patrick S. Carmack, William R. Schucany, Jeffrey S. Spence, Richard F. Gunst, Qihua Lin, Robert W. Haley

Research output: Contribution to journalArticle

7 Scopus citations

Abstract

Cross-validation has long been used for choosing tuning parameters and other model selection tasks. It generally performs well provided the data are independent, or nearly so. Improvements have been suggested which address ordinary cross-validation's (OCV) shortcomings in correlated data. Whereas these techniques have merit, they can still lead to poor model selection in correlated data or are not readily generalizable to high-dimensional data. The proposed solution, far casting cross-validation (FCCV), addresses these problems. FCCV withholds correlated neighbors in every aspect of the cross-validation procedure. The result is a technique that stresses a fittedmodel's ability to extrapolate rather than interpolate. This generally leads to better model selection in correlated datasets. Whereas FCCV is less than optimal in the independence case, our improvement of OCV applies more generally to higher dimensional error processes and to both parametric and nonparametric model selection problems. To facilitate introduction, we consider only one application, namely estimating global bandwidths for curve estimation with local linear regression. We provide theoretical motivation and report some comparative results from a simulation experiment and on a time series of annual global temperature deviations. For such data, FCCV generally has lower average squared error when disturbances are correlated. Supplementary materials are available online.

Original languageEnglish (US)
Pages (from-to)879-893
Number of pages15
JournalJournal of Computational and Graphical Statistics
Volume18
Issue number4
DOIs
StatePublished - Dec 28 2009

    Fingerprint

Keywords

  • Dependent data
  • Optimistic error rates
  • Prediction
  • Temporal correlation
  • Tuning parameter

ASJC Scopus subject areas

  • Statistics and Probability
  • Discrete Mathematics and Combinatorics
  • Statistics, Probability and Uncertainty

Cite this

Carmack, P. S., Schucany, W. R., Spence, J. S., Gunst, R. F., Lin, Q., & Haley, R. W. (2009). Far casting cross-validation. Journal of Computational and Graphical Statistics, 18(4), 879-893. https://doi.org/10.1198/jcgs.2009.07034