IALSA and Reproducible Research. The primary purpose of the IALSA network is the evaluation of the reproducibility of results from longitudinal and life course studies. The IALSA network was motivated, in part, by the recognition that results across different methods and models of change may not be directly comparable. A coordinated (i.e., parallel) analysis approach provides the opportunity to compare results across studies based on similar models of change. IALSA projects also provide an exciting opportunity to critically evaluate alternative methods and their intrepretation and to make recommendations of best practice standards for evaluating change hypotheses. Analysis scripts for each project will be made available so that results can be reproduced from particular study data and to enable new projects to efficiently build on past research.
Open science. The analysis of longitudinal observational data can take many forms and requires many decisions, including choice of statistical model, selection of covariates, and selection of final model. Sensitivity of results to minor differences in model, potential for confirmation bias, and overfitting of model to data weaken the opportunity for replication of results. We are working with the Center for Open Science to administer IALSA analysis projects through the Open Science Framework, providing an efficient and rigorous approach for collaborative project management.
Replication through coordinated analysis. Comparison of the pattern of results from particular models of change is made possible by conducting parallel analysis of independent studies. This is the primary approach used by the IALSA network and described in this paper appearing in Psychological Methods (Hofer & Piccinin, 2009).
Power to detect change and covariation in change. Rast and Hofer (2014) demonstrated that typical longitudinal study designs have substantial power to detect both variances and covariances among rates of change in a variety of cognitive, physical functioning, and mental health outcomes. The relation between growth rate reliability (GRR) and effect size to the sample size required to detect power greater than or equal to .80 was nonlinear, with rapidly decreasing sample sizes needed as GRR increases. The results presented here stand in contrast to previous simulation results and recommendations.
Measurement harmonization. The IALSA Metadata and Harmonization Platform is being developed to provide item-level metadata to both determine the potential for inclusion in coordinated analysis and as a basis for measurement harmonization. This platform will permit harmonization of existing data when possible and planning for new data collection to enable cross-linkage of measures. IALSA has partnered with Maelstrom Research (Core B Co-PL: Isabel Fortier) to build on existing international efforts for study comparison and harmonization.
Sensitivity of results across variations in models of change. An ongoing aim of the IALSA network is to examine the impact of alternative statistical models (Piccinin et al., 2011) on the interpretation and replicability of study results. Several projects within IALSA are evaluating the impact of statistical model on questions and conclusions regarding associations between health and cognition. A potential approach, demonstrated by Rast et al., (in press), is the application of the J-N technique within a curvilinear longitudinal model with higher-order terms to probe moderators and to identify regions of statistical significance of covariate effects.
Importance of distinguishing within-person and between-person effects. The distinction of between-person age differences from within-person age changes is necessary for understanding aging-related change processes (Sliwinski et al., 2010). Although longitudinal studies are required to address issues relating to within-person change, most studies begin with age-heterogeneous samples and conclude using survival-heterogeneous samples. Given the numerous potential confounds associated with age-heterogeneous samples, careful treatment of between-person age differences is essential to obtain the useful inferences regarding within-person age change. A good example on the effect of blood pressure on cognition is Thorvaldsson et al., (2012).
Retest effects. Gains in performance are often observed in cognitive outcomes, with the largest across the first and second occasion, and can confound the estimate of change. Several IALSA investigators have pointed out the inherent confounds in statistical and design-based adjustments (Hoffman, et al., 2011; Thorvaldsson et al., 2006; 2008) and have made the case that relative individual differences (within-study) are likely to be less confounded. Measurements obtained across different time scales provide the opportunity to distinguish short-term retest gains from long-term changes in performance (Sliwinski, et al., 2010) but this is not helpful for existing long-term studies. This is an important topic for IALSA projects to consider in terms of retest model (e.g., adjustment for baseline difference), interpretation, and comparison of results across variables and studies.
Incomplete data and mortality. Particularly in the study of older adults, missing occasions and dropout are increasinlgy due to illness and death. Mortality selection is also an issue for initial sample selection as individuals who have died are no longer available to be sampled. Longitudinal data containing information regarding death dates and methods tailored to specific research questions (see Kurland, et al., 2009) are required to adequately address the impact of mortality and proximity to death (Piccinin et al., 2011) on other sources of change. Inference regarding aging and health-related changes require the estimation of population parameters that are conditional on morbidity, mortality, and other attrition processes.
Here is a brief (but growing) list of statistical resources for multilevel and longitudinal statistical analysis, integrative data analysis, and measurement harmonization.