Stages of research

Click the links to the right to read more about each specific stage of research.  Or, continue reading below about how stages within a roadmap can help us to track progress, to better account for our investments in research, and to understanding the cascading effects of research on knowledge, practice, and outcomes.

Why propose stages in a research roadmap?

As research (and researchers) become increasingly specialized, roadmaps become essential to ensuring that individual research studies will converge to achieve progress towards the larger goal, of improving outcomes for people living with conditions such as ASD.  A more specific roadmap that breaks research down into specific stages can therefore help to track this progress, identify gaps, set new priorities, and re-allocate resources as needed.

STAGES OF RESEARCHBasic ResearchApplied ResearchClinical ResearchImplementation ResearchOther Applied ResearchStages can help to track progress

A research roadmap can help to track our progress on both a large and a small scale.  Consider the significance of the transition from Clinical to Implementation Research. On a larger scale,  for example, this begins to signify that a practice may be valid, reliable, and useful enough to merit considering its integration into daily usage by practitioners in the community. Such a milestone is an important watershed for researchers, signaling the transition from Clinical to Implementation Research. Such a milestone is an even more important watershed for service providers, signaling an emerging opportunity to develop specific, research-based practice recommendations.

Stage: Clinical Research Phases within a stage1. Pilot practice2. Then, demonstrate its validity and reliability,3. Finally, adapt it for use in community settings.Later phases extend its use to other populations and settingsOn a smaller scale, this roadmap can identify gaps in specific phases within one stage of research. Perhaps a practice has yet to be widely integrated into daily usage because it has not been tested across a range of populations (for example, a screening tool has not been validated for use with children with ASD from other cultures). Progress becomes a journey unfolding from one stage or milestone to the next over a period of years, with each stage in the journey requiring different kinds of research, and each stage incorporating specific phases.

Stages can help us to better account for our investments in research

There have been remarkably few attempts to try to assess how much we have invested in different stages or phases of research, let alone the progress from one to the other. Consider the strategic plan adopted by the InterAgency Autism Coordinating Committee (IACC), which not only provides the framework for the summaries of the grants allocated and the progress made, but shapes the priorities for future research in the US. Some of the major questions within the IACC's strategic plan can be linked to broad stages. For example, Question 2 (Biology) and Question 3 (Risk Factors) clearly focus on Basic Research while Question 5 (Services) would appear to fit clearly under Applied Research.  Initial analyses that also classified Question 1 (Screening and Diagnosis) and Question 4 (Treatment) under Applied Research concluded that 42% of funding focused on Basic Research.  When we and others looked more closely, however, we found subquestions with very significant funding allocations that were more clearly aligned with Basic Research. This resulted in a much greater emphasis on Basic Research (about 75% of total funding) relative to Applied Research (Doehring, 2019a). These findings are comparable to re-analyses of IACC data conducted by others.

We have since conducted other analyses based a project-by-project re-classification based on the Roadmap presented here, within IACC subquestions otherwise associated with Applied Research.  These have suggested that the relative effort devoted to Basic Research is even greater. For example, our review of projects focused on ASD Identification (Question 1) between 2008 and 2015 suggested that only about 14% of funds - or about 1 out of every 7 dollars spent - intended to improve ASD identification are actually allocated to Applied Research (Doehring, 2019b & c). These analyses revealed specific gaps in Implementation Research; prior to 2013, less than 1 out of 12 dollars spent on Applied Research focused on Implementation Research.  Earlier analyses of projects classified under Treatments (Question 4), funded by the National Institutes of Health (NIH) between 2008 and 2013, and using a more inclusive definition of Implementation Research,  revealed similar gaps in research focused on improving treatment (Doehring, 2018). Taken together these suggest that the total percentage of funding allocated to Basic Research is very likely to be even greater than 75%

Stages can help to assess the cascading impact of research on knowledge, practice, and outcomes

Given that these data only reflect funding allocations, what do we know about the actual impact? How do we assess the return on these investments in research? Is there evidence that specific research grants intended to improve outcomes actually did so? More broadly, how do we know if research is making progress towards its overall goals? Remarkably, we have very few answers to these important questions - indeed, it seems that we have yet to even ask them!

We have begun to ask these questions by first imagining how research projects in different stages in the roadmap may have different possible impacts on outcomes. In the process, we are discovering that the extent to which research is able to achieve a broad range of impacts depends on a number of factors related to research designs, research institutions, and the researchers themselves. We will describe these impacts in greater detail elsewhere, and so will only summarize the issues here.

Let's begin with a simple observation: all research projects propose to test a specific set of hypotheses or systematically explore a specific set of phenomena. These are captured in the stated objectives of the study, typically listed in journal articles at the  end of the introduction, just before the methods of the study are described. Thus, the first impact of projects at every stage of the research roadmap is the publication of the results of these explorations and these tests.  We call this a knowledge impact.

This knowledge impact is important at multiple levels.  First, these findings must first be published (preferably in a peer-reviewed journal) if future research to build on these findings. A basic research grant might hypothesize that a given characteristic (head circumference) distinguishes a specific population of children at high risk for ASD (infant siblings). A publication confirming this hypothesis might become the basis for a clinical research grant that seeks to validate a new screening tool that incorporates this indicator.

The knowledge impact, as reflected in the publication record, is also immensely important in advancing the career of researchers. It is one of the most important factors in decisions to hire and promote them, and to award grants to them.  The knowledge impact continues as these findings begin to influence the work of other researchers, as indicated by the number of times the findings of a given research study are cited by others in their own publications.

STAGE        IMPACTSBasic          KnowledgeClinical      PracticeImplem-    Program &entation            Population                            OutcomeWe have imagined, however, other kinds of impacts that might result from specific research projects, depending upon the stage of research involved. For example, while clinical research results in specific findings related to the validity and applicability of a clinical practice, these do not necessarily translate immediately and directly into a practice impact. Sometimes the results of an individual study are too specific, or lingering questions surrounding the methods used or populations studied leave front-line clinicians uncertain about the conclusions.  The evidence-based practice (EBP) reviews that we (Reichow, Doehring, Cicchetti, & Volkmar,  2011) and others have undertaken are intended to close this gap, by integrating findings across only those studies that meet a specific standard of quality. Thus, these EBP reviews begin to achieve a practice impact, by yielding more specific, research-based recommendations for practice that can be used by front-line clinicians.

Remarkably, there have been few efforts to document these different possible impacts of research, and the results obtained to date indicate significant gaps. For example, surveys of front-line clinicians suggests that the practice impact of outcome research is limited; many still struggle to incorporate EBPs into their daily use. We have also struggled to find evidence of clear policy impacts; i.e., examples of of national policies shaped by research with a clearly documented impact on outcomes. Even the systematic and intense state-by-state advocacy in the US to ensure that more ASD services are covered through private health insurance have had marginal impacts in rates of service use, let alone evidence of improvements in outcomes (Doehring, 2019a).

Significant gaps are evident even at the most fundamental level, with respect to the impact on knowledge. In a recent pilot study, we sought to identify publications that specifically reported the results obtained relative to the primary objective of research projects funded in the US to improve ASD identification.  Our systematic electronic literature searches yielded relevant publications for only about one-half of the projects funded. In two-thirds of these cases, no publications were found even though funding for the projects had ended more than three years ago (Doehring, 2019b & c).

We are not aware of any other studies that have sought to rigorously document the distribution of research funding across the stages of a roadmap, or the specific knowledge impacts resulting from these investments in research.  To date, IACC’s reports have been limited to summaries of expenditures broken down into the 7 research questions and subquestions, summaries of the number of publications and collaborations that have resulted, and annual reports highlighting specific research findings.

What is the relationship with other research roadmaps?

Research Roadmap: ASD Screening Research Roadmap: ASD DiagnosisWhile this review is focused on a research roadmap for ASD screening, we must also consider how different research roadmaps might focus on different, and sometimes inter-related, sets of objectives. For example, progress in screening research may ultimately depend on progress in other areas of research and practice.  You cannot test strategies to scale up screening to achieve population-level impact until reliable and accurate diagnoses can be accessed by individuals who screen positive (e.g., at elevated risk for ASD).

As we consider the possibility of other research roadmaps, we might encounter some similarities and some differences in their structure.  For example, other research roadmaps focused on assessment practices would probably make similar distinctions between these different stages and different steps. While other research roadmaps focused on intervention practices might make similar distinctions between the different stages of research, they might require the differentiation of the other steps within different stages of research.

Other Applied Research

Coming soon!

Basic Research

Coming soon!

APPLIED RESEARCH