X

A Screening Research Roadmap

Basic Research Causes, Characteristics, Trajectories, Sub-types...

Clinical Research Pilot practice, then validate it, then demonstrate its reliability and usefulness, and then extend its use to other populations

TRAINING, POLICY, AND ADVOCACY

 

Implementation Research Assess and then improve use with respect to delivery of practice, known gaps in access, and capacity to scale its use

SERVICES

Other Applied Research Population impact and Integrated care

Stage of Research

Steps  within a stage of research

Stories about ASD

A Screening Research Roadmap

ASD Roadmap

Stages of research

Each stage of research influences elements of the integrated network in different ways, leading to better services and better outcomes

January 31, 2019

 

ImplementationResearchClinicalResearchSo, we know that no single program will achieve a big outcome like the rapid and accurate diagnosis of ASD in all children affected. In the last essay, I described a broad roadmap in which research, services, training, and other elements could influence one another within an integrated network. Within the research sector, it is also unrealistic to expect that a single research study will elucidate causes, propose interventions, AND demonstrate improved outcomes on a meaningful scale. A more specific roadmap for research that breaks research down into specific stages can therefore help to track progress, identify gaps, set new priorities, and re-allocate resources as needed.

Hover over the diagram below to see how these stages of research might interact with the other elements of an integrated network to support better services and outcome. And read more below.

Hover to view details

Related Content

On this site

The stages of research

A research roadmap is essential to imagining and then marking progress on both a large and a small scale.  On a larger scale, a roadmap can help to identify when a practice is valid, reliable, and useful enough to merit integrating into daily usage by practitioners in the community. We can define milestones that mark a watershed; for example, signaling the transition from Clinical to Implementation Research.

On a smaller scale, this roadmap can identify gaps in specific steps within one stage of research. Perhaps a practice has yet to be widely integrated into daily usage because it has not been tested across a range of populations (for example, a screening tool has not been validated for use with children with ASD from other cultures). Progress becomes a journey unfolding from one step and milestone to the next over a period of years, with each stage in the journey requiring different kinds of research, and each stage incorporating specific steps.

STAGES OF RESEARCHBasic ResearchApplied ResearchClinical ResearchImplementation ResearchOther Applied ResearchI propose a complete research roadmap for ASD screening that begins by distinguishing between the broad stages of Basic and Applied Research, and within the latter between the specific stages of Clinical, Implementation, and Other Applied Research. These proposed distinctions were developed through an initial review of the categories of research adopted by the IACC for their own internal review purposes.  Additional distinctions between the steps of implementation research were further informed by a systematic review of all research on ASD identification conducted by the NIH between 2008 and 2013.

‚ÄčStage                               Steps within a stage      Clinical        Pilot practice, validate it, demonstrate its reliability and use-Research                 fulness, extend its use to other populations and settings

I hope to more clearly define these and other distinct steps within each stage of applied research through the systematic review now underway of all original, peer-reviewed research articles on the CHAT+ family of screening tools. A working definition of each of these stages of applied research, and the steps within each stage, is offered below.  As this review is completed, specific references will be added to each of the sections listed below.

Other essays later in this series provide other important information about these definitions, including more detailed descriptions of the components of the research roadmap (practices, steps, milestones, and outcomes), the critical watersheds (between basic and clinical research, and between clinical and implementation research), and the different resources and expertise required at each stage of research.

Basic Research

Clinical ResearchBasic ResearchCauses   Characteristics Trajectories   Sub-typesAs this series is focused on applied research, we will pay little attention to the steps possibly involved in basic research, but some observations may nonetheless be interesting. Basic research starts with a necessarily broad goal of discovery; in the case of ASD screening, to identify early markers of ASD in children who are subsequently diagnosed. This broad goal invites research in all types of physical and behavioral characteristics. Indeed, in my review of all NIH research focused on early identification over a 6 year period, I informally noted that many abstracts classified as basic research suggested that the proposed study could have implications for early screening or diagnosis. These abstracts considered all kinds of variables as potential markers, from behavioral to neurological to genetic characteristics.

In the stage of Basic Research, we might have a destination (earlier identification for all children with ASD) but do not yet have a roadmap, and in fact do not even have a vehicle - the clinical practice - that might get us to our destination.  The purpose of Basic Research is to define the core features of the practice. The development of a prototype for the practice marks the watershed between Basic and Clinical Research, a prototype we can test drive via Clinical Research.

How did this occur in the case of the CHAT? The first CHAT drew directly from basic research involving careful behavioral observations of specific characteristics; in this case, the joint attention and pretend play of very young children with ASD.  These observations revealed the potential for an approach to assessment based on simple behavioral observations by a parent or a professional: no traditional medical procedures were required.  Despite countless studies considering the full range of behavioral, neurological, and genetic indicators since the first CHAT study more than 25 years ago, no screening tool - indeed, no indicators - have approached the power of those captured by the CHAT+.

And so a body of basic research studies effectively identified the characteristics or indicators that the practice of screening might exploit (deficits in joint attention and pretend play) as well as the methods of assessment used in screening(behavioral observation by parents and professionals).  This sets the stage for applied research to transform these observations into a clinical practice to test, with the development and pilot testing of the first iteration of the screening tool marking the transition.  Our intrepid research explorer has identified what kind of car we needed for our research road trip, and now we are ready for a test drive into Applied Research.

Applied Research

In sum, applied research here refers to any studies that directly demonstrate the  effectiveness of a specific practice used by professionals providing services to an identified population of people with ASD, as well as studies evaluating the use of the practice on a large scale. Applied Research begins once we have built a prototype of car (a practice to pilot) that we are ready to take on the road.  More information about the specific watershed from Basic to Clinical Research is available elsewhere.

Clinical Research

The first stage of Applied Research is Clinical Research.  I have begun to define some of the specific steps within clinical research through the ongoing review of more than 110 original, peer-reviewed, empirical research studies on CHAT+ since 1992. In fact, most of this research appears likely to qualify as Clinical Research. These definitions may continue to evolve as this review is completed.

Basic ResearchClinical Research1. Pilot practice2. Validate it3. Demonstrate its reliability & usefulness4. Extend it to other populations & settingsTraining, Policy, and AdvocacyIn Clinical Research, we begin to test a specific practice with individuals identified with ASD (or, in the case of a screening tool, considered at risk for ASD).  The first steps of clinical research occur on a relatively smaller scale with respect to the number of participants, professionals, and settings involved. These first steps, exploring the practice's validity and reliability, and considering different combinations of test items, are familiar to most test developers.  Once this has been established, the potential usefulness of the practice to individual clinicians outside of the highly controlled research setting can be explored in subsequent steps.  In these later steps, the usefulness of the practice can tested with a greater range of populations, professionals, and settings.  And so we begin to approach another watershed, where the widespread use of the practice in real-world settings can begin to be tested via implementation research conducted at the program level.

How did these steps of clinical research evolve in the case of the CHAT+? Here, we are fortunate to be able to draw on a wide range of relevant studies published over the past quarter century. The first steps of clinical research revealed a fundamental psychometric limitation of the original CHAT; its low sensitivity meant that it was likely to miss more children than it identified.  The Modified CHAT (M-CHAT) sought to address this issue by converting items into a questionnaire completed by a parent, and then establishing risk based on relying on a summed score of indicators rather failure on a specific pattern of key items. This change also greatly simplified its use, by eliminating the need for observations by a clinician.  Later changes to the M-CHAT capitalized on a feature of the original CHAT: the use of a second administration to confirm risk status.  These changes captured in the M-CHAT R/F further strengthened perceptions of its sensitivity, specificity, and general utility.  Still later steps of clinical research, including pilot testing in community settings and a greater range of potentially eligible populations.

Leaving Basic ResearchOutlining these steps can help scientists begin to imagine the program of research studies needed to prepare an emerging practice for more widespread adoption by individual clinicians. This final step of clinical research helps to establish that a practice has the ease of use and breadth of applicability to warrant exploring its potential on a larger scale.  This final step can begin to feed back into the other components of the network; training, policy, and advocacy. For example, the screening guidelines formally adopted by the American Academy of Pediatrics (AAP Screening Guidelines) summarize the available clinical research and make very specific recommendations to integrate the CHAT+ and other measures into daily practice. This step can also guide efforts to secure the funding needed to undertake the next stage of Implementation Research.

And our roadmap metaphor? With new practice (the CHAT+) as our prototype, our intrepid research explorer took it to a real road to test it out. Is it reliable? Does it handle well enough? And so we take it on short trips to some familiar destinations under controlled conditions.  Once it proves to be reasonably reliable and useful, we feel comfortable taking it to the next level: testing it on all kinds of real trips with all kinds of real people, and scaling it up for a real world market via Implementation Research.

Implementation Research

Clinical ResearchImplementation ResearchAssess & then improve use with respect to:Delivery of practice; Known gaps in access;Capacity to scale its useTraining, Policy, and AdvocacyServicesOther Applied ResearchThe second stage of Applied Research is Implementation Research.  This preliminary definition of Implementation Research, and initial distinctions between the steps of implementation research, evolved through the review of categories of research funding adopted by the Inter-Agency Autism Coordinating Committee, and the assessment of NIH-funded research between 2008 and 2013. I also drew on the challenges I encountered seeking to deploy evidence-based practices within and across school- and hospital-based programs.

In Implementation Research, we consider what is needed for a practice to be deployed on a larger scale, in real-world settings. The level of analysis moves from the work of individual clinicians to the activities of whole programs, as well as the policies that guide these programs. Steps within Implementation Research include, for example, demonstrating that clinicians within practices outside of specialized research settings can be efficiently and effectively trained to use the practice reliably. Given the gaps in access for specific groups, either by virtue of their poverty, education, culture, or where they live, we must also be prepared to undertake a second step; overcoming barriers to their access to the practice. In a final step, we must also demonstrate that we can do each of the above at the scale needed to impact an entire population.  This can include, for example, the development of train-the-trainer programs. My preliminary review of existing grants and publications also suggests that it may be prudent to distinguish research within each of these steps, between studies that evaluate a step (e.g., that assess the barriers to accessing the practice in the community) and studies that seek to demonstrate how to address them (e.g., that demonstrate how to overcome a barrier to access).

ImplementationResearchClinicalResearchWhat did these steps of implementation  research look like in the case of the CHAT+? The limited number of relevant studies make these distinctions more speculative, at least for now. A principle goal of the review currently underway is to map existing studies using these distinctions, and to identify any other distinctions that might prove helpful.  On the one hand, clinical research studies must often be very specific to a given practice; for example, tests of the validity and reliability of a screening tool cannot be generalized to other tools. On the other hand, implementation research studies can begin to look beyond the specific practice to include other related practices. For example, many of the barriers that traditionally underserved groups face to accessing screening using CHAT+ will probably be similar to those evident for other screening tools. This process of mapping out the range of possible research studies may itself help researchers become more aware of other important research opportunities.

This final step of implementation research is to gather the insights learned, and recommend changes to training, policy, and program design before scaling up to full-scale implementation through programs of services. For example, the AAP Screening Guidelines suggest a great focus on barriers to ASD screening within the health care system. Outlining these steps can help scientists to deepen the partnerships with advocacy, program and policy leaders needed to support any broad, system-level changes to promote the practice.  This support can include commitments to undertake the additional research needed in the final stage of research to confirm the population-level impact intended.

And our roadmap metaphor? Now that our new practice has proven to be useful and reliable enough for daily use, our research explorer can test it under all conditions.  He can take it for a real roadtrip, and demonstrate the capacity of his vehicle for efficient mass production.  If we are successful, we can take it to the final stage of Other Applied Research, to evaluate whether it in fact has met all of our needs, and gotten us to where we wanted to go.

Other Applied Research

Implementation ResearchOther Applied ResearchPopulation impact Integrated careServicesThe final stage is Other Applied Research.  These initial distinctions between the steps of Other Applied Research evolved through the review of categories of research funding adopted by the Inter-Agency Autism Coordinating Committee, and the assessment of NIH-funded research between 2008 and 2013. These reviews revealed other research studies that were clearly important but that did not fit easily into the earlier stages defined above.

For example, consider studies describing overall prevalence or associated impact. These have play an increasingly powerful role not just in guiding research, but also in in raising awareness and concern among advocates. In the case of ASD screening, persistent and significant gaps in ASD identification only became evident through continued funding of the ADDM network, but this kind of research fell outside of the definitions of other stages offered above.  These kinds of studies provide the only meaningful measure of population impact at the regional, state, or national level. Studies addressing population impact therefore constitute one other important step of other applied research.

Population ImpactAnother important step involves the assessment of the practice's broad impact: how often do we test whether the specific target of the practice actually leads to the larger, long-term life outcome envisioned? By necessity, clinical research is often very specific to a particular practice and must focus on immediate or proximal outcomes (e.g., are children identified as high risk with the CHAT+ subsequently shown to have ASD?). While implementation research can begin to encompass other related practices, it remains focused on proximal outcomes (e.g., does a training program succeed in helping community-based practitioners become proficient in the use of  CHAT+?).

But what about the larger goal? The more distal outcomes? For example, early and accurate screening only has its intended impact when children identified as heightened risk for ASD are (a) actually diagnosed earlier, and (b) consequently benefit from earlier intervention.  If either (a) or (b) fail to occur, then the most effective screening tool is useless. This suggests the need for another step of applied research: integrated care. To what extent is a practice successfully integrated into a broader system of care, and in fact contributes to the larger, longer-term outcomes envisioned? For example, the AAP Screening Guidelines  considered the need for research demonstrating the impact of early screening on mortality.  What about demonstrating that early screening helps children subsequently diagnosed with ASD actually access early intervention more quickly? An ACE center grant awarded in 2017 is in fact intended to address that

And our roadmap metaphor? In this final step, our research explorer has arrived, and now must decide if the trip was worth it.  Did it meet his goals, or are there other destinations that might prove better?  And if so, if this still the right vehicle for getting him there?

The relationship with other research roadmaps?

The logo is adapted from Joaquim Alves Gaspar's drawing of Pedro Reinel's compass rose.

X

My Presentations and Publications

( 2019). The Impact of ASD Research on National Policy (Ch. 9). In Volkmar, F. (Ed.). Autism and Pervasive Developmental Disorders (3rd Ed.). Cambridge University Press, Cambridge, UK.

(2014). Translating Research into Effective Social Policy. In Volkmar, F. Paul, R., Rogers, S., & Pelphrey, K. Handbook of Autism and Pervasive Developmental Disorders: Assessment, Interventions, Policy, the Future (4th ed.). John Wiley & Sons, Inc., Hoboken, New Jersey

(2018). Priorities Established by the Combating Autism Act for Improving ASD Identification: Looking Beyond Ideas and Instruments Towards Implementation. View handout. International Society for Autism Research, Rotterdam, NL. May.

(2013). Autism Services Across America: Roadmaps for Improving State and National Education, Research, and Training Programs. Paul H. Brookes Publishing Co., Baltimore, MD