Search the Library

The integration of data across randomized controlled trials (RCTs) testing interventions and treatments for substance use disorder (SUD) offers a rich opportunity for improving the evidence base and analytic methods used in SUD research. With over 50 completed trials of the National Institute on Drug Abuse (NIDA) National Drug Abuse Treatment Clinical Trials Network (CTN) available for secondary analysis and harmonization, the possibilities are extensive, but the effort to harmonize and document datasets demands complex analytic formulation and methodology. This commentary discusses strengths and challenges of data harmonization, sharing clinical and data science considerations based on four exemplar studies that harmonized data across multiple CTN trials. We offer recommendations for others planning data harmonization for secondary analysis, discuss guiding principles for research data management, outline suggestions to bridge gaps in the context of the CTN, and finally frame considerations for using state-of-the-art tools such as generative AI and integration of data from clinical trials and electronic health records to enhance the promise of data harmonization.

There is an urgent need within the substance-use-disorders (SUD) treatment field to develop and implement consensus-based common core data elements (CDEs) with standardized vocabularies relevant to drug addiction treatment that could be incorporated and widely adopted into harmonized electronic medical record systems (EMRs). This will benefit patients by improving the quality of care and will assist in integration of specialty addiction treatment into disciplines of mainstream medicine. To achieve these aims, the NIDA Clinical Trials Network (CTN) has collected and collated dozens of treatment-form-related information and standardized instruments to develop a treatment-relevant set of CDEs. These CDEs were refined following a consensus-based meeting of federal, state, and community-based treatment stakeholders and providers. This poster describes the collaborative “Mind Map” used for developing and implementing core questions as CDEs for EMRs on SUD in primary care and SUD specialty treatment settings. Current progress in developing EMR core questions as CDEs for use in those settings is also provided, as well as implications of this project for the future of drug abuse treatment. NIDA is especially interested in input from College on Problems of Drug Dependence (CPDD) members on data collection hierarchy and core data elements and on the overall strategy in regards to other sources of input, other stakeholders who should be consulted, and other “next steps” as this project moves forward.

Several large-scale, pragmatic clinical trials on opioid use disorder (OUD) have been completed in the National Drug Abuse Treatment Clinical Trials Network (CTN). However, the resulting data have not been harmonized between the studies to compare the patient characteristics. This paper provides lessons learned from a large-scale harmonization process that are critical for all biomedical researchers collecting new data and those tasked with combining datasets.

Researchers harmonized data from multiple domains from CTN-0027 (N = 1269), which compared methadone and buprenorphine at federally licensed methadone treatment programs; CTN-0030 (N = 653), which recruited patients who used predominantly prescription opioids and were treated with buprenorphine; and CTN-0051 (N = 570), which compared buprenorphine and extended-release naltrexone (XR-NTX) and recruited from inpatient treatment facilities. Patient-level data were harmonized and a total of 23 database tables, with meticulous documentation, covering more than 110 variables, along with three tables with “meta-data” about the study design and treatment arms, were created. Domains included: social and demographic characteristics, medical and psychiatric history, self-reported drug use details and urine drug screening results, withdrawal, and treatment drug details.

In this paper, the authors summarize the numerous issues with the organization and fidelity of the publicly available data which were noted and resolved, and present results on patient characteristics across the three trials and the harmonized domains, respectively. A systematic harmonization of OUD clinical trial data can be accomplished, despite heterogeneous data coding and classification procedures, by standardizing commonly assessed characteristics. Similar methods, embracing database normalization and/or “tidy” data, should be used for future datasets in other substance use disorder clinical trials.

Related protocols: CTN-0027, CTN-0030, CTN-0051

The efficacy of treatments for substance use disorders (SUD) is tested in clinical trials in which participants typically provide urine samples to detect whether the person has used certain substances via urine drug screenings (UDS). UDS data form the foundation of treatment outcome assessment in the vast majority of SUD clinical trials. However, existing methods to calculate treatment outcomes are not standardized, impeding comparability between studies and prohibiting reproducibility of results.

In this study, the researchers extended the concept of a binary UDS variable to multiple categories: “+” [positive for substance(s) of interest], “–” [negative for substance(s)], “o” [patient failed to provide sample], “*” [inconclusive or mixed results], and “_” [no specimens required per study design]. This construct can be used to create a standardized and sufficient representation of UDS datastreams and sufficiently collapses longitudinal records into a single, compact “word”, which preserves all information contained in the original data.

Researchers developed the R software package CTNote (available on CRAN) as a tool to enable computers to parse these “words”. The software package contains five groups of routines: detect a substance use pattern, account for a specific trial protocol, handle missing UDS data, measure the longest period of consecutive behavior, and count substance use events. Executing permutations of these routines result in algorithms which can define SUD clinical trial endpoints. As examples, the authors provide three algorithms to define primary endpoints from seminal SUD clinical trials

Conclusions: Representing substance use patterns as a “word” allows researchers and clinicians an “at a glance” assessment of participants’ responses to treatment over time. Further, machine readable use pattern summaries are a standardized method to calculate treatment outcomes and are therefore useful to all future SUD clinical trials. The paper includes discussion of some caveats when applying this data summarization technique in practice and areas of future study.

Related protocols: CTN-0094

The purpose of this study was to estimate how results would have varied if a substance abuse clinical trial had been conducted with nationally representative adults with substance use and with representative adults receiving substance use treatment. Results were analyzed from NIDA Clinical Trials Network protocol CTN-0044, a multisite clinical trial comparing the effectiveness of the Therapeutic Education System to treatment as usual for outpatient addiction treatment (n = 507). Patients were recruited between June 2010 and August 2011. Abstinence was the primary outcome. The general population sample and general population-treated samples were derived from Wave 1 of the National Epidemiologic Survey on Alcohol and Related Conditions (NESARC) (n = 43,093). Propensity scores provided a standardized measure of the difference between clinical trial participants and the 2 NESARC samples. The clinical trial was reanalyzed by reweighting the sample with propensity scores derived from the 2 samples to obtain generalizable estimates of treatment effects.

Before the clinical trial sample was reweighted, the odds ratio (OR) of response to Therapeutic Education System versus treatment as usual in the trial was 1.62 (95% CI, 1.12-2.35). After the sample was reweighted to be representative of the 2 NESARC groups, ORs were 1.33 (95% CI, 0.34-5.26) for the representative sample with any substance use and 1.64 (95% CI, 0.82-3.27) for the representative treated sample. The effect size of the original study was statistically significant; the estimate effect size for the nationally representative sample was not. This does not necessarily mean that the Therapeutic Education System is not efficacious for the treatment of substance use disorders. Instead, the width of the confidence intervals reflects increased uncertainty associated with extrapolating the results of the clinical trial sample to broader populations.

Conclusions: Applying propensity score weighting to clinical trial results provides a method for estimating the population generalizability of clinical trial findings that relies on effect moderators observed in the study sample and population. Broader confidence intervals in the reweighted samples do not necessarily indicate lack of efficacy of the Therapeutic Education System but rather greater uncertainty concerning effectiveness in general population samples.

Related protocols: CTN-0044

This webinar provided an update on CTN-0062-Ot, a phased feasibility and proof-of-concept study seeking to incorporate addiction-specific screening and assessment of common data elements (CDEs) into a widely used electronic health record (EHR), explore the logistics and time required to do this, and assess impacts on the frequency of identification, diagnosis, and referral to treatment in large healthcare organizations.

The webinar included these components (click each title for the individual slides):

• A Phased-Implementation Feasibility and Proof-of-Concept Study to Assess Incorporating the NIDA CTN Common Data Elements into the Electronic Health Record in Large Primary Care Settings (CDE-EHR-PC Study, CTN-0062-Ot). Jennifer McNeely, MD, MS, New York University School of Medicine
• Usability: An Introduction. Joseph Kannry, MD, Mount Sinai Health System
• Usability in Healthcare IT: Data Collection and Analysis Approaches. Andrew Kushniruk, PhD, School of Health Information Science, University of Victoria
• Lessons Learned and Conclusion.

Related protocols: CTN-0062-Ot

The mHealth revolution brings exciting possibilities for both researchers and clinicians, but the use of this technology also raises unique ethical concerns. The purpose of this talk is two-fold. First, we will be discussing some of the most pressing ethical challenges around mHealth. This will involve looking carefully at issues around data ownership, privacy, transparency, and consent. Second, we will introduce some possible solutions. Come discuss what researchers, clinicians, and institutions can do to address ethical risk.

Dr. Tiffany Cvrkel is a bioethicist, philosopher, and lecturer in UCLA’s Department of Molecular, Cell, & Developmental Biology. The primary focus of her work is categorizing the impact and limitations of deliberation as it occurs in the contexts of biomedical ethics. Her particular area of expertise is the ethics of emerging biomedical technologies, including the ethical challenges around mHealth, eHealth, and Big Data. In addition to being an award-winning teacher, she serves as a consultant to scientists and clinicians working with bioethical questions. She specializes in both bringing clarity to bioethical challenges and to assisting in the creation of practical solutions.

Additional Resources:

• Download slides (pdf)

The National Drug Abuse Treatment Clinical Trials Network (CTN) of the National Institute on Drug Abuse (NIDA) recently launched a public portal which provides a single-source repository for CTN-recommended common data elements (CDEs) for substance use disorders (SUD) for use in electronic health record systems (EHRs) and clinical research. A CDE in this context is a data element consisting of a question and enumerated set of possible values for responses precisely defined by standardized metadata descriptors. CDEs consisting of individual question/answer pairs can be combined into more complex questionnaires and case report forms or used when gathering medical information in the context of providing clinical care. Thus, CDEs describe semantic characteristics for a discrete piece of data, which will be collected, stored, or exchanged during the course of a study or health examination. This will facilitate exchange of standardized data because of the use of CDEs. In this manner, NIDA CDEs can be commonly applied to multiple data collection systems whether in research or clinical care and across different institutions, such that their intentional commonality with use of common data standards can improve data quality, facilitate data repurposing, and promote data sharing.

This paper describes objectives and importance of the CTN CDEs initiative and portal to translational psychiatric research: To support harmonized use of EHR-compatible common data elements to enable exchange and integration of data to answer clinical meaningful questions of broad interest to SUD treatment research, thereby facilitating big-data biomedical science crossing boundaries between research and clinical care.

The purpose of this study was to examine sensitivity to missing data procedures on treatment effects in a randomized controlled trial (RCT) of osmotic-release methylphenidate (OROS) for adolescents with co-occurring attention-deficit/hyperactivity disorder (ADHD) and substance use disorders (SUD). Data came from a National Drug Abuse Treatment Clinical Trials Network study (CTN-0028, N=303), which evaluated the safety/efficacy of a 16-week RCT of OROS vs. placebo in adolescents aged 13-18 with ADHD who were also receiving cognitive-behavioral therapy for their SUD. The two primary outcomes were clinician-reported ADHD symptoms and self-reported past 28 days of substance use (SU). A parallel grow model was used to assess the effect sizes assuming missing at random (MAR) compared to two missing not at random (MNAR) models: Diggle-Kenward (DK) selection model and Wu-Carroll (WC) selection model.

The MAR model found no significant treatment effect on ADHD or SU, and the effect sizes were small for both ADHD and SU. The MNAR DK model also produced non-significant treatment effects with similar effect sizes of ADHD and SU. The MNAR WC model evidenced a significant effect of OROS relative to placebo on SU, and the effect sizes for both ADHD and SU were larger than reported in the other models.

Conclusions: While the MAR model and one MNAR model found similarly sized effects as the original RCT, the second MNAR model produced different results for both of the outcomes. This sensitivity analysis highlights an important need for future RCTs of co-morbid mental illness and SUDs to carefully evaluate the missing data assumptions made when assessing treatment effects.

Related protocols: CTN-0028

Use of psychosocial measures with different conceptual meanings across cultural groups may render treatment outcome analyses invalid in social work research. Determining measurement invariance allows researchers to assess whether the construct of a measure is similarly comprehended and measured across participant groups (e.g., based on race, ethnicity, gender, age, etc.). Nonequivalence is introduced when groups of participants experience or conceptualize a construct differently, or use distinctive criteria to describe the concept. Measurement nonequivalence across cultural groups is posited to occur due to (a) cultural differences in norms and relevance of the constructs being assessed; (b) language of assessment; or (c) potential differences in participants’ environments and opportunity structures to engage in certain behaviors or develop beliefs due to contextual differences, racism, or other forms of discrimination.

To illustrate this statistical procedure, this poster presents measurement invariance properties across racial groups for two commonly used instruments in social work and substance abuse treatment research (the Revised Helping Alliance Questionnaire (HAq-II) and the Short Inventory of Problems (SIP-R)), using data from the National Drug Abuse Treatment Clinical Trials Network protocol CTN-0004 (“Motivational Enhancement Treatment to Improve Treatment Engagement and Outcome in Subjects Seeking Treatment for Substance Abuse”). Analysis shows that use of measures with different conceptual meaning across racial and ethnic groups may render invalid analyses comparing such groups. Conclusions drawn from invalid findings can lead to ineffective treatments and policy initiatives.

Conclusions: Findings support the comparable understanding of therapeutic alliance and consequences of substance as measured by the HAq-II and SIP-R in African American and non-Latino white participants. Difference in reliability caused by the identified items needs verification in future studies to ensure use of reliable HAq-II and SIP-R latent factors.

Related protocols: CTN-0004

There are many benefits of data sharing, including the promotion of new research from effective use of existing data, replication of findings through re-analysis of pooled data files, meta-analysis using individual patient data, and reinforcement of open scientific inquiry. A randomized controlled trial is considered the “gold standard” for establishing treatment effectiveness, but clinical trial research is very costly — sharing data is an opportunity to expand the investment of the clinical trial beyond its original goals at minimal cost. This article describes the goals, developments, and usage of the Data Share website for the National Drug Abuse Treatment Clinical Trials Network (CTN) in the U.S., including lessons learned, limitations, and major revisions, and considerations for future directions to improve data sharing. Since its inception in 2006 and through October 2012, nearly 1700 downloads from 27 clinical trials have been accessed from the Data Share website, with use increasing over the years. Individuals from 31 countries have downloaded data from the site, and there have been at least 13 publications derived from analysis of data obtained through the public Data Share website. Limitations of the website include minimal control over data requests and usage, which has resulted in little information and lack of control regarding how the data from the website are being used, and a lack of uniformity in data elements collected across CTN trials, which has limited cross-study analyses.

Conclusions: The Data Share website offers researchers easy access to de-identified data files with the goal to promote additional research and identify new findings from completed CTN studies. To maximize the utility of the website, ongoing collaborative efforts are needed to standardize the core measures used for data collection in the CTN studies with the goal of increasing their comparability and facilitating the ability to pool data files for cross-study analyses.

Supported by the Duke Clinical Research Institute (CTN DSC 1).

A review of substance use clinical trials indicates that sub-optimal methods are the most commonly used procedures to deal with longitudinal missing information. In this study of data from the National Drug Abuse Treatment Clinical Trials Network buprenorphine protocol CTN-0003, listwise deletion (i.e., using complete cases only), positive urine analysis (UA) imputation, and multiple imputation (MI) were used to evaluate the effect of baseline substance use and buprenorphine/naloxone tapering schedule (7 or 28 days) on the probability of a positive UA (UA+) across the 4-week treatment period. The listwise deletion generalized estimating equations (GEE) model demonstrated that those in the 28-day taper group were less likely to submit a UA+ for opioids during the treatment period, as did the positive UA imputation model. The MI model also demonstrated a similar effect of taper group, but the effect size was more similar to that of the listwise deletion model.

Conclusions: The missing data situation described in this investigation generalizes to many other substance use psychopharmacology clinical trials wherein there is missing data on the outcome of interest only. Future researchers may find utilization of the MI procedure in conjunction with the common method of GEE analysis as a helpful analytic approach when the missing at random assumption is justifiable.

Related protocols: CTN-0003

While substance use problems are considered to be common in medical settings, they are not systematically assessed and diagnosed for treatment management. Research data suggest that the majority of individuals with a substance use disorder do not use treatment or delay treatment-seeking for over a decade. The separation of substance abuse services from mainstream medical care and a lack of preventive services for substance abuse in primary care can contribute to under-detection of substance use problems. When fully enacted in 2014, the Patient Protection and Affordable Care Act 2010 will address these barriers by supporting preventive services for substance abuse (screening, counseling) and integration of substance abuse care with primary care. One key factor that can help to achieve this goal is to incorporate the standardized screeners or common data elements for substance use and related disorders into the electronic health records (EHR) system in the health care setting. NIDA has asked its Clinical Trials Network (CTN) to lead the effort to develop a set of such data elements for drug abuse research that could also be used in EHRs for patient care.

This commentary focuses on recent evidence about routine screening and intervention for alcohol/drug use and related disorders in primary care. Federal efforts in developing common data elements for use as screeners for substance use and related disorders are described. A pressing need for empirical data on screening, brief intervention, and referral to treatment (SBIRT) for drug-related disorders to inform SBIRT and related EHR efforts is highlighted.

Supported by the Duke Clinical Research Institute (CTN DSC 1).

Substance abuse treatment research is often characterized by having a non-trivial amount of missing data, especially in longitudinal, randomized clinical trials. A review of substance abuse clinical trials indicates that listwise deletion and single imputation (i.e., imputing missing values with a positive urine analysis) are the most commonly used procedures to deal with missing information. Due to the prevalence and complex nature of missing data in substance abuse research, the best missing data procedures need to be employed and tailored to each unique missing data situation so as to maximize the likelihood of arriving at the most accurate conclusions. This poster reports on a study that compared the outcomes from two common missing data handling procedures, using data sets from protocol CTN-0003, with the outcomes from the multiple imputation (MI) procedure in the context of generalized estimating equations (GEE) in order to demonstrate that interpretations of treatment effectiveness and other baseline covariate effects can change as a function of how the missing information is handled.

Conclusions: The investigation demonstrated how treatment efficacy can vary as a function of how the missing information is treated when there is a significant amount of it. Missing data theory suggests that listwise deletion and single imputation procedures should not be used to account for missing information, and that MI has advantages with respect to internal and external validity when the assumption of missing at random can be reasonably supported. Future researchers may find utilization of the MI procedure in conjunction with the common method of GEE analysis as a helpful analytic approach compared to commonly used procedures when the missing at random assumption is justifiable.

Related protocols: CTN-0003

Because the analysis of a treatment’s efficacy can vary as a function of how the missing data in a substance abuse clinical trial is handled analytically, there is a potential problem of not thoroughly evaluating and reporting the mechanism of missing information as part of the analytic strategy. Appropriate decision-making regarding how the missing data is handled is critical in order to make sound clinical inference based on randomized trials of substance abuse treatment, but missing not at random (MNAR) methodologies have no been explored in this context with real data. This investigation uses data from protocol CTN-0003 to compare 3 different modeling strategies for the handling of missing values (i.e., missing at random (MAR) model versus 2 different missing not at random (MNAR) models; Diggle-Kenward and Wu-Carroll selection modeling), to determine whether the treatment effect (i.e., impact of trial arm on linear urine analysis (UA) slope) is dependent on the missing data strategy used.

Conclusions: The trial arm effect on the UA slope (i.e., UA change over time), and other covariate effects, changed in a meaningful way across the MAR and MNAR growth models, indicating that missing data assumptions are critical to understand and explain in clinical trials. It is not only important for the research team to consider what the most likely missing data assumption is (i.e., MAR or MNAR), but also to consider whether or not the additional assumptions associated with each MNAR and MAR model are reasonable. This investigation highlights the potential for these modern approaches to missing data to shed new light on outcomes of interest (e.g., time-specific dropout) other than the primary outcome of UA.

Related protocols: CTN-0003