Search the Library

Abstract: Motivational interviewing (MI) is an evidence-based practice shown to be effective when working with people in treatment for substance use disorders. However, MI is a complex treatment modality optimized by training with feedback. Feedback, assessment, and monitoring of treatment fidelity require measurement, which is typically done using audiotaped sessions. The gold standard for such measurement of MI skill has been an audiotaped interview, scored by a rate with a detailed structured instrument such as the Motivational Interviewing Treatment Integrity 2.0 (MITI 20.0) Coding System. The Helpful Responses Questionnaire (HRQ) is a pen-and-paper test of empathy (a foundational MI skill) that does not require an audiotaped session. A randomized trial of three different regimens for training counselors in MI (live supervision using teleconferencing, tape-based supervision, and workshop only) offered the opportunity to evaluate the performance of the HRQ as a measure of MI ability, compared to the several MITI 2.0 global scores and subscales. Participants were counselors (N=97) from 26 community-based substance use treatment programs affiliated with the Long Island and New York Nodes of the NIDA Clinical Trials Network, whose MI proficiency was measured at 4 time points: baseline (before an initial 2-day MI workshop), post-workshop, 8 weeks post-workshop (i.e., post-supervision), and 20 weeks post-workshop with both MITI 2.0 and HRQ HRQ total scores correlated significantly with the Reflection to Question Ratio from the MITI 2.0 at post-workshop, week 8, and week 20, and with the Spirit and Empathy global scores at week 20. Correlations of HRQ with other MITI 2.0 subscales and time points after workshop were small and not significant. As predicted, HRQ scores different between training conditions, with counselors assigned to live supervision achieving better HRQ scores than those in Workshop only. Conclusions: The HRQ is a modestly accurate measure, mainly of the Reflection to Question ratio, considered a core marker of MI skill. It is sensitive to training effects and may help identify counselors needing more intensive supervision. Given its ease of administration and scoring, HRQ may be a useful marker of MI skill during training efforts.

Abstract:

Learning motivational interviewing (MI) is an ongoing process, involving much more than attendance at a single workshop. Once proficiency is achieved, therapists benefit from ongoing coaching with individual feedback based on observed practice to ensure continued fidelity. The aim of this study was to assess outcomes of the unique training and supervision model employed in a recent trial of MI. The intervention tested in the six-site National Drug Abuse Treatment Clinical Trials Network protocol CTN-0047 trial was a 30-minute MI-based brief intervention delivered in the emergency department followed by two telephone booster calls delivered from a centralized call center. Thirty-one counselors and 3 booster counselors were trained in the intervention using a 2-stage process: local training in the MI process of engagement followed one month later by a 2-day training in MI. We employed a two-level model in which the formal coding was separated from the clinical supervision. One audio file per interventionist per week was coded using the MITI 3.1.1 coding system. This written feedback was available to clinical supervisors, who reviewed coding results during telephone supervision sessions.

Eleven percent of sessions were coded on an ongoing basis during the trial, with a total of 380 initial sessions (90%) and 83 booster sessions (20%) coded upon completion of the trial. Mean global scores for initial sessions ranged from 4.25 to 4.67, and for the booster sessions from 4.64 to 4.86, well above the proficiency benchmark of 4.0. Inter-rater reliability assessed on a random sample of 124 tapes was excellent, with ICCs averaging 0.81 for global scores and 0.93 for behavior counts. On a therapist level, MITI scores tended to improve over time, demonstrating the strategies employed helped with adherence and continued learning in MI.

Conclusions: A comprehensive strategy for successfully learning and maintaining skills in MI emerged from the CTN-0047 study, which employed a rigorous and novel plan for ensuring therapists adhered to the style of MI.

Related protocols: CTN-0047

Abstract:

This study examined associations of therapeutic alliance and treatment delivery fidelity with treatment retention in Stimulant Abusers to Engage in Twelve-Step (STAGE-12), a community-based trial of 12-Step Facilitation (TSF) conducted within the National Drug Abuse Treatment Clinical Trials Network (CTN). The STAGE-12 trial randomized 234 stimulant abusers enrolled in 10 outpatient drug treatment programs to an eight-session, group and individual TSF intervention. During the study, TSF participants rated therapeutic alliance using the Helping Alliance questionnaire-II (HAq-II). After the study, independent raters evaluated treatment delivery fidelity of all TSF sessions on adherence, competence, and therapist empathy. Poisson regression modeling examined relationships of treatment delivery fidelity and therapeutic alliance with treatment retention (measured by number of sessions attended) for 174 participants with complete fidelity and alliance data. Therapeutic alliance (p=.005) and therapist competence (p=.010) were significantly associated with better treatment retention. Therapist adherence was associated with poorer retention in a nonsignificant trend (p=.061).

Conclusions: Stronger therapeutic alliance and higher therapist competence in the delivery of TSF intervention were associated with better treatment retention whereas treatment adherence was not. Training and fidelity monitoring of TSF should focus on general therapist skills and therapeutic alliance development to maximize treatment retention. This is the first study to show a relationship between therapeutic alliance and retention in TSF with substance abusers, and to identify a significant fidelity-retention relationship for manual-guided TSF treatment, a finding that has important implications for treatment delivery.

Related protocols: CTN-0031

Abstract:

This ancillary investigation of data from National Drug Abuse Treatment Clinical Trials Network protocol CTN-0031 (“Stimulant Abuser Groups to Engage in 12-Step (STAGE 12)”) investigated the correspondence among four groups of raters on adherence to STAGE-12, a manualized 12-step facilitation (TSF) group and individual treatment targeting stimulant abuse. The four rater groups included the study therapists, supervisors, study-related (“TSF expert”) raters, and non-project-related (“external”) raters. Results indicated that external raters rated most critically Mean Adherence — the mean of all the adherence items — and global performance. External raters also demonstrated the highest degree of reliability with the designated expert. Therapists rated their own adherence lower, on average, than did supervisors and TSF expert raters, but therapist ratings also had the poorest reliability.

Conclusions: Findings highlight the challenges in developing practical, but effective methods of fidelity monitoring for evidence-based practice in clinical settings. While funding and licensing agencies increasingly call for use of evidence-based treatments, community-based organizations implementing them will seek the simplest, most reliable and cost-effective ways of monitoring their delivery. These results suggest that there may be a role for on-site therapists or supervisors rating adherence, and that raters unaffiliated with the treatment being tested may provide the most objective ratings. Future research should examine the impact of training therapists on self-rating to determine whether this group and achieve acceptable reliability and objectivity in ratings.

Related protocols: CTN-0031

Abstract:

This study examined the relationships between treatment fidelity and treatment outcomes in a community-based trial of 12-Step Facilitation (TSF) intervention. In a prior multi-site randomized clinical trial, National Drug Abuse Treatment Clinical Trials Network protocol CTN-0031, 234 participants in 10 outpatient drug treatment clinics were assigned to receive the Stimulant Abuser Groups to Engage in 12-Step (STAGE-12) intervention. This secondary analysis reviewed and coded all STAGE-12 sessions for fidelity to the protocol, using the Twelve Step Facilitation Adherence Competence Empathy Scale (TSF-ACES). Linear mixed-effects models tested the relationship between three fidelity measures (adherence, competence, empathy) and six treatment outcomes (number of days of drug use and five Addiction Severity Index (ASI) composite scores) measured at 3 months post-baseline. Adherence, competence, and empathy were robustly associated with improved employment status at follow up. Empathy was inversely associated with drug use, as was competence in a non-significant trend (p=.06). Testing individual ASI drug composite score items suggested that greater competence was associated with fewer days of drug use and, at the same time, with an increased sense of being trouble or bothered by drug use.

Conclusions: Greater competence and empathy in the delivery of a TSF intervention were associated with better drug use and employment outcomes, while adherence was associated with employment outcomes only. Higher therapist competence was associated with lower self-report drug use, and also associated with greater self-report concern about drug use. The nature of TSF intervention may promote high levels of concern about drug use even when actual use is low. This study is suggestive, but not conclusive, that higher fidelity intervention is associated with improved treatment outcome.

Related protocols: CTN-0031, CTN-0031-A-3

Abstract:

Therapist characteristics may be associated with variation in consistency, quality and effectiveness of treatment delivery. This ancillary investigation of National Drug Abuse Treatment Clinical Trials Network protocol CTN-0031 (Stimulant Abuser Groups to Engage in 12-Step (STAGE-12)) examined associations between treatment fidelity and therapist education, experience, treatment orientation, and perceived skills. Raters scored audio-recorded TSF sessions (n=966; 97% of TSF sessions) from 32 community-based, trained therapists for adherence, competence, empathy, and global session performance. Therapists with graduate degrees had significantly higher adherence and global performance fidelity ratings. Therapists reporting more positive attitudes toward 12-Step groups had lower adherence ratings. Being in recovery was associated with lower fidelity in univariate tests, but higher adherence in multivariate analysis. Fidelity was higher for therapists reporting self-efficacy in basic counseling skills and lower for self-efficacy in addiction-specific counseling skills. Fidelity was also superior in group relative to individual TSF sessions.

Conclusions: Results from this study have implications for therapist selection, training, and supervision in community-based, effectiveness trials and community implementation of evidence-based treatments. To obtain high fidelity and improve outcomes, it may be preferable to choose masters level therapists who are open to learning new treatments and have good, general counseling skills.

Related protocols: CTN-0031, CTN-0031-A-3

Abstract:

A fidelity measure was developed for use with Real Men Are Safe-Culturally Adapted (REMAS-CA), an HIV prevention intervention for ethnically diverse men in substance abuse treatment developed as an outgrowth of National Drug Abuse Treatment Clinical Trials Network protocol CTN-0018 (Reducing HIV/STD Risk Behaviors: A Research Study for Men in Drug Abuse Treatment). The aims of this ancillary investigation of data from that project were to: 1) assess the reliability of the Fidelity Rating and Skill Evaluation (FRASE); 2) measure improvement in therapist competence and adherence over time while delivering REMAS-CA; and 3) identify which modules of REMAS-CA were most difficult to deliver.

Conclusions: Results showed that the FRASE was a reliable instrument for measuring the fidelity of REMAS-CA delivery, and therapists achieved adequate adherence and competence after training, demonstrating significant improvement over time. Sessions 4 and 5 of REMAS-CA were found to contain the most challenging modules for therapists to deliver. These findings offer some guidelines for increasing counselor competence in implementing REMAS-CA for research or clinical practice. Specifically, more effort should be spent on training the counselors to implement the emotionally charged discussion and the specific skill building present in Sessions 4 and 5.

Related protocols: CTN-0018

Abstract:

Effective training and ongoing coaching in psychosocial treatment modalities is critical to maintaining fidelity in both research and practice. Maintaining fidelity may be particularly challenging in emergency department settings due to the fast pace and competing urgent and emergent priorities. This presentation describes intervention training, certification, supervision and fidelity monitoring procedures used in the NIDA CTN six-site “Screening Motivational Assessment and Referral to Treatment in Emergency Departments (SMART-ED)” study. Interventionists received a 2-day training in basic motivational interviewing skills, followed 1 month later by a 2-day training in the specific intervention used in this trial. Practice sessions with consenting ED patients were reviewed by expert raters, using the Motivational Interviewing Treatment Integrity scale (MITI, v.3.1.1), to determine if interventionists had reached benchmark scores and were therefore certifiable. Clinical supervision of interventionists was conducted independently of fidelity monitoring; centralized fidelity monitors reviewed 12% (n=96) of interventionists’ sessions and reported MITI scores to clinical supervisors to offer objective feedback regarding their supervisee’s performance. Clinical supervisors conducted bi-weekly phone supervision, discussing MI fidelity and clinical issues. Following completion of the trial, 20% of the interventionist sessions (n=161), of which 30% (n=55) were coded by two independent raters to assess inter-rater reliability, were randomly selected and coded for overall trial fidelity. Participating interventionists were 21 females and 12 males with little experience in addiction counseling (M=1.58 +/- 2.5 years). Fidelity monitoring during the trial successfully prevented drift and identified only one interventionist in need of remedial supervision. Bi-weekly coaching continued throughout the trial and interventionists found these sessions useful in maintaining their skills. Results from fidelity monitoring indicate above average performance on MITI scores.

Conclusions: The two-stage interventionist training, bi-weekly supervision, and ongoing monitoring produced excellent results and prevented drift. This model may bestow an advantage for learning and implementing brief interventions based on an MI approach.

Related protocols: CTN-0047

Abstract:

Twelve step facilitation (TSF) is an emerging, empirically supported treatment, the study of which will be strengthened by rigorous fidelity assessment. This report describes the development, reliability, and concurrent validity of the Twelve Step Facilitation Adherence Competence Empathy Scale (TSF ACES), a comprehensive fidelity rating scale for group and individual TSF treatment developed for the National Drug Abuse Treatment Clinical Trials Network study, “Stimulant Abuser Groups to Engage in 12-Step” (STAGE-12, CTN-0031). Independent raters used TSF ACES to rate treatment delivery fidelity of 966 (97% of total) TSF group and individual sessions. TSF ACES summary measures assessed therapist treatment adherence, competence, proscribed behaviors, empathy, and overall session performance. TSF ACES showed fair to good overall reliability; weighted kappa coefficients for 59 co-rated sessions ranged from .31 to 1.00, with a mean of .69. Reliability ratings for session summary measures were good to excellent (.69–.91). Internal consistency for the instrument was variable (.47–.71). Relationships of the TSF ACES summary measures with each other, as well as relationships of the summary measures with a measure of therapeutic alliance, provided support for concurrent and convergent validity. Fidelity instruments such as the TSF ACES can be used in clinical implementation to train and supervise counselor adherence and skill. Implications and future directions for the use of the measure in clinical trials and community treatment implementation are discussed.

Related protocols: CTN-0031, CTN-0031-A-3

Abstract:

Pharmacotherapy trials for treating tobacco dependence would benefit from behavioral interventions providing treatment consistent with clinical practice guidelines but not directing participants to treatment not evaluated in the trial. The Smoke Free and Living It behavioral intervention manual includes participant and interventionist guides and is designed to provide both practical counseling and intra-treatment support. In protocol CTN-0029, researchers utilized this intervention manual as part of their multicenter, randomized clinical trial of smokers with attention deficit hyperactivity disorder. In the study, they evaluated how the interventional manual performed in a “train-the-trainer” model requiring uniform counseling across 6 sites and 15 interventionists. The skill-adherence of the interventionists and the intervention-adherence of the participants was analyzed. The 2555 randomized participants completed 9.3 +/- 2.8 sessions (mean +/- SD), with 157 participants (61.6%) completing all 11 of the sessions and 221 (86.7%) completing at least 6 of the 11 sessions. Of the 163 sessions for which the study interventionists were evaluated, 156 (95.7%) were rated as adherent to protocol and “meeting expectations” on at least 6 of 7 established criteria, illustrating that fidelity can be maintained with minimal supervision.

Conclusions: The self-help and interventionists guides of the Smoke Free and Living It manual can thus be used to provide behavioral intervention with a high rate of adherence by both the interventionists and the participants. This manual can be used as a self-help guide, an interventionist guide, or both, and can be tailored to a specific research protocol, regardless of the size of the study or number of study sites. The Smoke Free and Living It manual meets the requirements of the United States Public Health Service Clinical Practice Guideline, can be adapted to specific research protocols, and provides a useful option for behavioral intervention during clinical trials for smoking cessation.

Related protocols: CTN-0029

Abstract:

Training research staff to implement clinical trials occurring in community-based addiction treatment programs presents unique challenges. Standardized patient walkthroughs of study procedures may enhance training and protocol implementation. The objective of this study was to examine and discuss cross-site and cross-study challenges of participant screening and data collection procedures identified during standardized patient walkthroughs of multi-site clinical trials. Actors portrayed clients and “walked through” study procedures with protocol research staff (the study completed 57 walkthroughs during implementation of four clinical trials). Observers and walkthrough participants then identified three areas of concern (consent procedures, screening and assessment processes, and protocol implementation) and made suggestions for resolving the concerns.

Conclusions: Standardized patient walkthroughs capture issues with study procedures previously unidentified with didactic training or unscripted rehearsals. Clinical trials within the National Drug Abuse Treatment Clinical Trials Network are conducted in addiction treatment centers that vary on multiple dimensions. Based on walkthrough observations, the national protocol team and local site leadership can modify standardized operating procedures and resolve cross-site problems prior to recruiting study participants. The standardized patient walkthrough improves consistency across study sites and reduces potential site variation in study outcomes.

Abstract:

Effective training in psychosocial treatment modalities is critical to maintaining fidelity in both research and practice. This poster reports on the effectiveness of a two-stage training for interventionists participating in a NIDA Clinical Trials Network study, protocol CTN-0047: Screening, Motivational Assessment, Referral, and Treatment in Emergency Departments (SMART-ED). Interventionists at the first two sites initiating the study received a 2-day training in basic Motivational Interviewing skills, followed 1 week later by a 2-day training in the specific counseling intervention used in the trial. They then completed practice sessions with consenting ED patients. Audiotapes of these sessions were reviewed by an expert rater, using the Motivational Interviewing Treatment Integrity scale (MITI, v. 3.1.2) and content checklists. To become certified, interventionists had to score at least 4.0 (“competent”) on the 5-point Global Clinician Rating from the MITI and 80% on content checklists, for 2 out of 3 sessions. Participating interventionists were 5 females and 3 males, 4 of whom were licensed or certified as counselors, with 2 years mean counseling experience (range 0-7). Their self-reported pre-training understanding of MI was moderate (5.5 mean rating on Likert scale 0-9). All participants met criteria for certification after only 2 practice sessions. In conclusion, the two-stage interventionist training used in this study produced excellent results across the first 8 interventionists trained. Focusing first on fundamentals and later on specific intervention content may bestow an advantage for learning and implementing brief interventions based on a motivational interviewing approach.

Related protocols: CTN-0047

Abstract:

This CTN platform study assessed the extent of implementation of the Public Health Service tobacco cessation guidelines among a national sample of counselors working in five different types of substance abuse treatment programs. Implementation patterns among counselors were also identified using cluster analysis and considered differences in counselor characteristics based on their cluster membership. Data were obtained from the 2008 Managing Effective Relationships in Treatment Services (MERITS I) project. Counselors (N = 615) working in National Drug Abuse Treatment Clinical Trials Network (CTN) affiliated community treatment programs completed paper-and-pencil surveys. Implementation of the guidelines was inconsistent and selective. Counselors could be grouped into low versus high implementers. What distinguishes the high implementers from the low implementers? More high compared to low worked in methadone maintenance programs, which is similar to previous findings. Additionally, high implementers tended to report greater annual incomes than low implementers, which may be a function of where these counselors are working. For example, counselors with higher incomes may be more likely to work in upscale and state-of-the-art treatment programs, have patients with greater insurance coverage, and have patients who are more involved in and demanding in terms of their treatment. This study adds to the sparse by emerging literature on the implementation of tobacco cessation guidelines in substance abuse treatment programs. Counselors are in a prime position to capitalize on the crucial opportunity for preventing tobacco related deaths, reducing health care costs, and reaching public health goals by routinely integrating these guidelines into their daily practice.

Abstract:

Although psychotherapy involves the interaction of client and therapist, mutual influence is not typically considered as a source of variability in therapist adherence and competence in providing treatments assessed in clinical trials. This study examined variability in therapist adherence and competence in Motivational Enhancement Therapy (MET) both within and between caseloads in a large multi-site clinical trial (CTN-0004: Motivational Enhancement Treatment to Improve Treatment Engagement and Outcome in Subjects Seeking Treatment for Substance Abuse). Three-level multilevel models (repeated measures, nested within clients, nested with therapists) indicated significant variability both within and between therapists. There was as much and sometimes more variability in MET adherence and competence within therapist caseloads than between therapists. Variability in MET adherence and competence within caseloads was not consistently associated with client severity of addiction at baseline. However, client motivation at the beginning of the session and days of use during treatment were consistent predictors of therapist adherence and competence.

Results raise questions about the nature of therapist adherence and competence in treatment protocols. Accordingly, future analysis of clinical trials should consider the role of mutual influence in measures of therapist performance.

Related protocols: CTN-0004

Abstract:

Therapist adherence has been shown to predict clinical outcomes in family therapy. In prior studies, adherence has been represented broadly by core principles and a consistent family (vs. individual) focus. To date, these studies have not captured the range of clinical skills that are represented in complex family-based approaches or examined how variations in these skills predict different clinically relevant outcomes over the course of treatment. In this ancillary study, the authors examined the reliability and validity of an observational adherence measure and the relationship between adherence and outcome in a sample of drug-using adolescents receiving Brief Strategic Family Therapy as part of protocol CTN-0014, “Brief Strategic Family Therapy (BSFT) for Adolescent Drug Abusers.” Results supported the proposed factor structure of the adherence measure, providing evidence that it is possible to capture and discriminate between distinct dimensions of family therapy.

Analyses demonstrated that the mean levels of the factors varied over time in theoretically and clinically relevant ways and that therapist adherence was associated with engagement and retention in treatment, improvements in family functioning, and reductions in adolescent drug use. Clinical implications and future research directions are discussed, including the relevance of these findings on training therapists and studies focusing on mechanisms of action in family therapy. Although this measure was developed specifically for the BSFT intervention, it is possible that the domains identified may be present in other empirically based family interventions and, as such, future studies may consider validating the use of this or similar measures with other clinical models and populations. The results of this study provide a necessary first step by validating a measure that can be used to capture the complexity of family therapy, and can be used to understand mechanisms of change in family therapy with drug-using youth.

Related protocols: CTN-0014