Search the Library

Abstract: Background: The National Institute on Drug Abuse (NIDA) Clinical Trials Network (CTN) has supported clinical trials of substance use disorder (SUD) interventions for 25 years. This review describes the use of implementation outcomes across CTN trials, characterizes outcomes included, and identifies gaps and potential opportunities to strengthen implementation research within the CTN and the field of SUD treatment. Methods: This systematic review included active or completed studies listed on the CTN Dissemination Library webpage as of August 18, 2021, and approved by the CTN for development by January 1, 2022. Study summaries and protocols were reviewed if they: 1) measured at least one implementation outcome and 2) examined a practice change, intervention, or process. Extracted data elements included trial design characteristics, implementation frameworks, and outcome assessment domains informed by the RE-AIM and Proctor Implementation Outcomes Frameworks. Results: 114 protocols were considered, 42 full-text protocols were screened, and 25 were included for data extraction. Start dates of trials spanned a 20-year period (2004–2024) with latter studies including more implementation outcomes. Fidelity (n = 29) and reach/penetration (n = 26) were the most included implementation outcomes. Equity was not identified in any protocols. Methods of defining, capturing, and evaluating outcomes data varied across trials and outcomes. Conclusions: The inclusion of implementation outcomes increased over time, perhaps reflecting a growing emphasis on implementation research. Incorporating measures of equity could advance knowledge about differential receipt or effectiveness of SUD interventions. Future research should seek to improve the consistency and comprehensiveness in descriptions of implementation science elements. Related protocols: CTN-0016, CTN-0056, CTN-0062-Ot, CTN-0064, CTN-0065, CTN-0069, CTN-0074, CTN-0074-A-1, CTN-0075, CTN-0076-Ot, CTN-0079, CTN-0079-A-1, CTN-0088, CTN-0090, CTN-0091, CTN-0095, CTN-0096, CTN-0097, CTN-0098, CTN-0099, CTN-0102, CTN-0103, CTN-0107, CTN-0116, CTN-0121

Abstract:

The Patient Feedback Survey, developed and evaluated in National Drug Abuse Treatment Clinical Trials Network protocol CTN-0016, is a performance improvement measure designed to assess the quality of outpatient substance abuse treatment. In this study, researchers modified and administered this measure to 500 individuals at AdCare Hospital, one of the New England Consortium Node’s community treatment providers (CTPs). Although the feedback scores were high in general, analyses of variance showed score variability in relation to type and length of treatment. Moreover, respondents who reported any use of marijuana, cravings for substances, or mutual-support group attendance (i.e., Alcoholics Anonymous or Narcotics Anonymous) had lower feedback scores than respondents without these experiences.

Conclusions: The results of the PFS study show that patients seeking outpatient services at a multisite community treatment program report positive evaluations of their treatment sessions. These findings are similar to the results of the original CTN study. The modified PFS appears to be an efficient measure that has the potential to gain broader applicability. This article highlights the importance of investigating treatment evaluations in the context of other recovery experiences.

Related protocols: CTN-0016

Abstract:

Therapeutic alliance, treatment satisfaction, and treatment participation have been associated with patient outcomes in substance abuse treatment programs. Available literature contends treatment outcomes are impacted on multiple levels by the interaction of patient-, clinician-, and program-level variables. Evidence suggests relationships between: patients’ engagement in treatment and program characteristics and functioning; treatment quality and clinicians’ caseload size; and organizational functioning and patients’ treatment satisfaction and therapeutic alliance.

Results from this study, which used baseline data from the Patient Feedback study (protocol CTN-0016), support previous findings on the relationship between organizational functioning and patient outcomes, operationalized as patients’ ratings of therapeutic alliance, treatment satisfaction, treatment duration, and self-reported substance use. Results provide further evidence on clinician-level variables, such as clinicians’ background and demographics, impact on patient engagement in treatment and abstinence from substances.

Related protocols: CTN-0016

Abstract:

Protocol CTN-0016 (“Patient Feedback: A Performance Improvement Study in Outpatient Addiction Treatment”) examined the feasibility of the implementation of a semiautomatic performance improvement system directed at clinicians in addiction treatment facilities called “patient feedback.” The study found that implementation of the program was generally feasible from both a research and a clinical perspective. This article reports on the results of a randomized, controlled trial evaluating the actual efficacy of the patient feedback system. The study involved 118 clinicians working at 20 community-based outpatient substance abuse treatment clinics in the northeastern United States (including some CTPs from the CTN). Ten clinics received 12 weeks of the patient feedback performance improvement intervention, and 10 clinics received no intervention during the 12 weeks. More than 1,500 patients provided anonymous ratings of therapeutic alliance, treatment satisfaction, and drug/alcohol use. There was no evidence of an intervention effect on the primary drug and alcohol use scales. There was also no evidence of an intervention effect on secondary measures of therapeutic alliance. Clinician-rated measures of organizational functioning and job satisfaction also showed no intervention effect. Possible insights from these findings and alternative methods of utilizing feedback reports to enhance clinical outcomes are proposed.

Related protocols: CTN-0016

Abstract:

Modifiable variables such as length of treatment, therapeutic relationship, and treatment environment are associated with successful outcomes for substance and alcohol dependent individuals, and permit and support incorporation of quality improvement (QI) systems in treatment settings to enhance consumer satisfaction. Initiated in 2006 and based on completed CTN protocol CTN-0016 (“Patient Feedback : A Performance Improvement Study in Outpatient Addiction Treatment Settings”), the Patient Feedback (PF) study, is a randomized effectiveness trial implementing a QI system at 20 outpatient, substance abuse treatment programs in Pennsylvania and New York. Patients in group therapy sessions complete anonymous surveys on a weekly basis evaluating treatment satisfaction and therapeutic alliance with their group counselors. Surveys are processed and two types of feedback reports are generated for clinicians to download via a password-protected website. Caseload reports display aggregated feedback from group clients for individual clinicians. Clinic reports display aggregated feedback from all group clients for all participating clinicians. During monthly staff meetings, clinic reports are discussed and strategies are developed to address specific areas where there is opportunity for improvement.

Key findings from staff interviews supporting the acceptability of the PF system include: 1) endorsement of PF and its usefulness in providing constructive feedback, 2) clinic reports are valued as an effective tool to identify areas for improvement, and 3) team meetings are useful in cultivating open discussions regarding significant therapeutic issues. Additional findings reflecting differences in clinic funding, technology utilization, and clinician caseloads will be presented. In conclusion, differences in clinic structure, organization, and available treatment services present unique opportunities for PF study implementation. These differences impact staff involvement, interest, and motivation of the PF system.

Related protocols: CTN-0016

Abstract:

Patient and staff turnover are significant therapeutic and management concerns in substance abuse treatment programs. Some literature suggest a positive correlation between patient satisfaction and successful outcome, while other studies are inconclusive. The treatment environment (location, physical space, and cultural milieu) is also associated with successful outcomes. Although providers often represent that they are satisfied with their jobs this seems to be contradicted by high staff turnover. Initiated in 2006 and based on completed CTN protocol CTN-0016 (“Patient Feedback : A Performance Improvement Study in Outpatient Addiction Treatment Settings”), the Patient Feedback study is a randomized, effectiveness trial, implementing a quality improvement (QI) system at 20 outpatient, substance abuse treatment programs. Patients’ baseline assessments were self-reported on survey instruments capturing ratings of treatment satisfaction. Clinic demographic forms and four self-report surveys assessing job satisfaction captured baseline data from providers along the following domains: 1) quality of director-employee relationships, 2) organizational characteristics, and 3) intrinsic/extrinsic satisfaction. Preliminary analysis from surveys of 679 patients reveals very high treatment satisfaction across patients of various treatment durations, including 227 patients in treatment for 1 month or less. Job satisfaction amongst 76 clinicians will be presented based on scores from the LMX-7 (evaluates interactions between directors and employees), the ORC (18 domains assessing organizational characteristics), and the MSQ (scale rating job satisfaction). Additional findings will be presented on gender, ethnicity, and length-of-treatment on patient and staff satisfaction. In conclusion, these findings have implications for the development and implementation of QI systems to enhance treatment outcomes and the work environment in substance abuse programs.

Related protocols: CTN-0016

Abstract:

Quality Improvement (QI) plays a significant role in accreditation in the healthcare industry, yet there is little research published on the effectiveness of QI in addictions treatment. In 2003, as part of NIDA’s National Drug Abuse Treatment Clinical Trials Network, a pilot study was launched to test the feasibility of a QI system, Patient Feedback (PF) (protocol CTN-0016, “Patient Feedback: A Performance Improvement Study in Outpatient Addiction Treatment”).

The study was conducted at six sites nationwide, with the results establishing the feasibility and acceptability of the PF system at addiction treatment facilities. In 2006, a randomized clinical trial testing the effectiveness of the PF system was initiated and is now being conducted at 35 adult outpatient substance abuse clinics in the New York City and Philadelphia areas. On a weekly basis, patients complete brief, anonymous surveys over a 12 week intervention to rate the quality of therapeutic alliance and treatment satisfaction in their drug counseling group sessions, and report weekly drug and alcohol use, and attendance to group sessions. The surveys are faxed to a data management unit and clinicians obtain real-time feedback of patient ratings through a password protected website. The supervisor and clinicians meet monthly to discuss the feedback and develop methods of enhancing patient ratings in the group sessions. Clinics are randomly assigned to one of two treatment conditions: experimental and delayed (control). Both conditions will have the opportunity to participate in a Sustainability Phase of the study consisting of 52-weeks of continued use of the PF system after completing the 12 week intervention. It is proposed that employment of the PF system will improve patient rates of therapeutic alliance, treatment satisfaction, attendance, and abstinence from drugs and alcohol, particularly among patients who are new to treatment.

Related protocols: CTN-0016

Abstract:

This article reports on the feasibility of implementing a semiautomated performance improvement system — Patient Feedback (PF) — that enables real-time monitoring of patient ratings of therapeutic alliance, treatment satisfaction, and drug/alcohol use in outpatient substance abuse clinics. The Patient Feedback system comprises four main components: feedback surveys, feedback reports, feedback newsletters, and team meetings. The system was implemented in six clinics within the National Institute on Drug Abuse Clinical Trials Network (as part of protocol CTN-0016), ultimately involving a total of 39 clinicians and 6 clinic supervisors. Throughout the course of the study (consisting of five phases: training period, baseline, intervention, postintervention assessment, sustainability), there was an overall collection rate of 75.5% of the clinic patient census. In general, the clinicians in these clinics had very positive treatment satisfaction and alliance ratings throughout the study. However, one clinic had worse drug use scores at baseline than other participating clinics and showed a decrease in self-reported drug use at postintervention.

This project demonstrated that the implementation of the PF system is generally feasible from both a research and a clinical perspective. Moreover, it is notable that clinics continued to use the PF system after the intervention phase, without any additional support from the research infrastructure, suggesting that the implementation of the intervention is sustainable. However, it is unclear whether this sustainability can only be achieved after completing a research study during which research infrastructure was substantially involved. Therefore, before clinical implementation can be recommended, more evaluation of the sustainability of the system and its positive intervention effects needs to be done.

Related protocols: CTN-0016

Abstract:

Patient Feedback (PF) is a web-based quality improvement (QI) system designed to monitor patient ratings of therapeutic alliance and other quality indicators in order to empower clinical staff. Improvement in therapeutic alliance has been shown to be associated with improvement in a variety of clinically important outcomes including attendance, retention, and abstinence. This poster reports on a feasibility study that was designed and implemented at six CTN clinics beginning in April 2004. It involved the administration of a 12-item confidential Patient Feedback Survey completed by clients at the close of group every other week for three months. The survey asked clients to rate their satisfaction with sessions, including whether or not they felt respected by their clinicians and whether they felt the sessions were helpful. The surveys were then converted into feedback reports, which each clinician and clinic could access securely on the Patient Feedback web site.

Overall, patients reported clinicians were performing very well, with relatively low average levels of patient drug and alcohol use. Similarly, alliance and treatment satisfaction ratings were high across all assessments. There was also evidence of improvements in average patient therapeutic alliance scores over time during the intervention phase for those clinicians who had initially poor treatment satisfaction ratings. This study demonstrated that the implementation of a semi-automatic quality improvement system for clinicians in addiction treatment facilities is feasible from both a research and clinical perspective and can be a low-cost, high impact way of conducting performance improvement studies.

Related protocols: CTN-0016

Abstract:

This manual was used in CTN Protocol CTN-0016. The feedback system in this manual is designed to help individual clinicians and clinical teams monitor attendance and two factors believed to contribute to attendance: therapeutic alliance and treatment satisfaction. Additionally, this system provides an objective, supportive and hopefully enjoyable way for clinicians and supervisors to work together to improve patient care. The Patient Feedback system is composed of four main components: feedback surveys, feedback reports, feedback newsletters, and team meetings, all of which are detailed in the manual. The Patient Feedback Survey instrument can be downloaded separately.

Related protocols: CTN-0016