Search the Library

Abstract: The importance of implementing evidence-based practices has taken over health care, whether in the mental health, medical, or substance use treatment realms. While well-intentioned and important, the implementation of evidence-based practices is only one aspect of quality care. In fact, there are many other elements of treatment that are at least as important as offering evidence-based practices, and this is particularly true in levels of care other than traditional outpatient. Other elements of care essential to quality treatment include: 1) Monitoring and ensuring a positive therapeutic alliance between clients and providers – Much research has documented the relationship between positive treatment outcome and a positive therapeutic alliance; 2) Monitoring and ensuring high customer satisfaction: Regardless of the specific treatment offered, it is essential to monitor and address customer satisfaction, as without this, clients will not obtain benefit from treatment; 3) Monitoring administrative discharges – In residential treatment programs, clients are often discharged for non-compliance with treatment. This needs close monitoring, as from experience, administrative discharges have more to do with staff than with clients; 4) Ensuring readily-available treatment access – When individuals are ready to enter treatment, treatment needs to be available. Treatment not being available or long waiting periods to enter treatment will result in lost opportunities; 5) Monitoring no-show rates and overall treatment retention – People have to come to treatment and be retained for treatment to be effective. Thus, a focus on this is necessary; 6) Ensuring clients – multiple treatment needs are addressed – Either in-house or through linkages, clients – other problems need to be addressed; and 7) Ongoing quality improvement – To ensure quality treatment, a culture of continuous quality improvement must be developed. This presentation will highlight other important aspects of care that must not be forgotten while programs are focused on implementing evidence-based practices.

Abstract:

This is the Primary Outcomes Article for CTN-0103.

National opioid-related overdose fatalities totaled 650,000 from 1999 to 2021. Some of the highest rates occurred in New Hampshire, where 40% of the population lives rurally. Medications for opioid use disorder (MOUD; methadone, buprenorphine, and naltrexone) have demonstrated effectiveness in reducing opioid overdose and mortality. Methadone access barriers disproportionally impact rural areas and naltrexone uptake has been limited. Buprenorphine availability has increased and relaxed regulations reduces barriers in general medical settings common in rural areas. Barriers to prescribing buprenorphine include lack of confidence, inadequate training, and lack of access to experts. To address these barriers, learning collaboratives have trained clinics on best-practice performance data collection to inform quality improvement (QI). This project sought to explore the feasibility of training clinics to collect performance data and initiate QI alongside clinics’ participation in a Project ECHO virtual collaborative for buprenorphine providers.

Eighteen New Hampshire clinics participating in a Project ECHO were offered a supplemental project exploring the feasibility of performance data collection to inform QI targeting increased alignment with best practice. Feasibility was assessed descriptively, through each clinic’s participation in training sessions, data collection, and QI initiatives. An end-of-project survey was conducted to understand clinic staff perceptions of how useful and acceptable they found the program.

Five of the eighteen health care clinics that participated in the Project ECHO joined the training project, four of which served rural communities in New Hampshire. All five clinics met the criteria for engagement, as each clinic attended at least one training session, submitted at least one month of performance data, and completed at least one QI initiative. Survey results showed that while clinic staff perceived the training and data collection to be useful, there were several barriers to collecting the data, including lack of staff time, and difficulty standardizing documentation within the clinic electronic health record.

Conclusions: Results suggest that training clinics to monitor their performance and base QI initiatives on data has potential to impact clinical best practice. While data collection was inconsistent, clinics completed several data-informed QI initiatives, indicating that smaller scale data collection might be more attainable.

Related protocols: CTN-0103

Abstract:

Emergency clinicians are on the front lines of responding to the opioid epidemic and are leading innovations to reduce opioid overdose deaths through safer prescribing, harm reduction, and improved linkage to outpatient treatment. Currently, there are no nationally recognized quality measures or best practices to guide emergency department quality improvement efforts, implementation science researchers, or policymakers seeking to reduce opioid-associated morbidity and mortality. To address this gap, in May 2017, the National Institute on Drug Abuse’s Center for the Clinical Trials Network convened experts in quality measurement from the American College of Emergency Physicians’ (ACEP’s) Clinical Emergency Data Registry, researchers in emergency and addiction medicine, and representatives from federal agencies, including the National Institute on Drug Abuse and the Centers for Medicare & Medicaid Services. Drawing from discussions at this meeting and with experts in opioid use disorder treatment and quality measure development, we developed a multistakeholder quality improvement framework with specific structural, process, and outcome measures to guide an emergency medicine agenda for opioid use disorder policy, research, and clinical quality improvement.

Abstract:

Protocol CTN-0016 (“Patient Feedback: A Performance Improvement Study in Outpatient Addiction Treatment”) examined the feasibility of the implementation of a semiautomatic performance improvement system directed at clinicians in addiction treatment facilities called “patient feedback.” The study found that implementation of the program was generally feasible from both a research and a clinical perspective. This article reports on the results of a randomized, controlled trial evaluating the actual efficacy of the patient feedback system. The study involved 118 clinicians working at 20 community-based outpatient substance abuse treatment clinics in the northeastern United States (including some CTPs from the CTN). Ten clinics received 12 weeks of the patient feedback performance improvement intervention, and 10 clinics received no intervention during the 12 weeks. More than 1,500 patients provided anonymous ratings of therapeutic alliance, treatment satisfaction, and drug/alcohol use. There was no evidence of an intervention effect on the primary drug and alcohol use scales. There was also no evidence of an intervention effect on secondary measures of therapeutic alliance. Clinician-rated measures of organizational functioning and job satisfaction also showed no intervention effect. Possible insights from these findings and alternative methods of utilizing feedback reports to enhance clinical outcomes are proposed.

Related protocols: CTN-0016

Abstract:

Modifiable variables such as length of treatment, therapeutic relationship, and treatment environment are associated with successful outcomes for substance and alcohol dependent individuals, and permit and support incorporation of quality improvement (QI) systems in treatment settings to enhance consumer satisfaction. Initiated in 2006 and based on completed CTN protocol CTN-0016 (“Patient Feedback : A Performance Improvement Study in Outpatient Addiction Treatment Settings”), the Patient Feedback (PF) study, is a randomized effectiveness trial implementing a QI system at 20 outpatient, substance abuse treatment programs in Pennsylvania and New York. Patients in group therapy sessions complete anonymous surveys on a weekly basis evaluating treatment satisfaction and therapeutic alliance with their group counselors. Surveys are processed and two types of feedback reports are generated for clinicians to download via a password-protected website. Caseload reports display aggregated feedback from group clients for individual clinicians. Clinic reports display aggregated feedback from all group clients for all participating clinicians. During monthly staff meetings, clinic reports are discussed and strategies are developed to address specific areas where there is opportunity for improvement.

Key findings from staff interviews supporting the acceptability of the PF system include: 1) endorsement of PF and its usefulness in providing constructive feedback, 2) clinic reports are valued as an effective tool to identify areas for improvement, and 3) team meetings are useful in cultivating open discussions regarding significant therapeutic issues. Additional findings reflecting differences in clinic funding, technology utilization, and clinician caseloads will be presented. In conclusion, differences in clinic structure, organization, and available treatment services present unique opportunities for PF study implementation. These differences impact staff involvement, interest, and motivation of the PF system.

Related protocols: CTN-0016

Abstract:

Patient and staff turnover are significant therapeutic and management concerns in substance abuse treatment programs. Some literature suggest a positive correlation between patient satisfaction and successful outcome, while other studies are inconclusive. The treatment environment (location, physical space, and cultural milieu) is also associated with successful outcomes. Although providers often represent that they are satisfied with their jobs this seems to be contradicted by high staff turnover. Initiated in 2006 and based on completed CTN protocol CTN-0016 (“Patient Feedback : A Performance Improvement Study in Outpatient Addiction Treatment Settings”), the Patient Feedback study is a randomized, effectiveness trial, implementing a quality improvement (QI) system at 20 outpatient, substance abuse treatment programs. Patients’ baseline assessments were self-reported on survey instruments capturing ratings of treatment satisfaction. Clinic demographic forms and four self-report surveys assessing job satisfaction captured baseline data from providers along the following domains: 1) quality of director-employee relationships, 2) organizational characteristics, and 3) intrinsic/extrinsic satisfaction. Preliminary analysis from surveys of 679 patients reveals very high treatment satisfaction across patients of various treatment durations, including 227 patients in treatment for 1 month or less. Job satisfaction amongst 76 clinicians will be presented based on scores from the LMX-7 (evaluates interactions between directors and employees), the ORC (18 domains assessing organizational characteristics), and the MSQ (scale rating job satisfaction). Additional findings will be presented on gender, ethnicity, and length-of-treatment on patient and staff satisfaction. In conclusion, these findings have implications for the development and implementation of QI systems to enhance treatment outcomes and the work environment in substance abuse programs.

Related protocols: CTN-0016

Abstract:

Quality Improvement (QI) plays a significant role in accreditation in the healthcare industry, yet there is little research published on the effectiveness of QI in addictions treatment. In 2003, as part of NIDA’s National Drug Abuse Treatment Clinical Trials Network, a pilot study was launched to test the feasibility of a QI system, Patient Feedback (PF) (protocol CTN-0016, “Patient Feedback: A Performance Improvement Study in Outpatient Addiction Treatment”).

The study was conducted at six sites nationwide, with the results establishing the feasibility and acceptability of the PF system at addiction treatment facilities. In 2006, a randomized clinical trial testing the effectiveness of the PF system was initiated and is now being conducted at 35 adult outpatient substance abuse clinics in the New York City and Philadelphia areas. On a weekly basis, patients complete brief, anonymous surveys over a 12 week intervention to rate the quality of therapeutic alliance and treatment satisfaction in their drug counseling group sessions, and report weekly drug and alcohol use, and attendance to group sessions. The surveys are faxed to a data management unit and clinicians obtain real-time feedback of patient ratings through a password protected website. The supervisor and clinicians meet monthly to discuss the feedback and develop methods of enhancing patient ratings in the group sessions. Clinics are randomly assigned to one of two treatment conditions: experimental and delayed (control). Both conditions will have the opportunity to participate in a Sustainability Phase of the study consisting of 52-weeks of continued use of the PF system after completing the 12 week intervention. It is proposed that employment of the PF system will improve patient rates of therapeutic alliance, treatment satisfaction, attendance, and abstinence from drugs and alcohol, particularly among patients who are new to treatment.

Related protocols: CTN-0016

Abstract:

This article reports on the feasibility of implementing a semiautomated performance improvement system — Patient Feedback (PF) — that enables real-time monitoring of patient ratings of therapeutic alliance, treatment satisfaction, and drug/alcohol use in outpatient substance abuse clinics. The Patient Feedback system comprises four main components: feedback surveys, feedback reports, feedback newsletters, and team meetings. The system was implemented in six clinics within the National Institute on Drug Abuse Clinical Trials Network (as part of protocol CTN-0016), ultimately involving a total of 39 clinicians and 6 clinic supervisors. Throughout the course of the study (consisting of five phases: training period, baseline, intervention, postintervention assessment, sustainability), there was an overall collection rate of 75.5% of the clinic patient census. In general, the clinicians in these clinics had very positive treatment satisfaction and alliance ratings throughout the study. However, one clinic had worse drug use scores at baseline than other participating clinics and showed a decrease in self-reported drug use at postintervention.

This project demonstrated that the implementation of the PF system is generally feasible from both a research and a clinical perspective. Moreover, it is notable that clinics continued to use the PF system after the intervention phase, without any additional support from the research infrastructure, suggesting that the implementation of the intervention is sustainable. However, it is unclear whether this sustainability can only be achieved after completing a research study during which research infrastructure was substantially involved. Therefore, before clinical implementation can be recommended, more evaluation of the sustainability of the system and its positive intervention effects needs to be done.

Related protocols: CTN-0016

Abstract:

Patient Feedback (PF) is a web-based quality improvement (QI) system designed to monitor patient ratings of therapeutic alliance and other quality indicators in order to empower clinical staff. Improvement in therapeutic alliance has been shown to be associated with improvement in a variety of clinically important outcomes including attendance, retention, and abstinence. This poster reports on a feasibility study that was designed and implemented at six CTN clinics beginning in April 2004. It involved the administration of a 12-item confidential Patient Feedback Survey completed by clients at the close of group every other week for three months. The survey asked clients to rate their satisfaction with sessions, including whether or not they felt respected by their clinicians and whether they felt the sessions were helpful. The surveys were then converted into feedback reports, which each clinician and clinic could access securely on the Patient Feedback web site.

Overall, patients reported clinicians were performing very well, with relatively low average levels of patient drug and alcohol use. Similarly, alliance and treatment satisfaction ratings were high across all assessments. There was also evidence of improvements in average patient therapeutic alliance scores over time during the intervention phase for those clinicians who had initially poor treatment satisfaction ratings. This study demonstrated that the implementation of a semi-automatic quality improvement system for clinicians in addiction treatment facilities is feasible from both a research and clinical perspective and can be a low-cost, high impact way of conducting performance improvement studies.

Related protocols: CTN-0016