The Journal of Science and Business Research


 

Decreasing Adverse Drug Events in Montana

Frontier Critical Access Hospitals Through

Health Information Technology

 

Kipman Smith, Principal Investigator

Allen, S; Carter, J; Cook, A; Eichler, M; Espeland, D; Foster, C; Hoas, H

Langbehn, C; Meissner, B; Meredith, M; Mitchell, S; Pfaff, C; Pleshar, S

Robertson, G; Shreffler-Grant, J; Wilcox, K

 

 

EXECUTIVE SUMMARY

 

 Purpose

 

The Montana Rural Healthcare Performance Improvement Network initiated a study, funded through the Agency for Health Research and Quality (AHRQ) Transforming Healthcare Through Information Technology Planning Grant, to assess the opportunities to decrease adverse drug events (ADEs) in frontier Montana critical access hospitals (CAHs), and to identify appropriate, cost-effective health information technology (HIT) solutions to the medication use challenges of these facilities.

 

 

Scope

 

The study involved 39 of Montana’s 45 Critical Access Hospitals which are located in 39 of 56 Montana counties. 

 

 

Methods

 

A baseline assessment of current medication management systems and the ADE reporting culture was conducted in December 2004.  Based on the results of this assessment, a six month pilot project to test the impact of a web-based ADE reporting system on the numbers and types of ADEs reported, patient impacts and organization benefits of HIT implementation was conducted in 2006.

 

 

Results

 

Implementation of the web-based ADE reporting system appears to have reduced voluntary ADE reporting in IT sites to a level which is counter-productive to improving patient safety.  However, the system appears to have improved the overall reporting of prescription, transcription and dispensing process errors, clarified the impact of ADEs on patients, and reduced fear of censure or discipline as a barrier to reporting ADEs.  Low patient and ADE volumes are barriers to achieving sufficient economies of scale to implement HIT in many Montana CAHs.  The study identified financial, technical and human resource factors which appear to have contributed to the successful installation of a web-based ADE reporting system in four of the five IT pilot sites. 

 

 

Key Words

 

Adverse drug events (ADEs)

Critical access hospitals (CAHs)

Health Information Technology (HIT)

 


 

PURPOSE

 

 

The AHRQ Transforming Healthcare Through Information Technology Planning Grant, in support of AHRQ’s long-term health information technology (HIT) initiative goals to improve clinical and administrative outcomes, funded the two broad goals of this project:

 

 

Specific objectives for successfully completing the HIT Planning Project were clarified with the AHRQ Program Official in November 2004:

 

 

The completion of the baseline assessment in December 2004 with evaluation of the initial study results the first quarter of 2005 led to the identification of a need to test proposed HIT solutions prior to broad-based implementation.  Specific objectives related to a one-year, no cost project extension, approved by AHRQ in September 2005, to conduct a pilot project were identified at that time:

 

 

 

 

SCOPE

 

Background              

           

Broadwater Health Center, on behalf of members of the Montana Rural Healthcare Performance Improvement Network (PIN), applied for the AHRQ Transforming Healthcare Through Information Technology Planning Grant in order to assess opportunities to decrease adverse drug events (ADEs) and medication errors in frontier Montana Critical Access Hospitals (CAHs). The planning process was designed to identify appropriate, cost-effective health information technology (HIT) solutions to the medication use challenges of low-volume, resource-deficient frontier hospitals.

 

The intended outcomes of the HIT planning process were to identify IT solutions which PIN members might pursue collaboratively or individually to provide clinical and organizational benefits to frontier CAHs, the priority populations they serve and the community-wide health infrastructure. It was believed that successful completion of the planning process would lead to application for and securing of an AHRQ Information Technology Implementation Grant for the following year for interested PIN participants. Although no application was submitted for an Implementation Grant, a one-year, no cost extension was received to conduct a pilot project on medication event reporting which is the primary focus of this report.

 

Context

 

The Montana Rural Healthcare Performance Improvement Network (PIN), founded in February 2001, is a collaborative effort of Montana’s Critical Access Hospitals (CAHs).  The PIN was originally formed to assist Montana CAHs to achieve compliance with the CAH Medicare Conditions of Participation related to quality assurance and performance improvement.  Geographic isolation, low patient volumes and scarce financial and human resources make Montana a unique and difficult place to implement traditional information technology solutions which are often costly to install and maintain. With little or no HIT support services readily available to most Montana CAHs, significant HIT capital investments could potentially sit idle or be significantly underused. Participation in the PIN provides a very necessary and valuable collaborative support mechanism for Montana’s CAHs to jointly pursue creative and cost-effective HIT patient safety innovations.

 

Settings

 

Critical Access Hospital is a federal designation of small, rural hospitals created as part of the Balanced Budget Act of 1999. CAHs must be located in a federally-designated rural area.  They are limited to a maximum of 25 licensed beds and must maintain an average annual length of stay of 96 hours or less.  The forty-five CAH members of the Montana PIN are extremely diverse, ranging in capacity of from 2 to 25 licensed beds.  PIN membership represents more than two-thirds of Montana’s licensed and certified hospitals with members scattered across the state, having a presence in 39 of Montana’s 56 counties.   Several CAHs have a single mid-level provider, providing only emergency and basic family practice medicine.   Others have 15 or more licensed practitioners on their medical staff and provide obstetrical and/or surgical services.  At the time this study was initiated, Montana had 39 licensed CAHs.

 

Montana’s CAHs remain vulnerable to the negative influences arising from the geographic isolation, low population density and persistent negative net operating margins which characterize them.  Early discussions by the PIN HIT Planning Committee centered on how to address the wide disparity of information technology, human and financial resources among the CAHs as they relate to efforts to leverage HIT in medication use challenges.

 

 

Participants

           

Ninety-five percent of the 39 PIN members in 2004 volunteered to complete the baseline assessment portion of this study in December 2004.  Thirteen members, representing each geographic region of the state, went on to participate as volunteers in the AHRQ Pilot Project as an IT test site, a non-IT test site or as a control group member.  CAHs participating in the pilot project included:

 

Clark Fork Valley Hospital                                       Pondera Medical Center

Fallon Medical Complex                                           Powell County Medical Center

Garfield County Medical Center                                Roosevelt Medical Center

McCone County Medical Center                              St. John’s Lutheran Hospital

Mountainview Medical Center                                  Stillwater Community Hospital

North Valley Hospital                                               Wheatland Memorial Medical Center

Pioneer Medical Center

 

The PIN’s HIT Planning Committee was composed of three CEO members of the PIN Advisory Board, three at-large CAH CEO members, one at-large Performance Improvement Director, and two members of the PIN staff.

 

An eight member Expert Panel provided technical assistance.  The panel is composed of knowledgeable professionals with relevant expertise from the University of Montana Skaggs School of Pharmacy, Montana State University College of Nursing, the National Rural Bioethics Project in Missoula, the Montana Board of Pharmacy and the PIN Clinical Consultant.

 

 

Incidence and Prevalence                

 

There were no published data available concerning the incidence and prevalence of ADEs in Montana’s critical access hospitals prior to beginning this project.

 

 

METHODS

 

Study Design

 

This study was developed and led by the members of the PIN’s HIT Planning Committee with technical assistance and expertise provided by members of the Expert Panel.   In the original study design, opportunities to improve patient safety and outcomes using HIT were to be identified from a baseline assessment of current practice in PIN-member CAHs.  The identification of HIT sources and funding were to follow, with broad implementation during the AHRQ Implementation Grant phase of the project.   However, based on the outcomes of the baseline assessment, the Planning Committee and Expert Panel members agreed that a pilot project to test the impact of HIT on adverse drug event (ADE) identification and reporting processes in a small sample of frontier CAHs should be conducted prior to a broad attempt to implement an HIT solution in CAHs across the state.  The designs of both the baseline assessment and pilot project will be discussed in this section.

 

A semi-quantitative baseline self-assessment tool was developed and distributed to all 39 PIN-member CAHs in Montana in December 2004.  Baseline assessment information was collected from the Chief Executive Officer (CEO), Medical Director, Pharmacy Director, Risk Manager/Patient Safety Officer, Director of Nursing/Nurse Administrator and two to three nursing staff members from each participating facility.  Unique assessments were developed for each professional role to elicit specific expertise and experience.  The assessment asked respondents to provide information about staffing and resources, medication management and practices, the facility’s patient safety culture, institutional pharmacy compliance, and HIT utilization and plans for investment.  Specific information was collected about the types of ADEs occurring, their impact on patients, staff perceptions about whether medication events were being appropriately identified and reported, and whether or not barriers to effective ADE reporting were perceived to exist.  Completed assessments were returned to the PIN for data aggregation and analysis. 

 

The findings from the baseline assessment were evaluated by the Planning Committee and Expert Panel members in a joint meeting in February 2005.  The results were assessed by comparing reported facility practices with the requirements of Institutional Pharmacy law in Montana and the recommendations of the literature from nationally recognized sources, including the IOM’s Six Aims for the 21st Century Health Care System, IOM’s 2004 Keeping Patients Safe Report, the National Coordinating Council on Medication Error Reporting and Prevention (NCCMERP), and the FDA’s MedWatch.

 

Based on the results of the baseline assessment, the Planning Committee and Expert Panel worked together to develop a pilot project to trial an HIT intervention.   The selected intervention was the SafeRatings, Inc. web-based ADE reporting and data aggregation software program.  The pilot project was structured to capture retrospective and concurrent ADE data from users of the program and those who did not use the program, as well as reassess the ADE reporting culture in the participating facilities at the end of the project.  The pilot project was conducted from January 1, 2006 through June 30, 2006.

 

To test the impact of the HIT intervention, a standardized paper medication event reporting tool was developed and provided to the software vendor for replication in a web-based, electronic format.  Nine CAHs from among those completing the baseline assessment volunteered to continue the AHRQ study as pilot project participants.  Based on current ability to technically implement the web-based ADE reporting system and a commitment to report all ADEs using the IT system during the pilot, each of the facilities was assigned to either the IT site test group or the non-IT site test group.  A standardized set of ADE policies, terms, definitions and reporting requirements was developed which all participants agreed to implement in their facility prior to the project launch on January 1, 2006.  All participants were required to attend training in the use of the policies, definitions, electronic and paper tools and to train their facility staff in their use prior to the project launch.

 

At the end of the pilot period, retrospective ADE data was collected from a control group.  Control group participants were selected based on matching facility size and the complexity of services they provide with test site participants.  These participants also needed the ability to provide the required data rapidly.  Pilot site ADE rates were adjusted using the control group’s three-quarter mean ADE rate to minimize the influence of variables unrelated to the project’s goals.  The pilot concluded with a semi-quantitative survey which participating facilities completed to evaluate the pilot and their progress in improving ADE identification and reporting.

 

 

Data Sources and Collection

 

A comprehensive literature review was conducted as described above and formed the basis of the development of the baseline assessment and other project policies and tools.  The baseline assessment was conducted in December 2004.

 

Pilot project sites submitted retrospective medication events data for the first quarter of 2005 and concurrent medication events data for the first and second quarters of 2006.  They collected and submitted facility ADE reporting culture reassessment data in July 2006, immediately after the end of the test period.  Each site participated in a qualitative telephone survey July 2006 conducted to collect anecdotal information about facility ADE rate changes.  Each site participated in an anonymous semi-quantitative project evaluation survey conducted electronically in August 2006.

 

Control group sites submitted retrospective medication events data for the first quarter 2005, first and second quarters 2006 and a facility reassessment of ADE reporting culture in August 2006.

 

           

Interventions and Interactions with Pilot Sites

 

The PIN HIT Planning Committee announced the AHRQ study and initial baseline assessment activity by letter to PIN members in October 2004.  Assessments were mailed to facilities December 2004.  The Principle Investigator, PIN staff and the PIN Clinical Consultant were available by telephone as needed to answer questions about the study and completing the assessment.

 

After receiving approval for the no-cost, one year pilot project extension from AHRQ, PIN staff announced the pilot project and solicited volunteer participants in September 2005. Volunteers submitted written letters of agreement to the PIN and designated in-house project leads at this time.  The Principal Investigator consulted the CEO of each volunteer site to determine assignment to either the IT or non-IT test group.

 

IT sites joined the HIT Planning Committee and Expert Panel in evaluating three software products and selecting the ADE software vendor for the project throughout October 2005.

 

The pilot project facility leads attended a one day, face-to-face training provided by PIN staff and the PIN Clinical Consultant in November 2005.  The training included background information about ADE identification and reporting, review and use of the set of standardized policies, procedures, tools, and the reporting and evaluation forms to be used during project.  Instruction was provided for in-house staff training to be given by the facility project leads.  SafeRatings Inc., the selected software vendor, provided IT sites training in the use of their web-based ADE reporting software.

 

PIN staff followed up by telephone with the project leads in December 2005 to verify in-house training progress, software installation and access, and to identify and address potential barriers to a timely project launch on January 1, 2006.  Identified barriers included difficulties accessing the web-based reporting form, gaining internal IT support and establishing the non-punitive reporting policy.  PIN staff worked with the software vendor and facility leaders to resolve these issues prior to the project launch date.

 

PIN staff hosted monthly conference calls from December 2005 through June 2006 with the project leads.  Individual telephone and onsite support was provided by PIN staff upon request throughout the pilot.

 

ADE data aggregation and report generation training were provided in April 2006.  The software vendor provided IT sites training in the use of the software for these purposes.  PIN staff provided non-IT sites training in the use of Excel for data aggregation and report generation.

 

PIN staff contacted potential volunteers for the control group by telephone in August 2006. Control group participants were selected to reflect the size and range of services of the test site participants.   The control group included one site already using a web-based ADE reporting program and three sites that were using a paper reporting system.

 

 

Measures

 

Two sets of performance measures related to identifying patient and system-level outcomes were defined as part of the baseline assessment and pilot project activities.

 

The baseline assessment measure set was targeted to evaluate the initial study goals of assessing the opportunities to decrease adverse drug events (ADEs) and to identify appropriate, cost-effective HIT solutions to CAH medication use challenges.  The selected measures were:

 

 

The measure set was refined for the pilot project and targeted to evaluate the impact of HIT on the medication event reporting processes in participating CAHs.  The selected measures were:

 

 

 

Limitations

 

The outcomes of the AHRQ study and subsequent pilot project were limited by grant and pilot project restrictions, and the low-volume, resource-deficient characteristics of Montana CAHs.

 

The requirement for matching funds in the AHRQ Implementation Grant phase of the project immediately discouraged many CAH CEOs from participating in the pilot project.  Ongoing negative net operating margins, limited cash resources and the limitations of the rural and priority population communities and counties in which these facilities operate make raising matching funds extremely difficult.

 

Many of the CEOs whose facilities completed the baseline assessment did not believe they had the necessary technical and human resource skills available to complete the pilot project.  Some of these were not able to provide the necessary computers and internet access for staff at their work sites for web-based ADE reporting.  Other CEOs felt the required staff time and skill to fully participate in a pilot project were not available.  Still others determined that the project did not fit their immediate priorities for improvement. 

 

Data analysis was limited by the low number of participants, the low inpatient volumes typical of Montana CAHs, and the corresponding low numbers of ADEs identified and reported.  These factors limited the ability of the project analysis to demonstrate the statistical significance of its findings although measurable change was detected.  One participant, having no inpatients for four of the six pilot months, did not identify any ADEs during the pilot period.  The constraints of the AHRQ one-year no cost extension precluded extending the pilot to collect additional data.  The analysis was also limited by the inability of several participants to retrieve the number of billable drug doses for ADE rate calculations because of low financial system IT penetration in the facility.

 

 

 

RESULTS

 

 

Principal Findings of the Baseline Assessment

 

The purpose of the baseline assessment was to assess opportunities to decrease ADEs in Montana CAHs, and to identify appropriate, low cost HIT solutions to meet the medication use challenges of these facilities.  The response rate to the December 2004 baseline assessment survey was 95% (37 of 39) of the PIN-member CAHs.  The 282 individual responses were aggregated in the analysis.  Each responding facility submitted data collected from the Chief Executive Officer (CEO), Medical Director, Pharmacy Director, Risk Manager/Patient Safety Officer (RM), Director of Nursing/Nurse Administrator (DON) and two or three staff nurses. 

 

1.  Pharmacy Volume and Staff:     The volume of medications dispensed varies substantially among Montana CAHs and there is a significant shortage of qualified pharmacists available to meet the pharmaceutical demands of these rural facilities.

 

 

2.  Medication Management & Institutional Pharmacy:     Substantial variance from the expectations of Institutional Pharmacy and best-practice medication management processes were identified.  These variances might be linked to the shortage of qualified pharmacy staff described above and the low operating budgets of these facilities.

 

 

In addition, only 3% of the respondents had all of the recommended and/or required policies identified below implemented at the time of the baseline survey:

 

 

Policy in Place

Patient med history includes OTC, vitamins, herbals, illicit drugs

75 %

Standardized inpatient medication administration time

73 %

Reconcile all ED medications dispensed

70 %

Medication recall process

65 %

Standardized prescribing practices

62 %

Pre-printed medication order sheets of any kind

59 %

High risk drug protocols

49 %

High risk medications with identified dosing limits

46 %

Acceptable abbreviations list approved by medical staff

41 %

Look alike, sound alike drugs management

24 %

 

 

3.  Patient Safety Culture:     The essential processes for improving the patient safety culture as it relates to medication use are only beginning to be implemented in Montana CAHs.  Establishing a non-punitive ADE reporting environment has been especially challenging, and leadership and staff perspectives on this issue varied greatly. 

 

 

4.  HIT Utilization and Plans:     Nearly 60% of the responding CAHs have invested in some form of HIT to support their medication use systems.  A much smaller percentage, 11-16%, have budgeted for higher technology medication system support investments over the next couple of years.  None of the CAHs had implemented computerized provider order entry (CPOE) at the time of the survey.

 

                  

 

The majority of CAH medical directors (MDs) were not using decision support tools for safe medication use in their daily practice in 2004.  Only 25% were using PDAs for drug decision support and only 19% routinely used the internet to access drug information.  Sixty-five percent of the responding MDs had no experience using CPOE.

 

 

Baseline Assessment Discussions

 

The PIN HIT Planning Committee and project’s Expert Panel met to review the results of the baseline assessment in February 2005.  The baseline assessment revealed numerous opportunities for improvement in the medication use systems in Montana critical access hospitals. 

 

The range of demand for pharmaceuticals and the availability of trained pharmacy staff to provide them vary substantially among Montana CAHs.  Overall, a significant shortage of in-house licensed pharmacy staff and technicians exists.  This shortage creates tremendous barriers in leadership, expertise and manpower to implementing nationally-accepted medication use patient safety practices and the HIT tools that might support them.   At the same time, the written policies and procedures needed to support safe medication use practices and a patient safety culture focused on preventing ADEs were frequently either not available or not yet fully implemented.  With only 42% of surveyed CAHs reporting that they had a standardized definition of an ADE and process for reporting that event, a statewide attempt to leverage HIT to reduce ADEs is premature.  A solid foundation for the reporting of ADEs and understanding ADE trends is an essential first step to ensuring targeted HIT acquisitions are effective in supporting specific aspects of the medication use system in our rural health care settings.

 

 

Conclusions

 

A pilot project testing the efficacy of one targeted HIT intervention should be conducted prior to a broad-based HIT implementation project.  In view of the low penetration of and planning for medication system HIT use among the surveyed CAHs, the meager pharmacy expertise and leadership available within facilities, and the absence of essential supporting processes for understanding ADEs, the pilot project was focused on increasing the quality of ADE reporting.   Our expectation was that ADE reporting would increase as we focused on it[1].  Our approach to improving ADE reporting quality was to standardize ADE definitions and reporting processes, reduce barriers to reporting, and to increase the understanding of the patient impacts of ADEs[2].   We believed that this approach would lead to an enhanced ability of the CAHs to target appropriate, effective HIT interventions that ultimately would lead to the reduction of ADEs within Montana CAHs.

 

 

Principal Findings of the Pilot Project

 

Eight of nine CAHs initiating the six-month pilot project were able to complete it.  One IT test site was not able to submit ADE data due to a lack of in-house technical support and the staff skills needed to implement a web-based software system.  One non-IT test site experienced such low hospital inpatient volume that there were no reported hospital-based ADEs during the entire six months of the pilot.  All nine participants completed the culture reassessment.  The ADE data in sections one and two below are based on the data submitted from the seven participants submitting ADE data.

 

1.  Little data was available about the types of ADEs occurring in Montana CAHs prior to this project.  However, as a result of standardizing the definitions for types of events, the types of events submitted by reporters as ‘unknown’ was reduced 50% over the course of the pilot.  The data from a total of 193 ADEs reported in the first 2 quarters of 2006 suggest:

 

2.  The data suggest that the type of reporting system in place impacts the number and types of events identified and reported.  The graph below shows the change in the adjusted ADE rate during the pilot for all participants, the IT sites and the non-IT sites.

 

In contrast to our expectations[3]:

 

 

IT sites reported fewer ADEs than non-IT sites in all three quarters of the pilot.  The adjusted ADE rate fell in three of the four IT sites and two of three non-IT sites completing the project.  Although these decreases are substantial, we were unable to demonstrate this change was significant when tested statistically due to the small number of pilot project participants.   Additional data were collected in a qualitative telephone survey at the end of the project in an attempt to identify potential factors contributing to decreased ADE reporting.   Potential contributors identified include:

 

 

The one IT site experiencing an increase in adjusted ADE rate attributed the increase to improved identification and reporting of near miss events.  The one non-IT site experiencing an increase in adjusted ADE rate attributed the increase to implementation of the standardized reporting process.

 

The data also suggest that the type of reporting system in place impacts the identification of the type of events identified and reported.  To determine type of event, participants identified one or more process errors involved in each ADE reported.  IT sites identified 226 process errors associated with reported ADEs during the first two quarters of 2006.  Non-IT sites identified 143 process errors associated with reported ADEs in the same period.

 

As shown in the graph below, IT sites identified more prescription, transcription and dispensing events than the non-IT sites during the pilot period.  At the same time, IT sites categorized substantially more event types as “unknown” than non-IT sites during the same period.  This might be linked to the limited ability to follow up on event reports when using an anonymous reporting system.  Both IT sites and non-IT sites identified drug administration process errors as the primary contributor to the ADEs occurring in their facilities.  Transcription and MAR documentation errors were also major contributors to the reported ADEs for both test sites.

 

 

 

3.   The data suggest that HIT impacts the event reporting process in Montana CAHs in several ways:

 

 

As discussed in section 2 above, the adjusted rate of ADEs reported by IT sites fell to a mean adjusted rate 0 events/100 PD.  This suggests IT sites may be under-reporting their ADEs.  The IT sites also reported more prescription, transcription and dispensing errors than the non-IT sites during the first two quarters of 2006, but left more process errors categorized as ‘unknown’ compared to non-IT sites.  This suggests that IT may be helpful in clarifying certain process errors but may also be a barrier to following up with the individual reporting the event when additional process information might be useful for understanding it.

 

Very little data about the clinical impact on patients from ADEs were collected by participants prior to the project.  As part of the pilot project, participants were required to use the NCCMERP scale to categorize the patient impacts of reported ADEs.  The relative frequency of patient impacts identified by both groups is shown in the graph below.

Although the majority of patient impacts for both groups were identified in the less severe categories (A-C), IT sites appear to have done a better job of reporting moderately severe (D-F) and severe (G-I) patient impacts than the non-IT sites.  At the same time, as with the identification of type of event above, the IT sites categorized more event impacts as ‘unknown’ than the non-IT sites.  This could be related to the barriers to follow up investigation of events when using an anonymous reporting system. 

 

 

The change in staff perception that all ADEs are identified and reported was assessed.  At the end of the pilot, participating staff perceived that 62% of all ADEs in their facilities were being reported, a small gain from 58 % in December 2004.  The IT and non-IT sites perceived 59% and 66% of their ADEs were being reported respectively.  Over the course of the project, aggregate nurse administrator perception increased 9%, and risk manager perception increased 15%.  Staff nurse perception remained unchanged at 57%. 

 

Potential barriers to ADE reporting were originally identified in the 2004 baseline assessment based on respondent perception and reassessed at the end of the project.  The implementation of the anonymous, web-based ADE reporting system essentially eliminated fear as a barrier to reporting ADEs.  Both IT and non-IT sites experienced reductions in the event reporting process itself as being a barrier.  This finding seems somewhat inconsistent with the substantial reduction in events reported.  The differences between IT site and non-IT site barrier perception at the end of the project are shown in the graph below.

The implementation of the web-based reporting system appears to have had the following impacts on the perception of barriers in IT sites:

 

 

In contrast, the clarification of the paper-based reporting system appears to have had these impacts on non-IT site perceived barriers:

 

 

4.  The organizational benefits of implementing a web-based ADE reporting system were assessed in the pilot project by evaluating improved medication system work processes and improved ADE data collection and reporting in light of the financial impact of maintaining a subscription web-based ADE reporting system.

 

Improvement in medication system work processes and ADE data collection and reporting were assessed using the results of the pilot project evaluation completed by test site project leaders in July 2006.  Seven of nine project leaders completed the project evaluation.  The majority of respondents reported improvement in the following medication use processes:

 

 

Seventy-one percent of the participants indicated they foresee using the medication use policies and procedures developed for the pilot project on a permanent basis.  One IT site plans to continue using the web-based ADE reporting system and one non-IT site plans to implement the system.

 

The financial impact of the system was estimated by calculating an approximate average cost per reported ADE during the pilot for a one year subscription to a web-based reporting system comparable to the system used in the pilot.  The baseline cost per ADE to use a paper reporting system and Excel software for ADE data aggregation was presumed to be $0 per event for both IT and non-IT participants.  All of the projects’ participants had Excel on their in-house computers prior to the project.  For the web-based system, based on the ADE report rates during the pilot and the vendor’s quoted price to continue the subscription, the average cost to use the system was approximately $48 per reported ADE.

 

 

Pilot Project Discussions

 

Data describing the types of ADEs occurring in Montana CAHs were not consistently available prior to the pilot project.  This suggests the processes needed to collect this information either were not in place or not well implemented prior to the project.  Both IT and non-IT pilot project participants made substantial improvement in the identification of the types of events occurring within their facilities.  Data from the pilot suggest that the majority of ADEs in Montana CAHs occur in medication administration and the documentation of administration on the patient’s medication administration record. 

 

IT sites reported fewer ADEs than non-IT sites at baseline and throughout the two quarters of the pilot project.  Overall, IT site event reporting dropped 100%, and the non-IT site adjusted ADE rate fell 80% over the course of the pilot.  One non-IT site took action during the pilot to reduce the number of omitted dose events in the facility.  However, the project was not able to demonstrate that the overall reduction in reported ADEs was statistically significant for either IT or non-IT sites due to the small number of pilot participants.  The event data also suggest that HIT appears to have helped IT sites do a better job of reporting prescription, transcription and dispensing events than non-IT sites.  One IT site qualitatively reported substantially improved capture of near miss events within the facility. 

 

Confidence in the ADE reporting process improved in both IT and non-IT participating sites.  The IT and non-IT nurse administrator and risk manager perceptions that ADEs are reported increased 9-15%.  Interestingly, staff nurse perception that events are reported was unchanged.  IT sites were able to essentially eliminate fear as a barrier to ADE reporting; this barrier persisted among non-IT sites.  In contrast to these perceptions, the IT site adjusted reporting rate was essentially zero at the end of the project, suggesting the possibility of voluntary under-reporting of ADEs in the IT sites.  Should this be the case, it would be counter-productive to efforts to improve patient safety in medication use.  IT sites identified persistent software problems, staff reluctance to use software for near miss reporting, staff reluctance to enter multiple reports for omitted doses for the same patient, slowness of staff to gain skill using IT reporting due to infrequent use, software barriers to investigative follow up with individuals submitting reports when using anonymous reporting systems, and a persistent staff perception that “paper is just easier” as barriers to using a web-based system for ADE reporting.

 

Both IT and non-IT sites improved their identification of the impact of ADEs on patients.

It is unclear whether the improvement is related to IT implementation, or the provision of the NCCMERP scale for assessing patient impact.  In the first two quarters of 2006, IT sites identified more moderate and severe patient impacts than non-IT sites.  This may be due to the higher level acuity patients served by the IT sites as well as the generally greater IT infrastructure within them, enabling detection of patient impacts through clinical monitoring systems.  This patient impact information will be valuable to all of the participants in their ongoing efforts to improve patient care quality and safety.

 

The organizational benefits of implementing HIT in Montana CAHs are less obvious.  IT and non-IT sites alike made improvement in their ADE reporting, data analysis and communication processes.  Whereas the potential cost to human life of an ADE should not be underestimated, at an average incremental cost of $48 per ADE to implement a web-based ADE reporting system, the inability of rural CAHs to achieve the economies of scale needed to make HIT affordable presents a clear and ongoing barrier to HIT implementation in this area of the country. 

 

 

Conclusions

 

Implementation of the web-based ADE reporting system in a small number of Montana CAHs appears to have reduced voluntary ADE reporting in IT sites to a level which is counter-productive for improving patient safety.  At the same time, it appears to have improved the reporting of prescription, transcription and dispensing errors and reduced fear of censure or discipline as a barrier to reporting in the facilities testing it.   Financial, technical and human resource factors which appear to have contributed to successful IT implementation include:

 

 

A longer period of study is needed to observe whether or not voluntary event reporting in IT sites would return to expected rates as the internal culture continues to adjust to a new reporting system.  Additional study is also needed to evaluate the impact of an anonymous IT ADE reporting system on individual reporting accountability.  Reduced reporting accountability may be a contributor to the reduced ADE reporting observed in the IT sites. 

 

 

SIGNIFICANCE

 

In contrast to the CAHs in which IT penetration was established prior to the pilot project, those facilities participating in the project as non-IT sites or who were not able to successfully complete the project as an IT site are only beginning to acquire the basic IT infrastructure needed to support their patient health care processes.  In some instances, this includes acquiring high-speed internet access in the community and/or facility as well as providing internet access at work sites to enable routine web-based technology integration with established work processes.  None of the non-IT sites have employed IT professionals on staff.  The absence of in-house, ongoing hardware and software support is a barrier to further penetration of HIT in these facilities.  IT has not been integrated into the daily work lives of these clinical care staff.  Low patient volumes, infrequent use and lack of familiarity with HIT systems appear to have encouraged staff resistance to IT use in some rural Montana CAHs.

 

These facilities tend to be located in the more remote areas of the state, serving the smallest and poorest populations in Montana.  It will be extremely difficult for these hospitals to generate the staff, capital and cash resources needed to acquire and maintain HIT in their facilities.  Given their low patient volumes, continued use of a non-IT, paper ADE reporting system in a non-punitive, patient safety focused environment may be appropriate for them.  The significant incremental costs of implementing a stand alone web-based ADE reporting system may not be justifiable to their patients, governing boards and communities.

 

 

IMPLICATIONS

 

Substantial ongoing financial, technical and human resource assistance will be needed for frontier Montana CAHs to successfully implement HIT in support of providing safer, higher quality patient care, and to reap the organizational benefits of doing so.  This support will be essential in their ability to successfully meet President Bush’s electronic medical record challenge by 2010.
 

 

Grant Reports

 

Smith, K.  Montana Rural Healthcare Performance Improvement Network. 2006. Interim and Final Progress Reports to AHRQ.  Grant No: 1 P20 HS014995-01

 

 

Resources

 

Web-based tool for Reporting Montana CAH ADEs.  SafeRatings, Inc. Copyright 2005. Gilbert, AZ.  www.saferating.com

 

[1]    Cook, A., Hoas, H. (2004).  You have to see errors to fix them.  Modern Healthcare.  December 6, p21.

[2]    O’Leary, D. (2000).  Statement of the Joint Commission on Accreditation of Healthcare Organizations before

        the Committee on Health, Education, Labor and Pensions, U.S. Senate and the Subcommittee on Labor, Health

        and Human Services, and Education of the Senate Committee on Appropriations.  Feb 22, p1.

[3]    Ibid.

 

 

Acknowledgment of Agency Support

 

This study was supported by the Agency for Health Research and Quality Transforming Healthcare Quality Through Information Technology Planning Grant, Grant Award Number  1 P20 HS014995-01.   Specific copyright protections may apply.

 

 

       

Copyright © 2006, SBR Publications.  All rights reserved.

 

Back to the Index of this Issue

 

Contact us

 

The Journal of Science and Business Research

 

The Online Journal

for the Publication of Contemporary Articles

Concerning Scientific and Business Research and Practice

 

www.sbrjournal.net