Curated by: Aanchal Gupta, Program Coordinator, NaRCAD

Tags: ADvice, Evaluation, Data
 
In this edition of our AD-vice series, we explore the importance of data collection, program evaluation, and leadership buy-in for AD programs. Gathering and analyzing data not only helps in understanding the challenges faced by clinicians, but also in driving meaningful change by demonstrating the impact of your detailing efforts. Read insights from experts in the field on planning and executing your AD program evaluations.

PRACTICE TOOLS & STRATEGIES FOR DATA COLLECTION

• "For a resource-constrained department, having that one-page Word document that allows detailers to chart their interactions is more than enough data. Charting for just five minutes after a detailing visit about everything that took place becomes a wealth of information. You can also use an Excel document to input data from provider surveys." -Kristefer Stojanovski, PhD, MPH, San Francisco Department of Public Health

 

• "When I’m detailing on one topic, I’m always thinking about future topics by gathering data from providers so that I can better understand their challenges. I ask providers their biggest concerns and I’ve been really surprised at how ready they are to share gaps in resources and information."

            -Jacki Travers, PharmD, Pharmacy Management Consultants

• "For every campaign, we conduct an initial and follow-up visit where we assess providers’ practices. This allows us to see if there has been a change in practice from the initial to the follow-up visit. Additionally, we rate what providers intend to adopt in terms of the key recommendations, supporting tools, and resources. We also collect a large amount of qualitative data because it's also critical to gain a more complete picture of the campaign’s success, especially when reporting on barriers, access, and materials." -Michelle Dresser, MPH, New York City Department of Health and Mental Hygiene

MAKING THE CASE TO LEADERSHIP & FUNDERS

• "It’s important to track a mix of quantitative and qualitative data, and the critical components that should be tracked are the outcomes and the process of detailing. Data is absolutely critical for getting leadership buy-in, especially if it can tell a story." -Kristefer Stojanovski, PhD, MPH, San Francisco Department of Public Health

 

• "It’s always been useful for us to evaluate and publish our results, especially when we approach the government for more funding. While we can’t do every campaign as a research project, I’ve been working on evaluating a campaign we did on diabetes and also a study on the impact of group visits vs. 1:1 detailing." -Harald Langaas, MPharm, MPH, KUPP - RELIS

 

• "It would be a strong statement if someone was able to go to leadership with a story about how providers have changed their practices. Using concrete results and showing leadership that detailing is making a change is extremely helpful for buy-in." -Kristefer Stojanovski, PhD, MPH, San Francisco Department of Public Health

IMPACT & VALUE OF EVALUATION 
​
Key Insights from Melissa Christopher, PharmD & Mark Bounthavong, PharmD, MPH, VA Pharmacy Benefits Management Academic Detailing Services
​

• "Measuring program work builds a case not just for the success of one academic detailing intervention, but for the success of future programs--a case for sustainability. Evaluation measures the quality of a program, analyzing results to look at a program’s impact, and allowing for process improvement adjustments to be made to streamline efforts and strengthen that impact. Evaluation cannot be optional, especially when lives are at stake."

 

• "Leveraging results from well-designed evaluation is essential for academic detailing interventions to illustrate success, share value, and provide stakeholders and community members with a clear ‘Yes!’ in answer to their overarching question: ‘Was the investment worth it?’"

 

• "We encourage other academic detailing programs to prioritize program evaluation as we have at the VHA—no matter the size of your program, if you’re thinking, ‘we can’t afford to do program evaluations,’ we stress that you can’t afford NOT to do them."

For additional information, check out our AD Evaluation Toolkit for guides on planning an evaluation intervention, adaptable surveys, detailing visit tracking sheets, and more!
 
Best,
The NaRCAD Team

Overview: Harald Langaas speaks with NaRCAD about his experiences in co-founding Norway’s first national AD program, KUPP. KUPP, which loosely stands for “Knowledge-Based Updating Visits” in Norwegian, has been actively serving Norwegian General Practitioners (GPs) for several years.

by: Winnie Ho, Program Coordinator

​Tags: Chronic Illness, Detailing Visits, Evaluation, International, Program Management

Winnie: Hi Harald, thank you for joining us all the way from Norway to talk about KUPP, the Norwegian Academic Detailing program! Can you tell us a little bit more about yourself and the work that KUPP does?
 
Harald: I’m a pharmacist by training, with experience in working at hospital pharmacies and as a pharmacy manager in the private sector. My interest has always been in how to better provide independent information about the use of medicines to healthcare professionals to improve the quality of healthcare. I work for one of the four Regional Medicines Informational Centers in Norway, one for each of the four health regions.
 
The Norwegian AD Program is strongly connected with those Centers, so my position is split between the regional center and as Director of KUPP, which operates at the national level. KUPP is a small organization – it’s myself and a consultant in clinical pharmacology handling the administration of AD.

W: I can absolutely see why AD fits a lot of your interests! Let’s talk about how KUPP got its start. You were part of the founding of a brand new AD program. What was that like, and what did you learn?
 
H: My colleague Roar Dyrkorn had visited Australia, met the NaRCAD team in Boston, and was very inspired by AD. He saw it as an opportunity to improve the quality of prescribing in primary care and began lobbying to acquire funding for an AD program.
 
Our first campaign in 2015 was put together within a month or two, focusing on NSAIDs (Non-steroidal anti-inflammatory drugs) for GPs. We were extremely fortunate to have Debra Rowett from Australia, who has been pioneering AD in Adelaide for many years, fly out to train our first detailers because we were still novices to this work. This campaign went quickly – maybe too quickly -- but we were able to implement it well, and we had success with the campaign. We’ve been continuing to detail ever since.

W: You also mentioned that Norway is divided into four health regions. Can you tell us a little bit more about these regions and the communities that you serve in each?
 
H: The four jurisdictions, all funded and overseen by the government, are responsible for hospital services in that region. However, primary care services are overseen at the national level. In each region, KUPP has between 5-10 people that are trained as detailers and conduct visits in addition to working at the Regional Medicines Information Centers or at a Clinical Pharmacology department at a hospital.
 
W: On average, how many clinicians does KUPP work with per year?
 
H: There are about 5,000 total GPs in Norway, and we visit between 1,000-1,200 GPs a year, which is about 20%. We have limited resources while trying to reach as many providers as we can nationally. For the funding we have, we’re happy with our work, but of course, we are ambitious! We want to be able to visit everybody.

W: That’s a pretty sizable population that you reach, especially on limited resources! Can you provide some context about Norway’s healthcare system that help us better understand the context in which KUPP operates?
 
H: In Norway, we have universal healthcare, which is fully funded by the government. It means that our healthcare system is quite homogenous across the country. The GPs that we focus on are mostly self-employed, but fully funded by the government. This does mean that when we make arrangements to schedule detailing visits, we have to contact GPs one at a time.
 
They have no financial incentive to see us, and since we take up their time instead of them seeing a patient, they actually lose money by seeing us. This means that we have to ensure that a visit from us is useful and that it’s a valuable investment towards improving the treatment of their patients. We keep all visits to 30 minutes or less.
 
Another important thing to mention is that patients are designated to their GPs. You can’t shop around for providers, so you have to see the same one each time or apply to change to another one. This means that a GP follows their patients for a long period of time, and have a lot of history with their patients.

W: That’s useful background information to know about. Since patients often stay with the same and only GP, how does that impact a GP’s insight into their patient population?
 
H: Because of the long shared history, it means that when we are talking with clinicians, they know their patient pool very well. Even when a GP has taken over a practice, they will be very knowledgeable about who they are serving.
 
W: I imagine that when detailing on chronic conditions, this is an advantage because a GP and a detailer can follow a patient population over time and offer continuous support! Now, we’ve been able to follow KUPP’s work for a while, especially with a lot of your recent presentations and research. How are things going with research and evaluation lately?
 
H: It’s always been useful for us to evaluate and publish our results, especially when we approach the government for more funding. While we can’t do every campaign as a research project, I’ve been working on evaluating a campaign we did on diabetes and also a study on the impact of group visits vs. 1:1 detailing.

We’re also working on a small qualitative evaluation of our virtual visits at the moment. It’s been exciting to be contacted by other research groups who want to work with us. It’s really inspiring for us to know there are groups who want to learn more about AD because of us, and that we’re being noticed and seen as a good research partner.

​W: It’s always exciting to see where AD travels and how many borders it can cross. We always learn more because the AD community is constantly growing and innovating. As someone who has been at the forefront of establishing AD abroad, what are your hopes for the international AD community at large?
 
H: I would sure hope to see more AD programs emerging in Europe. It would be very helpful to fully connect the AD initiatives that are ongoing around us, to build the same kind of network that North America has had between the United States and Canada.
 
W: We hope to see more programs emerge too! Last question – any final words of advice for detailers and programs?
 
H: The main advice would be to not give up. There will be resistance, and you will run into some troubles, but keep on working. If you believe in the method, and you believe in the work you do, it will pay off. The 1:1 approach is something that separates AD from other tactics, and makes it easier for both clinicians and funding organizations to see you as unique. This work is worth it.

Have thoughts on our DETAILS Blog posts?
You can head on over to our Discussion Forum to continue the conversation!

​Harald Langaas has been the director of a hospital-based medicines information centre (RELIS) in Trondheim, Norway since 2013. Together with colleagues at St. Olavs Hospital he started the first academic detailing program in Norway in 2015, and has been involved in AD work both as administrator and active detailer since then. Since 2018 he has been the director for KUPP – The Norwegian Academic Detailing Program. He is currently working on a PhD based on evaluation of academic detailing.

An interview with Jacki Travers, PharmD, Clinical Academic Detailing Pharmacist, Pharmacy Management Consultants (PMC). PMC operates out of the University of Oklahoma College of Pharmacy and has been providing educational and consultative services for the Oklahoma Medicaid Pharmacy Program for 25 years. PMC began its academic detailing program in 2014 and Jacki was onboarded in 2015 as the first detailer. The academic detailing work is funded primarily by the Health Services Initiative Grant received by Oklahoma Medicaid from the Children's Health Insurance Program. Jacki also serves as an expert training facilitator for the NaRCAD team.

by Anna Morgan, MPH, RN, PMP, NaRCAD Program Manager

Tags: Detailing Visits, Evaluation, ​Primary Care

Anna: Hi Jacki! We can’t wait to hear about your academic detailing work in Oklahoma! Can you tell us a little bit about your program and the clinical topics you detail on?
 
Jacki: Absolutely. Our program is smaller than other AD programs – we have one FTE dedicated to AD and that’s me! Most of our topics are pediatric-focused, based on our funding from the Children’s Health Insurance Program. I've detailed providers on topics including ADHD, appropriate use of atypical antipsychotic medications, treatment of upper respiratory infections, use of psychosocial interventions for mental health needs, implementation of shared decision-making tools, and immunizations. I've ​established a relationship with more than 800 providers and their staff across primary care and specialty care settings in the state of Oklahoma.

​Anna: That's impressive, Jacki. You detail on such unique clinical topics – can you share some challenges related to these topics and how you’ve overcome them?
 
Jacki: We're primarily a rural state, so resources can be hard to access. There's no point in asking a provider to commit to a change and then have them hit this continued wall of unforeseen lack of resources. As a result, I spend a fair bit of my time collecting information that will help providers bridge the implementation gap. I try to connect providers with resources like care management teams, electronic referral platforms, or other providers in their area who are implementing particular services like parent-child interaction therapy, medication therapy management, and applied behavioral analysis. If I'm asking a provider to do something, I want to make sure that they have the tools and the bandwidth to carry it out!
 
Anna: What a great point! You can’t expect a clinician to make changes if they don’t have the resources to do it. Are there additional challenges that have come up in your academic detailing work?
 
Jacki: Most of the challenges I face are the same as those faced by all detailers - access to clinicians, scheduling visits, handling objections, overcoming barriers, gaining commitment to change, and getting access to resources. I overcome these barriers by getting warm handoffs from previously detailed providers, using champions whenever I can, and putting myself in the mind of providers to anticipate what specific resistance there might be so that I can come up with enablers.
 
Sometimes I do a bit of out-of-the-box thinking for the specific challenges and always bring a healthy dose of flexibility. In one case, I dusted off my high school French and used Google translate to ask a French-speaking researcher for permission to modify one of her shared decision-making tools. I could have asked her in English, and it would have been just fine, but I felt like it was going to demonstrate my respect for her work if I did my best to communicate in her preferred language.
 
Whenever possible, I want to try to connect with people in the way that is the most seamless for them. And that's absolutely true for detailing too! I've had providers who want to meet over coffee and muffins at 6:00 AM because it's the only time they have in their day, or providers who need to pump their breast milk during our visit. I roll with it all and make sure I’m meeting providers where they’re at. 

​Anna: Being flexible certainly makes for effective detailing visits. Can you tell us a little more about how you meet providers where they’re at and customize your detailing approach?
 
Jacki: When I’m detailing on one topic, I’m always thinking about future topics by gathering data from providers so that I can better understand their challenges. I ask providers their biggest concerns and I’ve been really surprised at how ready they are to share gaps in resources and information. That’s how our antibiotic topic materials came about. Many providers shared that they often have patients who ask for antibiotic prescriptions and how they have to battle against patient satisfaction surveys that seem to penalize them for not prescribing antibiotics.

The antibiotic detailing materials we created had some scripting to help reduce antibiotic prescribing while also increasing patient satisfaction. The materials included shared decision-making tools with a breakdown of non-antibiotic treatment evidence like humidifiers, honey, saline spray, etc. It's all about identifying why the providers might not be feeling empowered to follow the evidence and then helping them find that empowerment through knowledge, motivation, and resources. 
 
Anna: I love that you continually assess the needs of the providers in your state to inform future work and strengthen relationships. I’m sure with the 800 providers and staff you’ve detailed that you’ve had some success stories– can you share one with us?

Jacki: Of course! I have a great story that shows how important it is to assess needs, really listen, and empathize with clinicians and what they’re going through. I was at a pediatric practice and it had taken me four solid months to get in the door. There had been some pretty strong reluctance even to schedule a visit. Once I did get on the calendar, they canceled and rescheduled multiple times. There was a sense of defensiveness, as though the staff may have been concerned that the visit would be punitive.
 
As I started my needs assessment questions, one of them mentioned that they didn't understand how this topic related to the other meetings they’ve had. I explored that comment a bit more with them and it turned out they had just undergone an extensive audit by the state agency. The whole process left them feeling examined and analyzed.
 
Once I learned about their negative experience, I put their fears to rest, let them know that our time together was completely unrelated to that audit, and that I was there to help them get the best evidence in a digestible format. You could almost see the pressure leaving the room at that point - their body language changed, they were engaged, and they were asking questions and strategizing.
 
By the end of the meeting, the practice manager told me I wouldn’t have any more trouble getting on their calendar. She said, "this was not a waste of my time, and make no mistake, I would tell you if it was!”
 
Anna: That's amazing – what a great story. Before we wrap up, let’s focus on evaluating success. I know you’ve had some remarkable results with your ADHD topic, including a cost savings of more than $226,000 - can you share some of your most recent evaluation data with us?

Jacki: We’re most proud of outcomes from when we worked with providers to reduce the number of unnecessary antibiotic prescriptions, particularly for upper respiratory infections. The providers who were receiving detailing reduced their antibiotic prescribing by more than 17% and they also reduced their use of non-first line agents for upper respiratory infections by more than 16%.
 
However, we wanted to make sure that there weren’t any unintended consequences and that patients weren't having longer or more serious infections when the antibiotics were scaled back. To accomplish this, first, we looked at the prescribing in the previous five years and then one year after the detailing campaign. We identified an oral antibiotic prescription and then looked at the following two weeks after that antibiotic was prescribed to see whether or not there was a hospitalization or an emergency department visit. We found 90% fewer emergency department visits and more than 50% fewer hospitalizations after our detailing campaign. Of the hospitalizations, patients had shortened stays by more than 50%.
 
Even though the patient outcome is, of course, the goal, you can also demonstrate cost savings for your funders and stakeholders. We looked at the dollar amounts for the avoided hospital stays and ER visits and found a total annual cost savings of more than $834,000.

Anna: Wow, that’s impressive! Thank you for taking the time to share your insights and your program’s challenges, successes, and data. You’re an asset to all the communities you detail in and bring so much value to clinicians throughout Oklahoma. We’re also extremely lucky to have you as part of our extended team and larger AD community. We’re looking forward to catching up again soon!
 
Hear more about Jacki’s reflections on the impact of AD here.

Have thoughts on our DETAILS Blog posts?
You can head on over to our Discussion Forum to continue the conversation!

Biography. Jacki joined Pharmacy Management Consultants (PMC) in 2015 and serves as the chair of the academic detailing committee. She has been active in the development and implementation of PMC’s academic detailing program in service to Oklahoma Medicaid providers. Prior to joining PMC, she served in the practice settings of independent, hospital, and clinical pharmacy. She currently develops detailing materials, delivers detailing services, and analyzes program results for multiple topics as part of a statewide plan. Her program efforts focus on bridging the gap between information and application in order to provide quality health care in a fiscally responsible manner.

By: Winnie Ho, Program Coordinator

2020 was a year of many hard-earned lessons. We’re so fortunate to have an AD community that’s committed to sharing best practices, tips, and experiences. This communal knowledge base is what makes us stronger and allows us to all grow together.

​Here’s a collection of the great advice some of our DETAILS Best Practice Blog and Discussion Forum guests have given us this past year:

Tags: Detailing Visits, Evaluation, Program Management

"The most critical thing is to allow enough time for the planning process – ideally, 18 months before you’re looking to launch. This allows you to gather resources, make partnerships internally and externally. If you can reach out to colleagues in the field, learn about what are good mistakes to avoid. It’ll save you a lot of time!"
-Carla Foster, NYC Dept. of Health and Mental Hygiene (NYC DOHMH)

"My best tip would be to create a standard operating procedure (SOP) or some type of guidebook for your visits. Our team developed a SOP which discusses how to conduct a needs assessment, conversational tips, how to weave in key messages, and how to address barriers. Developing the SOP really allowed me to understand the intricacies that need to be addressed before launching the campaign. It works as such a good practice guide, and you can always refer back to it whenever you need it."
-Julie Anne Bell, NYC Dept. of Health and Mental Hygiene (NYC DOHMH)

"One thing I’ve learned about AD is that it’s only as effective as your intervention across an entire system. Any work that I’m doing is irrelevant unless I’m addressing the culture of the entire system. If the front desk staff isn’t on board, or the clinical staff isn’t a believer, or the CEO doesn’t understand – there will be challenges that will be harder to overcome."
-Andrew Suchocki, Clackamas County, Oregon, Medical Director

"Building relationships with key stakeholders has made all the difference. They’ve helped me curate my detailing aids and key messages, and have even allowed me to practice my detailing sessions with them."
-Kelsey Bolton, Gundersen Health System, Wisconsin

"A strong team is an important part of a detailing campaign. Strong teamwork means supporting each other through tough detailing sessions, communicating well, and keeping a positive attitude. During virtual times, turning the camera on during staff meetings can also help keep the team spirit alive!"
-Marlys LeBras, RxFiles Academic Detailing Service in Saskatchewan, Canada

"There are numerous external pressures when it comes to AD, but the most important part is keeping the human aspect in check when reaching out to providers. We can get bogged down into the guide posts, the bench posts, or the numbers – but the COVID-19 era reminds us that it’s all about empathy."
-Tara Hensle, University of Illinois at Chicago/Illinois ADVANCE
​
"You may find it helpful to create an e-Detailing materials packet and see if you can grab some time with providers over a virtual platform. It can be a helpful foot-in-the-door for future in-person detailing!"
-Jess Alward, New Hampshire Division of Public Health, Dept. of Health and Human Services
 
"Lunch time is still the best time for visits. They were the most popular when I did it, and they’re still the most popular now, as my team tells me."
-Terryn Naumann, British Columbia Provincial Academic Detailing (BC PAD) Service

"There’s a lot of listening that happens in AD. You might spend all this time learning about the topic before you meet the providers, but if you take the time to really listen to them, you might learn more than you came with. There is so much to learn from all the incredible people you meet in AD."
-Debra Rowett, Drug and Therapeutics Information Service (DATIS) in South Australia

"The big thing I’ve learned through networking with detailers is to be flexible and be prepared for any situation, especially in the virtual environment. You might have one idea of how your session will go, and it could go in the opposite direction, which is part of the charm of detailing. Also, practice mock detailing with your colleagues!"
-Vishal Kinkhabwala, Michigan Dept. of Health and Human Services
 
"It’s important to have several different ways of presenting information to providers and to use varied approaches to barriers or objections that come up. I typically focus on emotional connection, financial concerns, and the evidence behind the key messages I’m delivering."
-Brandon Mizroch, Louisiana Dept. of Health

"No visit is ‘one-size-fits-all’. You need to consider the provider, their situation, and their environment and decide what will be the best way to deliver the evidence. It’s critical that you’re attentive to the provider you’re detailing and that you continue to focus on the needs assessment at all times."
-Mary Liz Doyle-Tadduni, Alosa Health in Pennsylvania
 
"I was delivering an in-person visit, and the skeptical questions about AD from the provider kept coming. I tried not to be defensive, but I answered everything I could. Eventually, the provider allowed me to get to the topic, and that changed everything! By the end of the visit, the opposition took an about-turn. I gained a professional friend and ally and ended up seeing this person with virtually every topic over the next 20 years. Never write someone off because of some seemingly extreme pushback – you just never know!"
-Loren Regier, Centre for Effective Practice (CEP) and Canadian Academic Detailing Collaboration (CADC)
 
"Confidence is key. You can study and practice everything with your team, but at some point you have to get out there and just do it! You have something valuable to offer and a few opportunities a year to capitalize on that value. A strong relationship can overcome a difference in clinical background or even a rough start. It just takes enough of your effort to show that you’re really there to be of service. Remember, you wouldn’t have been hired in this role if you weren’t qualified!"
-Amanda Kennedy, Vermont Academic Detailing Program

"When addressing stigma, it’s important to note that tough conversations can produce some cognitive dissonance in people. All providers are human. They care about their patients. What helps is not overwhelming them with data, but repeated snippets of information over time to help reinforce the message."
-Elisabeth Mock, Maine Independent Clinical Information Service (MICIS)
 
"Don’t be afraid to ask for a specific behavior change and remember to follow up to make sure that change occurs. The ‘ask’ can be hard for detailers, so I always tell them to frame it as, “based on what you’ve heard today, what is one thing you’d do differently?”
-Tony de Melo, Alosa Health in New England

"We encourage providers to complete a post-visit survey. We ask them to share their level of agreement that they were given new/different information, and they intend to implement practice changes as a result of AD conversations."
-Jacki Travers, Pharmacy Management Consultants in Oklahoma
 
"It’s important to track a mix of quantitative and qualitative data, and the critical components that should be tracked are the outcomes and the process of detailing. Data is absolutely critical for getting leadership buy-in, especially if it can tell a story."
-Kristefer Stojanovski, San Francisco Dept. of Public Health
 
"Once you’ve identified the problem you’re addressing and done the work to understand it, jump in! AD works!"
-Jennifer Pruskowski, University of Pittsburgh Medical Center

Thank you to the AD community for your resilience, compassion, and incredible work through a tumultuous year. We hope the AD community continues to share its pearls of wisdom with us through the new year. We are excited by all the progress made in 2020, and look forward to a brighter 2021 with you all.
 
Best,
The NaRCAD Team

An interview with Alok Kapoor, MD, MSc a cardiovascular investigator at the University of Massachusetts, Worcester about his work on the SUPPORT-AF II Study. 

​By Mike Fischer, MD, MS, NaRCAD Director and written by Winnie Ho, Program Coordinator.

​Tags: Cardiovascular Health, Data, Detailing Visits, Evaluation, Primary Care

Mike: We’re glad to have you join us to talk about your recent work using AD to improve anticoagulant use in patients with atrial fibrillation (AF). Could we start out by getting an understanding of your work and the goal you had set for your SUPPORT AF II intervention?
 
Alok: I am an internist doing cardiovascular outcomes research, and for the last few years I have been really laser-focused on how to fill the gap in anticoagulation use for patients with AF who have an elevated risk for stroke. These patients tend to be older adults with multiple co-morbidities, which presents certain challenges for primary care providers and cardiology specialists. The goal of our particular AD intervention was to provide evidence and patient case scenarios to show some of the common situations where patients go untreated for stroke prevention despite experts’ suggestions that therapy is warranted.

M: The underuse of anticoagulants is more common than we would like, and the impact of that underuse is substantial. What made you decide to utilize AD as a part of the intervention for your study?
 
A: I was thinking about an intervention that would be more than a simple reminder to providers, and thought that perhaps something more customized that would take into consideration the individual provider’s practice and experience with prescribing anticoagulants made more sense. AD was suggested as a potential strategy by our grant sponsor to address those concerns, so I began to read more into it. The SUPPORT AF II intervention is a combination of the audit and feedback reminders given in our original study, SUPPORT AF I, plus the new offering of AD.

M: How did you anticipate that those different components of the SUPPORT AF I and II interventions would work together? Were there any unanticipated surprises during the implementation?
 
A: I believed that the reminders would encourage providers to reach out to their subspecialty colleagues and also remind them to have discussions about anticoagulation with their patients. Then, AD would allow us to get closer to the underlying belief and resistance factors that might be making it more difficult to prescribe in challenging situations, such as a patient with prior falls, bleeds, or on other medications that can make bleeding more common. Some of these barriers included also unfamiliarity with initiating direct oral anticoagulants and guiding patients to coverage information for the cost of newer anticoagulants.
 
There were some specialists who were not necessarily enthusiastic about receiving messages from us. There were also providers during the course of messaging that indicated that they did not think that these messages were helpful for them, so we adapted. However, most people were appreciative or otherwise silent when receiving messages. The harder work was the convincing needed during the AD visit that could help lead to a more impactful intervention.

M: Yes, an impactful intervention is the goal. In your paper, you talked about the importance of patient choice as a factor in anticoagulant use, and this has been consistent with a few other studies of anticoagulation in AF that highlighted similar challenges. Are there any ways that you’ve thought about to adapt an AD intervention to address the importance of patient choice?
 
A: As part of our AD intervention, we gave prescribers a Jeopardy-type menu where you could choose which themes to explore, and one of those was a shared decision making module with resources including an app designed by my co-collaborator David McManus. This app allowed patients to input their unique conditions and circumstances into our risk stratification algorithm. Knowing the patient risk level, the provider would then be shown questions frequently asked by AF patients that would presumably help the provider address certain concerns during the next patient visit.

M: As you reflect on your experience implementing this intervention, were there themes that especially connected with the primary care providers or cardiology specialists who were receiving AD?
 
A: I was responsible for AD with the primary care physicians while my collaborator worked with cardiology specialists. The providers I spoke with seemed to be really drawn to the evidence in the guidelines and often requested support from me in identifying specific evidence that would be helpful as they developed their own improved management strategy around anticoagulants. I think where we could have added something more robust would be to offer providers a way to deliver these messages to their patients and how to do motivational interviewing with patients who are resistant to start a recommended therapy.  

​M: Support AF II is an impressive piece of work that provides many insights. Do you see other topics in cardiovascular care, or other clinical specialties where it might be useful to do similar studies to test AD to increase the use of evidence-based care?
 
A: There are other types of adherence issues in cardiovascular medicine that are potential targets such as blood pressure management. The issue doesn’t seem to be starting the medication, but in continuing to take it on a daily basis. The AD intervention would be done directly to the providers, but there might be value in also directly approaching the patients.
 
M: It would certainly be interesting to understand whether management issues are based on clinical inertia and hesitation in taking the next step, versus barriers in patient adherence itself. Thank you for taking the time to speak with us today, you’ve given us all a lot to think about!

​Have thoughts on our DETAILS Blog posts?
You can head on over to our Discussion Forum to continue the conversation!

Alok Kapoor, MD, MSc is an investigator who has developed several projects related to anticoagulation and conditions requiring anticoagulation. He is one of the former directors of the medical consultation service at Boston Medical Center. In that role, he routinely educated other providers on the need for anticoagulation versus potential harm, particularly for underserved populations. At the University of Massachusetts, he has established a focus on filling the gap in anticoagulation of patients with atrial fibrillation. This started with SUPPORT-AF, an audit and feedback project funded to give providers a snapshot of their AC prescribing rates relative to their peers and to a national benchmark. In SUPPORT-AF II, he expanded the team's previous efforts to include educational outreach in the form of academic detailing. In his subsequent efforts, he have collaborated with informatics experts to understand the potential for electronic health record-based decision support to fill the gap in AC use.

An interview with Kristefer Stojanovski, MPH, PhD(c), Public Health Researcher and Evaluation Specialist, Capacity Building Assistance Program, San Francisco Department of Public Health
​by Anna Morgan, RN, BSN, MPH, NaRCAD Program Manager

​Tags: Data, Evaluation, HIV/AIDS, PrEP, Sexual Health

NaRCAD:  Hi, Kristefer!  Thanks for taking the time to chat with us today.  Can you tell us a bit about your background and the work you’re currently doing as it relates to academic detailing?

​Kristefer: Thanks for having me. I’m a public health researcher and an evaluation specialist with the Capacity Building Assistance Program at San Francisco Department of Public Health.  I serve as a specialist and a technical assistance provider for the West region of the United States.  My work is focused on data and evaluation of academic detailing programs that are working on topics like sexual health, HIV, and PrEP.  My main goal is to help folks measure, or think about, what “success” may look like for an academic detailing program.

NaRCAD: What data do you think is most important for academic detailers to track during their visits when measuring or thinking about success?

Kristefer: I see evaluation data and detailing efforts as one complete package.  Detailers should think about their data at a high level and focus on the information they’re collecting and how that information serves the overall goal of detailing, which is to improve knowledge, attitudes, intentions, and behaviors of providers and clinics. 
​
At the same time, detailers should think about how they can show that they’re achieving that goal.  For example, it’s useful to track how many providers they’ve seen, how much time was spent with providers, what they talked about during the visit, the resources that were provided, how the providers plan to use those resources, if a follow-up visit was scheduled, and the purpose of that follow-up visit.  It’s important to track a mix of quantitative and qualitative data, but the critical components that should be tracked are the outcomes and the process of detailing.  

NaRCAD: What about academic detailing programs?  What data should they collect?

Kristefer:  In a typical detailing program, detailers have a longitudinal, continuous outreach to providers.  There’s an interesting conversation to be had about how we can use that temporal matrix as a tool and strategy for evaluation.  I’m interested in how we can use the work academic detailers are doing – the actual visits themselves - as pieces of data over time. 

​For example, if detailers are collecting some of the rich conversation that they’re having with providers or clinics, it’s fascinating to track those conversations over time and see how the detailing program is changing the knowledge, attitudes, intentions, and behaviors of those providers and clinics.  There’s a lot of work that detailers are already doing that can be easily turned into data sources for both the detailing effort and the evaluation effort.  I like to think about how we can make things as simple as possible.

NaRCAD: Is there a specific platform that makes things simple and is best for collecting data for academic detailers and programs?

​Kristefer:  That certainly becomes a little bit more individualistic when thinking about the needs, challenges, and abilities of each jurisdiction.  Detailing is a conversational effort that is attempting to make concrete behavior change, so it becomes more convoluted when you think about how to track a conversation.  Some jurisdictions might have a place to track conversations in their electronic health record, where others may not. 
 
At the most basic level, detailers can chart their conversations on one-page Word documents.  The detailers I worked with charted their conversations with providers over time and eventually put it in one large PDF that could be easily uploaded into a qualitative data analysis software.  You have to be creative when it comes to tracking this information. 

NaRCAD: How would you recommend that programs with limited resources go about data collection and evaluation?

Kristefer:  A lot of times we say we want to have high-tech solutions when we don’t actually need them.  For a resource constrained department, having that one-page Word document that allows detailers to chart their interactions is more than enough data.  Charting for just five minutes after a detailing visit about everything that took place becomes a wealth of information.  You can also use an Excel document to input data from provider surveys. 
​
There are many low-tech ways to track information and it’s important to be aware of the low hanging fruit in terms of data collection.  You want to be able to easily collect data that serves the detailing efforts, the program, and the evaluation and improvement process.  

NaRCAD: That’s a great way to look at it.  What are some best practices for using data for leadership buy-in?

Kristefer: I can’t stress enough how important data is in getting leadership buy-in.  Data is not only quantitative and qualitative, but also using the information gathered to tell a story.  It would be a strong statement if someone was able to go to leadership with a story about how providers have changed their practices based on the detailing effort.  Using concrete results and showing leadership that detailing is making a change is extremely helpful for buy-in. 
​
Being able to show crazy big outcomes with your data won’t happen, but sharing stories from providers and clinics about how detailing has helped them is quite moving.  I’ve heard some amazing stories during my evaluation.  For example, detailers helped providers at some clinics to provide patients with directly observed therapy for PrEP at the same time that they were providing them with medication-assisted treatment for opioid use disorder.  It’s impressive that detailing at those sites was able to make the clinics think creatively and be able to provide PrEP to these patients.

NaRCAD: That certainly is impressive! What has surprised you the most about the academic detailing data you’ve evaluated?

Kristefer:  I can’t help but to think that pharmaceutical companies spend millions of dollars and resources on this model and they certainly wouldn’t have been doing this for decades if it didn’t work.  We’re almost a little late to the game as public health practitioners, but through my experience evaluating some of this work and reading other evaluations, I’ve been shocked by how much providers truly value detailers.
​
It’s fascinating to see how these health systems and departments are viewed as trusted partners by providers and clinics and how detailing has served as a role to improve that partnership and collaboration.  Providers have often said how crucial this information has been in getting access to Department of Public Health resources they didn’t even know existed, which is pretty sad.  Seeing public health and the medical system working side by side in this kind of way has been breathtaking.    

Biography. 
Kristefer Stojanovski is a PhD Candidate in the Department of Health Behavior & Health Education, School of Public Health at the University of Michigan. Kristefer has been doing community-based mixed methods research since 2010. His research explores the social and structural determinants to sexual health and HIV outcomes among key populations in the U.S. and in Southeastern Europe. Kristefer’s work interrogates how stigma drives HIV risk and infection using complex systems theory, structural equation, agent-based and multilevel modeling. Kristefer also translates his research into policy and decision-making. He is an evaluation specialist with the Capacity Building Assistance program with the San Francisco Department of Public Health.  

Tags: Detailing Visits, Evaluation, Program Management

​The NaRCAD Team is excited to kick off the latest episode in our C.O.r.E. Podcast Series, this time featuring the insights of Program Director Rebecca Edelberg, MPH, from the Boston-based non-profit academic detailing organization Alosa Health, as she shares her experiences managing field programs in clinical outreach education.

​This episode's 15-minute interview with Rebecca hones in on the "how-to's" of strong AD program management, including:

• ​​Hiring the right detailing candidates that have a combination of clinical know-how and communications savvy;
• Ensuring regular supportive check-ins with her team; &
• Planning ahead when thinking about collecting evaluation data.

Tune in here, and sound off on Twitter or in the comments section below with insights, questions for Rebecca, or topics you'd like to see featured on our next podcast. Learn more about Rebecca and Alosa Health below!

Tags: Data, Detailing Visits, Evaluation, Opioid Safety, Substance Use

In 2015, the Department of Veterans Affairs (VA) invested in the National Academic Detailing Service to improve the health of our Veterans to address the call to action for the opioid crisis. Through the Opioid Overdose Education and Naloxone Distribution (OEND) Program, our goals were to reduce harm and risk of life-threatening opioid-related overdose and deaths among Veterans.

Key components of the OEND program include raising awareness about the epidemic, 1:1 academic detailing visits with clinicians to provide education and training regarding opioid overdose prevention, opioid overdose rescue response, and issuing naloxone products. We developed direct-to-consumer marketing and other e-resources, including a video, Introduction to Naloxone for People Taking Prescribed Opioids.

We also created implementation tools, including population management dashboards to aid staff in evaluating risk factors of their patient population and distributing naloxone accordingly. Academic Detailers demonstrated to VA providers these resources to help raise awareness of opioid overdose risk for their patient panel.  

Decision-makers believed that funding this program would yield a good return on investment. As part of the National Academic Detailing Service, it’s our responsibility to collect data and supply decision-makers with evidence on the value and success of our program. In other words, we’re accountable for answering the question, “Is academic detailing worth it?” 

To answer this question, we performed several program evaluations of the National Academic Detailing Service from 2015 to 2017, one of which we just published in the Journal of American Pharmacists Association (JAPhA) (Trends in naloxone prescriptions prescribed after implementation of a National Academic Detailing Service in the Veterans Health Administration: A preliminary analysis.) 
​
The evaluation found that our program improved naloxone distribution rates at a seven times greater increase for Veterans at risk for opioid overdose. These results provided key empirical evidence that VA’s strategy of academic detailing was working. Just as important, these findings also gave decision-makers what they needed—proof that their investment in an area of high risk to Veterans’ health paid off by improving care.

​Sharing program results with the clinicians in this intervention also encouraged these providers to share their own results, many of which were stories of patients returning to the clinic to relate their experiences of using naloxone to reverse an overdose. These stories, along with reversal reports from the field that tracked the outcomes of naloxone kit distribution and subsequent use, also created a tangible “return on investment” for everyone involved.

We encourage other academic detailing programs to prioritize program evaluation as we have at the VHA—no matter the size of your program, if you’re thinking, “we can’t afford to do program evaluations,“ we stress that you can’t afford not to do them.

Measuring program work builds a case not just for the success of one academic detailing intervention, but for the success of future programs—a case for sustainability. Evaluation measures the quality of a program, analyzing results to look at a program’s impact, and allowing for process improvement adjustments to be made to streamline efforts and strengthen that impact. Evaluation cannot be optional, especially when lives are at stake.

​We also recommend that the results from program evaluations are shared with other stakeholders, such as clinicians, in order to encourage and sustain their behavior changes. Leveraging results from well-designed evaluation is essential for academic detailing interventions to illustrate success, share value, and provide stakeholders and community members with a clear “Yes!” in answer to their overarching question: “Was the investment worth it?”

Thanks for taking the time to share the great clinical outreach education work that’s being done by the NYC Department of Health and Mental Hygiene, Michelle! Tell us a bit about yourself and how you got involved in public health, specifically public health detailing.
​
Michelle: Thank you for the opportunity to speak about the Public Health Detailing Program. I have over 20 years of public health experience in both the non-profit and government setting, with the last 12 here at the New York City DOHMH. Throughout my professional career, my specialty has been in healthcare marketing and provider education, emphasizing how providers and consumers can better communicate with each other by tailoring complex messages using health literacy principles.

It’s essential our reps have excellent selling and communications skills, so when they engage providers and get their buy-in, providers are then equipped to get their patients “on board”. One-on-one provider engagement helps them understand how important it is to have a 2-way communication with patients.

How can an outreach representative encourage providers to “get on board” and think about care as a dialogue?
​
Michelle:  Let’s use obesity as an example. With obesity, both providers and patients are frustrated, for different reasons. Providers may be frustrated that patients’ comorbid conditions are being exacerbated or don’t have the same kinds of tools to treat obesity as they do other conditions; patients might feel that providers aren’t using great communication techniques, like motivational interviewing (MI), to help them set goals and take small steps towards the goal.

​If a patient is only told, “You need to lose weight,” which is such a broad and overarching goal, they’ll be frustrated, and frankly, non-adherent. I know I would be.

Encouraging providers to have specific dialogues using a customized approach for each patient is important. This kind of dialogue takes into account patients’ literacy beyond the written and spoken word—it looks at scientific, fundamental, health and cultural literacy, too.

We work on “coaching scripts”, which take the key recommendations and reframes them in order to custom-tailor the conversation for each patient.

One thing that’s unique about public health detailing is that we detail the whole team through one-on-one interactions. ​Evidence shows these types of interactions with providers and staff are more effective at changing behavior; however, sometimes due to the makeup of the practice we must conduct group presentations.  It’s not ideal, but it still allows us to get the messages and materials out there. 

So when an outreach representative goes into an office, they detail...everyone?

Michelle:  If there are 15 people who work in an office, we’re going to detail all 15 of them. It’s a lot! Sometimes, the person who is the champion of a new behavior or workflow isn’t going to be the provider. We see the front desk staff as instrumental; they’re interacting with all of the patients. We work with our teams to ensure even the front desk staff receives the materials and information, rather than seeing them merely as a “gatekeeper” to get to the providers.
​
Sounds like a lot of training goes into preparing for your campaigns, and for thinking about the entire process of effective outreach.  Tell us more about your trainings, and about how you prepare outreach representatives on disease content training, as well as in marketing and communications skills.

Michelle:  On average, our trainings are about 5 days in length and take place the week prior to launching a new campaign. About 40 percent of the training is disease content, so we work with our internal Health Department experts, as well as external experts, where we learn about prevention strategies, treatment strategies, epidemiology and the landscape around the key recommendations chosen based on the evidence of that topic. We need to know the ‘why’ behind the campaign.
​
​Once we have that under our belt, we shift to sessions on how to frame the issue, how to promote the materials, figuring out the “features and benefits” as well as the “barriers and objections” and finally “gaining a commitment”, which are phrases that come from pharmaceutical marketing.  We’re “selling” and promoting public health interactions, so we work on those skills.



​We also do a great deal of role playing, including videotaped analysis of each rep. We look at body language, what communication skills are effective, we do knowledge assessments, quizzes—we make sure our team is well-prepared to go out and detail. We take this seriously—they’re representing the New York Department of Health and Mental Hygiene.

What’s a major barrier your program has faced, and how have you tackled it?

Michelle:  A big challenge, when starting a detailing program, is access. The landscape of healthcare systems in NYC has drastically changed over the past few years. As an example, several years ago, the majority of our Brooklyn territory was almost entirely made of up of small practices where access wasn’t an issue.

What’s changed since then?
​
Michelle:  Now, many of these sites have become part of larger institutions, so there’s corporate buy-in that needs to happen for people to come in and talk to the staff. As I mentioned before, although we try and limit group presentations, this has proven to be an effective strategy when entering into a new relationship. Once they get to know us and recognize the value of the program, they’re engaged in having us come back to conduct 1:1 visits on the follow-up and subsequent campaigns.

How do you know when a campaign is working and becoming successful?
​
Michelle:  Evaluation is always on the top of our priorities, and can be a challenge for any program to evaluate effectiveness.  For every campaign we conduct an initial and follow-up visit where we assess provider practice. 

This allows us to see if there has been a change in practice from the initial to the follow-up visit.  Additionally, we rate what providers intend to adopt in terms of the key recommendations and supporting tools and resources. We also collect a large amount of qualitative data because it's also critical to gaining a more complete picture of the campaign’s success, especially when reporting on barriers, access and materials. 

by Joy Leotsakos, PharmD

Tags: Cardiovascular Health, Detailing Visits, Evaluation, Program Management, Training

Who We Are. The Academic Detailing Service (ADS) of the Atrius Health Clinical Pharmacy Program provides clinically appropriate, evidence-based, cost-effective medication management in a multidisciplinary team setting.  Our Clinical Pharmacy Program includes 15 clinical pharmacists (CPs) serving nineteen Internal Medicine and Family Medicine (IM/FM) ambulatory care practice locations. In the past four years, our program has evolved and transformed through evaluating our impact, absorbing and implementing internal feedback, and collaborating with others in the field, including NaRCAD.

Our Start. As the program manager of our ADS, I’ve seen our service grow and change. When we began our program in 2011, it was as an administrative mandate to meet with all IM/FM prescribers once per fiscal quarter to deliver messages about cost-effective prescribing and clinical quality.  We started by formulating a menu of topics to cover in our ADS work each quarter, including individual clinician prescribing reports reflecting performance on prescribing initiatives from the Pharmacy & Therapeutics Committee, specific questions to survey clinicians on a clinical topic, targeted education for low performers on prescribing initiatives, and various other ‘hot topic’ clinical issues. CPs detailed individual clinicians via formal 1:1 scheduled appointments, and also did so less formally (such as by catching them in the hallways) or in larger groups during department meetings.

Is it Working? We documented our ADS activities by checking off the individual clinicians we detailed each quarter. At that time, there was no formal training for our CPs on how to conduct a detailing meeting. Unfortunately, this method of creating content for visits soon resulted in a large menu of topics so varied that each quarter’s detailing became unwieldy and too broadly focused. And our documentation, while it gave us a general sense of the number of clinicians detailed, did not tell us anything about the quality of this detailing.

Room for Improvement. Our group is fortunate in that our ADS activities have always been accepted and even expected by our IM/FM clinicians. We experienced almost no clinician resistance to our educational meetings. But in 2013, when attending one of NaRCAD’s 2-day Academic Detailing Training sessions, I learned that we could make changes to improve our services, as well as my own skills as a detailer. As a result, we altered the format of our ADS program, choosing to detail clinicians in a 1-1 or small group format of less than 4. We also selected a goal of 90% of clinicians receiving detailing at least once every quarter.

Evaluating Impact. We began evaluating the impact of the changes we’d made to our ADS, specifically choosing to look at its impact on a discretely measurable topic: reducing the unnecessary ordering of an ALT test (alanine transaminase) in patients on the ’statin’ cholesterol-lowering medications.  We were able to demonstrate that our detailing of all IM/FM clinicians led to significant reductions in ALT ordering and meaningful cost avoidance for our organization.

Asking for Feedback. With NaRCAD’s support, we further refined our program in 2014 based upon feedback from an internal focus group. By soliciting honest feedback from the CPs about their detailing experiences, I discovered considerable variation in how they approached the menu of topics provided each quarter and came to understand that the continuous process of visiting with each clinician at their sites often felt stale and repetitive.

New Approach, New Results.  We revised our ADS workflow to tie each round of clinician appointments directly to a specific and single P&T prescribing initiative. Furthermore, we developed a method to tag low performing clinicians for an ‘intense’ ADS visit and higher performers for a ‘touch’ ADS visit. We began this new workflow with an initiative to improve the use of evidence-based beta-blockers in patients with heart failure, a quality measure for the Medicare Pioneer Accountable Care Organization (ACO) project. Using this new approach, clinical pharmacists were able to deliver a fresh and meaningful message to the right prescribers, resulting in a change from 73.6% to 97.8%prescribing of evidence-based beta-blockers in this patient population.

Partnering with NaRCAD for Ongoing Learning. In March 2015, we coordinated with NaRCAD again, and they provided our group of clinical pharmacists with a 2.5 hour workshop to enhance our AD skills. I’d encourage anyone who does this type of educational outreach to make use of this invaluable resource. Of course, our Atrius Health Academic Detailing Service will continue to grow and change as we find additional ways to improve our workflows and messages. I look forward to continued collaboration with NaRCAD and with others in the field, so that we can all keep learning from each other and improve health outcomes through effective academic detailing.

​Bio: Joy Leotsakos is a senior clinical pharmacist and the program manager for the Academic Detailing Service (ADS) of the Atrius Health Clinical Pharmacy Program.   Joy joined Atrius Health in 2007 and became the program manager for the ADS program in 2012. Prior to joining Atrius Health, Joy worked as an assistant professor at Massachusetts College of Pharmacy and Health Sciences University in Boston, MA and provided ambulatory care pharmacy services to the South End Community Health Center also in Boston.  Joy graduated with a Doctor of Pharmacy degree from Virginia Commonwealth University School of Pharmacy and then completed her residency in Ambulatory Care and Community Pharmacy at the University of Florida College of Pharmacy.  Joy is the mother of one son, and enjoys salsa dancing, cycling and running in the summer and skiing in the winter.  You can reach Joy by email at [email protected].