Curated by: Aanchal Gupta, Program Coordinator, NaRCAD

Tags: ADvice, Evaluation, Data
 
In this edition of our AD-vice series, we explore the importance of data collection, program evaluation, and leadership buy-in for AD programs. Gathering and analyzing data not only helps in understanding the challenges faced by clinicians, but also in driving meaningful change by demonstrating the impact of your detailing efforts. Read insights from experts in the field on planning and executing your AD program evaluations.

PRACTICE TOOLS & STRATEGIES FOR DATA COLLECTION

• "For a resource-constrained department, having that one-page Word document that allows detailers to chart their interactions is more than enough data. Charting for just five minutes after a detailing visit about everything that took place becomes a wealth of information. You can also use an Excel document to input data from provider surveys." -Kristefer Stojanovski, PhD, MPH, San Francisco Department of Public Health

 

• "When I’m detailing on one topic, I’m always thinking about future topics by gathering data from providers so that I can better understand their challenges. I ask providers their biggest concerns and I’ve been really surprised at how ready they are to share gaps in resources and information."

            -Jacki Travers, PharmD, Pharmacy Management Consultants

• "For every campaign, we conduct an initial and follow-up visit where we assess providers’ practices. This allows us to see if there has been a change in practice from the initial to the follow-up visit. Additionally, we rate what providers intend to adopt in terms of the key recommendations, supporting tools, and resources. We also collect a large amount of qualitative data because it's also critical to gain a more complete picture of the campaign’s success, especially when reporting on barriers, access, and materials." -Michelle Dresser, MPH, New York City Department of Health and Mental Hygiene

MAKING THE CASE TO LEADERSHIP & FUNDERS

• "It’s important to track a mix of quantitative and qualitative data, and the critical components that should be tracked are the outcomes and the process of detailing. Data is absolutely critical for getting leadership buy-in, especially if it can tell a story." -Kristefer Stojanovski, PhD, MPH, San Francisco Department of Public Health

 

• "It’s always been useful for us to evaluate and publish our results, especially when we approach the government for more funding. While we can’t do every campaign as a research project, I’ve been working on evaluating a campaign we did on diabetes and also a study on the impact of group visits vs. 1:1 detailing." -Harald Langaas, MPharm, MPH, KUPP - RELIS

 

• "It would be a strong statement if someone was able to go to leadership with a story about how providers have changed their practices. Using concrete results and showing leadership that detailing is making a change is extremely helpful for buy-in." -Kristefer Stojanovski, PhD, MPH, San Francisco Department of Public Health

IMPACT & VALUE OF EVALUATION 
​
Key Insights from Melissa Christopher, PharmD & Mark Bounthavong, PharmD, MPH, VA Pharmacy Benefits Management Academic Detailing Services
​

• "Measuring program work builds a case not just for the success of one academic detailing intervention, but for the success of future programs--a case for sustainability. Evaluation measures the quality of a program, analyzing results to look at a program’s impact, and allowing for process improvement adjustments to be made to streamline efforts and strengthen that impact. Evaluation cannot be optional, especially when lives are at stake."

 

• "Leveraging results from well-designed evaluation is essential for academic detailing interventions to illustrate success, share value, and provide stakeholders and community members with a clear ‘Yes!’ in answer to their overarching question: ‘Was the investment worth it?’"

 

• "We encourage other academic detailing programs to prioritize program evaluation as we have at the VHA—no matter the size of your program, if you’re thinking, ‘we can’t afford to do program evaluations,’ we stress that you can’t afford NOT to do them."

For additional information, check out our AD Evaluation Toolkit for guides on planning an evaluation intervention, adaptable surveys, detailing visit tracking sheets, and more!
 
Best,
The NaRCAD Team

Curated By: Aanchal Gupta, Program Coordinator, NaRCAD

Tags: Stigma​, Primary Care, Data

Time and time again we’ve heard about the challenges detailers face when tackling clinician stigma. Detailers have shared comments from clinicians such as, “We don’t take those types of patients” or “I don’t want to be known as the gay doctor.”
​
Addressing stigma and fostering understanding with clinicians can often feel overwhelming for detailers. In this edition of “AD-vice” we shine a light on these issues and share experiences from our community on how they managed stigma during detailing visits. 

Understanding Stigma
​

• “Whenever I address something uncomfortable with a provider, stigma or otherwise, I try to create an environment where we can work collaboratively on the issue and the provider feels as though I am a resource for them. – Jessica Alward, MA, New Hampshire Bureau of Infectious Disease Control

​ 

• “One of the biggest myths about stigma is that some people carry it, and some people don’t. However, stigma is not binary, and we all possess the ability to stigmatize another group that we perceive to be an ‘outsider’ group. Historically, stigma has appeared mainly in the form of social inclusion, with those identified as ‘others’ being treated as societal outcasts.” - NaRCAD

 

• “Imagine that the clinicians or people you detail hold stigma not because they want to stigmatize others, but for some more relatable reasons: they’re impressionable, they’re naïve, they’re vulnerable. In the same way that a clinician wouldn’t expect someone with a substance use disorder to suddenly recover if harshly confronted, we can’t expect the person who stigmatizes to respond to similar tactics.” – Zack Dumont, BSP, ACPR, MSPharm, RxFiles Academic Detailing Service

​

• “Stigma isn’t something that folks are actively choosing, it’s more of what they’ve been taught. Changing that culture of practice is much more difficult compared to asking prescribers to prescribe cholesterol-lowering therapy. There’s very little societal baggage when it comes to improving cholesterol than there is when it comes to destigmatizing addictions or chronic pain.” – Andrew Suchocki, MD, MPH, Clackamas Health Centers

Addressing Stigma through Education and Conversations
​

• “We’re approaching [stigma] with education and lots of conversations, since we’ve found that helping our staff to get a better sense of addiction as a disease is really invaluable to making them more open to [medications for opioid use disorder (MOUD)] and treating people with opioid use disorder (OUD).” – Carol Furlong, LCMHC, MAC, MBA, Elliot Hospital

 

• “When we detail in groups, we focus on small group discussions. One method I use involves flashcards with myths or biases about OUD, and asking two or three of the attendees to discuss that amongst themselves. We have also used a language sheet that guides providers in what to say.” – Elisabeth Fowlie Mock, MD, MPH, Maine Independent Clinical Information Service

 

• “A lot of the older language around OUD identifies with “bad choices” and “bad people”. For example, relapse is associated with a fault of the person. When we are talking about a person with OUD, we are talking about someone with a disease and relapse is a natural course of the disease. When a patient’s blood sugar goes up, we don’t call it a relapse. Just like people with diabetes, we will never cure a person with OUD, but we help them manage [it].” – Don Teater, MD, MPH, Teater Health Solutions

Addressing Stigma through Data and Resources

• “Our team provides statistics that relate to the provider’s specific neighborhoods and specialty, giving them real-time pictures of what’s happening with the patients they see. We know that [cocaine use] is still a difficult topic to bring up, so we help address this with our action kit resources on stigmatic language and counter-top brochures that signal to patients that the provider’s office is a safe place to discuss these issues.” – Carla Foster, MPH, New York City Department of Health and Mental Hygiene

 

• “Know your patient population: Understand who the patients are, the trauma they’ve faced, and the stigma they may endure. Look at the experiences of your team, the clinicians, and the patients you’re working with and try to understand how these different perspectives all influence one another as you develop your resources.” – Trish Rawn, BScPhm, PharmD, Centre for Effective Practice

Our team at NaRCAD is here to learn and support you as we combat stigma and continue to promote inclusivity. Check out our new Healthcare Inclusivity Toolkit for detailers for additional resources. 

Best,

​The NaRCAD Team

By Anna Morgan-Barsamian, MPH, RN, PMP, Senior Manager, Training & Education, NaRCAD
 
An interview with José Peña Bravo, PhD, Health Educator, Florida Department of Health in Duval County.
 
Tags: Detailing Visits, Evidence-Based Medicine, Opioid Safety, Data

Anna: Hi, José! Thanks for joining us on our DETAILS blog today. Your journey that led you to the academic detailing community is unique – can you share that journey with us?

José: Yes! My background is not originally in public health. I’m a biomedical researcher by training, specifically preclinical research using animal models. My dissertation work was on understanding the neurophysiological changes and different brain regions involved in behaviors related to substance use.
​
My intention was to stay in academia and start my own lab, but my plans changed during COVID-19 and the opportunity to work with the public health department in Duval County presented itself. It’s been a learning curve for me to switch my perspective from preclinical research to public health—it’s been an enjoyable journey so far!

Anna: I’m sure your biomedical research skills have a positive impact with clinicians during your detailing visits too, especially when clinicians want to discuss the neurobiology of substance use disorder. Speaking of visits, your detailing work is funded through CDC’s Overdose Data to Action (OD2A) grant, which seeks to prevent overdoses. Can you tell us about what the work for this grant looks like in Florida?

José: The OD2A project is a team effort across three CDC-funded jurisdictions in Florida. The health departments share that funding with various community organizations, and we all work toward linking patients with substance use disorder to treatment, mental health care, and care coordination services.

Our detailing team is closely connected to organizations and resources within our community, and we share these resources with clinicians during our detailing visits. We also have access to aggregate prescription data from our jurisdiction and are continuing to find ways to present and incorporate this data at our visits with clinicians. We share this data and other resources across our three jurisdictions.

Anna: We’ve found that many AD programs have been successful when they are closely connected to community resources. NaRCAD recently hosted a detailing training for OD2A recipients that you attended. What was it like to train with other jurisdictions working on the same project?

José: It was helpful to hear from other jurisdictions because they’ve all approached their AD work differently based on the gaps in care in their own communities. I was able to hear from AD programs in rural areas and the specific challenges that their patients face with lack of access to care (long travel times, stigma, etc.).
​
I also enjoyed practicing my detailing skills in a space where I felt comfortable making mistakes. It’s valuable to try things out and see how they’ll go before going out in the field. I learned a lot at the training and am excited to try out some of my new skills at my next visit. ​

Anna: Hearing from other detailers who are doing this important work with you is so helpful as you continue to think about and grow your own program. What advice would you tell other detailers working on the OD2A project?

José: If you’re just starting out, reach out to community partners and get a sense of what patients with substance use disorder are experiencing and the challenges they’re facing before you start detailing clinicians. You’ll better be able to represent what is happening in the community and the resources that exist when you’ve done your research first!

Anna: That’s terrific advice – a key piece of being an effective detailer is understanding the patient experience for the clinical topic you’re detailing on. So, what’s next for your work and Duval County?

José: We’re currently working with our epidemiology team to collect population-level data and present it concisely. We want to be able to efficiently share this data with clinicians in a way that gets their attention and has them compare it to what they’re experiencing in their clinics to ensure an interactive dialogue during detailing visits.
​
Anna: Using data to tell a story helps clinicians see the impact that they have in preventing overdoses and starts a conversation about organizations and resources that exist within communities for patients with substance use disorder. Thanks for sharing your OD2A work with us, José. We look forward to connecting with you and the other OD2A recipients at our conference in November! 

Have thoughts on our DETAILS Blog posts?
You can head on over to our Discussion Forum to continue the conversation!​

Biography. José’s background is in Neuroscience preclinical research with over 10 years of experience in the field. His graduate work focused on the study of rodent models of substance abuse and the neurophysiological changes associated with controlled-substance experience. He has additional experience as an undergraduate and graduate level lecturer in different biomedical research topics. José recently transitioned to a position as health educator as part of the Overdose Data to Action (OD2A) program at the Florida Department of Health in Duval County. His role involves the implementation of the academic detailing program including outreach to clinics, integrating novel data and information to education materials, and keeping track of different metrics associated with outreach and AD sessions.

This interview features Carla Foster, MPH, who leads the conceptualization, implementation, and evaluation of Public Health Detailing as an Epidemiologist within the Bureau of Alcohol and Drug Use Prevention, Care and Treatment (BADUPCT) at the New York City Department of Health and Mental Hygiene (NYC DOHMH). She is currently activated for the COVID-19 emergency response as Lead Analyst managing the Reporting Unit within the Integrated Data Team of DOHMH’s Incident Command System.
 
By Winnie Ho, Program Coordinator

Tags: Data, Detailing Visits, Evidence-Based Medicine, Health Disparities, Program Management, Stigma, Substance Use, Training

Winnie: Hi Carla! You’ve certainly had a lot on your plate with so many diverse campaigns. Can you walk us through the conceptualization process for your detailing campaigns, and how your team came to choose cocaine use as your current detailing topic?
 
Carla: We can start with some data on this. In 2018, more New Yorkers died from drug overdose than from homicide, suicide, and motor vehicle crashes combined. Cocaine – in both crack and powder forms – has played an increasingly prominent role in this crisis. The mortality rate from overdose deaths involving cocaine more than doubled between 2014 to 2018, amounting to 52% of all drug overdose deaths in NYC. Some of the associated risks are serious - increased exposure risk to fentanyl, cardiovascular disease events and death.

W: That’s stunning data. Especially in the midst of the opioid crisis, it’s important that we don’t lose sight of other substance use issues going on right now. I’d love to learn a little more about the challenges and lessons that your team has learned by detailing on cocaine use.
 
C: First, we have to be aware that fentanyl, a powerful opioid 50 to 100 times stronger than morphine may be found in many substances, including cocaine. We’re very concerned about fentanyl and cocaine because people who use cocaine do not have tolerance to opioids and are at even higher risk for overdose.

It’s also important to address the perception of who is most impacted by high mortality rates. There’s this idea that cocaine use is more prominent in younger populations, but our data show that it’s actually impacting an older population more than many might expect. In particular, residents age 55-84 in the Bronx Borough have experienced the largest increase in cocaine overdose death rates in New York City from 2014 to 2018.

That’s why it’s critical for us to raise awareness in an effort to mitigate misconceptions and stigma around risky use and those who may have a substance use disorder (SUD). In addition to shame, there are still very real potential socioeconomic and legal consequences from disclosing substance use, which can deter folks from even seeking help.
 
We take into account the unjust consequences of policies applied unevenly according to race, and how this impacts implicit biases in terms of which patients are thought to use substances, which types of substances they might use and even more critically, which type of treatment, if any, they are offered. Implicit biases combine with the effects of systemic racism to compound these consequences. It’s important to note that it’s not race that drives poor health outcomes, but racism.

​W: Challenging stigma is one of the most powerful ways that detailing campaigns can combat the damage done by the War on Drugs, because stigma can make the difference of whether or not people receive dignified care. With a campaign so focused on addressing stigma and with a topic this important, how do you prepare your detailers for this task?
 
C: We devote a significant amount of time towards training our detailing reps – a week-long training, 8 hours a day. We spend a large amount of that time talking in detail about stigma as related to cocaine use. It’s critical to us that our detailers are comfortable and knowledgeable when speaking about this topic, because it sets the tone for the providers who then set the tone for their patients.

We ensure that our representatives are prepared to respond to a wide range of questions or comments, because this builds the provider-detailer relationship and enhances the value of the detailing visit. We’ve found during our follow-up visits that this support has led to high provider engagement with the campaign and providers reporting incorporation of the key recommendations into their daily practice, which is the aim of our public health detailing campaigns.

W: How have providers responded when detailed on a topic that carries so much stigma?
 
C: The good news is that we’ve found NYC healthcare providers to not only be receptive to our work on substance use, but they’re eager to partner with us to support their patients once they learn about the severity of the issue.
 
Our team provides statistics that relate to the provider’s specific neighborhoods and specialty, giving them real-time pictures of what’s happening with the patients they see. We know that it’s still a difficult topic to bring up, so we help address this with our action kit resources on stigmatic language and counter-top brochures that signal to patients that the provider’s office is a safe place to discuss these issues.

W: It gives me tremendous hope to hear about that there’s been enthusiastic response from providers. It means that things are changing.
 
Let’s also talk a bit about program sustainability. Your team has worked extensively on campaigns across multiple topics. What have you learned from implementing past campaigns?
 
C: Each public health detailing campaign is different, but we’ve learned some key strategies that support the growth and success of subsequent campaigns:

• ​Sending an introductory letter, about 2 weeks in advance, signed from the Deputy Commissioner of our health department outlining the campaign and encouraging participation, helps gain interest.
• Using a standardized visit workflow that all of our detailing representatives follow to ensure that key components are covered is key. Ours is still flexible enough for the detailers to meet the specific needs of each provider, which really helps with the uptake of the action kit and clinical practice recommendations.
• We meet as a team and evaluate our program weekly, allowing us to continuously adapt and to implement any necessary adjustments.
• Incorporating promotional items inscribed with the campaign’s logo and NYC DOHMH logo. For the cocaine campaign we distributed hand sanitizer and pens to detailing targets. These items serve as highly visible reminders of the campaign mission and have the potential to be incorporated into daily usage by the targeted health care providers.

 
Our overall goal is to do everything we possibly can to improve the health of our fellow New Yorkers. I like to remind our detailers of this James Baldwin quote that informs our public health detailing mission: “Not everything that is faced can be changed, but nothing can be changed until it is faced.”

Have thoughts on our DETAILS Blog posts?
You can head on over to our Discussion Forum to continue the conversation!

Carla Foster, MPH is an Epidemiologist at the New York City Department of Health and Mental Hygiene (NYC DOHMH). Her research focuses on the implementation and evaluation of public health detailing campaigns across New York City with the aim of reducing overdose mortality. Prior to joining the NYC DOHMH, she led development of clinical practice guidelines at the American Urological Association. She received dual Bachelor of Arts degrees in Africana Studies and Neuroscience from Wellesley College. Carla also obtained her Master of Public Health Degree in Epidemiology from Columbia University.

An interview with Alok Kapoor, MD, MSc a cardiovascular investigator at the University of Massachusetts, Worcester about his work on the SUPPORT-AF II Study. 

​By Mike Fischer, MD, MS, NaRCAD Director and written by Winnie Ho, Program Coordinator.

​Tags: Cardiovascular Health, Data, Detailing Visits, Evaluation, Primary Care

Mike: We’re glad to have you join us to talk about your recent work using AD to improve anticoagulant use in patients with atrial fibrillation (AF). Could we start out by getting an understanding of your work and the goal you had set for your SUPPORT AF II intervention?
 
Alok: I am an internist doing cardiovascular outcomes research, and for the last few years I have been really laser-focused on how to fill the gap in anticoagulation use for patients with AF who have an elevated risk for stroke. These patients tend to be older adults with multiple co-morbidities, which presents certain challenges for primary care providers and cardiology specialists. The goal of our particular AD intervention was to provide evidence and patient case scenarios to show some of the common situations where patients go untreated for stroke prevention despite experts’ suggestions that therapy is warranted.

M: The underuse of anticoagulants is more common than we would like, and the impact of that underuse is substantial. What made you decide to utilize AD as a part of the intervention for your study?
 
A: I was thinking about an intervention that would be more than a simple reminder to providers, and thought that perhaps something more customized that would take into consideration the individual provider’s practice and experience with prescribing anticoagulants made more sense. AD was suggested as a potential strategy by our grant sponsor to address those concerns, so I began to read more into it. The SUPPORT AF II intervention is a combination of the audit and feedback reminders given in our original study, SUPPORT AF I, plus the new offering of AD.

M: How did you anticipate that those different components of the SUPPORT AF I and II interventions would work together? Were there any unanticipated surprises during the implementation?
 
A: I believed that the reminders would encourage providers to reach out to their subspecialty colleagues and also remind them to have discussions about anticoagulation with their patients. Then, AD would allow us to get closer to the underlying belief and resistance factors that might be making it more difficult to prescribe in challenging situations, such as a patient with prior falls, bleeds, or on other medications that can make bleeding more common. Some of these barriers included also unfamiliarity with initiating direct oral anticoagulants and guiding patients to coverage information for the cost of newer anticoagulants.
 
There were some specialists who were not necessarily enthusiastic about receiving messages from us. There were also providers during the course of messaging that indicated that they did not think that these messages were helpful for them, so we adapted. However, most people were appreciative or otherwise silent when receiving messages. The harder work was the convincing needed during the AD visit that could help lead to a more impactful intervention.

M: Yes, an impactful intervention is the goal. In your paper, you talked about the importance of patient choice as a factor in anticoagulant use, and this has been consistent with a few other studies of anticoagulation in AF that highlighted similar challenges. Are there any ways that you’ve thought about to adapt an AD intervention to address the importance of patient choice?
 
A: As part of our AD intervention, we gave prescribers a Jeopardy-type menu where you could choose which themes to explore, and one of those was a shared decision making module with resources including an app designed by my co-collaborator David McManus. This app allowed patients to input their unique conditions and circumstances into our risk stratification algorithm. Knowing the patient risk level, the provider would then be shown questions frequently asked by AF patients that would presumably help the provider address certain concerns during the next patient visit.

M: As you reflect on your experience implementing this intervention, were there themes that especially connected with the primary care providers or cardiology specialists who were receiving AD?
 
A: I was responsible for AD with the primary care physicians while my collaborator worked with cardiology specialists. The providers I spoke with seemed to be really drawn to the evidence in the guidelines and often requested support from me in identifying specific evidence that would be helpful as they developed their own improved management strategy around anticoagulants. I think where we could have added something more robust would be to offer providers a way to deliver these messages to their patients and how to do motivational interviewing with patients who are resistant to start a recommended therapy.  

​M: Support AF II is an impressive piece of work that provides many insights. Do you see other topics in cardiovascular care, or other clinical specialties where it might be useful to do similar studies to test AD to increase the use of evidence-based care?
 
A: There are other types of adherence issues in cardiovascular medicine that are potential targets such as blood pressure management. The issue doesn’t seem to be starting the medication, but in continuing to take it on a daily basis. The AD intervention would be done directly to the providers, but there might be value in also directly approaching the patients.
 
M: It would certainly be interesting to understand whether management issues are based on clinical inertia and hesitation in taking the next step, versus barriers in patient adherence itself. Thank you for taking the time to speak with us today, you’ve given us all a lot to think about!

​Have thoughts on our DETAILS Blog posts?
You can head on over to our Discussion Forum to continue the conversation!

Alok Kapoor, MD, MSc is an investigator who has developed several projects related to anticoagulation and conditions requiring anticoagulation. He is one of the former directors of the medical consultation service at Boston Medical Center. In that role, he routinely educated other providers on the need for anticoagulation versus potential harm, particularly for underserved populations. At the University of Massachusetts, he has established a focus on filling the gap in anticoagulation of patients with atrial fibrillation. This started with SUPPORT-AF, an audit and feedback project funded to give providers a snapshot of their AC prescribing rates relative to their peers and to a national benchmark. In SUPPORT-AF II, he expanded the team's previous efforts to include educational outreach in the form of academic detailing. In his subsequent efforts, he have collaborated with informatics experts to understand the potential for electronic health record-based decision support to fill the gap in AC use.

An interview with Debra Rowett, BPharm, Adv Prac Pharm, FPS, the Director of the Drug and Therapeutics Information Service (DATIS) in Adelaide, Australia.

Debra joins Winnie Ho, NaRCAD Program Coordinator in a two-part conversation about a 30-year career of pioneering academic detailing in Australia and reflects on the past, the present, and the future of the field. In Part One, we introduce you to DATIS and academic detailing in the Australian context.  Stay tuned for Part Two!

Tags: Data, Evidence-Based Medicine, International, Program Management

Winnie: We’re so glad to have a chance to chat with you about your long career in AD, Debra!

Debra: This is a great part of the job, talking to other people in the AD community. It really is a great privilege to be working with academic detailing organizations in different countries. I’ve loved getting to meet and learn from so many different people.

W: I would agree! I’m sure you have some great stories of what it’s like to work internationally in this field.

D: One of the things that it has  really highlighted is the nuance of language. At a training workshop in the U.S early in my career, I was saying how we would meet with doctors in their "surgery", and how important it was to meet them in their surgeries close to where they make decisions. The workshop participants really politely said to me, but isn’t surgery a really bad time to detail? Oh! Surgery – I meant their office.

W: We really would have quite a different communication model if we had detailers visiting providers mid-surgical operation! This does gives us a good starting point into discussing how Australian AD is unique. Besides the context of the word "surgery", what else would our non-Australian colleagues need to know about Australia to understand the context of what you do with the Drug and Therapeutics Information Service (DATIS)?

D: I have been involved with DATIS since its formation in 1991. I was a clinical pharmacist at the Repatriation General Hospital, a teaching hospital, when Jerry Avorn’s paper was published. His work on how AD could influence clinical decision making really resonated with us in Adelaide. One of the big things to know about Australia is that we have a National Medicines Policy, which aims to improve positive health outcomes for all Australians through access to and quality use of medicines. DATIS was one of the first programs funded through the Quality Use of Medicines initiative, and in 1998 NPS MedicineWise (formerly the National Prescribing Service) was funded.

W: Australia is enormous - it must be a challenge to cover. What is the geographical coverage of your AD program?

D: South Australia has a population of about 1.4 million people and a vast geographic reach – the furthest of my AD visits is about 800km (500 miles) away from where we are! We work to provide AD to over 85% of all family physicians in South Australia, so about 1,300 General Practitioners (GPs) each year. We provide AD services to aged care, primary care and hospital providers. We also work in partnership with NPS MedicineWise who have implemented AD at the national level.  

W: That’s certainly an enormous coverage zone, especially for those core 12 people! How has this work manifested in South Australia?

D: At the heart of our program is service, and we build our program to emphasize that. There are three aspects of DATIS: service delivery of detailing visits, training of detailers, and research. We have a core team of 12 people who carry this work out alongside our colleagues who join us for various projects. Because of our multiple different contracts, the clinicians we provide services to can see us for multiple reasons in a year.

Between visits, providers will ring us with clinical questions about therapeutic issues that have arisen in their practice. We have also developed interprofessional communication training to support interprofessional practice with a focus on pharmacists and physicians. Our AD programs usually seek to address a therapeutic area or clinical issue however a recent AD program we developed with our hospital pharmacy colleagues was to support pharmacist preceptors implement a performance outcome framework based on entrustable professional activities for interns and undergraduate pharmacy students.

W: It's incredible that DATIS has such a focus on this three-pronged approach, because it continues to help push our understanding of best practices in AD through implementation, study, and training others to carry on the work. Can you tell us a little more about the foundation that DATIS is built upon?

D: Behavior change theory and implementation science has informed our work from the outset in 1991, including the development of the training program which was designed in collaboration with psychologists and experienced GP medical educators. Social marketing frameworks, an adult learning approach, the concept of cognitive biases, clinical reasoning all recognize the many interacting and complex influences on behavior. We try to learn from these and apply to the design and implementation of AD.

We also use pharmacopidemiology methods to understand evidence to practice gaps and for evaluation. As AD evolves and changes, something I really emphasize is staying true to the principles of AD – this is a rigorous process.

W: We’re seeing innovations all over the world and across so many clinical topics. Are there any unique innovations that you feel differentiates Australia AD from other AD programs?

D: One innovation that we're exploring is applying the principles of AD to patient behavior change interventions.  As part of person-centered care, it is important for patients to understand their medicines, and to be involved and empowered in shared decision making. We haven’t called this work AD, but have applied the principles of AD in this research.

Complex clinical decisions need to be made each and every day by providers, and it's a privilege to be able to bring providers the best available evidence through academic detailing services,  part of the power of AD is the adaptability and personalization to providers along the continuum of care. We are seeing the world of healthcare change, and we have so much to learn as it does.
​
(Part One of Two)

Have thoughts on our DETAILS Blog posts?
You can head on over to our Discussion Forum to continue the conversation!

​Debra Rowett,BPharm, Adv Prac Pharm, FPS, has led an academic detailing team for over 20 years and is a member of the team which designed, developed and delivered the “Best Practice in Educational Visiting” training for academic detailers in Australia. Debra has worked closely with NPS Medicinewise since their inception and has provided consultancies to other national and international academic detailing programmes. Debra is an experienced academic detailer with expertise in designing, developing, training, implementing and evaluating academic detailing programmes. Debra has served as the President of the Australian Pharmacy Council and is currently the Vice President of the Council of Pharmacy Schools. Debra has worked extensively in the area of quality use of medicines, inter-professional practice, policy and health workforce development in Australia. Debra is a member of the national Drug Utilisation Sub-Committee of the Australian Pharmaceutical Benefits Advisory Committee (PBAC).

An interview with Kristefer Stojanovski, MPH, PhD(c), Public Health Researcher and Evaluation Specialist, Capacity Building Assistance Program, San Francisco Department of Public Health
​by Anna Morgan, RN, BSN, MPH, NaRCAD Program Manager

​Tags: Data, Evaluation, HIV/AIDS, PrEP, Sexual Health

NaRCAD:  Hi, Kristefer!  Thanks for taking the time to chat with us today.  Can you tell us a bit about your background and the work you’re currently doing as it relates to academic detailing?

​Kristefer: Thanks for having me. I’m a public health researcher and an evaluation specialist with the Capacity Building Assistance Program at San Francisco Department of Public Health.  I serve as a specialist and a technical assistance provider for the West region of the United States.  My work is focused on data and evaluation of academic detailing programs that are working on topics like sexual health, HIV, and PrEP.  My main goal is to help folks measure, or think about, what “success” may look like for an academic detailing program.

NaRCAD: What data do you think is most important for academic detailers to track during their visits when measuring or thinking about success?

Kristefer: I see evaluation data and detailing efforts as one complete package.  Detailers should think about their data at a high level and focus on the information they’re collecting and how that information serves the overall goal of detailing, which is to improve knowledge, attitudes, intentions, and behaviors of providers and clinics. 
​
At the same time, detailers should think about how they can show that they’re achieving that goal.  For example, it’s useful to track how many providers they’ve seen, how much time was spent with providers, what they talked about during the visit, the resources that were provided, how the providers plan to use those resources, if a follow-up visit was scheduled, and the purpose of that follow-up visit.  It’s important to track a mix of quantitative and qualitative data, but the critical components that should be tracked are the outcomes and the process of detailing.  

NaRCAD: What about academic detailing programs?  What data should they collect?

Kristefer:  In a typical detailing program, detailers have a longitudinal, continuous outreach to providers.  There’s an interesting conversation to be had about how we can use that temporal matrix as a tool and strategy for evaluation.  I’m interested in how we can use the work academic detailers are doing – the actual visits themselves - as pieces of data over time. 

​For example, if detailers are collecting some of the rich conversation that they’re having with providers or clinics, it’s fascinating to track those conversations over time and see how the detailing program is changing the knowledge, attitudes, intentions, and behaviors of those providers and clinics.  There’s a lot of work that detailers are already doing that can be easily turned into data sources for both the detailing effort and the evaluation effort.  I like to think about how we can make things as simple as possible.

NaRCAD: Is there a specific platform that makes things simple and is best for collecting data for academic detailers and programs?

​Kristefer:  That certainly becomes a little bit more individualistic when thinking about the needs, challenges, and abilities of each jurisdiction.  Detailing is a conversational effort that is attempting to make concrete behavior change, so it becomes more convoluted when you think about how to track a conversation.  Some jurisdictions might have a place to track conversations in their electronic health record, where others may not. 
 
At the most basic level, detailers can chart their conversations on one-page Word documents.  The detailers I worked with charted their conversations with providers over time and eventually put it in one large PDF that could be easily uploaded into a qualitative data analysis software.  You have to be creative when it comes to tracking this information. 

NaRCAD: How would you recommend that programs with limited resources go about data collection and evaluation?

Kristefer:  A lot of times we say we want to have high-tech solutions when we don’t actually need them.  For a resource constrained department, having that one-page Word document that allows detailers to chart their interactions is more than enough data.  Charting for just five minutes after a detailing visit about everything that took place becomes a wealth of information.  You can also use an Excel document to input data from provider surveys. 
​
There are many low-tech ways to track information and it’s important to be aware of the low hanging fruit in terms of data collection.  You want to be able to easily collect data that serves the detailing efforts, the program, and the evaluation and improvement process.  

NaRCAD: That’s a great way to look at it.  What are some best practices for using data for leadership buy-in?

Kristefer: I can’t stress enough how important data is in getting leadership buy-in.  Data is not only quantitative and qualitative, but also using the information gathered to tell a story.  It would be a strong statement if someone was able to go to leadership with a story about how providers have changed their practices based on the detailing effort.  Using concrete results and showing leadership that detailing is making a change is extremely helpful for buy-in. 
​
Being able to show crazy big outcomes with your data won’t happen, but sharing stories from providers and clinics about how detailing has helped them is quite moving.  I’ve heard some amazing stories during my evaluation.  For example, detailers helped providers at some clinics to provide patients with directly observed therapy for PrEP at the same time that they were providing them with medication-assisted treatment for opioid use disorder.  It’s impressive that detailing at those sites was able to make the clinics think creatively and be able to provide PrEP to these patients.

NaRCAD: That certainly is impressive! What has surprised you the most about the academic detailing data you’ve evaluated?

Kristefer:  I can’t help but to think that pharmaceutical companies spend millions of dollars and resources on this model and they certainly wouldn’t have been doing this for decades if it didn’t work.  We’re almost a little late to the game as public health practitioners, but through my experience evaluating some of this work and reading other evaluations, I’ve been shocked by how much providers truly value detailers.
​
It’s fascinating to see how these health systems and departments are viewed as trusted partners by providers and clinics and how detailing has served as a role to improve that partnership and collaboration.  Providers have often said how crucial this information has been in getting access to Department of Public Health resources they didn’t even know existed, which is pretty sad.  Seeing public health and the medical system working side by side in this kind of way has been breathtaking.    

Biography. 
Kristefer Stojanovski is a PhD Candidate in the Department of Health Behavior & Health Education, School of Public Health at the University of Michigan. Kristefer has been doing community-based mixed methods research since 2010. His research explores the social and structural determinants to sexual health and HIV outcomes among key populations in the U.S. and in Southeastern Europe. Kristefer’s work interrogates how stigma drives HIV risk and infection using complex systems theory, structural equation, agent-based and multilevel modeling. Kristefer also translates his research into policy and decision-making. He is an evaluation specialist with the Capacity Building Assistance program with the San Francisco Department of Public Health.  

Tags: Data, Detailing Visits, Evaluation, Opioid Safety, Substance Use

In 2015, the Department of Veterans Affairs (VA) invested in the National Academic Detailing Service to improve the health of our Veterans to address the call to action for the opioid crisis. Through the Opioid Overdose Education and Naloxone Distribution (OEND) Program, our goals were to reduce harm and risk of life-threatening opioid-related overdose and deaths among Veterans.

Key components of the OEND program include raising awareness about the epidemic, 1:1 academic detailing visits with clinicians to provide education and training regarding opioid overdose prevention, opioid overdose rescue response, and issuing naloxone products. We developed direct-to-consumer marketing and other e-resources, including a video, Introduction to Naloxone for People Taking Prescribed Opioids.

We also created implementation tools, including population management dashboards to aid staff in evaluating risk factors of their patient population and distributing naloxone accordingly. Academic Detailers demonstrated to VA providers these resources to help raise awareness of opioid overdose risk for their patient panel.  

Decision-makers believed that funding this program would yield a good return on investment. As part of the National Academic Detailing Service, it’s our responsibility to collect data and supply decision-makers with evidence on the value and success of our program. In other words, we’re accountable for answering the question, “Is academic detailing worth it?” 

To answer this question, we performed several program evaluations of the National Academic Detailing Service from 2015 to 2017, one of which we just published in the Journal of American Pharmacists Association (JAPhA) (Trends in naloxone prescriptions prescribed after implementation of a National Academic Detailing Service in the Veterans Health Administration: A preliminary analysis.) 
​
The evaluation found that our program improved naloxone distribution rates at a seven times greater increase for Veterans at risk for opioid overdose. These results provided key empirical evidence that VA’s strategy of academic detailing was working. Just as important, these findings also gave decision-makers what they needed—proof that their investment in an area of high risk to Veterans’ health paid off by improving care.

​Sharing program results with the clinicians in this intervention also encouraged these providers to share their own results, many of which were stories of patients returning to the clinic to relate their experiences of using naloxone to reverse an overdose. These stories, along with reversal reports from the field that tracked the outcomes of naloxone kit distribution and subsequent use, also created a tangible “return on investment” for everyone involved.

We encourage other academic detailing programs to prioritize program evaluation as we have at the VHA—no matter the size of your program, if you’re thinking, “we can’t afford to do program evaluations,“ we stress that you can’t afford not to do them.

Measuring program work builds a case not just for the success of one academic detailing intervention, but for the success of future programs—a case for sustainability. Evaluation measures the quality of a program, analyzing results to look at a program’s impact, and allowing for process improvement adjustments to be made to streamline efforts and strengthen that impact. Evaluation cannot be optional, especially when lives are at stake.

​We also recommend that the results from program evaluations are shared with other stakeholders, such as clinicians, in order to encourage and sustain their behavior changes. Leveraging results from well-designed evaluation is essential for academic detailing interventions to illustrate success, share value, and provide stakeholders and community members with a clear “Yes!” in answer to their overarching question: “Was the investment worth it?”

The exponential increase in computing power and data storage capacity, coupled with the sharp decrease in data processing costs, have made possible an era of ‘big data’ that is transforming many aspects of life and commerce.  In health care, this evolution is enabling access to information that was impossible to imagine in the era when I first began this work when most prescriptions and test orders were still written on little scraps of paper. As applied to academic detailing, this growing capacity opens up a veritable armory of double-edged swords.

Knowing what doctors are ordering: This information has always been an important advantage of the pharmaceutical industry, which routinely buys the detailed prescribing records of specific physicians from intermediaries such as IMS, who in turn purchase these records from nearly every pharmacy in the nation. In the hands of an agile pharmaceutical representative, knowing a doctor’s drug preferences can be a powerful tool in shaping a promotional message tailored to that person. 

Many of us in have had mixed views about the use of such data. On the one hand, it can make possible a more precisely focused discussion about optimal ordering of tests and treatments that is based on a given practitioner’s actual behavior. On the other hand, the approach comes with several risks. One is the concern that clinicians may feel “spied upon” – a problem that doesn’t seem to come up much in industry visits. This in turn can divert the conversation to discussion of “Why are you visiting me?” rather than a conversation about optimal patient care. 

​Data feedback to clinicians also degenerates frequently into he said-she said debates that often come down to “My patients are different!”  We welcome feedback from academic detailing programs on how this use of data has worked (or hasn’t) in their own settings.

​What patients are (or aren’t) doing:  The computerization of dispensing records opened an era of hitherto-difficult research on patient adherence to their medication regimens, with generally depressing findings of low adherence.  The full import of this rampant epidemic of non-compliance is still not well understood by most prescribers. The rapid growth in mobile and wearable technologies that capture physical activity and other lifestyle choices provides another potential source of data on patient behavior, but the best applications of this information are even less well understood. 

In principle, academic detailing programs embedded in health care organizations can provide feedback to clinicians on how much or how little their patients are taking medications as directed or complying with other medical advice, and – more important – what to do about it. Is this a useful component of the educational encounter?  Again, we would welcome hearing how this use of big data to provide feedback on adherence or patient behavior does or doesn’t fit into the work of ongoing academic detailing programs.
 
In the coming years, we will see even greater access to terabytes of data on who is ordering what, and what patients are doing with their prescriptions and other treatments. Used well, this technological revolution can provide added power to programs designed to improve that clinical decision making.