The Complete Package: Translating Detailing Efforts into Measurable Outcomes

An interview with Kristefer Stojanovski, MPH, PhD(c), Public Health Researcher and Evaluation Specialist, Capacity Building Assistance Program, San Francisco Department of Public Health
​by Anna Morgan, RN, BSN, MPH, NaRCAD Program Manager

​Tags: Data, Evaluation, HIV/AIDS, PrEP, Sexual Health

NaRCAD:  Hi, Kristefer!  Thanks for taking the time to chat with us today.  Can you tell us a bit about your background and the work you’re currently doing as it relates to academic detailing?

​Kristefer: Thanks for having me. I’m a public health researcher and an evaluation specialist with the Capacity Building Assistance Program at San Francisco Department of Public Health.  I serve as a specialist and a technical assistance provider for the West region of the United States.  My work is focused on data and evaluation of academic detailing programs that are working on topics like sexual health, HIV, and PrEP.  My main goal is to help folks measure, or think about, what “success” may look like for an academic detailing program.

---

NaRCAD: What data do you think is most important for academic detailers to track during their visits when measuring or thinking about success?

Kristefer: I see evaluation data and detailing efforts as one complete package.  Detailers should think about their data at a high level and focus on the information they’re collecting and how that information serves the overall goal of detailing, which is to improve knowledge, attitudes, intentions, and behaviors of providers and clinics. 
​
At the same time, detailers should think about how they can show that they’re achieving that goal.  For example, it’s useful to track how many providers they’ve seen, how much time was spent with providers, what they talked about during the visit, the resources that were provided, how the providers plan to use those resources, if a follow-up visit was scheduled, and the purpose of that follow-up visit.  It’s important to track a mix of quantitative and qualitative data, but the critical components that should be tracked are the outcomes and the process of detailing.  

NaRCAD: What about academic detailing programs?  What data should they collect?

Kristefer:  In a typical detailing program, detailers have a longitudinal, continuous outreach to providers.  There’s an interesting conversation to be had about how we can use that temporal matrix as a tool and strategy for evaluation.  I’m interested in how we can use the work academic detailers are doing – the actual visits themselves - as pieces of data over time. 

​For example, if detailers are collecting some of the rich conversation that they’re having with providers or clinics, it’s fascinating to track those conversations over time and see how the detailing program is changing the knowledge, attitudes, intentions, and behaviors of those providers and clinics.  There’s a lot of work that detailers are already doing that can be easily turned into data sources for both the detailing effort and the evaluation effort.  I like to think about how we can make things as simple as possible.

---

NaRCAD: Is there a specific platform that makes things simple and is best for collecting data for academic detailers and programs?

​Kristefer:  That certainly becomes a little bit more individualistic when thinking about the needs, challenges, and abilities of each jurisdiction.  Detailing is a conversational effort that is attempting to make concrete behavior change, so it becomes more convoluted when you think about how to track a conversation.  Some jurisdictions might have a place to track conversations in their electronic health record, where others may not. 
 
At the most basic level, detailers can chart their conversations on one-page Word documents.  The detailers I worked with charted their conversations with providers over time and eventually put it in one large PDF that could be easily uploaded into a qualitative data analysis software.  You have to be creative when it comes to tracking this information. 

NaRCAD: How would you recommend that programs with limited resources go about data collection and evaluation?

Kristefer:  A lot of times we say we want to have high-tech solutions when we don’t actually need them.  For a resource constrained department, having that one-page Word document that allows detailers to chart their interactions is more than enough data.  Charting for just five minutes after a detailing visit about everything that took place becomes a wealth of information.  You can also use an Excel document to input data from provider surveys. 
​
There are many low-tech ways to track information and it’s important to be aware of the low hanging fruit in terms of data collection.  You want to be able to easily collect data that serves the detailing efforts, the program, and the evaluation and improvement process.  

---

NaRCAD: That’s a great way to look at it.  What are some best practices for using data for leadership buy-in?

Kristefer: I can’t stress enough how important data is in getting leadership buy-in.  Data is not only quantitative and qualitative, but also using the information gathered to tell a story.  It would be a strong statement if someone was able to go to leadership with a story about how providers have changed their practices based on the detailing effort.  Using concrete results and showing leadership that detailing is making a change is extremely helpful for buy-in. 
​
Being able to show crazy big outcomes with your data won’t happen, but sharing stories from providers and clinics about how detailing has helped them is quite moving.  I’ve heard some amazing stories during my evaluation.  For example, detailers helped providers at some clinics to provide patients with directly observed therapy for PrEP at the same time that they were providing them with medication-assisted treatment for opioid use disorder.  It’s impressive that detailing at those sites was able to make the clinics think creatively and be able to provide PrEP to these patients.

NaRCAD: That certainly is impressive! What has surprised you the most about the academic detailing data you’ve evaluated?

Kristefer:  I can’t help but to think that pharmaceutical companies spend millions of dollars and resources on this model and they certainly wouldn’t have been doing this for decades if it didn’t work.  We’re almost a little late to the game as public health practitioners, but through my experience evaluating some of this work and reading other evaluations, I’ve been shocked by how much providers truly value detailers.
​
It’s fascinating to see how these health systems and departments are viewed as trusted partners by providers and clinics and how detailing has served as a role to improve that partnership and collaboration.  Providers have often said how crucial this information has been in getting access to Department of Public Health resources they didn’t even know existed, which is pretty sad.  Seeing public health and the medical system working side by side in this kind of way has been breathtaking.    

---

Biography. 
Kristefer Stojanovski is a PhD Candidate in the Department of Health Behavior & Health Education, School of Public Health at the University of Michigan. Kristefer has been doing community-based mixed methods research since 2010. His research explores the social and structural determinants to sexual health and HIV outcomes among key populations in the U.S. and in Southeastern Europe. Kristefer’s work interrogates how stigma drives HIV risk and infection using complex systems theory, structural equation, agent-based and multilevel modeling. Kristefer also translates his research into policy and decision-making. He is an evaluation specialist with the Capacity Building Assistance program with the San Francisco Department of Public Health.  

---