[MUSIC]

>>SINGER: I’m Steve Singer, I’m the director of Education and Outreach at the Accreditation Council for Continuing Medical Education.

>>MENCIA: Hello, I’m William Mencia and I’m Vice-President of Education and Medical Affairs at Med-IQ.

>>SINGER: Welcome back, William.

>>MENCIA: Thank you Steve.

>>SINGER: We’re going to talk in this video about the opportunity that you’ve found as an accredited CME provider to actually publish the results of the work that you did in a CME activity or two CME activities to improve performance improve care and I think in one of them to actually show the impact of the CME activity on improving the efficiency of care and perhaps lowering cost. So, in the context of talking about this I want to sort of point out that in the spectrum of what CME providers do sort of within the framework of the ACCME’s requirements we talk a lot to accredited providers about the fact that planning CME activities does not have to be complicated. And that you can work with stakeholders in order to simply build upon the framework of the Plan, Do, Study, Act process. Sort of the improvement, learning and improvement process, where you say what’s the problem we’re trying to solve? What do we want to change? Were we effective in making changes? And have we made progress on the problem? And really that’s still going to be the framework about what we talk about but the added component that you’ll discuss is the fact that it doesn’t mean that you can’t get robust data out of it that is publishable and can help to evidence the value proposition that you as an accredited CME provider and you as a partner to health care can make about contributing  education as a tool to improve care and lower costs. OK. So, let’s start by, tell me about the two projects that we’re going to discuss.

>>MENCIA: Certainly, so one project looked at treatment of atrial fibrillation and it was done as a web cast. And the other project came out of a performance improvement activity where we wanted to see impact of PI on specific patient care issues.

>>SINGER: OK So, let’s talk about the web cast first, you mentioned atrial fibrillation, so the web cast, just tell me how it was conducted.

>>MENCIA: Sure, this was a standard one hour web cast where we looked at increasing knowledge around the treatment of atrial fibrillation and identifying those factors that would require specific types of treatment based on individual patient situations.

>>SINGER: OK. So, you’re communicating information to, to your physicians that gives them information from which they can improve their strategies for treating these patients.

>>MENCIA: That is correct. Through both didactic and case based learning we were delivering information to improve knowledge and competencies. So, hopefully through that activity seeing how physicians are translating that new knowledge into their clinical practice.

>>SINGER: OK. So, how did you connect that to the care delivered?

>>MENCIA: Well, we hear all the time how some of the more traditional CME models may not elicit the same type of quality outcomes that performance improvement, or quality improvement type models have. And so, we wanted to see what truly is the impact of knowledge and competency based activity within practice performance and patient care. So, we partnered with a group out of the University of Miami. It’s the University of Miami Humana Research Group. So, the reason that we picked this particular collaborator was to have that direct access to the administrative medical claims data. So, this group traditionally looks at that type of data to report back to the managed care organization different aspects of quality of care of health care utilization. But we saw them as an opportunity to also see what impact that education had on those different claims metrics that they typically employ.

>>SINGER: OK. So, so you didn’t have to create from scratch a construct through which to measure your activity. So, that, for those in our audience those people watching they know that part of our ACCME requirements is looking at change and, and seeing are you effective in conducting education? And that, that’s part of what motivated sort of the genesis of this you wanted to know how effective is this web cast? But what’s interesting you specifically didn’t have to measure that change directly. You engaged or collaborated with a partner sort of a research approach, research focus partner who would look at the kind of data that they look that sort of the construct that they live within in order to get information sort of to connect the dots between what you did and what they measure.

>>MENCIA: Yes. So, this was a national level CME web cast we wanted to work with this group to identify those physicians within the Humana managed care system that had participated in the national level program and see what impact the activity had on their practice of care. So, we worked with them on identifying some specific codes

>>SINGER: medical codes

>>MENCIA: medical codes, some specific codes related to atrial fibrillation care and then we did a retrospective analysis six months prior and six months post their participation in the activity. And we had a little under a hundred Humana physicians within the larger global population of the participants in the program.

>>SINGER: OK. And, and, and do you, do you at, at your organization, do you sort of have the in house expertise to be able to say, How do we look at those hundred physicians and the measures that Humana was looking at in order to say do we, do we have, you know how do we connect the effect or the change in code coding to the education that we did because I would imagine you needed to use controls and sort of all the research method approaches. Is that something in house that you brought to the table or was this something you sought from this partner?

>>MENCIA: It’s a combination

>>SINGER: OK

>>MENCIA: of all of the above. It’s, it’s things that we had done in the past our statistical team that we work with out of NYU and the group out of the University of Miami.

>>SINGER: OK. So, again, collaborations.

>>MENCIA: Yes.

>>SINGER: So, so one of the, one of the opportunities that I want the accredited providers other stakeholders that watch this video to know is that they don’t necessarily have to have all those skills in house that they can also utilize the opportunities for collaboration or integration with other systems to, to be able to successfully navigate how to do the research methodology and what measurement tools are available to them and I would imagine that it is also resource intensive to engage those sort of partners.

>>MENCIA: That is correct. Not only in identifying those partners but then ensuring that everyone is on the same page in terms of what the goals of the activity are trying to achieve.

>>SINGER: So, so let’s talk about the results of the study that you did.

>>MENCIA: As a web cast that was looking at improving knowledge and competency the goal of utilizing administrative claims data was to be able to again see what impact that was having on actual physician changes within the practice. So, we looked at several different factors. We looked at health care utilization. We looked at, we looked at readmissions and we looked at ED visits as well as clinic visits. And what we were able to demonstrate is statistically significant improvements across all of those areas.

>>SINGER: So, so you found statistical significance were you expecting that you would have those results?

>>MENCIA: Honestly, Steve, no. Actually we were a little, we were actually a little surprised at the breadth and the depth of the quality of the information that we were able to obtain. It was a small study, as I mentioned earlier, slightly less than a hundred participants from the Humana system, so, again this is a pilot level initiative, but it gives us the information now to say there’s opportunity here to make this to expand this to make us feel

>>SINGER: OK so you’re, we talked in the other video about a sort of a business model about improving quality and using CME as part of that process. So you have now a little bit of data that’s significant data that surprised you. I would imagine that that part of what you’re looking for as a both a medical education provider who wants to improve patient care that’s a admirable goal, but you’re also a business and  part of what you wanted to do is to say is how do we expand this? How do we do this more and in other places? So out of is that where the sort of the publication strategy came from? To say, We have these results we’d like to share them.  

>>MENCIA: Yes. Once we saw the quality of the data the next logical step was to get this published. So, we wanted to validate the quality of our study by submitting this to a peer review journal. And we selected the American Journal of Managed Care because of the type of activity that we had done within a managed care framework. The purpose for publishing was one, again, to validate, but also, to establish that there is a model for research that can elicit data out of more traditional types of CME activities. And that we can build upon that to do more studies, larger studies beyond just this pilot.

>>SINGER: So, let’s now just sort of change tracks and talk about the other activity that you did and sort of the genesis of that and again the connection to the a research focus for it.

>>MENCIA: Certainly. So, in the first video we spoke about how a PI activity served unwittingly as a needs assessment

>>SINGER: a needs assessment right.

>>MENCIA: Exactly. For a QI initiative. Well, we had been doing a diabetes performance improvement activity for about two and a half three years and we wanted to see what was really the impact that that performance improvement activity was having on actual patient care. Because performance improvement has traditionally been more around the processes of care, but how does that translate to actual patient care? So, we established a research protocol where we identified a series of physicians where we could do a same patient analysis  retrospective analysis pre to post of that particular physician’s participation in the program.

>>SINGER: OK. So, what did you find as a result?

>>MENCIA: So, again, in the spirit of a pilot, of doing a pilot initiative we looked at roughly about 45 physicians who had completed all the way through the different stages of performance improvement.

>>SINGER: OK. So, so, and those are taking sort of a baseline measure of the care or the performance measure that they were conducting having an educational intervention, a series of interventions and then a re-measurement point. So this is part of this Plan, Do, Study, Act process that we talked about.

>>MENCIA: Correct and we actually take it one step further in our performance improvement activities wherein that middle stage we actually encourage the development of an improvement plan.  .

>>SINGER: OK. OK.

>>MENCIA: So, we want to see exactly what plan is going to be established. So, that aside, we identified these 45 physicians that had participated in the program and who wanted to participate in the study out of a pool of around 150 total physicians that we had to date. And those 45 physicians translated to roughly 335 actual patient charts pre to post, same patient pre to post. We went into the study wanting to see how the PI would impact A1C levels hemoglobin A1C levels, cholesterol, LDLC and HDLC and blood pressure. So, we picked three of the standard risk factors for the development

>>SINGER: clinical measures

>>MENCIA: Correct. Three of the standard clinical risk factors that will lead to complications of diabetes.

>>SINGER: OK.

>>MENCIA: So, we approached the study three ways. We wanted to see if participation at all stages of performance improvement has higher impact than someone who participates in the first or second stage of PI. And we also wanted to compare how participation in a PI type program would differ from someone who participates in a traditional type program.

>>SINGER: OK.

>>MENCIA: And that was our control group.

>>SINGER: I understand ok, so, so it’s interesting as an accredited provider you you’re doing sort of quite a introspective sort of a methodological approach to say, How effective are we? What strategies or what approaches, what educational approaches have the best effect? And, and again in the context of the ACCME’s requirements those are all things for which the system rewards you for engaging in that sort of improvement of your own product. It wasn’t necessarily that there was an external stakeholder they were saying to you which part of this was most effective or beneficial but you’re looking trying to create the best product that you can to have the best effect. What you’re describing is a real internal analysis of how effective as an accredited provider how effective are we being with the educational approaches that we’re having and with the measures of our success? And sort of looking at the different combinations the permutations of it to see sort of going forward what approaches what educational approaches are likely to be most effective?

>>MENCIA: And it’s also a way of validating our mission. Our mission as an organization is to not only focus on improving knowledge and competence that leads to improvement in performance but our mission actually states that we want to improve patient care. So, this was a way for us being able to validate that the education that we’re delivering does have an impact on patient care. And through this type of research we found some very interesting data. The first point is statistically significant improvement as you can imagine in A1C levels in LDLC levels and in blood pressure that was what we went into expecting to find. The other thing we expected to find in our hypothesis is that traditional types of education would not have such a strong impact versus performance improvement that we were able to demonstrate as well. But, what was surprising to us is that there were similarities, very strong similarities, in the results of those physicians who participated in all stages of the performance improvement and those physicians that participated in just the first two stages and they dropped out. And what we have deduced from that is that there are barriers to the performance improvement model. There are some significant participation barriers. But physicians may be having their aha moment from that initial baseline assessment and they may be realizing there are areas that I have to improve within my practice. I’m going to choose not to continue to do this program, but they may still be developing improvements within their practice that is just not showing up in our traditional PI outcomes model. But through this data study we were able to see that there are similarities in both groups.

>>SINGER: Yeah. And, and you know the, that, that adds value to a perspective that you can have as an accredited provider to see that your physicians or the health care professionals that you engage with, may sort of touch your education in different ways and at different times and maybe not everybody goes through all three stages but that your engagement with them brings educational value and can bring practice focused value to them and I would imagine you can sort of see yourself as an educational home for them as you sort of again to your mission as you try to help support their improvement of performance improvement of patient care sort of down the line. So, having these results in hand did you take a similar strategy in terms of approaching a publication a place where you could share this data?

>>MENCIA: Yes we did, Steve. The first step was taking that data and working with the Endocrine Society who had been a partner with us throughout the life of the diabetes PI initiative and presenting that as an abstract during a session at the Endocrine Society annual meeting earlier this year. The next step is to develop the manuscript and submit it for publication peer review clinical trial

>>SINGER: OK. So, that’s sort of on its way and, and you’ll hope to see a publication on that in the future.

>>MENCIA: Yes. And through most of our performance improvement and quality improvement programs and even some of our traditional programs we’ve emphasized trying to get that information published whether it be in a clinical peer review journal, a quality journal, an education journal, it’s important to advocate the value that CME can have and all of these programs that providers are developing regardless of the complexity or the simplicity of the design we can all learn from the results of those programs and be able to continue to evolve the quality of the education that we’re delivering.

>>SINGER: Fantastic. And that’s a great way sort of an imperative to your colleagues a great way to end the interview. So thank you again William for your time.

>>MENCIA: Thank you, Steve. Appreciate it.

[MUSIC]

This is a transcript of Demonstrating CME’s Value through Outcomes Research and Publishing.

© 2013 Accreditation Council for Continuing Medical Education; all rights reserved. For non-commercial educational use only. For permission to reproduce and/or distribute for other purposes, please contact info@accme.org.