Insights

Evaluation of return on patient engagement: How pharma leadership can create a rising tide to lift all boats

Need a PDF version of this article? Download the report below

Measuring the impact of patient engagement is complex, contentious and absolutely necessary. While some innovative pharma companies are taking proactive steps to tackle the challenge, an industry-wide effort to build an evidence base of value is key to leveling up patient engagement in the eyes of all stakeholders.

Summary

In the burgeoning field of patient engagement, measurement of returns has always been a thorny issue

  • A retrospective analysis of conversations with A Life in a Day clients over the last 12 months gave us a rare glimpse of what patient engagement and patient-centricity look like through the lenses of
    governance, strategy and culture within their organisations today While it is a real test to identify the right KPIs and metrics for a given initiative, we learned that clients are nevertheless striving to do the
    best they can
  • Encouragingly, in numerous companies, the focus on patientcentricity now extends as far up as the CEO and the board of directors
  • Our parallel review of information in the public domain revealed that developments in the external environment, principally through public-private collaborations, are impacting the measurement landscape more broadly
  • There is clearly pent-up demand for an evidence base demonstrating returns on patient engagement
  • We offer some ideas around what pharma leadership in this area might look like

Section 1: The Measurement Problem

It’s fair to say that the pharmaceutical industry loves to measure things: metrics and KPIs abound. And that’s understandable. The huge sums needed to discover and develop new molecules, and then to maximize the reach of commercialized therapies warrants rigorous tracking and analysis of return on investment.

The job of measuring returns is more straightforward in some pharmaceutical functions than others. Commercial leaders can look at the extent to which investment in sales teams and in direct to-
consumer advertising is translating into increased market share. In clinical development, leaders can create dashboards to monitor spending on clinical trials against success rates in late-phase studies
and regulatory submissions.

In the burgeoning field of patient engagement, things are more problematic. For many people, it’s often not clear what to measure or how to go about it. There are many reasons for this – from the absence of robust measurement tools and the lack of well-defined endpoints to the subjective nature of what success looks like. It’s such a thorny issue that the very term “return on engagement” (and variations on it) remains contentious for some pharma stakeholders, perhaps because of perceived negative connotations or unfavorable optics (see Box 1). And this is despite the prevailing sense that patient engagement is the right thing to do.

Box 1: Definitions of terminology

Section 2: What clients are telling us

The pharmaceutical trade media, conferences and scientific journals are all great places to follow recent, high-level discourse on this topic. Meanwhile, A Life in a Day’s position as a provider of patient-centric
services offers a unique opportunity to learn directly from pharma clients what’s happening on the front line of operations.

To create this report, we took a retrospective look at selected conversations with A Life in a Day clients that took place over the last 12 months. The discussions covered a range of subjects, so we extracted
specific insights that were pertinent to patient engagement and patient-centricity. Note that we hadn’t prompted clients to talk about specific measurement tools. Instead, our retrospective analysis aimed
to summarize what they chose to volunteer, and this proved to be a useful way to gauge the current temperature around measuring value.

We also got a rare glimpse of what patient engagement and patient centricity look like through the lenses of governance, strategy and culture within their organisations today. Are clients satisfied with where their companies are right now? Or is there still a way to go?

How clients measure value from patient engagement today

On the topic of measuring value, some standout messages stuck with us:

  • It’s a real test to identify the right KPIs/metrics for a given initiative
  • Mindful there’s a way to go, clients are nevertheless striving to do the
    best they can
  • Tangibles showing impact are the ideal
  • Innovative clients are taking proactive steps to improve matters

A selection of client comments follows below – both to back up these messages and to illustrate the realities of operationalizing, and evaluating, patient-centric activities on the ground.

Decorative purpose only. team working to measure a result.

Identifying the right KPIs/metrics no mean feat

One client said it’s been very difficult for her to show metrics/KPIs. “I find it hard to identify metrics for patient engagement.” [Global Head Innovation]


Another said she’d never seen figures that quantify returns on patient centricity
“It’s a very tricky measure.” [Director of Global Oncology Commercial Engagement]

Clients’ best (current) efforts in reporting

It was clear that qualitative measures of reporting continue to overshadow quantitative ones.

One director of global oncology commercial engagement client said she primarily shows value through feedback from participants in patient engagement programs.

A head of learning client reports back on value through testimonials, how people feel about a program, as well as any advocacy plans or other actions they’ve committed to as a result.

A global patient engagement lead client reports on patient engagement across functions, considering factors such as the extent to which a patient focus is included in global brand strategies, as well as in customer plans at country level. While she uses KPIs relating to patient engagement, she admits she has reservations about them.

We heard the task of obtaining patient engagement metrics described as “challenging yet necessary … a work in progress”. [Patient Engagement Leader]

Also noteworthy was the comment from a director of global oncology commercial engagement client that with any educational program, it’s very hard to quantify value in concrete terms or ROI.

“If you don’t provide tangible metrics, people will lose belief in what we are doing”

Desire for tangibles as evidence of impact

While one patient engagement leader client acknowledged that qualitative measurements are much easier to identify than quantitative ones, he emphasised that “it’s data and evidence that trumps
everything else … Qualitative is good. But if you can say ‘I moved from X to Y or Z’, then that’s even better”.

To the extent that metrics may be available, one head of learning client’s evaluation process naturally involves reviewing data before and after a patient-centric program. She wants to be able to see an
improvement between the two time points, especially if the point of implementing a program was to resolve an issue previously identified within a specific function.

And why are tangibles so important to clients?

One global patient engagement lead client told us that tangible facts and figures are ideal because this is how the company rates staff performance in general.

Conversely there’s a risk if figures and data are not forthcoming, as captured in the following comments: “If you don’t provide tangible metrics, people will lose belief in what we are doing. We work in the
for-profit sector, and we want to see what value and what impact we are driving and so we have to provide some numbers.” [Patient Engagement Leader]

While a few clients pointed to commercial metrics as the ideal tangible, they acknowledged the challenge of making a link between patient centric activity and revenue.

We also heard some other ideas with a non-commercial flavor: “Will I create new tools because of an insight I learned? Do I need to think about using a different type of channel to interact or engage with
patients because of an insight I have learned? Is there a particular symptom that I have over- or under-emphasised?” [Global Head Innovation]

Proactive steps to improve the measurement landscape

On the value of patient engagement, one global patient partnerships leader client told us: “How to measure is the challenge”, an indication that in her view methodologies may be lacking or inadequate. Hinting that change may be on the way, she added: “There is at least the desire to make it [value] measurable and a clear commitment to hard KPIs.”

Among the most positive signs, one patient engagement leader client is taking a proactive approach to improve the measurement landscape. His team is creating a dashboard to capture both qualitative and quantitative metrics across the lifecycle. The plan is to use the dashboard to report up to senior management and potentially also to external stakeholders.

Persistent challenge of convincing skeptical co-workers

As buyers of patient-centric services, the people we spoke to need no convincing of the value of patient engagement. Making this message stick with their co-workers is another matter. Clients offered their thoughts on this and why they believe things are less than ideal.

Flawed self-belief in existing knowledge

We heard that part of the challenge lies in many pharmaceutical professionals coming from backgrounds in medicine and clinical practice. The sentiment was best summed up in this quote from a global patient partnerships leader: “The problem is the doctors that really have a lot of experience don’t understand that interacting with a person as their treating physician is very different from trying to really empathise with the patient experience. So, they do know in a very technical way, but there’s still a lag.”

Generational divide

One global medical director client told us he thinks the younger demographic is 100% on board with patient-centricity but that this isn’t true for all of the older generation. This is important in his view because the latter group tend to be more commercially driven, and they are still the people that make a lot of decisions in pharma.

Short-termism

The delayed nature of impact is a very real problem for companies accustomed to monthly and quarterly results. We heard from one patient engagement leader client that senior managers at his organisation are often very focused on the short term, whereas in patient engagement it is difficult to show short-term results.

Patient-centricity not a legal requirement

Aside from regulators’ requests to incorporate patient-experience data in submissions for new products, there are no legal requirements to undertake patient-centric activity or patient engagement programs
more broadly.

The upshot, in the view of one head of commercial training client, is that some patient engagement programs are never going to fall into the category of a ‘must have’, because the must haves are what companies are legally obliged to do. “While everyone agrees [a program might be] useful, because of udgets, it might be hard to place in terms of priorities.”

Similarly, a regular battle for one patient advocacy manager client is showing co-workers that patient-centric activity is business critical and a creator of value rather than a ‘nice to have’. Many of her colleagues make the false assumption that patient-centered work is handled outside of the company.

Patient-centric bosses and boards

An encouraging finding of our analysis was that for numerous employers, the focus on patient-centricity extends as far up as the CEO and the board of directors. In only a couple of cases was this not true or
questionable.

One client noted that patient-centricity is a regular conversation topic for the board, adding: “It’s one of our eight guiding principles of the business unit and one of which is mentioned the most.” [Global Patient
Engagement Lead]

For another, it’s notable that the CEO reports on patient-centricity more and more often, and that: “It’s actually been called out by the highest echelons and committees of our company as being a critical success factor.” [Patient Engagement Leader]

A head of commercial training client saw the arrival of a new CEO a few years ago, and ever since then the focus on patient-centricity has come from the top down. External quarterly and annual reports now
include elements of the company’s patient-centric efforts.

Patient-centricity and corporate life

We observed great variability in the extent to which patient-centricity is art of each company’s culture.

Part of the corporate fabric

For some clients, there is no doubt the two are intertwined.

One company has strategically embedded patient-centricity across the entire lifecycle, even before a molecule enters clinical development. The client explained: “That’s where we believe you get the biggest bang for your buck.” [Patient Engagement Leader]

One global patient partnerships leader client summed up a companywide commitment to putting patients at the center in the following quote: “Everybody’s really empowered to ensure patient inclusivity
across the organisation: it’s in each and everyone’s job description.”

A head of learning client told us succinctly: “Patient centricity has always been a cornerstone of the business”, adding that every aspect of its operations is in line with this strategic objective.

Work in progress

Some pharma players appear to be part way along a journey towards a more patient-centric future.

As an example of this, one global head innovation client pointed to an evolution in recent years that has seen patient engagement progress from a unique or grassroots approach to one that’s more crossfunctional, strategic and structured.

And a director of global oncology commercial engagement’s employer has set itself an aspirational corporate goal to become the most patient-centric [pharma] company.

Guiding principle without individual accountability

One global medical director client had recently joined a rare disease specialist where patient-centricity is a guiding “North Star”. We learned that he and his colleagues work in a patient-centric way because it’s the right thing to do. That being said, he has found it difficult to articulate patient-centered targets, or “what they are on the hook for when it comes to patient-centricity”.

What could help move the needle

Despite the challenges, clients had some ideas on how to ‘level up’ patient engagement, enabling it to stand on a par with other activities where recognition of value is more entrenched.

The problem is the doctors that really have a lot of experience don’t understand that interacting with a person as their treating physician is very different from trying to really empathise with the patient experience. So, they do know in a very technical way, but there’s still a lag.

Evidence base

In a science-based industry it was unsurprising that clients want to see evidence of what industry peers have achieved through patient engagement. Peer-reviewed research papers top the wish list, since
for scientists this would represent a “seal of approval”. [Global Head Innovation]

Exposure for decision-makers and budget-holders

Other clients felt that having senior leaders and those who hold the purse strings buy into, and ideally undergo, patient-centered learning programs is a powerful way to facilitate organisation-wide adoption.
In one conversation, a head of learning client recalled a senior clinical leader going through a potent experience that taught him people with chronic conditions “can’t turn it off when they go home”. The same client heard from another leader: “I want this [program] for my whole team, how do I make this happen?”

Another client hopes to see participation from colleagues in corporate functions “because it drives more disease understanding and how to create [and] give guidance and develop strategy”. [Global Patient Engagement Lead]

Market access catalysts

Powerful catalysts for change may also come from the pricing and reimbursement environment. From the starting point that regulators are already demanding patient-experience data, one global patient engagement lead client suggested that if health technology assessment bodies were to do the same, the returns from patient engagement might become more tangible. In turn this might smooth the way for approvals from budget-holders.

Every company tells a different story

Regarding awareness and usage of available metrics, every client was perhaps unsurprisingly in a different position, suggesting a lack of standardisation across the industry. Companies also differ in terms of the strategic nature of patient-centricity, and the extent to which the focus on patients is part of the corporate culture. Boxes 2-8 include highlights of our conversations with selected clients to illustrate the differences.

Download PDF to view boxes 2-8:

Section 3: Where do we go from here?

The insights outlined in Section 2 came from a discrete sample of pharma professionals sourced from the A Life in a Day client base. So, there may well be practices, opinions and developments prevailing elsewhere in pharma that we haven’t been able to capture in this report.

Another key limitation of our retrospective analysis is that we could only rely on what those same clients had already told us, without the opportunity to ask more probing follow-up questions. So, while we
would like to be an enabler of knowledge-sharing, we haven’t been able to describe any specific tools clients may be using to measure returns on engagement.

To address the second limitation, we carried out a parallel landscape analysis of information in the public domain. We wanted to understand whether any developments in the external environment may be impacting the measurement of patient engagement more broadly.

It turned out that much has happened in just the last few years, principally as a result of public-private collaborations. One important name to look out for is PARADIGM, a respected public-private partnership whose latest contribution to this field is its so-called “Patient Engagement Metrics Framework”. First released in 2022, the PARADIGM tool seeks to address the weaknesses of previous patient engagement frameworks; namely that they were heavy on concepts but not operationally useful.

An equally reputable endeavor has been unfolding under the direction of the Drug Information Association (DIA) and the Tufts Center for the Study of Drug Development, the latter being a Boston group that regularly churns out data and reports that many in pharma rely on. In 2020, the DIA and Tufts published a toolkit with “standardised metrics” that they hope will enable stakeholders to assess the impact of patient engagement and other patient-centric initiatives.

The PARADIGM and DIA/Tufts initiatives are discussed in detail in our accompanying report “Novel return-on-engagement tools poised for pharma validation”.

Critique of the status quo

Absorb our retrospective analysis of A Life in a Day client conversations alongside our parallel review of what’s happening in the public-private space and two things will become evident.

First, there is pent-up demand for an evidence base demonstrating returns on patient engagement. Anecdotes, case studies and ad hoc reports may have filled a void until now, but they no longer suffice.

Robust data, captured at scale and incorporating quantitative as well as qualitative measures, are needed to shift the dial.

The second finding is less visible but no less pertinent: there has been a real lack of pharma leadership in this area. PARADIGM notes that only a minority of research papers on the theme of evaluating patient engagement are authored by (and for) the pharmaceutical industry.

A Life in a Day’s observation is that in the PARADIGM work that led to the Patient Engagement Metrics Framework, pharma companies participated as contributors, including through ‘working group’ representation, but not as leaders. The same is true of the DIA/Tufts project. In both cases, academics played the lead role.

The biggest opportunity for pharma is to step up and address both of these issues head on. So, what might pharma leadership in building an evidence base in patient engagement look like? We offer three ideas.

Validation of new tools, longitudinally in the real world

Neither the PARADIGM nor the DIA/Tufts teams released their offerings as definitive finished products. PARADIGM researchers made it clear that their metrics had “hardly been tested in practice”, so at the time of making the framework available they had little to say in terms of optimal methodologies for measurement. This thinking must have played into their decision to put a beta version of their Metrics selector out for public consultation in 2022.

Similarly, DIA/Tufts described their new resource as “an initial set of standardised return-on-engagement metrics”, which we interpret as meaning it was supposed to be a foundation on which to build over time.

Forward-thinking, early adopters in pharma should use the new tools from PARADIGM and DIA/Tufts in real-world industry applications. And they should publish their findings, as lead authors, in peer-reviewed journals. This will provide much-needed validation of their utility outside of the academic setting, as well as potentially revealing ways to refine them.

They should also take some lessons from the real-world data movement in patient-experience data. Professionals in this field know that the richest datasets are those compiled longitudinally over extended
periods, rather than captured at a single point in time. This is also true in measuring returns on patient engagement, because of the time lag between execution and impact. In fact, half of the 87 metrics in the
PARADIGM tool are so-called “impact metrics” intended for use in the years after a patient-centered initiative. So, pharma needs to think long term, not only in terms of action but also dissemination of the results.

Exploration of “value-impact”

In 2022, a research team described a pharma-led initiative that came to life after a paradoxical discovery: no previously published studies had systematically captured patients’ views on patient-centered
impact measures.

Aiming to remedy this, the team interviewed representatives of patient advocacy groups about their experiences in this area. They then used the learnings to adapt a set of 23 impact measures, in order to make them more meaningful to patients.

Beyond this, they also discovered that for patients to consider a program as having delivered value important to them, the impact needs to be “a positive, tangible or useful outcome”. The researchers coined the phrase “value-impact” to encapsulate this insight. Now they want to ensure patients play a part in future implementation, so that any new tools developed to capture data are effective at measuring
value-impact in patient-relevant ways.

While one particular pharmaceutical company led this project to improve its own internal processes, it also wanted to promote sectorwide learning. The revised impact measures are not proprietary; in fact, the findings are detailed in full in an open-access research paper.

Nobody could argue that the patient voice doesn’t have a role in determining the value derived from patient engagement. And the phrase value-impact may turn out to be less contentious than return
on engagement. So other pharma companies should adopt this team’s recommendations and publish their findings, adding even more credence to the evidence base.

Measurement of value creation after R&D

Through our conversation with one patient engagement leader, we learned that his company had strategically embedded patientcentricity across the entire lifecycle (see Box 5). This client was the only one to volunteer this information. But the relevance extends to any other pharma company that has commercial as well as clinical operations, since the value of patient engagement (and the imperative to measure it) doesn’t end at the point of regulatory approval or payer reimbursement.

Interestingly, this is where both of the newly introduced tools fall short. PARADIGM’s is limited to three areas: setting of priorities in R&D; clinical trial design; and early discussions with regulators and payers. And because that team’s study captured less input from the first and third buckets in this list, they acknowledged that the metrics in their framework are mostly focused on clinical trials. The strongest message from the 2022 public consultation on their beta Metrics Selector was that to broaden its utility, it needs metrics to measure value beyond clinical research settings.

Similarly, the DIA/Tufts team focused their efforts on patient-centric initiatives used in clinical R&D.

This points to an opportunity for pharma leadership to engage the numerous functions involved in maximising the value of pharmaceutical products when they are on the market and reaching patients. Sales is one of the most obvious choices. Our conversation with a head of commercial training revealed what the end goal might look like: at his company, the sales model has shifted from physician orientation to patient orientation (see Box 8). This client now spends a lot of his time enabling “unbranded conversations” with doctors. This means explaining to HCPs in “patient language” people’s complaints, their greatest concerns, and how to meet their needs.

To give another example, pharma professionals responsible for designing and implementing patient-support programs are prime candidates to explain to leadership what they consider to be of value, and possible ways to measure this.

The collective input from workers employed further along the chain than R&D would help inform future iterations of patient engagement frameworks and toolkits. To build a robust evidence base, all this work should be the topic of pharma-led, peer-reviewed, published research.

Time to show true leadership

Pharma has a formidable track record of finding solutions to once intractable scientific and medical challenges.

The solutions to the problem of demonstrating return on patient engagement – a willingness to help validate new measurement tools, to demonstrate value in patient-relevant ways, to look beyond clinical research, and to publish findings as lead authors in peer-reviewed journals – are all well within pharma’s reach.

And leaders in pharma shouldn’t feel they need to tackle the problem on their own. Each of the three ideas we outlined above could be the focus of collaboration rather than work driven by a single entity.

Given the potential benefit to all stakeholders, pharma should now step up and help create a rising tide to lift all boats.

When evaluating patient engagement, questions around what to measure, and how, will soon become less bewildering than they have been since patient-centricity emerged as a trend. An even more pertinent question for pharma in 2024 is who will be the first to take the lead.

FURTHER READING
Gorbenko, O. et al. (2022) ‘Co-creating with patients an impact framework across the medicine’s life cycle: A qualitative study exploring patients’ experiences of involvement in and perceptions of impact measures’, Research Involvement and Engagement, 8(1). doi:10.1186/s40900-022-00334-0.

Stergiopoulos, S. et al. (2020) ‘Measuring the impact of patient engagement and patient centricity in clinical research and development’, Therapeutic Innovation & Regulatory Science, 54(1), pp. 103–116. doi:10.1007/s43441-019-00034-0.

Vat, L.E. et al. (2021) ‘Evaluation of patient engagement in medicine development: A multistakeholder framework with metrics’, Health Expectations, 24(2), pp. 491–506. doi:10.1111/hex.13191

Get in Touch

If you are interested or have questions about A Life in A Day, please reach out and contact us.


    You can select more than one by holding down the Ctrl key on selection