Jason Hotchkiss, author, chaplain and psychologist, has unveiled new research on the ways online reviews can help drive hospice quality and staff morale.
He is currently a chaplain and behavioral health specialist at the Center for Elders’ Independence and a professor of psychology at Cornerstone University in Michigan. Prior to this work, he spent more than a decade as a hospice chaplain working for national and regional providers.
The research is published in his new book, “Love Letters from Caregivers: Grateful Testimonials to Inspire and Empower Palliative Care Professionals.”
Hospice News spoke with Hotchkiss about the implications of his new research for hospices and how they can apply feedback from patients’ families to fine-tune their services and motivate clinical employees.
Jason HotchkissRegarding your newest book, “Love Letters from Caregivers.” How would you encapsulate the thesis of that work?
The footprint of a hospice’s online reviews tells the story about what a hospice values.
We can do well in hospice. We can make money in hospice, and that’s okay as long as we’re providing excellent care. My overall learning from this is that families and caregivers will tell us that it’s the people that make a difference in the services.
Online reviews do tell a story about a hospice’s care, and I think making peace with it is important. Being able to see that you’re going to get a review now, sometimes negative, and that doesn’t always seem fair. But even that has something for us too.
I remember my own hospice, with the online reviews, my experience of it was an approach of, “We’ll deal with the really negative things when they come in, but we’ll just sort of shelve that.” But other hospices have begun to post testimonials on their site. There’s something to be gained from that.
My book, I hope, is really a praise of what hospice is doing in America. As a psychologist, I’m into behavioral shaping and behavioral interventions, and my behavioral intervention for hospice would be similar to what’s being done in the [Medicare Advantage hospice] carve-in program, rewarding hospices for quality care. In some of the five-star reviews, people would describe their hospice like it was like one of the best gifts they ever got. That’s heartening.
When you say reviews, are you talking about things like Yelp or those kinds of public websites?
The two that we used in the data gathering were Google and Yelp. We looked at more than 5,000 reviews. We also drew on Facebook. The way this works is, we get a review star rating the program, and then we use the natural language processing, the sentiment analysis, to see what is actually being said in that review.
You get two elements from that. One is what you call a sentiment score, which is sort of your minus-one to plus-one. Minus-one would just be a terrible experience up to a positive one. There is a little bit of nuance. You might have a five-star review that praised people, but also had a comment about responsiveness near the end of life or something that was more of a critique.
How did you go about conducting this research?
We gathered all the quality studies that I could find about hospice. One of the interesting things is that even in research, there’s a lot of studies that want to show hospice in its best light.
I really began with going to all the providers in each of their programs and grabbing reviews using a sampling technique to make sure that I’m balancing the reviews from all different parts of the programs.
One of the things that you’ll find online is that there’s many reviews that are just star ratings. I gathered every review that had any narrative component. And another thing that’s important to understand is that for all these reviews, Google and Yelp also have their own quality process for being able to rule out reviews that just look fake, that are not really authentic.
Then I looked at what are the themes that are within those reviews and was able to draw upon that as well. But my method, as far as the sampling is really analyzing using natural language programming. The tools are there, but no one’s ever looked at reviews in this quantity before to answer those kinds of questions.
What were some of the key factors that you examined?
Three of the main factors were star ratings, emotional intensity and caregiver sentiment.
Google and Yelp reviews of hospices have the same elements, a star rating and a narrative component. Review star ratings (1-5), a third global measure of caregiver satisfaction, was operationally defined as the mean of star rating of all of a hospice’s reviews.
Caregiver Sentiment was operationally defined as the mean of all 25 topic sentiments of caregivers weighted by prevalence. Sentiment scores ranged from –1 to +1. Scores closer to +1 express positivity, closer to -1 express negativity.
A sentiment score greater than +0.40 is defined as an excellent experience, between +0.25 and +0.40 is considered a good experience and zero to .25 defined as neutral. Scores between zero and –0.25 is considered a dissatisfied experience, and finally lower than –0.25 is operationalized as frustrated-to-angry.
Emotional Intensity measures the magnitude of the feeling expression in the review and includes the review length in this calculation. Magnitude ranged from zero to infinity. Scores closer to zero express little emotion, the higher the score the more emotional intensity and the longer the review length. Raw scores were used in order to better differentiate overall emotional expressions by hospice and by topic.
I also had the work looked at by third parties, a peer review.
What were some of the most significant findings?
If I’m talking to hospices today and asking them how they reach people, I think one of the big things I learned was that the hospices that have done the best in my study were the smaller ones that have been around for a while. They’re carved out their path. They’ve done their homework, and they’ve got that reputation.
I looked at this from a user-experience perspective, as a way of improving services. When you read what people actually write, you develop a survey out of what you think is important to people. But what people say is important to them.
This is actually quite revealing in the [Consumer Assessment of Healthcare Providers and Systems (CAHPS)] process. For instance, there’s no measure that asks, “Did you feel like someone cared about you?” There’s no question like that. But it seems the most important question that caregivers ask is, “Did someone care about me as I have my loved one in hospice?”
That’s one of the reveals that pops out. When a service is provided in the hospital, it’s often the patient-rated experience. Patients don’t really rate their experience in hospice. It’s the caregivers that rate the experience. People that want to do better in hospice need to embrace that side of it.
Your comments about CAHPS and that question of “Does somebody care?” really struck me. Clinical care is obviously very important, but I think there’s that distinction when the care is just procedural. You can go into a hospital, and sometimes they’ll give you your medication and walk out of the room, for example. With hospice, you have that more direct, compassionate relationship.
One of the things that was revealing is how, in the CAHPS survey there’s eight indicators. There’s two indicators that are oriented around respect and spiritual, emotional support. And what’s interesting about them, is that even with an underperforming hospice, that’s not answering their phone calls, they often score high in those categories. The CAHPS process asks people if your support was too much, too little or just right.
By asking if it’s too much, it’s almost like it underestimates the amount of emotional support that people need. Yet, spiritual and emotional support is the area where the high-touch programs do so well.
I think you’ll be seeing more tools that help to put a hospice’s people in front of patients and families, with less time spent on documentation, which can be so overwhelming. In hospice, your whole thing is to be with people as they’re dying. And yet, it’s the non-medical side. It’s holding the person’s hand. It’s therapeutic relationships.
Reading good feedback is kind of an under-utilized resource. It’s the part that I think is sustaining for professionals that have told us that when families get to tell the stories about the good care they received, it’s the sustaining element for the work.
