The State of Mediocrity in Medicine: Just OK is not OK
I’ve mentioned this problem before, but it manifests in more than one way in health care.
There are plenty of articles reporting that the Patient Safety Movement, which is now over 20 years old, has gained little movement in those 20+ years.
If you go to a surgeon and ask them what their surgical complication rates are, they quote the national averages and that their rates are similar to the national average.
Have you seen those commercials where everything is “OK”? One has a doctor walking in saying his privileges were, maybe, just reversed and the nurse says he is “OK”. Or the waitress at a sushi joint who says the sushi is “OK” and the chef got sick after eating the sushi but a replacement is “OK” (then the kid preparing the sushi says, “Hey, this fish is raw!”)
You see, being OK with being average or having average performance is being OK with being OK. Even in those commercials, being OK is not OK. Right? Don’t you want better than a “C” performance? From everything? Your car? Your smartphone? Your surround sound system? YOUR HEALTH CARE??? Just OK is not OK!
Try it sometime. Ask your health care practitioner what their performance scores are. Ask your PCP what his/her hypertension control or diabetes control rate is. What is that performance compared to national performances. More than likely s/he hasn’t a clue. Maybe if they actually read a practitioner performance profile they get from a health insurer they might know their own performance data, but, most practitioners throw them into the circular file (anyone under 30 know what that is?).
Here’s an example. I had to have a major abdominal surgery. I chose a surgeon I knew and whom I respect (she is a terrific surgeon!). At the pre-op visit, I asked her to go through the risks and complications of the surgery. She was very knowledgeable in the national averages, which I already knew. So, then, I asked her what her personal complication rates were and she said her hospital rates were the similar to the national averages. I said I was more interested in her personal rates. She said she didn’t know that. I told her that, since she was working in an integrated health care system (Physician Group + Hospitals + Health Plans) and that, especially for the system’s health plan members, they had all of the data necessary to calculate her own personal complication rates, she should know her personal rates. She said the surgeons had asked for that, but the system had never made that happen. The reason? The finance folks running the system didn’t perceive that that would help the health care system improve their bottom line (remember, it’s all about the money). First, it costs money to prepare those kinds of statistics. Second, people coming back in for treatment for complications adds to the bottom line….except for “capitated” patients (pre-paid monthly), of which this system has a significant number, but their thinking is still “fee-for-service” (the more you do the more you get paid). In actuality, being excellent in post-op complications would lower the cost per patient in their capitated population. But, of course, your expecting C-player MBA’s to understand this. And the doctors don’t even know how the system is getting paid.
So, let’s have a short lesson in math. To calculate the rate of something in a data set (for example, the cost of a house in a state) you have many data points. Using the house example, not every house has the exact same sale price, so the values for the sales prices in dollars are variable. You’ll see periodic articles in your local newspapers about house prices. Data sets have what’s called a distribution of values (sales prices). Some houses may cost as little as $100,000 and some as much as $2,000,000. When you graph the number of houses against the price of the houses you get a “distribution curve” which usually comes out looking like a bell, so, they call them “bell curves”. There are two basic types of bell curves: a normal one and non-normal one. A normal curve looks just like the Liberty Bell: symmetrical. A non-normal curve has a tail on it either on the right or left side. In the case of houses, the tail is usually on the right, since houses over $1,000,000 are usually less prevalent than houses less than $1,000,000. For example, in most states or cities, most houses are in the $200,000 to $700,000 range.
Now, there is a statistical test for “Normalcy” called the Anderson-Darling test. If that test says your bell curve is normal, then it is OK to calculate and use the average of the data set. If the test says your data is not normal, you can’t use the average because the average will be skewed towards the tail of the curve. In the house example, the average house price will be artificially high because the prices skew (or drags) out to the right (higher prices). Therefore, you have to use the “median”, which is defined as the value for which 50% of the values are lower and 50% are higher than the stated value. In other words, it is the value separating the higher half from the lower half of a data sample.
If you notice, house price reports usually state three values: The range of prices (e.g., $100,000-$3,500,000), the average, and the median. Almost without exception, the median is lower than the average and is more indicative of your experience in buying a house.
Having worked for years in health care data, often, health care data to have a non-normal curve. As a matter of fact, Medicare, about 25 years ago, figured this out relative to the DRG length of hospital stays (in days). A DRG is a Diagnosis Related Group (like, Pneumonias), and Medicare pays a lump sum (like, $8,000) for a hospital stay for a DRG (of, say, Pneumonia). They have an “outlier” payment if the length of stay exceeds what they historically called the average length of stay (ALOS). (actually the ALOS is used extensively in the health care insurance industry when it shouldn’t be, it should be the median) Medicare, after about 10 years, realized that hospital lengths of stay, when distributed, did not create a normal distribution. The tail is to the right, like house prices. The ALOS was skewed to the right. For the past 20 years or so, Medicare uses the geometric mean, which, incidentally, is the wrong statistical methodology for that purpose (I won’t get into this here as the explanation is several paragraphs long). It should be the median. Thankfully and coincidentally, the geometric mean approximates the median.
I’ve never seen data presented for surgical complications that detail if the bell curve is normal or not. Health care folks love averages, without testing for normalcy. I don’t know whether a complication data set has a higher or lower tail or no tail at all, so it is hard to figure out what it means when a surgeon or the hospital says they are average. But, even if you use the median, it’s really not where you want your health care provider to be. It means 50% of the health care providers in that bandwidth are better than the person you are seeing. Unfortunately, no one knows who those better 50% are. So, your care is a crap-shoot!
Whether they use average or median, is that where you want your surgeon (or any health care professional) to be? Wouldn’t you want your health care in the hands of a practitioner who is above average or median? Me, I’d like someone who is an “A” not a “C”. To me that would be 3 standard deviations better than average or median. They don’t report practitioners that way.
This is similar to the use of an “index” where one divides their performance rate by the average comparator rate (the comparator rate is usually the average performance of a number of similar entities; for example, the average performance of 400 hospitals who are reporting their data into the hospital data set; and no one figures out if it really should be the median). An index value of 1.0 means the person or hospital is performing at the average of the data set. A higher index score means they are worse if having a lower raw rate is better. For example, post-op infections are bad, so the lower the score the better. If your infection rate is 5% and the comparator average rate is 4%, your index score is 1.25.
The problem is, practitioners and health care administrators perceive an index of 1.0 or close to that, as being just fine, or “OK” when it is not. It is better to be as close to the best performing entity in the data set. In the above example, an index of 0.3 might be the best in the group.
Another one is scoring by “percentiles”. They like it when they can say they are at the 90thpercentile or better. Scoring by percentiles is the same as scoring school essays on a “curve”. You take the range of scores and take the best score, even if the actual best score is an F, and make that an “A”, then you adjust all of the other scores so at least 60% of the class passes. For example, if all patients with diabetes are supposed to get an annual eye exam, and only 50% get them, 50% becomes the 99thpercentile, 40% becomes the 75thpercentile, etc. People think they are doing “OK” if they are above the 75thpercentile because that is a “C+ and better”, when actually they have a failing performance.
OK, enough about statistics, but the above points actually hamper health care performance improvement, which brings us back to why health care continues to be generally mediocre and health care folks are OK with being average.
The main reason is, the practitioner, hospital, skilled nursing facility, etc. is not the one who suffers from the complication or medical error. They do not feel the pain. They don’t get the post-op infection, get put on antibiotics, then get C.difficile (a bacteria that causes horrible diarrhea that can be fatal), etc. The patient is the one who suffers. They don’t suffer when they cut off the wrong leg. The patient suffers. And the practitioner feels OK about it because the patient just suffered a complication at the average rate; it was bound to happen, right?
So, they will sanguinely report that they are “average”. They don’t try to find out who is best. They don’t strive to be best. No one reports who is best!! (FYI, US News and World Report doesn’t really know, as their results are largely based on process metrics, not outcomes metrics). No one in health care tries to figure out what the best performer is doing and tries to beat that performance to be the best! Even if they do find out who is the best, they usually don’t have the ability or the will to do what it takes to make the improvement.
I was in health care for 45 years and there are very few places that try to figure out who is doing it best and how to be as good as or better than the best. You can call me jaded but I don’t see that happening in the near future, if ever, because it is hard to do, and it is easier to settle to be average. Plus, the people running health care don’t have the will or knowledge to make it happen.
Perpetual mediocrity. Get used to it. Everything is “OK”.