Agile and Physical Health - Part 4 - Empirical Data
"The greatest obstacle to discovering the shape of the earth, the continents and the oceans was not ignorance but the illusion of knowledge." - Daniel J. Boorstin
You can manage what you measure. If you mis-measure, you're sure to mis-manage. Choose wisely.
Vanity Metrics
Any discussion about collecting data meant to measure progress toward important outcomes like personal health would be incomplete without covering the idea of "vanity metrics." It's vital to know what you're measuring and why. It's equally vital to know when something you're measuring no long applies to your objectives and it's time to stop tracking it.
A review of what you're measuring should be a key part of your ongoing health strategy. If it isn't, you're at risk of succumbing to Goodhart's Law: "When a measure becomes a target, it ceases to be a good measure." A tragic example of this on a national scale is public education's devolution toward teaching to the test - the inevitable outcome when teaching focuses on preparing students for a standardized test and neglects broader understanding of a variety of subjects so that higher standardized test scores can be achieved. End result: Grade inflation and test scores, even college degrees, that are no longer good metrics for competency or ability.
Perhaps a more relatable example, told in a way only David Sedaris could, is his experience with wearing a Fitbit fitness tracker. He quickly became obsessed with his step count. The intrinsic motivation for enjoying a good walk had disappeared. The number on the Fitbit was all that mattered. And when his Fitbit died:
"I was devastated when I tapped the broadest part of it and the little dots failed to appear. Then I felt a great sense of freedom. It seemed that my life was now my own again. But was it? Walking twenty-five miles, or even running up the stairs and back, suddenly seemed pointless, since, without the steps being counted and registered, what use were they? I lasted five hours before I ordered a replacement, express delivery. It arrived the following afternoon, and my hands shook as I tore open the box. Ten minutes later, my new master strapped securely around my left wrist, I was out the door, racing, practically running, to make up for lost time."
The concept of "vanity metrics" gained traction in the early days of Agile software development. The concept is simple, but over the course of my career as a software developer, manager, and Agile coach, it was a concept that wasn't particularly sticky. There is some sort of psychological algorithm that prioritizes the vanity metric path. Probably something akin to the path that prefers to stop at first-order thinking. Or prefers simple, familiar, or desired outcomes. There are a whole host of cognitive biases in play here. To ease cognitive load, people prefer to latch on to a single number that tells them how things are.
Medicine isn't immune to relying on vanity metrics. A clear example is the body mass index (BMI), a calculated value that estimates body fat from body weight relative to height. From this single number, medical and insurance types determine if someone is underweight, overweight, or obese. From there, they extrapolate the risk of heart attack, stroke, and other diseases associated with being overweight or obese.
BMI doesn't explicitly factor in things like sex, age, genetics, fitness, etc. These factors are assumed to average out over large populations, which may very well be helpful. But when large population metrics like BMI are then assumed to apply equally well to individuals they result in less quality decisions for the particular individual. Imagine a metric like the Foot Mass Index (FMI). The FMI determines optimal shoe sizes and if the size of your foot falls into the "overlong" or, God forbid, "protracted" you would be pressed into drastic measures for shortening your feet! For individuals, BMI is almost as useless as FMI.
At present, According to the CDC, my BMI is 26.4. In BMI-speak, I'm overweight. Factoring in other variables, however, I'm just 2.5 pounds off my ideal weight. In order to just make it into the top end of the BMI scale for "healthy weight" I'd have to weigh 210 pounds. Presumably, life in the mid-range of "healthy weight" (18.5 to 24.9) would be "ideal." To get there, I'd have to reduce my weight to 185 pounds. In college, as a starving student, I weighed 180 pounds. Looking back on those years, I was decidedly unhealthy - weak, under nourished, and low on energy. Unsurprisingly, I can cast around and find other BMI calculators that give my current weight as "healthy" and appropriate to my height and age. It's not at all clear how they accomplish this magic. The interpretation of the BMI number is more subjective than objective.
BMI, credit scores, and current temperature are examples of single numbers imbued with the same certainty and value. BMI tells the doc and insurance company I'm "overweight." My credit score tells the bank I'm a good credit risk. The temperature tells me I should wear a coat when I go outside. Easy-peasy, yes? No. The value and utility of each of these numbers are vastly different. BMI is derived from a ridiculously simple ratio of height to weight and yet has resulted in a swamp of bad advice or adverse motivated reasoning by politicians and special interest groups like the pharmaceutical and food industries.
Vanity metrics are for lazy, unhealthy people. Don't be one of them.
Quality Metrics
Quality metrics don't always tell you what you want to hear. Maybe they never do. But they'll tell you what you need to hear so that down the road you'll have more peace of mind and greater satisfaction in life. I have a set of criteria for distinguishing vanity from quality metrics. quality metrics...
...are specific.
...are relevant to me.
...aren't "squishy" in the way BMI is.
...are repeatable.
...are actionable.
...are informative.
...are reliable.
...are understandable. That is, they don't require some obscure "expert" knowledge to interpret.
...are free from opinion and aren't easily manipulated by the self-serving motivations, prejudices, and biases of others.
As science progresses, several stalwart tests have been shown to be less informative when compared to more recent tests. This is something we should expect. Science is a process, not a destination. The knowledge of today leads to better knowledge tomorrow. The metrics we track today should yield to better metrics tomorrow. This doesn't mean we completely swap out all the metrics. The process of science can also re-affirm past and present knowledge, giving us higher confidence in what the metrics are tracking. The tests for cholesterol are one example I'll cover in Part 6 of this series. (Nota bene: There's an obvious play for AI here. A tool that will accurately track validated advances and make recommendations accordingly would be tremendously helpful as it would allow people without a strong foundation of scientific training or busy medical practitioners to make better decisions.)
Sending the BMI number through the shredder one more time: There is a similar reference number proposed in 2013 known as the Body Roundness Index (BRI.) The BRI uses height, waist circumference, and hip circumference to estimate body fat distribution. This, I think, is a significant improvement as greater relevance to an individual can be drawn from the number. BRI is at least indicating where the weight, in terms of fat, is located on a person's body. (Excess belly fat has been shown to increase the risk of many health issues.) Sampling several BRI calculators, my BRI (2.7) measures well within the healthy range. While BRI is a better metric to follow, every one of the doctors I've seen in the past year still track BMI on my medical records.
Unlike BMI or even BRI, blood tests qualify as quality metrics. I've had comprehensive blood tests for each of the past 10 years. More recently, I've expanded the number of tests to include lipoprotein fractionation ion mobility, Apo A1, T4/T3, lactate dehydrogenase, gamma-glutamyl transferase, and iron. Until recently, my tests had been coming back as "normal," for the most part, with a few numbers put on the "things to watch" list. The usual recommendation was to continue to watch my diet (which I do) and recheck the numbers in a year.
As Dr. Attia has pointed out, “normal” is often erroneously used interchangeably with “optimal.” The outside edges of ranges are treated like tripwires - nothing's wrong until a line is crossed. I would have benefited more if ten years ago my docs had started saying something like "Your numbers aren't overly concerning today, but they're moving in a direction that portends significant problems down the road."
To be clear, I don't fault them. They were probably following best practices as they learned them from the fire hose that is medical school. Also, I certainly had plenty of exposure to science and medicine to grasp the importance of trends. Even so, I'd comfort myself with the idea I wasn't in trouble...yet.
Being the kind of person who vigorously hunts for the best information possible - good or bad - for any issue I'm working to resolve, I appreciate that the Early Medical program is out in front of this. The program does an excellent job of connecting seemingly small trends across combinations of blood markers to likely catastrophic consequences down the road, if not corrected. It's about being the captain of the Aoi Kumo. The presentations were enlightening to me, but I could imagine someone new to the details having the crap scared out of them.
Disclaimer
The author has Bachelor degrees in both biochemistry and cell biology but is not a licensed practitioner of medicine or psychotherapy and nothing presented on this website claims or should be construed to provide medical or psychotherapeutic advice. This series of articles is presented as a personal reflection by the author on work he's done to improve his health and as such is relevant to the author and no one else. The author makes no recommendations as to any course of action the reader may chose to follow other than to encourage the reader to work closely with qualified health professionals when making healthcare decisions relevant to their personal lives.
← Agile and Physical Health - Part 3 - Historical Data
Related Articles
If you have any questions, need anything clarified, or have something else on your mind, please send a DM or email me directly.
Image generated by Grok.
