Big Data was used to see if TCM was scientific, but the outcome of this study shouldn’t make us forget the implausibility of the practice
https://www.mcgill.ca/oss/article/m...nese-medicine-has-not-been-vindicated-science
People love to show that skeptics were wrong about something, especially when national pride hangs in the balance. The South China Morning Post published the following headline on November 3rd: “Scientists find traditional Chinese medicine is based on a complex network of proteins – 3,000 years before modern science.”
The article points out that respectable editorials in the scientific literature had repeatedly referred to traditional Chinese medicine (TCM) as “largely just pseudoscience” and “based on unsubstantiated theories.” Yet here was the believer’s vindication: that TCM really was rigorously scientific while predating the European origin of what we refer to as “modern science.”
Skeptics were bound to eat their hats.
The study itself, published in Science Advances, is certainly interesting, but its complexity makes it opaque to the average person. It’s one of those impenetrable bits of data wrangling that can easily be dismissed as nonsense by the TCM skeptic or blindly embraced as confirmatory by the TCM believer.
Let’s dive in.
Exploring links between symptoms, proteins, and herbs
The paper only focuses on the herbalism part of TCM, ignoring the many other interventions, like acupuncture, qigong, and cupping, that are often found under the TCM umbrella; hence, to claim that this study validates all of TCM is deceptive.
The authors here did not test any particular herb in a clinical trial or even in the laboratory. Their work was done on computers and they essentially created maps. They had a long list of symptoms, a long list of genes and their corresponding proteins, and a long list of herbs used in TCM. They wanted to know if there was a dependable link between a symptom and the herb that was supposed to treat it, and if that link could be found through proteins.
The thinking goes like this: a symptom like fever arises because certain genes are turned on, and these genes produce proteins which help create the fever. If the chemicals in the herb typically given to a feverish patient can be shown to interact with these fever-associated proteins more often than not, then TCM has some validity. It means that these herbs really are effective for scientific reasons that were unknown to the practitioners who started using them thousands of years ago.
The problem is that you end up with very large datasets, and finding associations is not hard. Symptoms can indeed be associated with genes and their protein products, but it’s not a one-to-one relationship. Fever, for example, has been linked to well over 100 different genes, according to the supplementary data published with the paper. The scientists here initially focused on 174 symptoms that were associated with at least 20 proteins, but only 108 of these symptoms formed a connection to proteins that was likelier than by chance alone.
As for the herbs and the proteins they seem to influence, it gets complicated by the number of potentially therapeutic chemicals in a single herb. On average, each herb they looked at had 62 potentially therapeutic chemicals in them (plus or minus 62, meaning that there was a lot of variability from herb to herb). Each chemical has on average 70 protein targets (with lots of variability here as well). You may have seen maps in an airline magazine that shows all of the routes the airline makes available, with bountiful lines emanating from London and Washington, D.C. and connecting airports throughout the world. This is the type of map we are talking about here.
Using this kind of network analysis, the authors report finding that, on average, a symptom, certain proteins, and an herb are closer together than to other parts of the network. They then wanted to validate this finding in actual patient data. They looked at nearly 2,000 patients with liver cirrhosis hospitalized in Wuhan, China. They extracted from their medical charts the symptoms they were complaining of and they looked at the herbs that had been prescribed to them. They had to decide if the herbs had worked or not. They did so by looking at how quickly patients had recovered when given a particular herb versus how quickly other patients with the same symptom had recovered when given a different herb. Patients usually received a combination of 16 to 18 different herbs. Less than 10% of all herb-symptom pairings showed a statistically significant effectiveness when analyzed in this way (86 pairings out of 888). Even when considering the small sample sizes of many of the pairings, this result does not look good for the clinical acumen of TCM practitioners. Still, the paper’s authors write that their study “reveals the scientific foundation of TCM.”
There are many limitations associated with this paper, and the authors were more open about them in the preprint version of their article. All of this data about which symptom is associated with which proteins, and which proteins are influenced by the ingestion of an herb, it’s very noisy. Moreover, liver cirrhosis patients are not a particularly representative slice of the population. And nobody got an herb prescription that was meant to be wrong or a placebo: every patient at that hospital received the herbs that a practitioner thought would treat their symptoms. Yet in most cases, those prescriptions seemed not to work.
Also worth mentioning is that simply looking at genes and their products to predict how someone will respond to treatment is not as easy as it sounds. Yes, there are clinically validated tests that are robust enough, but the literature is also littered with failed propositions. It is not an easy code to crack.
When we zoom out of this particular theoretical paper, we can contextualize these findings within what we already know about traditional Chinese medicine, because it will help us assess if these latest results are plausible or just the kind of noise we might expect from large, incomplete datasets being made to interact with each other.
The implausibility problem
Despite its name, traditional Chinese medicine is not particularly old. It is a modern reinvention spearheaded by Chairman Mao Zedong in the middle of the 20th century. As I’ve written before, access to healthcare in China in those days, especially outside of big cities, was hard, so Chairman Mao decided to revive and repackage disparate practices that had sprung up all over China over thousands of years and to train “barefoot doctors” in them in order to provide some sort of medical care in the countryside. He famously did not believe in it but thought it important to promote it. It worked, and when New York Times columnist James Reston had his appendix removed in China, the story that spread throughout the Western world was that he had not received any anesthetics before the surgery; instead, the surgeons had used acupuncture and he had felt no pain. (This story is not true.)
TCM notions are thus not aligned with our modern understanding of biology and are often the result of superstitions. An important difference with conventional medicine—and not one in TCM’s favour—is that TCM focuses on symptoms, not diseases. As the authors of the present paper write, “Connecting TCM to the modern biomedical literature is challenging, due to the absence of the concept of ‘disease’ in TCM.” This is ironic given the common accusation from fans of alternative medicine that conventional healthcare treats symptoms but never the underlying condition. In TCM, the focus on symptoms to the exclusion of underlying causes is baked in. This is because whatever is wrong with the body is claimed to be due to blockages in the flow of qi, a presumed life force for which there is no evidence.
The prescribing of herbs is itself another problem with TCM. Herbs contain unknown amounts of various chemicals, and these quantities vary depending on where and when the herb was grown. Buying acetylsalicylic acid or Aspirin from the local drugstore guarantees you are receiving a fixed dose of the active ingredient; sipping on an infusion of the bark of a willow tree will deliver an unknown dose of the related salicin and many other chemicals. When you’re throwing 16 to 18 herbs at a patient, you might as well include the kitchen sink.
Finally, we must contend with the questionable reliability of Chinese research into Chinese traditions. Widespread data fabrication in clinical trials was unearthed by China’s food and drug regulator a few years ago, and we also know that virtually all trials of acupuncture done in China report a benefit, which clashes with results from trials done elsewhere and from biomedical research results in general. Interventions simply are never that good.
TCM thus has a tall hurdle to overcome in terms of its plausibility, and a study that didn’t directly test its herbalism but instead relied on Big Data connections doesn’t cut it. Proponents of TCM are quick to point to the Nobel-Prize-winning malaria treatment artemisinin as a TCM success story, but as pharmacist Scott Gavura pointed out in Science-Based Medicine when the prize was awarded, artemisinin represents a triumph of modern scientific refinement. Yes, an old Chinese manuscript recommended the cold brewing of a tea with the plant that contains artemisinin to treat a fever; but artemisinin is eliminated too quickly from the body, which leads to relapse when used alone. It was a pharmaceutical company, Novartis, which mixed an artemisinin derivative, artemether, and lumefantrine into a beneficial medication for malaria. Pills are good not because the pharmaceutical industry benefits from them, but because they deliver a consistent dose of a well-studied molecule as opposed to the chemical chaos of whole herbs.
The authors behind the paper drawing connections between symptoms, proteins, and Chinese herbs are hopeful that their model will show which herbs used in TCM seem particularly promising. They claim that chemicals in some herbs are known to interact with the same proteins involved in a particular symptom, but that this herb-symptom association has so far been ignored by TCM practitioners. They give several examples, such as Aristolochia fangchi known colloquially as Fang Ji which, based on their computer work, could help with abdomen distention. Patients beware: that plant was used in the 1990s instead of the listed herb as part of a slimming regimen in Belgium, where it caused “an outbreak of terminal renal failure.” That is something that abstract maps of chemical interactions may not tell you, but we should not forget what we already know from experience.
Rejecting the recent reinvention that is TCM does not mean turning away from plants. Effective drugs are developed from plants all the time. It’s just that plants are the beginning of the process, not the end.
Take-home message:
- Researchers behind a new study claim to have revealed the scientific foundation of traditional Chinese medicine (TCM)
- Using large data sets, they looked for connections between symptoms, proteins, and herbs used in TCM and found some associations that were likelier than would be expected by chance
- This theoretical exercise needs to be balanced against the implausibility of TCM: its herbs are variable mixtures of chemicals and its practices are incongruous and were repackaged by Mao Zedong in the middle of the 20th century to provide some kind of healthcare in the countryside even though he did not personally believe in their validity
https://www.mcgill.ca/oss/article/m...nese-medicine-has-not-been-vindicated-science
People love to show that skeptics were wrong about something, especially when national pride hangs in the balance. The South China Morning Post published the following headline on November 3rd: “Scientists find traditional Chinese medicine is based on a complex network of proteins – 3,000 years before modern science.”
The article points out that respectable editorials in the scientific literature had repeatedly referred to traditional Chinese medicine (TCM) as “largely just pseudoscience” and “based on unsubstantiated theories.” Yet here was the believer’s vindication: that TCM really was rigorously scientific while predating the European origin of what we refer to as “modern science.”
Skeptics were bound to eat their hats.
The study itself, published in Science Advances, is certainly interesting, but its complexity makes it opaque to the average person. It’s one of those impenetrable bits of data wrangling that can easily be dismissed as nonsense by the TCM skeptic or blindly embraced as confirmatory by the TCM believer.
Let’s dive in.
Exploring links between symptoms, proteins, and herbs
The paper only focuses on the herbalism part of TCM, ignoring the many other interventions, like acupuncture, qigong, and cupping, that are often found under the TCM umbrella; hence, to claim that this study validates all of TCM is deceptive.
The authors here did not test any particular herb in a clinical trial or even in the laboratory. Their work was done on computers and they essentially created maps. They had a long list of symptoms, a long list of genes and their corresponding proteins, and a long list of herbs used in TCM. They wanted to know if there was a dependable link between a symptom and the herb that was supposed to treat it, and if that link could be found through proteins.
The thinking goes like this: a symptom like fever arises because certain genes are turned on, and these genes produce proteins which help create the fever. If the chemicals in the herb typically given to a feverish patient can be shown to interact with these fever-associated proteins more often than not, then TCM has some validity. It means that these herbs really are effective for scientific reasons that were unknown to the practitioners who started using them thousands of years ago.
The problem is that you end up with very large datasets, and finding associations is not hard. Symptoms can indeed be associated with genes and their protein products, but it’s not a one-to-one relationship. Fever, for example, has been linked to well over 100 different genes, according to the supplementary data published with the paper. The scientists here initially focused on 174 symptoms that were associated with at least 20 proteins, but only 108 of these symptoms formed a connection to proteins that was likelier than by chance alone.
As for the herbs and the proteins they seem to influence, it gets complicated by the number of potentially therapeutic chemicals in a single herb. On average, each herb they looked at had 62 potentially therapeutic chemicals in them (plus or minus 62, meaning that there was a lot of variability from herb to herb). Each chemical has on average 70 protein targets (with lots of variability here as well). You may have seen maps in an airline magazine that shows all of the routes the airline makes available, with bountiful lines emanating from London and Washington, D.C. and connecting airports throughout the world. This is the type of map we are talking about here.
Using this kind of network analysis, the authors report finding that, on average, a symptom, certain proteins, and an herb are closer together than to other parts of the network. They then wanted to validate this finding in actual patient data. They looked at nearly 2,000 patients with liver cirrhosis hospitalized in Wuhan, China. They extracted from their medical charts the symptoms they were complaining of and they looked at the herbs that had been prescribed to them. They had to decide if the herbs had worked or not. They did so by looking at how quickly patients had recovered when given a particular herb versus how quickly other patients with the same symptom had recovered when given a different herb. Patients usually received a combination of 16 to 18 different herbs. Less than 10% of all herb-symptom pairings showed a statistically significant effectiveness when analyzed in this way (86 pairings out of 888). Even when considering the small sample sizes of many of the pairings, this result does not look good for the clinical acumen of TCM practitioners. Still, the paper’s authors write that their study “reveals the scientific foundation of TCM.”
There are many limitations associated with this paper, and the authors were more open about them in the preprint version of their article. All of this data about which symptom is associated with which proteins, and which proteins are influenced by the ingestion of an herb, it’s very noisy. Moreover, liver cirrhosis patients are not a particularly representative slice of the population. And nobody got an herb prescription that was meant to be wrong or a placebo: every patient at that hospital received the herbs that a practitioner thought would treat their symptoms. Yet in most cases, those prescriptions seemed not to work.
Also worth mentioning is that simply looking at genes and their products to predict how someone will respond to treatment is not as easy as it sounds. Yes, there are clinically validated tests that are robust enough, but the literature is also littered with failed propositions. It is not an easy code to crack.
When we zoom out of this particular theoretical paper, we can contextualize these findings within what we already know about traditional Chinese medicine, because it will help us assess if these latest results are plausible or just the kind of noise we might expect from large, incomplete datasets being made to interact with each other.
The implausibility problem
Despite its name, traditional Chinese medicine is not particularly old. It is a modern reinvention spearheaded by Chairman Mao Zedong in the middle of the 20th century. As I’ve written before, access to healthcare in China in those days, especially outside of big cities, was hard, so Chairman Mao decided to revive and repackage disparate practices that had sprung up all over China over thousands of years and to train “barefoot doctors” in them in order to provide some sort of medical care in the countryside. He famously did not believe in it but thought it important to promote it. It worked, and when New York Times columnist James Reston had his appendix removed in China, the story that spread throughout the Western world was that he had not received any anesthetics before the surgery; instead, the surgeons had used acupuncture and he had felt no pain. (This story is not true.)
TCM notions are thus not aligned with our modern understanding of biology and are often the result of superstitions. An important difference with conventional medicine—and not one in TCM’s favour—is that TCM focuses on symptoms, not diseases. As the authors of the present paper write, “Connecting TCM to the modern biomedical literature is challenging, due to the absence of the concept of ‘disease’ in TCM.” This is ironic given the common accusation from fans of alternative medicine that conventional healthcare treats symptoms but never the underlying condition. In TCM, the focus on symptoms to the exclusion of underlying causes is baked in. This is because whatever is wrong with the body is claimed to be due to blockages in the flow of qi, a presumed life force for which there is no evidence.
The prescribing of herbs is itself another problem with TCM. Herbs contain unknown amounts of various chemicals, and these quantities vary depending on where and when the herb was grown. Buying acetylsalicylic acid or Aspirin from the local drugstore guarantees you are receiving a fixed dose of the active ingredient; sipping on an infusion of the bark of a willow tree will deliver an unknown dose of the related salicin and many other chemicals. When you’re throwing 16 to 18 herbs at a patient, you might as well include the kitchen sink.
Finally, we must contend with the questionable reliability of Chinese research into Chinese traditions. Widespread data fabrication in clinical trials was unearthed by China’s food and drug regulator a few years ago, and we also know that virtually all trials of acupuncture done in China report a benefit, which clashes with results from trials done elsewhere and from biomedical research results in general. Interventions simply are never that good.
TCM thus has a tall hurdle to overcome in terms of its plausibility, and a study that didn’t directly test its herbalism but instead relied on Big Data connections doesn’t cut it. Proponents of TCM are quick to point to the Nobel-Prize-winning malaria treatment artemisinin as a TCM success story, but as pharmacist Scott Gavura pointed out in Science-Based Medicine when the prize was awarded, artemisinin represents a triumph of modern scientific refinement. Yes, an old Chinese manuscript recommended the cold brewing of a tea with the plant that contains artemisinin to treat a fever; but artemisinin is eliminated too quickly from the body, which leads to relapse when used alone. It was a pharmaceutical company, Novartis, which mixed an artemisinin derivative, artemether, and lumefantrine into a beneficial medication for malaria. Pills are good not because the pharmaceutical industry benefits from them, but because they deliver a consistent dose of a well-studied molecule as opposed to the chemical chaos of whole herbs.
The authors behind the paper drawing connections between symptoms, proteins, and Chinese herbs are hopeful that their model will show which herbs used in TCM seem particularly promising. They claim that chemicals in some herbs are known to interact with the same proteins involved in a particular symptom, but that this herb-symptom association has so far been ignored by TCM practitioners. They give several examples, such as Aristolochia fangchi known colloquially as Fang Ji which, based on their computer work, could help with abdomen distention. Patients beware: that plant was used in the 1990s instead of the listed herb as part of a slimming regimen in Belgium, where it caused “an outbreak of terminal renal failure.” That is something that abstract maps of chemical interactions may not tell you, but we should not forget what we already know from experience.
Rejecting the recent reinvention that is TCM does not mean turning away from plants. Effective drugs are developed from plants all the time. It’s just that plants are the beginning of the process, not the end.
Take-home message:
- Researchers behind a new study claim to have revealed the scientific foundation of traditional Chinese medicine (TCM)
- Using large data sets, they looked for connections between symptoms, proteins, and herbs used in TCM and found some associations that were likelier than would be expected by chance
- This theoretical exercise needs to be balanced against the implausibility of TCM: its herbs are variable mixtures of chemicals and its practices are incongruous and were repackaged by Mao Zedong in the middle of the 20th century to provide some kind of healthcare in the countryside even though he did not personally believe in their validity
