A Blog by Virginia Hughes

My Risk-Benefit Ratio For Personal Genetics

In high school I took a wonderful course in bioethics. We read about the infamous breaches of ethics by Nazi doctors and the Tuskegee syphilis experiment. We watched Gattaca and talked about unintended consequences and worst-case scenarios. The bulk of the course, though, was writing research reports that 1) presented a tricky bioethical question; 2) outlined several reasonable answers; and 3) came down in favor of a particular action. Much to my delight, I found all of my old reports in a file today. My 17-year-old self decided, apparently, that human cloning is bad, cochlear implants are good, and that parents should make decisions about the fate of their premature baby only if the baby has less than 50 percent chance of survival. (WTF?)

Anyway, I’m certainly not an expert in bioethics. (If you’d like to follow a real bioethicist who writes smartly and beautifully, try Kelly Hills.) But I did learn from that high school class a fundamental bioethics concept that has become useful anytime I try to frame a sticky ethical situation: the risk-benefit ratio.

In theory, it’s not complicated: When making decisions regarding human research or medical treatment, risks must be minimized and benefits must outweigh risks. In practice, though, the ratio is often difficult to calculate. Last year I wrote about a 13-year research project comparing the development of Romanian orphans who were raised in orphanages with those who were put into foster care. On the one hand, the researchers were effectively denying foster care (long known in Western countries to be better than institutionalized care) to half of the kids. On the other hand, their study would provide empirical evidence for foster care that the Romanian government could then use for policy changes. In this case, the possible benefits outweighed the risks, and the study was deemed ethical.

The Romanian orphans study was a tough call. The ongoing controversy about consumer genetics tests sold by 23andMe is not a tough call, despite all of the bioethical hoopla surrounding it. It’s a very easy call, with a risk-benefit ratio greatly favoring 23andMe.

To recap recent events: Several weeks ago, the FDA smacked down 23andMe by sending them a cease-and-desist letter. Because of 23andMe’s lack of cooperation with the agency, the company was no longer allowed to sell its spit kits that gave customers (including me) a look at medical information regarding about 1 million DNA markers in their genome. (This pertains only to the medical stuff; the company is still allowed to sell infirmation about ancestry.) In a commentary published in today’s issue of Nature, geneticist Robert Green and legal expert Nita Farahany describe the FDA’s allegations:

The FDA ordered 23andMe, based in Mountain View, California, to “immediately discontinue marketing” its testing kit and personal genome services, which the agency says offer medical advice and so require regulatory approval. “Serious concerns are raised if test results are not adequately understood by patients or if incorrect test results are reported,” the FDA wrote. Notably, it said, genetic results could drive consumers to take extreme steps, such as having unnecessary surgery to prevent cancer. Consumers might also abandon or alter prescribed treatments without consulting health professionals, the letter alleged.

But as Green and Farahany explain in the commentary, these supposed risks are completely unfounded. Several surveys have shown that people do not get overly anxious as a result of finding out their genetic risks. And though some people do, of course, interpret their results incorrectly, this also happens in doctors’ offices every day. Even when customers get genetic results back from 23andMe about gravely serious mutations, it seems they act reasonably. According to Green and Farahany:

The FDA is particularly concerned about how people might respond to learning that they have BRCA gene mutations that increase their risk of breast and ovarian cancers. Early evidence suggests that consumers respond appropriately. In a study carried out by 23andMe, the company sent interview invitations to 136 customers who carried pathogenic mutations. Of the 32 who accepted, 14 men and 11 women had learned for the first time through consumer testing that they carried a high-risk BRCA mutation. All of the women had consulted health-care professionals with their results, and all but one (who elected for surveillance and not surgery) had their tests repeated. Their behaviour after learning this information was no different from that of people who discover these mutations through medical channels.

In my opinion, the risks of consumer genetics tests are minimal. But what about the benefits side of the equation?

As many people have pointed out, the medical relevance of genetic data that customers get from companies like 23andMe is questionable. Last month in the New York Times, a graduate student in bioethics wrote about how different genetic testing companies provided her with very different genetic results. 23andMe told her that her biggest risks were for psoriasis and rheumatoid arthritis, whereas another company, Genetic Testing Laboratories, told her that her lowest risks were for those two conditions. I had similarly confusing results even within the 23andMe service; it told me I have a higher-than-normal risk of melanoma because I carry three particular risk variants, but I also carry a variant that confers a lower-than-normal risk of the disease.

Experts will quibble over the medical value of this or that variant or combinations of variants. For the sake of argument, though, let’s say that the information has absolutely no medical value. Let’s say that every single risk prediction that 23andMe gives me turns out to be just as likely to be wrong as right. Even in that case, I think there’s a huge benefit of personal genomics that is rarely talked about: public education.

Genetic analysis isn’t going anywhere. Quite the contrary — it’s transforming the medical industry. Take, as arbitrary examples, two announcements from the past week. Industry giant Illumina now has a machine that can read entire genomes — that’s all 3 billion letters, as opposed to about a million that 23andMe scans — for $1,000 a piece. The Geisinger Health System in Pennsylvania just launched a research project in which it will collect genetic data from 100,000 patients and try to link it to various diseases and health profiles.

Our genetic information, in other words, is going to be used more and more to determine our medical care. And yet the vast majority of people don’t understand what genes are. They don’t know what genes can and can’t tell us about our future. Because of movies like Gattaca and scare-mongering articles, the average person is more likely to be afraid of their genes than curious. 23andMe, to its credit, explains this science clearly, thoroughly, and accurately, and provides forums for its customers to talk about it in depth. I recently wrote about a woman in rural Virginia who had little training in science but, thanks to her interest in genealogy and 23andMe, is now a chromosome aficionado.

Forcing 23andMe to stop selling its tests is a good way to make the public even more scared of this technology, and perhaps more scared of science and technology in general. And that’s a risk we can’t ignore.

16 thoughts on “My Risk-Benefit Ratio For Personal Genetics

  1. I agree with your points, however, their conflicting results make it clear that these companies need to be held to a higher standard of evidence. They are in fact giving medical advice, and it seems likely that many people will take that advice at face value rather than considering the statistics involved in genetic risk predictions. For these reasons I think it is a good idea for them take a step back, at least for a little while, and consider more carefully how they are going about their business.

  2. @Megan Have you been genotyped yourself or looked at 23andMe results? I would like a concrete example of what you find to be “medical advice”.

    Here are examples from my profile that certainly DON’T look like “medical advice” to me.
    – “33.9 out of 100 men of European ethnicity who share Alex Khomenko’s genotype will develop Atrial Fibrillation between the ages of 0 and 79.”
    – “Has one mutation in the F11 gene linked to factor XI deficiency. A person with one of these mutations typically does not have factor XI deficiency, but can pass the mutation to offspring. May have other mutations in the F11 gene (not reported here).”

    1. @Alex The medical advice is implied. If they are correct, you have a one in three chance of developing atrial fibrillation, a serious condition. At the very least, if their calculations are correct, that would mean you (and your doctor) should be vigilant for this condition, looking for symptoms or perhaps even having extra testing. Otherwise, what is the value of the information? But my point really is that at a minimum the information they are giving you should be correct, or as correct as possible based on well established statistical principles. The information these companies provide should be subject to some kind of outside review before it is provided to the public, whether it is by the FDA or by a professional organization that sets standards.

      1. Where is the advice implied? The language seems pretty clear. The medical advice is what my doctor will give me, not anything I see here.

        The value of that information to me is that it exists, I have unfettered access to it, and I can decide to use it as I please. I have access to the raw data that the information is based on (which I can interpret using any number of ways btw), to references based on which 23andMe is basing the estimates, and to the whitepaper describing the method used for computing risk estimates. There is no objective truth that’s being measured here, so the information can’t really be correct in a way that someone’s blood sugar level can be correct. It can be consistent, it can be well-founded, etc. The raw data has to be accurate to a known degree, and that comes from a CLIA lab (just like any non-FDA-regulated LDT) from a SNP chip with known properties.

        Bottom line – when you say “it should be subjected”, you have to provide some kind of justification that is missing here.

        1. Alex, I agree with everything you say. You and I know that there is no objective truth in this kind of data. However, many people lack the science or math literacy to understand that. They are counting on the medical/scientific establishment to ensure that the information reaching them is accurate and actionable. I’m just looking for some protections for the so-called “average consumer” of medical information and services.

  3. A good perspective, Virginia. I’ve felt that 23andMe’s service is of dubious value, but you’ve made me reconsider my position.

  4. Great article. I have always taken an interest in these subjects. The inderstanding of human genetics and our potential ability to manipulate them to treat health issues and cure disease is fascinating.

  5. Folks are trying really hard to frame this as “your data and your right to it”, but it ain’t. 23andMe is a slick Facebook-style business arrangement, by which they’re essentially giving away a product (these kits cost a lot more than what they sell them for) to collect data about millions of people, which they hope in turn to sell to insurance companies and drug companies for billions. And that’s fine. I don’t think the FDA really cares, except that they chose to describe medical aspects of that data without demonstrating that the data is correct. The FDA seems to have made an effort to provide a workable roadmap for this company to gain approval. To quote from their actual letter to 23andMe: “FDA has spent significant time evaluating the intended uses of the PGS to determine whether certain uses might be appropriately classified into class II, thus requiring only 510(k) clearance or de novo classification and not PMA approval, and we have proposed modifications to the device’s labeling that could mitigate risks and render certain intended uses appropriate for de novo classification. Further, we provided ample detailed feedback to 23andMe regarding the types of data it needs to submit for the intended uses of the PGS. As part of our interactions with you, including more than 14 face-to-face and teleconference meetings, hundreds of email exchanges, and dozens of written communications, we provided you with specific feedback on study protocols and clinical and analytical validation requirements, discussed potential classifications and regulatory pathways (including reasonable submission timelines), provided statistical advice, and discussed potential risk mitigation strategies. As discussed above, FDA is concerned about the public health consequences of inaccurate results from the PGS device; the main purpose of compliance with FDA’s regulatory requirements is to ensure that the tests work.” So – – are the kits appropriately accurate? Maybe, but they haven’t proven it. Instead, 23andMe ceased all communications with the FDA, pitched their kits like crazy, posted medical implications of data on their website and then went all indignant (on Facebook, seriously) when their hands were slapped. It will be interesting to see what they do moving forward. What they do will probably depend on what they’re seeing in all that genetic data.

  6. I must say that Megan and Alex made the argument in favor of 23andMe right here. That Alex has a gene configuration that puts him at higher risk of a-fib is valuable information that Alex can now share with his doctor. That’s valuable. The article mentions 23andMe customers who had the BRCA gene mutation but were not aware of it. Again, potentially life saving information that can be shared with one’s physician. Since in most cases, action taken in response to genetic testing will be through a physician, I see this as a win-win — it encourages communication with one’s physician and makes sure that your physician keeps up with the latest and greatest from the world of the genome. If there should be false conclusions from the data by 23andMe (or the like), or false expectations by the customer, or new ways to interpret the data, then these can all be clarified by one’s physician. So ethically, I can see no real downside and the potential (life saving potential) seems to far out weigh the few people who may over-react or garner false expectations from the services of 23andMe.

  7. There’s perhaps some conflation of complex & incomplete technology with perception of “evil” intent here, much like the distrust of GMOs due to association with companies like Monsanto.

    23&Me is giving back a certain type of editorialised/simplified data that *may* be informative & actionable. They seem to be doing this in a responsible manner, couched with warnings & caveats. However, they don’t test for the competence of the buyer in interpreting their statements or in comprehending the educational material. Do they have a responsibility if a customer does act on the information in appropriately? Does a motor bike salesman have a responsibility to ensure the youth can safety operate the machine? To some degree, yes. This is where the neutrality of language & the incentive for 23&Me to provide value (in order to expand their gross data) will clash.

    The rationale for the FDA in pushing back on 23&Me is also unclear. It seems based on potential risk rather than evidence of incorrect use of information. The FDA is protecting from anticipated damage. This is better than correction after an incident but they need to work with 23&Me & not simply decide any information on genetic disposition is too volatile to be given to a non-expert. Hopefully, a balance will be found that doesn’t result in denial of information – as interpretation is getting better every day. This is information that appreciates over time, unlike most things we buy.

  8. I enjoyed your article but wonder what the general feeling about the technology would be if people decided to not reproduce because of the risks they discovered they carry in their genes. I, for one, didn’t need a lab-generated analysis to convince me that the potential harm I could do creating another human being outweighed the potential positives. My decision to be child-free is based wholly on my genetic potential and yet I don’t know my actual genetic makeup. I think it would create a whole new level of ethics to consider if people really know for sure what they’re getting into making babies. Surely many would agree with me that the risks are simply too high and abstain. And if they knew of risks but didn’t abstain, would they be liable for the health and happiness of their offspring? It’s food for thought, for sure…

  9. The practice of genetic testing has become a very controversial issue. At the forefront of this issue is direct-to-consumer genetic testing. Many believe it is our First Amendment right to access our own genetic information, arguing that the government should not interfere, while others have voiced concerns over the quality of the interpretation of test results by the companies that provide and execute the genetic tests, and the risk of harm to consumers. We recently hosted a debate on this issue, and have written a number of articles discussing the various sides of the controversy: http://www.lifesciencesnow.com/genetic-testing/

  10. Re-reading the two-year-old piece “Who’s Afraid of Virginia’s Genes,” I found this little nugget:

    “Like a rebellious teenager, I loathe this kind of paternalism, and bought the test partly in defiance. If I waited too long, regulators might deny me the opportunity.”

    Ha. Ha.

  11. Do we have some evidence for the closing paragraph? “Forcing 23andMe to stop selling its tests is a good way to make the public even more scared of this technology, and perhaps more scared of science and technology in general. And that’s a risk we can’t ignore.”

    The positive arguments are well-made, but this last paragraph seems to be needless scaremongering with no apparent basis beyond the general fear from some in the science community that the public are fearful and vaguely hateful towards science (which itself seems poorly supported beyond various Frankenstein movies!).

    It seems bizarre, too, to assert that the public definitely behave rationally when dealing with 23andMe’s information, but to later in the same article, blithely assume that they will behave irrationally when they hear 23andMe is no longer allowed to give medical-style results. Is this the same public? I find it very hard to believe that they would be rational about the former but not the latter.

    As a side-note, this is clearly an area that does need regulation and monitoring. The FDA’s ban may be extreme, and 23andMe relatively well-behaved, but they will not be the last company to enter this market, and others may be less responsible. I feel that the sort of argument that appeals to fears of “paternalism” and “regulators” who “deny” people stuff heralds from a very self-regarding libertarian place, too. Relying entirely on the continued good behaviour of profit-driven organisations is a sure way to be disappointed, which is where regulation comes in.

  12. In reading this, and other, articles on this issue it seems that many people are missing the main point of the FDA letter.

    The FDA was concerned that they “still do not have any assurance that the firm has analytically or clinically validated the PGS for its intended uses”. The word “validate” is key. Assay validation is a very specific body of work that medical/pharma companies need to perform. The idea is to PROVE than the assay is accurate and does not result in false-positives or false-negatives. This is critical to the effectiveness of ANY assay and validation is required for ALL assays used in health care. Think of the harm that would be done if somebody received a result that said they DID NOT have a particular mutation, when in fact they did. Or the harm from somebody being told they had a high risk gene when they did not. That is the major concern here – not how people are going to react to the information, but whether the information is accurate.

    It is very confusing to me so see the number of people defending 23andme when all that is being asked of them is that they prove their product does what they claim it does. They are being held to the same standard as everyone else in the medical community. Until they can do that, they are just another snake-oil salesman.

Leave a Reply

Your email address will not be published. Required fields are marked *