How Reliable Are Genetic Association Studies?

by Margaret Wahl on Tue, 2009-09-01 11:01

These studies are powerful — and sometimes misleading

It sounds like a good idea, or at least a simple idea:

Human beings have diverse genetic backgrounds and diverse environmental exposures, compared to laboratory mice. It’s much harder to determine the effect of any specific factor on human disease than it is to determine its effect on rodent disease in a controlled laboratory environment.

Take a large group of people with a particular disease and look at their DNA; then take a similar group of people without the disease and look at their DNA. If there are differences in the DNA, those might be genetic risk factors for developing the disease, or might provide insights into the biochemical pathways of disease development, and ultimately could even help scientists in their quest for therapies.

In fact, dozens of such genetic association studies, in ALS and other diseases, have been conducted in the last several years, aided by tremendous advances in technology. It seems every day we’re hearing about another disease, ALS included, in which a new genetic factor has been identified.

The new genetic technologies have been rightly praised as powerful. And for diseases like ALS, for which causation remains largely a mystery (except in a small percentage of inherited cases), one can’t afford to leave any stones unturned.

But at the same time, pitfalls of genetic association studies have to be recognized, particularly in “genome-wide” or “whole-genome” association studies, in which all genes are analyzed at once.

Widening the lens

Until recently, most genetic association studies started with a biological hypothesis. For instance, if the distance between nerve and muscle fibers is abnormally large in a particular disease, investigators might hypothesize that there’s a problem with the proteins that hold things together at the junction of these fibers. They then would take a look at the genes for these proteins to see if they’re different in people affected by the disease.

This type of “narrow-lens” study still is done today. For example, a recent meta-analysis of 11 studies of variations in the genes for paraoxonase (PON) enzymes in ALS included six such narrow-lens studies that looked specifically at PON genes. The biological hypothesis underlying the search for PON gene differences was based on the known role of PON enzymes in detoxifying pesticides and possibly other environmental poisons. (For more on this analysis, see “Meta-analysis finds no ALS-PON gene connection.”)

The other five studies in the PON gene meta-analysis were whole-genome association studies — scans of the entire genome (all genes) in search of any differences at all, without the guidance (or prejudice) of a biological hypothesis. This kind of “wide-lens” study wouldn’t have been possible without the technical developments of the last five years or so.

Pitfalls and caveats

Although all genetic association studies have the possibility to lead us in productive directions in ALS research, there are pitfalls to be aware of, particularly when using a very wide lens.

1. ALS probably isn’t one disease.

If research has shown us anything over the last several decades, it's that there are almost certainly many ways to develop degeneration of the motor neurons, the cells in the brain and spinal cord that control muscles and mysteriously die off in ALS.

If, in a study of 400 people or even 4,000 people with ALS, there are actually five or 10 subtypes of the disease, each of which has different genetic associations, then chances are extremely small that any particular genetic association will stand out as statistically significant. True associations can be missed this way.

2. Human populations are genetically diverse.

In a population of laboratory mice, investigators ideally keep all variables constant, except the ones in which they’re interested. Mice in the experimental group and mice in the “control” group to which they’re being compared will have as close to the same genetic background as possible. In fact, mice even can be cloned to achieve this.

But humans, even from the same ethnic group or family, have diverse genetic backgrounds. And when you start talking about humans from varying ethnic groups around the globe, their genetic backgrounds can diverge significantly.

That factor matters in a genetic association study, because a variant form of a gene (for instance, a PON gene) in one genetic background may have one effect, while the same variant placed against a different genetic background may have another.

The significance of a variant that may have a significant impact on disease development in a small, genetically similar ethnic group may be lost when a study combines people of widely divergent ethnic backgrounds.

3. Human exposures are widely diverse.

It’s a principle of good laboratory practice that experimental animals are kept in an environment that’s as controlled as possible.

Laboratory mice can be kept in a constant temperature range, protected from infection and injury and fed the same diet. If the animals were in the wild, they’d be exposed to a variety of infectious agents, food types and amounts, temperature ranges and injuries, any of which could affect the course of a disease or the chance of developing it.

Humans, of course, don’t live in controlled environments, usually not even for the course of a clinical trial, let alone a lifetime.

A genetic variant, such as a PON gene change, that only matters in the context of certain environmental exposures, could well be overlooked in a study in which a large percentage of the population didn’t have those exposures.

4. The amount of data generated, particularly with whole-genome association studies, can be enormous and unwieldy.

Today’s genetic association studies are often conducted using computer chips that can analyze and compare hundreds of thousands of nucleotides (the building blocks of DNA) at a time.

How is a whole-genome association study like a food colander? Like a colander, these studies strain out data that aren’t significant, allowing passage of only the most significant differences between the disease-affected and unaffected samples. But experts aren’t certain how small the holes (how stringent the criteria) should be.

Although the first three points above focused on the possibility of overlooking a genetic variant that might actually be relevant (a type of error called a “false negative”), this fourth point addresses the possibility of finding an influence that isn’t really relevant (a “false positive”).

The possibility is high that at least one of some half-million DNA nucleotide comparisons made in a whole-genome association study could be judged statistically significant by the usual mathematical criteria for such judgments.

But some of the findings from such an analysis would be false positives that actually have no influence on the development of ALS but which, by chance, turned out to be different in the affected versus the unaffected populations.

In order to avoid false positives, statisticians working on whole-genome association studies use very stringent criteria to establish significance.

Those criteria perform the same function that a colander performs in food preparation, straining out unwanted elements. If the usual data “colander” has holes that allow the passage of peas but not potatoes, statisticians on whole-genome analysis studies are using colanders with holes so small that only the smallest peppercorns get through.

The problem is, no one knows exactly how stringent the statistical criteria for such studies should be. If they’re not stringent enough, a lot of false positive “peas” fall through the holes. If they’re too stringent, even significant peppercorns might get stuck and not be counted.

The bottom line

The bottom line about gene association studies, for those avidly following the progress of ALS research: Positive findings from gene association studies should be taken as leads to be followed, not as immediate therapeutic targets. And negative findings don’t always mean there’s no association between a genetic variant and ALS.

Margaret Wahl
No votes yet
MDA cannot respond to questions asked in the comments field. For help with questions, contact your local MDA office or clinic or email See comment policy