If you have wondered why a model that looks brilliant on paper stumbles on new molecules, machine learning bias in drug discovery data is a likely culprit. Skewed training sets teach shortcuts instead of biology, so reported accuracy can overstate real-world usefulness.
| Item | Value |
|---|---|
| Typical assumption for missing negatives in interaction databases | Unlabeled pairs treated as negatives, then randomly sampled |
| Bias found in the DUD-E benchmark | Analogue bias and decoy bias |
| Share of women in trials for approved drugs over six years (2022 commentary) | 40%-72% |
| Black or African American share of oncology trial participants, 2017-2020 | 2% to 5% |
| Hispanic or Latino share of oncology trial participants, 2017-2020 | 4% to 6% |
These figures come from peer-reviewed papers indexed by the NIH, such as the DUD-E bias study and the 2022 commentary on women in clinical research. They can change as new studies appear, so check the source before you rely on them.
What “bias” means in drug discovery data
In this field, dataset bias means the examples a model learns from do not fairly represent the situations where it will be used. The model then picks up patterns that come from how the data was collected rather than from chemistry or biology.
Bias can enter at several points:
- Which compounds chemists chose to make and test in the first place.
- Which results were published, and which were quietly dropped.
- How “inactive” examples were defined or invented.
- Which patients were enrolled in the trials that produce clinical data.
Each of these produces a different kind of error. Understanding them helps you read claims about model performance with a sensible degree of caution, a theme that runs through the broader picture of how ai is used in drug discovery.
The missing-negatives problem
A study in the International Journal of Molecular Sciences describes the situation plainly: “An additional issue arises when the databases only contain positive examples of ( m, p) pairs known to interact, but no negative examples of ( m, p) known not to interact” (PubMed Central).
The usual workaround is to treat untested pairs as negatives. The same paper notes that “it is classical to assume that most unlabeled interactions are negatives, and to randomly sample negative examples among them.” That shortcut is convenient, but it can quietly mislabel real binders as non-binders.
The authors also report that such databases “present high statistical bias, leading to a high number of false positives, thus increasing time and cost of experimental validation campaigns.” In plain terms, a biased model may send laboratory teams after targets that do not hold up. Their proposed fix is to balance how often each protein and each drug appears among positive and negative examples, with the aim of reducing false positives.
When benchmarks teach shortcuts
Benchmarks are shared datasets used to compare methods. If a benchmark contains hidden regularities, models can score well by exploiting them. The DUD-E set, a widely used collection of known binders and property-matched decoys, is a documented example.
Researchers testing convolutional neural networks on it reported: “We find that superior enrichment efficiency in CNN models can be attributed to the analogue and decoy bias hidden in the DUD-E dataset rather than successful generalization of the pattern of protein-ligand interactions” (PLOS ONE via PubMed Central).
Two terms matter here:
- Analogue bias: binders of the same or related targets are thought to be correlated in chemical space.
- Decoy bias: the “inactive” molecules were selected by rules that differ systematically from the actives, so a model can separate them without modelling any binding.
The authors conclude that “biases that could be present in constructed datasets should be thoroughly evaluated before applying them to machine learning based methodology development.” That is a useful rule of thumb for anyone reading a leaderboard result.
Bias in clinical and patient data
Bias is not only a chemistry problem. Models that predict safety, dosing or patient response draw on clinical data, and that data reflects who joined past trials. A 2022 commentary in Cell Reports Medicine states that “women, and specifically women of color, are underrepresented in clinical trials, limiting biological understanding and contributing to health inequities and social injustice” (PubMed Central).
The same commentary reports that over the preceding six years, the percentage of women in the trials it examined “has ranged from 40%-72%.” It adds that in oncology trials from 2017 to 2020, the share of participants who were Black or African American ranged from 2% to 5%, and Hispanic or Latino participants ranged from 4% to 6%.
A model trained mostly on one population may behave differently in another. That does not prove any particular model fails; it means the evidence needs checking for the group of interest, and uncertainty should be stated openly.
How teams try to reduce bias
No single method removes bias, but several habits help. Researchers commonly combine them:
- Audit the data first. Look at chemical diversity, label sources and how negatives were defined before training anything.
- Choose negatives deliberately. Balance how often each molecule and protein appears as positive and negative, as the interaction-database study proposes.
- Validate experimentally. Laboratory testing on genuinely new compounds is the check that data-only comparisons cannot replace.
The same logic applies to property prediction. Questions about training-set coverage matter just as much for admet prediction machine learning, where models are asked to judge compounds unlike those they learned from.
What this means when you read a result
For a related topic, see the primer on how does alphafold work. Across tasks, the practical advice is similar.
- Ask how negatives or decoys were chosen.
- Ask whether test molecules resemble training molecules.
- Ask which populations the clinical data represents.
- Treat a single benchmark score as a starting point, not proof.
This site is educational and independent. Nothing here evaluates any specific product, company or candidate, and it is not medical or investment advice. Ask a physician or other qualified clinician about any treatment decision.
Frequently asked questions
What is machine learning bias in drug discovery data?
Show answer
It is a systematic skew in the examples used to train a model, such as missing negative results, over-represented chemical series or unrepresentative trial participants. The model can then pick up patterns from the collection process instead of from biology.
Why are negative results so scarce?
Show answer
When databases contain only known interactions and no confirmed non-interactions, researchers commonly treat unlabeled pairs as negatives.
Can bias make a model look better than it is?
Show answer
Yes. In the DUD-E study, strong performance of neural networks was attributed to analogue and decoy bias in the dataset rather than successful generalization of protein-ligand interaction patterns.
Does fixing bias guarantee a model will work?
Show answer
No. The balancing scheme was proposed to reduce false positives, but models still need laboratory validation, and results from one dataset may not carry over to another.
This article is for information only. It is general education about science and the drug industry, and it is not medical, treatment or investment advice.
