5. Avoiding Bias

Statistical bias is avoided by:

1. Correct selection of the experimental unit  (as discussed previously)

2. Randomisation of the experimental units to the treatment groups in a method which depends on the experimental design. (a randomised block is different from a completely randomised design)

3. Randomisation of the order in which measurements are made and the animals are housed because there will be time and space variables which influence the results.

4. “Blinding” and the use of coded samples to ensure that the investigator or other staff can not easily influence the outcome of the experiment.

There are other types of bias which should be avoided where possible:

Selection bias occurs when an investigator manipulates the results so as to give a result which supports their hypothesis

Publication bias occurs when positive (usually) results are published but not negative ones. This might be due to journals not accepting papers with negative results (all well designed papers should be publishable), or because the authors do not bother to write up their negative results.

Randomisation

Randomisation ensures that each experimental unit has an equal probability of receiving a particular treatment. It reduces the chance of systematic differences between the treatment groups.

There will still be differences due to chance sampling errors and, by definition, in 5% of cases these differences will be “statistically significant” at the 5% level!

All good statistical packages provide ways of putting numbers or letters in random order. It can also be done using a spread sheet such as EXCEL, as shown here

Assume we want to randomise 12 subjects to three treatments A,B, & C. in a completely randomised design.

The treatment designations A-C were put in the first column, 4 subjects per treatment

A random number was put in the second one (as “values” in this case, though this is not essential)

The two columns were then sorted on the random number column to give column 3, the treatments in random order. The animal numbers are then added. In this case the first three animals will be assigned to A, the 4th. To C etc. Randomisation often does not look very random. In extreme cases the subjects can be re-randomised.

Randomising a randomised block design

In a randomised block design the experiment is split up into a number of small parts or “blocks”. Typically each block has one experimental unit of each treatment. So if there are four treatments, block size is four experimental units.

Randomisation is done within each block. One way of doing this with EXCEL is as shown here. Assume the aim is to randomise four treatments: A, B, C, D, in  four blocks.

Column one shows the animal number, column 2 is a random number (shown to two decimal places), and column three is the treatment assignment. The lowest number in the block is assigned to treatment A, the next to B and so on. The last column is the block number.

This randomisation can be done in the office, printed out and taken to the animal house.

Randomisation in a Latin square

In a Latin square experiment the number of rows= number of columns = number of treatments and every treatment should appear once in every row and every column.. Randomisation needs to maintain this structure.

The square on the right was written A,B,C,D on the first line, then A, B, C, D on the second line but shifted one column to the right, with the D recycled back to the first column and so on with the 3rd. and 4th. rows. Randomisation is subsequently done first by whole columns and then by whole rows (not shown). This will maintain the structure, while still allowing randomisation.

Classification variables

Some variables such as genotype, age, sex can not be randomly assigned to subjects. However, the order in which the animals (or other experimental units) are housed and measured should be randomised. If males and females are to be compared in an experiment, then they should be comparable in other ways. If old males were compared with young females it would be unclear whether any differences were due to age or sex.

How should the animals be caged?

There are various ways in which the animals can be caged (rodents are the most widely used and this section refers to them)

There is no one answer to the numbers of animals housed per cage. It depends on species and the nature of the experiment.

Single housing of mice and rats may be stressful and is strongly discouraged for welfare reasons. But male mice may fight, depending on the strain and husbandry conditions.

Very valuable animals such as those fitted with telemetry apparatus, or ones with a genetic modification are sometimes housed with a companion which is not part of the experiment.

Group housing poses problems if treatment is given in the food or water as the cage is then the experimental unit unless sophisticated apparatus is used so that each animal can have a different diet. This is sometimes done with farm animals.

Group housing may also be a problem if drug treatments are involved as rats and mice are coprophageous so control animals may consume metabolites of the test compound if animals of different treatment groups are housed together.

It is not a good idea to house all the controls in one cage, all of treatment A in a second cage etc. as then the cage becomes the experimental unit. There can be “cage effects” due to social interactions which could seriously bias the results (e.g. if all the controls are fighting, but the treated animals are not).

If animals receiving different treatments (or genetically modified and wild type animals) can be housed together, then a randomised block design might be used as shown at the bottom of the figure (above).

Blinding

We usually have a vested interest in the outcome of our experiments. We might want to find “significant” differences between groups, or in some cases no significant differences (particularly if we are toxicologists). So,  having done the randomisation, wherever possible use the animal numbers as codes to “blind” everyone to the treatment.

This is particularly important when making measurements, scoring histological sections or measuring behaviour. Blinding may be difficult in some cases such as when comparing two mouse strains which differ in coat colour.

Failure to randomise and blind can lead to false positive results

In this study (Bebata et al 2003 Acad. emerg. med. 10:684-687) 290 animal studies were scored for blinding, randomisation and whether the outcome was positive or negative, as defined by authors. The results are shown below:

Odds ratio
Blind/not blind                3.4  (95% CI 1.7-6.9)
Random/not random      3.2 (95% CI 1.3-7.7)
Both/neither                   5.2  (95% CI 2.0-13.5)

An odds ratio of one implies that blinding or randomisation was not associated with the outcome of an experiment. These positive odds ratios show that on average studies which were not blinded and/or randomised produced excessive numbers of (presumably false) positive results.

Studies where there was no blinding or randomisation were unreliable.      >