For centuries, therapies have been applied by trial and error, and varied according to the theory of disease at the time. The main measure of effectiveness has been the experience of the practitioner. But theories can be incorrect and experience faulty. The fact that something has been used for many years, even for centuries, is not assurance that it is helpful. The best example is bleeding, which was a mainstay of therapy for serious diseases for thousands of years, even though we now know it was probably more harmful than helpful.

Confidence that something is helpful in the treatment of MS requires evidence, and there are different levels of evidence. A neighbour’s claim that a particular medicine helped her or a story of someone who responded dramatically to a therapy (anecdotal evidence) constitute weak evidence. More people claiming the same thing adds weight, but is still not solid evidence, since almost any approach – useful or useless – has enthusiastic believers.

Stronger evidence would be provided from a study of a group of patients carefully assessed in an open trial – but since both the patient and physician are aware of the nature of the therapy, the results can be affected by bias and by the placebo effect. In addition, in an open trial patients may drop out if they do not do well or feel worse, so the therapist accumulates all the good responders and may have the impression that most of the treated patients do well.

Defining how patients will be treated and assessed over a future period – a prospective trial – is stronger evidence than a retrospective trial – a study that looks back in time to see how people fared from a treatment.

Monumental advances have been made in the last half century in the development of a scientific approach to measurement of effectiveness versus risk for a therapy. The most obvious is the development of the double-blind, placebo-controlled, randomised, clinical trial (RCT). The randomised clinical trial can be used to study new drugs, but also surgical procedures and other methods of care.

Trials can be made stronger by using a placebo control group to compare with the group on the proposed therapy. The two groups are matched for as many factors as possible (similar age, sex, duration, type, and severity of disease, and any other factors that seem important), and by keeping the therapist and the patient blinded as to whether they are receiving the treatment or the placebo. Only an overseeing safety committee is able to access information as to what patients are taking in the trial.

Many alleged treatments can be put forward as beneficial for patients with MS, but it is humbling to realise that what seems like a good idea may not be helpful, and may even be harmful. Some approaches may seem to be helpful because we want them to be, and we take any sign of improvement to be due to the treatment. We are realising that no matter how objective we may try to be, our biases and our hopes influence what we think we are seeing. Clinical trials are designed to reduce bias as much as possible.

When using a therapy, the benefit seen may be due to the placebo effect, which is a real effect that has been shown to be significant in therapies used for MS. The interesting and complex nature of the placebo effect is now better understood, and the design of clinical trials allows for assessment of a study drug in comparison to the effect one might see with a placebo.

Clinical trials also use statistical methods to indicate how many patients need to be studied in equal groups and for how long, in order to demonstrate a significant effect if there is one. The process of randomisation of patients assures that the groups to be compared, one on the study drug and the other on a similar appearing placebo, are similar on defined characteristics.

In recent years, a technique called meta-analysis has added even stronger evidence about therapies. Meta-analysis takes all of the well designed trials that meet defined criteria and analyses their results in order to reach a conclusion about the safety and effectiveness of a treatment.