Okay! I've had like 2 hours of sleep in the last 48 hours as I finish this post. So, if there are any glaring grammatical errors, I can't see them because my eyes are kinda crossed. I'll trust my fellow bloggers to fix any obvious ones. :)
Continuing on my series of articles outlining a recently published two-part special issue of the journal Homeopathy on the efficacy of homeopathy, this post is on an article [1] testing the effectiveness of application of homeopathy to chronic primary insomnia.
This research was a double-blind, placebo-controlled study, the primary purpose of which was to "evaluate the efficacy of homeopathic similimum in the treatment of chronic primary insomnia in terms of the patient's perception of the treatment, using a Sleep Diary (SD) and the Sleep Impairment Index (SII).
The Sleep Diary is defined in the study as a daily written record of the subject's sleep patterns, including sleep-wake times, time in bed, estimated period of sleep, quality of sleep, number of sleep interruptions, and daytime naps. The SII is a "7-item measurement tool that yields a quantitative index of sleep impairment." It relies on self-report of the subject's perception of "insomnia, its severity, level of distress and impairment of daytime functioning"
For the study, subjects were recruited and then asked to fill out a SII to provide a baseline measurement of the severity of insomnia symptoms. They were also instructed to record sleep data in the sleep diary for one week to provide a baseline measurement of other sleep variables. Their full homeopathic case history and physical examination were performed. For each subject, the homeopath was allowed to prescribe any homeopathic remedy that he or she felt would be beneficial to the subject. Potencies were not limited, rather the homeopath determined the most suitable remedy. The dosage took the form of
"three single-dose lactose powder satchet per consultation, one of which was dissolved sublinguially each night consecutively before going to sleep… each active powder sachet comprised 10 medicated lactose granules which were placed into the sachets containing lactose powder. The medicated granules were produced in accordance with Method 10 of the German Homeopathic Pharmacopoeia. Lactose granules were triple-impregnated with centesimal potencies of the relevant remedy contained within 96% ethanol… Each placebo powder sachet comprised lactose powder and 10 lactose granules impregnated with 96% ethanol alone and were indistinguishable from the active sachets in appearance and taste"
Impressively, the researchers in this group put a great amount of forethought into the blinding and randomization process. The researcher and research supervisor would discuss each case and determine which homeopathic preparation was most appropriate, based on "repertorisation (RADAR version 9) of the totality of symptoms presented". Then, the prescription was dispensed by an independent dispenser according to the randomization list. So, the researchers and subjects did not know if the subjects were receiving homeopathic preparations or placebos. Participants were instructed to begin taking the medications a week after the initial consultation (presumably so that they could fill out their week-long sleep diary). After two weeks, they returned to the clinic, where they were reassessed. At this point, the homeopath could modify the treatment as needed, so long as treatment still fell within the principals of homeopathy. Again, subjects went to a dispenser for either the homeopathic preparation or a placebo.
The subject group consisted of an initial recruitment of 45 subjects. 12 did not meet the inclusion criteria. Thus, 16 were allocated to the treatment group while 17 were allocated to the placebo group. Of these subjects, 2 were lost to follow up from the treatment group (due to scheduling difficulties and compliance) and one was lost from the placebo group due to scheduling difficulties. This left a total of 14 subjects in the treatment group and 16 in the placebo group.
At the end of the study, researchers analyzed the SD in terms of total hours slept per week. For the SII, subjects graded their sleep symptoms in terms of a 4-point severity scale (none, mild, moderate, severe or very much). The authors stated that they analyzed this data "in the form of summary scores as well as per individual question".
So, let's talk about statistics a little. The researchers set the P-value at .05, meaning that the researchers would be 95% certain that the data they obtained from groups were significantly different from one another. This is important to note because, as Petter so eloquently pointed out, most published research results are wrong, and so a p-value of .05 might be acceptable for preliminary work but is absolutely unacceptable for conclusive work, especially clinical medical studies. The researchers state that the groups did not differ significantly in terms of their baseline measurements of hours per week slept or SII reports.
For the treatment group, the weekly hours slept was as such: Baseline: 35, week2: 45, week3: 43, and week4: 41.
For the placebo group, weekly mean scores for hours slept were: baseline: 34, week2: 32, week3: 35 an week4: 35.
For the SII measures, the treatment group was: baseline: 3.34 week2: 3.14, week3: 1.47.
For the placebo group the SII measure was: baseline: 3.53, week1: 3.41, week2: 3.35
A major problem with these numbers is that while it looks like the treatment group got more mean hours of sleep than the placebo group, the reader is offered no numbers for standard deviation. That means that I can't calculate effect size to see if these numbers have any clinical relevance. For those of you who aren't researchers or statisticians – this is a big deal. One might be able to show that there is a statistical difference between groups, but whether or not this difference is clinically relevant can spell the difference between useless statistical noise and an actual worthwhile treatment. On the surface, those numbers look impressive, but without a standard deviation, we have no way of knowing how distributed the data were – we have no way of knowing, for example, if one or two subjects seriously skewed the data.
Was there a significant difference in any of the other factors of the sleep diary? You'll recall that the SD measured at least six different factors of sleep, yet the authors only chose to report on the mean hours of sleep per week. Not per day, but per week. Why did they not report on the other items in the sleep diary? My guess is that they did not find that any of those factors were significantly different, or perhaps even found that some of those factors were significantly different, but showed that the placebo group had a more positive outcome. We do not know, because they do not say. Regardless, if researchers mention that they measured something, it is prudent that they report on the results. In my experience, journal reviewers frown on the practice of mentioning variables early in a paper and then not reporting on the outcomes of those variables.
Things are a little more complicated for the SII measure. The researchers report contradictory and vague results, stating:
- The SII is a 7-item measurement tool.
- There were "significant improvements in 6/11 questions after 1 week" for both groups
- There were "significant improvements in 10/11 questions" in the treatment group and significant improvements in 4/11 questions in the placebo group for week 2.
- "When comparisons were made between Baseline SII scores and those at Week 4 (trial entry and trial completion), significant improvements in all (11/11) questions within the verum group were observed. Within the placebo group no significant improvements were noted for any of the questions (0/11) over the same period.
It's interesting that a 7-item measurement tool has "11 questions" – what's the deal with that? How is it that there were improvements in the placebo group, but then there weren't? The researchers claim that there was a significant difference between treatment groups between baseline and each follow up consultation. This seems to jive with their numbers (3.34, 3.14, 1.47) but they say that while initial improvement occurred in the placebo group (moving from 3.53 to 3.41) there was "no difference between baseline and final SII scores".
Say what? The scores were 3.53, 3.41 and 3.35. There is clearly a greater difference between baseline (3.52) and week 2 (3.41) and week 4 (3.35), but they say this is non-significant. Why? Also, why is it that they took baseline SD measures and then 3 follow-up SD measures, but only took 2 follow-up SII measures? From my perspective, the data in this study is starting to unravel. Once again, they only give the mean and not the standard deviation, so I am unable to calculate effect size.
The researchers also have a table of the specific homeopathic preparations they used, and to what frequency they were prescribed. They do not say to what potency the preparations are given, just that 23 were given at 30CH, 63 given at 200CH, 38 given at 1M, and 5 given at 10M. They are: Lachesis muta (8), Nux vomica (7), Medorrhinum (5), Sepia officinalis (5), Lycoposium clavatum (4), Carcinosin (4), Sulphur (3), Natrum muriaticum (3), Calcarea carnonica (2), Coffea cruda (2), Ignatia amara (2), Silica terra (2), Mercurius solubilis (1), Arsenicum album (1), Cannabis indica (1), Calcarea arsinicosum (1), Kalium carbonicum (1), Tuberculinum (1), Thuja occidentalis (1) and placebo (5).
Yes, they prescribe placebos intentionally. Apparently this was given as the "second prescription only; only if the first prescriptions was considered to be still acting." I'll let you guys make of this what you will.
One of the more fascinating results of this study is that the placebo group does not mirror the placebo effects of typical drug trials for insomnia. Why is it that the results of the placebo group are so negligible? In this study, the placebo treatment resulted in no significant increase in the duration of sleep and only tiny improvements in the SII scores. Quite, frankly, this is surprising and does not agree with other research on the placebo effect of treatments for insomnia. A strong positive placebo (and nocebo, for that matter) effect has been established in the treatment of primary insomnia [2] As such, I am skeptical that the verum treatment has been adequately compared to a placebo treatment.
I also take issue with the author's statement that homeopathy is a viable treatment for insomnia, given the small size of treatment groups and respective probable lack of power. Of course, since the researchers did not specify the standard deviation, I can't calculate power either. I think the correct conclusion of this study is that given the small sample size, this study is a pilot study that can be said to have determined that further study on the effects effect of homeopathy on the treatment of chronic primary insomnia may be warranted. I think that it would also be valuable to explore the surprising lack of placebo effect in the placebo group, as this is contrary to established literature.
- Naude DF, Couchman IMS, Maharaj A. Chronic primary insomnia: Efficacy of homeopathic simillimum. Homeopathy 2009:99, 63-68
- Perlis ML, McCall WV, Jungquist CR, Pigeon WR, Matteson SE. Placebo effects in primary insomnia. Sleep Medicine Reviews 2005:9;381-389
Labels: research, science, skepticism