Home > The not so SMART study > The Not So SMART Study

The Not So SMART Study

I have borrowed this title from a comment in the journal, Lancet Infectious Diseases, entitled “Not so Smart?” by Justin Stebbing and Angus Dalgleish.

The SMART study as many will recall was a randomized comparison of two antiretroviral treatment strategies.

HIV infected individuals were randomized to receive either  continuous antiviral treatment or to receive it intermittently while the CD4 count had fallen below 250. This trial received a tremendous amount of publicity.  Deaths from all causes – including those that were not obviously related to HIV infection, were significantly increased in the group that were treated intermittently.  This seemed to dampen enthusiasm for treatment interruptions and brought attention to a possible relationship between HIV infection and deaths from causes previously not associated with it.

5,472 patients participated in this study at 318 sites in 33 countries.

There were a total of 85 deaths in the study.

79 of these 85 deaths occurred in the US where 55% of the patients were randomized.

There were only 6 deaths among the 45% of patients randomized in countries outside the US.


It would seem that treatment interruptions are quite safe, as long as they occur in countries outside the US.

Did I miss this information in the original report of the study published in 2006?

There were numerous discussions of the SMART study on websites and newsletters addressed to HIV infected people and their health care providers.  Did I miss those that reported on the fact that only 6 of the 85 deaths occurred in countries outside the US?

Of course I looked at the original report again but could not find this information – perhaps it was buried in a supplementary appendix?

For some reason, it seems that the authors of the report on the SMART study did not feel it necessary to draw attention to this information – at least not with the prominence that it deserved, if it was mentioned at all.

Most of the deaths on the study were not from AIDS associated opportunistic infections or malignancies.

With a presumption (maybe this  suggestion is too harsh) that despite this, the deaths were indeed related to HIV,  a possible relationship with this virus was sought. One obvious possibility of connecting these deaths with HIV was by linking them with the inflammation that is associated with HIV disease.

Thus, as a follow up to the SMART study, various markers of inflammation were looked at in both groups, and not surprisingly these were increased in the group with the most deaths, those receiving intermittent treatment rather than continuous treatment. As mentioned almost all of these deaths were confined to the US.

So, what we have is the observation that people who were to die within a relatively short period had increases in markers of inflammation. Of these, D-dimer, CRP and IL 6 had already been associated with all cause mortality, even in people not infected with HIV.

With respect to the cardiovascular deaths in the study, here is a quotation from PM Ridker:  “In apparently healthy men, elevated levels of IL-6 are associated with increased risk of future MI. These data thus support a role for cytokine-mediated inflammation in the early stages of atherogenesis”.

IL-6 is a pro inflammatory cytokine and levels were increased in those receiving intermittent therapy.  IL-6 promotes HIV replication,  and can be produced by HIV infected cells but also by many other stimuli.

So IL 6, which is associated with atherogenesis,  also directly increases the replication of HIV.  IL 6 secretion is increased by numerous and diverse factors. For example bacterial toxins induce IL 1 which in turn stimulates IL 6 release and  hepatitis C virus core proteins induce IL 6.  HIV infected cells can also release IL 6.

But with so many different agents able to do this it is difficult to attribute IL 6 production to HIV.  This is of course muddied by the fact that whatever stimulates IL 6 secretion, IL 6 itself will accelerate the replication of HIV.

But possibly the most intriguing feature of the report of this follow up study  is the first sentence of the Results section:

“Most of the deaths (79 of 85) occurred in the US”.

Having made this rather startling statement, the authors never return to it.  It remains undiscussed,  as if it is of no consequence!

Are we to believe that intermittent therapy with antiviral agents  promotes inflammation with its lethal consequences only in the US?

The outcome measurement of the SMART study included death from all causes. Only 8% were the result of opportunistic disease.

There were 16 deaths from cancer ( 11 in the intermittent therapy(IT) group and 5 in the continuous therapy group(CT)), 11 deaths from cardiovascular   disease  (7, IT, 4,  CT);

8 people died from substance abuse, 7 from violence.

18 deaths were from causes that could not be determined. Of these 18, 15 occurred in those on intermittent treatment and only 3 in those receiving continuous treatment.  This last rather large difference leads one to ask if it is possible that the two groups were treated differently. At least, in the US, where almost all the deaths occurred.

This may seem like an outrageous question. But unintentional bias in unblinded studies cannot be ignored and I will return to this.

Many of the deaths reported –  certainly far from all, were caused by  conditions that might have been ameliorated by appropriate medical care ( this does not only mean from the point of view of the physician. The patient is also involved – for example, were medical visits made? Did the patient pay attention to symptoms? Was there compliance with prescribed treatment?)

With almost all of the mortality confined to the US,  it looks like something else must be at play here, something other than the antiretroviral treatment strategies, and the first place to look is the overall quality of medical care – which,as mentioned, includes issues that may entirely be related to the patient – such as poor compliance with recommendations, despite adequate support.

There are two distinct  questions to be asked.

Firstly,  why was there such  a difference  in  the trial outcomes between US and non US sites?

Secondly, in the US can we reliably attribute the differences in outcomes in the two treatment arms to the differences in the antiviral treatment strategies?

The first two questions one would ask in trying to explain the difference between 6 deaths and 79 deaths is related to the quality of general medical care in the US as compared to the non US countries, and then to possible differences in the patient populations.   The patient populations may have differed for example in the extent of co- morbidities,  and in the degree of compliance with recommended treatments.

But  I don’t know that one can come up with an answer about the quality of medical care.   We must assume that there were probably no great differences.  However there was some information on co- morbidities such as Hepatitis b and C,  but not enough to attribute the differences in the number of deaths to this factor.  [Note`added on April 4 2010. The difference in co-morbidities is in fact probably  the reason for the striking difference in mortality between US and non US sites. Here is a link to a later post where a table is reproduced  from the paper describing the mortality difference referenced below. The population enrolled in US sites, where most of  the deaths occurred,  were much more likely to suffer from non HIV related health problems than those enrolled in non US sites.  Here are two sentences from the later post:  ”The study was conducted in US sites on what appear to have been a group of individuals in whom disorders unrelated to HIV were overrepresented.  As mentioned earlier, these disorders include diabetes, hepatitis B and C, high blood pressure and a history of heart disease:

I conclude with a few more comments on the SMART study with a possible explanation for the huge discrepancy in the number of deaths in US sites, 79, compared to only 6 in non US sites.   At least there is a very clear reason why the results observed in this study should not be generalized to all HIV infected individuals”. This current entry on the SMART study, which I’m leaving unchanged,  should be looked at in conjunction with my subsequent post. LINK TO LATER POST ]

The study was conducted in US sites on what appear to have been a group of individuals in whom disorders unrelated to HIV were overrepresented.  As mentioned earlier, these disorders include diabetes, hepatitis B and C, high blood pressure and a history of heart disease.

I doubt if information on compliance is available.

Even if one could show differences between the US and non US sites ,  how would  this  affect the study outcome?  More people died in the intermittent treatment arm compared to those receiving continuous treatment. So this is the second question.

Could there be an explanation for the differences noted between the study arms (albeit only in the US) other than the antiviral treatment strategies?

There could be a connection with general patient care.

In order to minimize bias in a study, where possible when treatments are compared, participants and those conducting the trial do not know what treatment is being received by particular participants.

The study is blinded, so that as far as it is possible, we can attribute any effects observed to the treatment, not to any anything else.  For example, if patients knew what they were or were not receiving in a treatment trial, they may behave differently, and  in ways that may affect the outcome, which then could not be attributed to the particular treatment being studied.

For example if a patient knew they were receiving a placebo, they may then take other medications that might affect the outcome of the trial, or if doctors knew patients were taking a medicine they believed worked they might treat their patients with greater care or with less care.  We do recognize that some behaviors that may alter the outcome of a study are  certainly not intentional.

It was impossible to blind the SMART study.  So, both participants and physicians knew which arm of the study patients were randomized to.

If the study doctor was also the person who provided  general  care than the specter of bias unfortunately is lurking and may confound interpretations.

This is not to say that differences in general care between both study arms, if indeed there were differences, were intentional.

To put the questions in another way:

1: Can general patient management strategies ( not the strategy of antiviral treatment being studied) have an impact on all cause mortality?  In other words, can the way health care providers manage the general  health of their patients make a difference to survival?

2: Can bias influence the ways physicians take care of their patients?

The answer is of course yes., although we may not like to admit this.    So bias might be a factor in an unblinded study and affect the outcome.

So we are still in the dark regarding regarding the value or danger  of treatment interruptions.

As a postscript, a similar problem hangs over the original AZT study – the study that led to the approval of this drug by the FDA.  Of course the dramatic life saving effect of zidovudine seen in this trial  has never been observed again.

This placebo controlled study was also in effect unblinded. Patients and doctors knew who was receiving placebo or active drug.

Deaths were mostly due to opportunistic infections. Patient management strategies can make the difference between life and death with regard to these infections. Rapid diagnosis, effective treatments obviously make a difference. Can bias influence patient management strategies?

I wrote about this in – I think 1989, and the article can be seen by clicking here.

I suppose that one must conclude that the fact that almost all the deaths in the SMART study occurred in the US was not known to journalists and those who specialize in informing us about issues related to AIDS.  I also missed it when it was  published in 2008 [iii].

The report of Dr Kuller may be the first public mention of this odd result. But it is just mentioned and not discussed at all.

Here is what Justin Stebbing and Angus Dalgleish wrote in the Lancet Infectious diseases about this report:

” The follow-on case-control study by Kuller and colleagues showed that it is apparently safer to be off  HAART outside the USA rather than on HAART within the USA”

As a clinician I don’t know what to make of the SMART results. In the lamentable absence of firm evidence one has to use one’s best judgment in caring for patients.  Numbers of my patients have – at their request and at my recommendation, temporarily interrupted their treatments, using a variety of strategies, with no harm, and with a better quality of life.

I imagine that some will have been persuaded to stop this practice  by their new physicians. But I am still in touch with one, who had a CD4 count of 0 when first seen, who still regularly interrupts his treatment.  He is extremely well, leading an active and productive life.

The Lancet Infectious Diseases, Volume 9, Issue 5, Pages 268 – 269, May 2009

The New England Journal of Medicine [NEJM 355(22): 2283-96 (2006)

PLoS Medicine 5 (10); e203.doi:10.1371/journal.pmed.0050203

Kuller LH, et al. (2008) Inflammatory and Coagulation Biomarkers and Mortality in Patients with HIV infection.

HarrisTB et al 1999  Association of elevated IL6 and CRP levels with mortality in the elderly, Am J Med 106: 506

Ridker PM et al 2000 Plasma concentrations of IL 6 and the risk of future myocardial infarction among apparently healthy men.  Circulation  101 1767

Shorr AF et al 2002 D-dimer corerelates with proinflammatory cytokine levels and oycomes in critically ill patients, CHEST 121: 1262

HIV disease is in fact characterized by multiple examples of positive feedback systems – a subject for another post.

  1. September 17, 2009 at 2:07 am


  1. June 12, 2009 at 4:04 pm

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: