[india-drug] Evaluation methodology of interventions

[Sunitha Srinivas <sunitha9srinivas@yahoo.com>]

(Enclosed is an example of Quasi-experimental design, discussed during
the training imparted in the course of Promoting Rational  Drug Use in
Community. Idrug participants who have conducted such studies can
kindly share the information in the Indian context, to keep the
discussion moving forward. Thanks!?SS)


Time-series design
---------------------------

In some cases a study design using controls is not possible. This is
the case, for example, when you implement a mass media campaign. The
whole population is then reached by the intervention. Or you may lack
resources to include a control group in your study. You can then
evaluate your intervention using a time-series design. For this you
collect information on your outcome measure at least three times:
before the intervention, and at least twice after the intervention.
This method is descriptive and does not provide strong scientific
evidence on the effectiveness of your intervention. When you have no
control groups, it is especially important to look carefully at what
changes have occurred which could provide alternative explanations for
observed changes in outcome measures. For this you need to develop a
conceptual framework which lists the factors affecting your outcome
measurement. By means of multivariate analysis (ask a statistician for
advice) you can determine which factors (including your intervention)
are correlated with the changes observed. You can also assess the
effect of interventions qualitatively by interviewing the target
audience on why they changed their behaviour ? was it because of the
interventions or were there other reasons?

The examples from Kenya below give the results of two intervention
studies using time-series designs.


I. Changing home treatment of childhood fevers by training shopkeepers
in rural Kenya.
This intervention, aimed to improve the treatment of childhood fevers,
took place in a malaria endemic area in Kenya. Research has shown that
the majority of early treatments of childhood fevers are self-medicated
with shop-bought, brand name drugs. These treatments are usually
incorrect or sub-optimal.
The intervention and its objectives: The aim of the intervention was to
train shopkeepers who sell drugs in Kenyan communities in giving advice
on the type and quantity of drugs to buy for childhood fevers, and on
how to use them. The ultimate objective was to improve the use of
antipyretic and antimalarial drugs in childhood fevers. Shopkeepers
were trained at a series of three workshops, each lasting three days.
The methods used encouraged active participation, practical training
and skill development. Shopkeepers were provided with dosage charts for
chloroquine and aspirin/paracetamol-based drugs, and sets of rubber
stamps depicting the correct way of using chloroquine in children of
different ages.
Evaluation methodology: The impact of the training programme was
evaluated in two rounds of observational studies and home interviews
during peak malaria seasons.
Results: Before the training workshops 32% of antimalarial sales
included an adequate dose of antimalarials. After the workshops this
percentage increased to 83% three months after the intervention and
then to 90% seven months post-intervention. Before the training, advice
was only given in 2% of antimalarial sales. This increased to 94% and
98% in the two subsequent observation rounds post-intervention. The
home interviews revealed that only 4% of childhood fevers treated with
chloroquine were given an adequate dose of chloroquine before the
training. This increased to 65% three months after the intervention and
75% seven months later. Appropriate dispensing and safe use of aspirin
also increased after the intervention. The researchers evaluated the
process and found major changes in the way the shopkeepers sold their
drugs and that the community viewed the changes positively.


II. Integrating research and evaluation in Kenya
The Youth Variety Show (YVS) in Kenya, a radio call-in for young people
on the subject of sexual behaviour, was guided by intensive formative
and evaluative research. This included: a national baseline survey of
youth and parents (6,300 interviews); focus group discussions with more
than 350 adolescents and parents in 5 districts; in-depth interviews
among leaders and gate keepers; a review of legislation and policy
environment; content analysis of newspaper coverage of youth issues;
and, once the programme started, content analysis of letters from young
people. During the radio broadcast, a panel of young people and a
separate panel of parents listening to the show carried out monitoring.
 Their critiques were used to improve the content of the next
programme.
The intervention and its objectives: The intervention aimed to increase
adolescent knowledge on sexual health matters, and encourage
adolescents to go to reproductive health clinics for their sexual
health needs.

Evaluation methodology: Evaluation was done through a follow-up
household survey conducted among adults and adolescents to assess
audience exposure to the YVS. This was conducted by a market research
firm that carries out omnibus surveys in the commercial sector several
times a year.  John Hopkins University Center for Communication
Programs bought some questions as part of this ongoing survey.
Results: Results showed that 38% of respondents listened to YVS but of
l5-24 year olds 55% listened.  Sentinel site surveys at clinics showed
that increasing numbers of adolescents attending the clinics had
listened to YVS and, along with friends, YVS was the most important
source of referral. Content analysis of letters and radio listener
panel studies corroborated this. 

Marsh VM et al. (1999) Changing home treatment of childhood fevers by
training shopkeepers in rural Kenya. Tropical Medicine and
International Health, 4(5):9999 383-389.




________________________________________________________________________
Missed your favourite TV serial last night? Try the new, Yahoo! TV.
       visit http://in.tv.yahoo.com
The INDIA-DRUG discussion group is a partnership between SATELLIFE
(www.healthnet.org), WHO Essential Drugs and Medicines Policy
(www.who.ch), and the Delhi Society for the Promotion of the
Rational Use of Drugs (DSPRUD) in India.

To send a message to india-drug, write to: india-drug@usa.healthnet.org
To subscribe or unsubscribe, write to: majordomo@usa.healthnet.org
in the body of the message type: subscribe india-drug OR unsubscribe india-drug
To contact a person, send a message to: india-drug-help@usa.healthnet.org