If we want more evidence-based practice, we need more practice-based evidence.*

CHAPTER 7:
EVALUATION AND THE ACCOUNTABLE PRACTITIONER

Table of Contents

Headlines, Timelines, & Milestones

Archived Headlines

Supplementary References

Books and Reviews

Other Reviews

Supplementary Resources

 


Headlines, Timelines, & Milestones


Archived Headlines


Supplementary References

 

Abubaker, WA., Al-Assaf, A.F., & Cleaver, V.L. (1999). Quality assurance and iron deficiency in Egypt. INTERNATIONAL JOURNAL FOR QUALITY IN HEALTH CARE 11(2): 163-168.
    ABSTRACT: Objective. To develop an intervention in rural Egypt to address the problem of iron deficiency anemia and to demonstrate the effectiveness of applying quality assurance (QA) methods in combating this. Design. Assessment of an intervention study utilizing QA methods. Setting. Rural primary care clinics in Egypt. Study Participants. One hundred and eighty pregnant mothers (and their 180 children) were randomly selected at two clinic sites from all those who were diagnosed as having an iron deficiency disorder. Intervention. Multi-disciplinary teams were formed to develop and deliver health promotional approaches related to iron deficiency to the study participants. By using QA techniques the teams were able to strengthen local capacity and participant compliance to the educational messages. Main outcome measures. Pre- and post-measurements of client satisfaction, results of hemoglobin lab tests, and the extent of retention of nutritional messages by the participants. Results. Eighty percent of the study population demonstrated satisfactory knowledge of the nutritional messages. There was a 75% improvement of client satisfaction with the clinic and an effective follow-up system of care was designed and implemented successfully for each clinic. On average, the number of children aged less than 5 years diagnosed with an iron deficiency disorder decreased from 37% to 5%. Similar success was achieved with the pregnant mothers: the prevalence of iron deficiency anemia was reduced from 100% to only 14%. Conclusions. The use of QA process improvement techniques was extremely effective in reducing iron deficiency anemia among the target population. There is an increasing need to include quality methods in micronutrient intervention techniques.

Backett-Milburn, K. & McKie, L. (1999). A critical appraisal of the draw and write technique.  HEALTH EDUCATION RESEARCH 14(3): 387-398.
    ABSTRACT: http://her.oupjournals.org/cgi/content/abstract/14/3/387 .

Berg-Smith, S. M., Stevens, V. J., Brown, K. M., Van Horn, L., Gernhofer, N., Peters, E., Greenberg, R., Snetselaar, L., Ahrens, L., & Smith K. (1999). A brief motivational intervention to improve dietary adherence in adolescents. HEALTH EDUCATION RESEARCH 14(3): 399-410.
    ABSTRACT: http://her.oupjournals.org/cgi/content/abstract/14/3/399 .

Card, Josefina J. (1999). Teen Pregnancy Prevention: Do Any Programs Work? ANNUAL REVIEW OF PUBLIC HEALTH 20: 257-286.
    ABSTRACT/Full-Text:
ABSTRACT.

Connell, C. M. (1999). Older adults in health education research: some recommendations.  HEALTH EDUCATION RESEARCH 14(3): 427-431.
    ABSTRACT: http://her.oupjournals.org/cgi/content/abstract/14/3/427 .

Davis, S.M., & Reid, R. (1999). Practicing participatory research in American Indian communities. AMERICAN JOURNAL OF CLINICAL NUTRITION 69 (4 SUPPL): 755S-759S.
    ABSTRACT: The purpose of this article is to explore the historical issues that affect research in American Indian communities and examine the implications of these issues as they relate to culturally sensitive, respectful, and appropriate research with this population. Methods include review and analysis of the literature and examination of our collective experience and that of our colleagues. Recommendations are given for conducting culturally sensitive, participatory research, We conclude that research efforts must build on the establishment of partnerships between investigators and American Indian communities to ensure accurate findings and analyses and to implement culturally relevant benefits.

Eades, S.J., & Read, A.W. (1999). The Bibbulung Gnarneep Project: practical implementation of guidelines on ethics in Indigenous health research. MEDICAL JOURNAL OF AUSTRALIA 170 (9): 433-436.
    ABSTRACT: Human rights and ethical principles justify Indigenous control of health research conducted in Indigenous communities, but also emphasize the need for scientific rigour, knowledge and expertise, much of which currently resides with the non-indigenous community In 1994, we began development of the Bibbulung Gnameep* Project on Aboriginal maternal and child health in Perth, Western Australia. We describe the process of developing a protocol to ensure ethical conduct of this research through consultation and negotiation between Aboriginal and non-Aboriginal people involved in the project.

Eyler, A.A., Mayer, J., Rafii, R., Housemann, R., Brownson, R.C., & King, A.C. (1999). Key informant surveys as a tool to implement and evaluate physical activity interventions in the community. HEALTH EDUCATION RESEARCH 14 (2): 289-298.
    ABSTRACT: Key informant surveys are important tools for planning and evaluating community health programs. A survey was conducted to gather views on policies toward physical activity from four sets of key informants: physicians, church leaders, business leaders and civic leaders. Surveys were mailed to 797 key informants who were selected from 12 southeastern Missouri counties. For comparison, data from a telephone survey of 2106 persons in the general population were also analyzed, The majority (>85%) in all four key informant groups were very supportive of required physical education in schools, but less supportive (<69%) of government funding for places where community members can exercise. Physicians perceived community members as having somewhat greater access to places to exercise relative to the other key informant groups. Comparisons of the key informant surveys to the population survey indicated similar levels of support for physical activity policy. The information from this survey has been useful in identifying support for physical activity policy and gaining access to potential influences for community change, Since key informant research in the area of physical activity policy and cardiovascular disease prevention is sparse, there is a need for future studies.

Graham, J. D., Corso, P. S., Morris, J. M., Segui-Gomez, M., & Weinstein, M. C.  (1998). Evaluating the cost-effectiveness of clinical and public health measures.  ANNUAL REVIEW OF PUBLIC HEALTH 19: 125-152.
    ABSTRACT: http://publhealth.annualreviews.org/cgi/content/abstract/19/1/125.

Green, J., & Tones, K. (1999). Towards a secure evidence base for health promotion. JOURNAL OF PUBLIC HEALTH MEDICINE 21 (2): 133-139, June.
    ABSTRACT: The importance of a sound evidence base for health promotion is recognized. Criteria for establishing evidence have, in the past, been heavily influenced by the bio-medical agenda. The problems in using experimental and quasiexperimental approaches and their limited relevance for evaluating health promotion interventions are examined. The multifactorial nature of the health promotion enterprise in relation to both inputs and outputs is recognized and a range of direct, indirect and process indicators are identified. The relevance of combining quantitative and qualitative methods for data collection is discussed and the paper concludes by advocating a 'judicial principle' for assessing evidence.

Jacob, P. III, Yu, L., Shulgin, A. T., & Benowitz, N. L. (1999). Minor tobacco alkaloids as biomarkers for tobacco use: Comparison of users of cigarettes, smokeless tobacco, cigars, and pipes. AMERICAN JOURNAL OF PUBLIC HEALTH 89 (5): 731-736.
    ABSTRACT: Objectives. This study (1) determined levels of various tobacco alkaloids in commercial tobacco products; (2) determined urinary concentrations, urinary excretion, and half-lives of the alkaloids in humans; and(3) examined the possibility that urine concentrations of nicotine-related alkaloids can be used as biomarkers of tobacco use. Methods. Nicotine intake from various tobacco products was determined through pharmacokinetic techniques. Correlations of nicotine intake with urinary excretion and concentrations of anabasine, anatabine, nornicotine, nicotine, and cotinine were examined. By using urinary excretion data, elimination half-lives of the alkaloids were calculated. Results. Alkaloid levels in commercial tobacco products, in milligrams per gram, were as follows: nicotine, 6.5 to 17.5; nornicotine, 0.14 to 0.66; anabasine, 0.008 to 0.030; and anatabine, 0.065 to 0.27. Measurable concentrations of all alkaloids were excreted in the urine of most subjects smoking cigarettes, cigars, and pipes and using smokeless tobacco. Correlations between nicotine intake and alkaloid concentrations were good to excellent. Conclusions. Anabasine and anatabine, which are present in tobacco but not in nicotine medications, can be used to assess tobacco use in persons undergoing nicotine replacement therapy. AUTHOR ADDRESS: P Jacob III, Univ Calif San Francisco, San Francisco Gen Hosp, Bldg 100,Room 235, San Francisco, CA 94110 USA

Jeffery, R. W., & French, S. A. (1999). Preventing weight gain in adults: The pound of prevention study. AMERICAN JOURNAL OF PUBLIC HEALTH 89 (5): 747-751.
    ABSTRACT: Objectives. This study examined whether weight gain with age could be prevented through the use of a low-intensity intervention. Methods. Participants, 228 men and 998 women recruited from diverse sources, were randomized to one of the following groups: (1) no-contact control, (2) education through monthly newsletters, or (3) education plus incentives for participation. All participants were weighed and completed questionnaires about behaviors and attitudes related to weight at baseline and annually for 3 years thereafter. Results. Individuals in intervention groups reported favorable changes over time in frequency of weighing and healthy dieting practices relative to those in the control group. These behavior changes were in turn related to a reduced rate of weight gain over time. However, weight gain over 3 years did not differ significantly by treatment group. Conclusions. This low-intensity educational approach to weight gain prevention sustained interest over a lengthy time period and was associated positively with behavior change, but it was not strong enough to significantly reduce weight gain with age.

Kawakami, N., Haratani, T., Iwata, N., Imanaka, Y., Murata, K., & Araki, S. (1999). Effects of mailed advice on stress reduction among employees in Japan: A randomized controlled trial. INDUSTRIAL HEALTH 37 (2): 237-242. KEYWORDS: stress reduction; breakfast; mental health; health education; intervention study.
    ABSTRACT: We conducted a randomized controlled trial (RCT) to examine the effects of mailed advice on reducing psychological distress, blood pressure, serum lipids, and sick leave of workers employed in a manufacturing plant in Japan. Those who indicated higher psychological distress (defined as having GHQ scores of three or greater) in the baseline questionnaire survey (n=226) were randomly assigned to an intervention group or a control group. Individualized letters were sent to the subjects of the intervention group, informing them of their stress levels and recommending an improvement in daily habits and other behaviors to reduce stress. Eighty-one and 77 subjects in the intervention and control groups, respectively, responded to, the one-year follow-up survey. No significant intervention effect was observed for the GHQ scores, blood pressure, serum lipids, or sick leave (p>0.05). The intervention effect was marginally significant for changes in regular breakfasts and daily alcohol consumption (p=0.09). The intervention effect was marginally significant for the GHQ scores among those who initially did not eat breakfast regularly (p=0.06). The study suggests that only sending mailed advice is not an effective measure for worksite stress reduction. Mailed advice which focuses on a particular subgroup (e.g., those who do not eat breakfast regularly) may be more effective.

Kaneko, M. (1999). A methodological inquiry into the evaluation of smoking cessation programmes. HEALTH EDUCATION RESEARCH 14(3): 433-441.
    ABSTRACT: http://her.oupjournals.org/cgi/content/abstract/14/3/433 .

Linnan, L. A., Fava, J. L., Thompson, B., Emmons, K., Basen-Engquist, K., Probart, C., Hunt, M. K., & and Heimendinger, J. (1999). Measuring participatory strategies: instrument development for worksite populations. HEALTH EDUCATION RESEARCH 14(3): 371-386.
    ABSTRACT: http://her.oupjournals.org/cgi/content/abstract/14/3/371 .

McBride, C.M., Curry, S.J., Lando, H.A., Pirie, P.L., Grothaus, L.C., & Nelson, J.C. (1999). Prevention of relapse in women who quit smoking during pregnancy. AMERICAN JOURNAL OF PUBLIC HEALTH 89 (5): 706-711.
    ABSTRACT: Objectives. This study is an evaluation of relapse prevention interventions for smokers who quit during pregnancy. Methods. Pregnant smokers at 2 managed care organizations were randomized to receive a self-help booklet only, prepartum relapse prevention, or prepartum and postpartum relapse prevention. Follow-up surveys were conducted at 28 weeks of pregnancy and at 8 weeks, 6 months, and 12 months post-partum. Results. The pre/post intervention delayed but did not prevention postpartum relapse to smoking. Prevalent abstinence was significantly grater for the pre post intervention group than for the other groups at 8 weeks (booklet group, 30%, prepartum group, 35%, pre/post group 39%; P = .02 [different superscripts denote differences at P < .05]) and at 6 months (booklet group, 26%; prepartum group, 24%; pre/post group, 33%; P = .04) postpartum. A nonsignificant reduction in relapse among the pre post group contributed to differences in prevalent abstinence. There was no difference between the groups in prevalent abstinence at 12 months postpartum. Conclusions. Relapse prevention interventions may need to be increased in duration and potency to prevent postpartum relapse.

McKenzie, J. F., Wood, M. L., & Kotecki, J. E. (1999). Establishing content validity: Using qualitative and quantitative steps. AMERICAN JOURNAL OF HEALTH BEHAVIOR 23 (4): 311-318, July-August.

Slade, M., Thornicroft, G., & Glover, G. (1999). The feasibility of routine outcome measures in mental health. SOCIAL PSYCHIATRY AND PSYCHIATRIC EPIDEMIOLOGY 34 (5): 243-249, May.
    ABSTRACT: Background: Standardised outcome measures are not being used in routine mental health care. Method: The importance of routine use of Standardised outcome measures is argued, and reasons for their lack of use suggested. Results: One reason for Standardised outcome measures not being used routinely is the lack of appropriate instruments. This property of being suitable for routine use is often called feasibility, but there is no consensus about the meaning of feasibility, or how it should be measured. We propose a definition of feasibility as a psychometric property of a Standardised outcome measure, provide criteria for assessing feasibility, and then present a framework for changing practice to increase the routine use of Standardised outcome measures. Conclusions: If mental health care is to maximize outcome, then more attention needs to be paid both to the process of developing and to facilitating the routine clinical use of feasible outcome measures.

St James, P.S., Shapiro, E., & Waisbren, S.E. (1999). The resource mothers program for maternal phenylketonuria. AMERICAN JOURNAL OF PUBLIC HEALTH 89 (5): 762-764.
    ABSTRACT: Objectives. The purpose of this study was to measure the effectiveness of resource mothers in reducing adverse consequences of maternal phenylketonuria. Methods. Nineteen pregnancies in the resource mothers group were compared with 64 pregnancies in phenylketonuric women without resource mothers. Weeks to metabolic control and offspring outcome were measured. Results. Mean number of weeks to metabolic control was 8.5 (SE = 2.2) in the resource mothers group, as compared with 16.1 (SE = 1.7) in the comparison group. Infants of women in the resource mothers group had larger birth head circumferences and higher developmental quotients. Conclusions. The resource mothers program described here improves metabolic control in pregnant women with phenylketonuria.

Wang, M.Q., Collins, C.B., Kohler, C.L., DiClemente, R.J., & Wingood, G. (1999). AIDS knowledge among Black drug users: An issue of reliability. PSYCHOLOGICAL REPORTS 84 (1): 121-124.
    ABSTRACT: This study examined the reliability of an inventory measuring AIDS-related knowledge among 553 black drug users. Data were collected from a socially high-risk community in Birmingham, Alabama. The KR-20 reliability scores were computed from the inventory for each drug-use group. The KR-20 reliabilities ranged from .39 to .57. These findings may suggest that the inconsistent reports of the predictive power of AIDS knowledge among drug users reported in previous studies might be related to the low internal consistencies of the inventory.

    Books and Reviews 

Research on Human Subjects - Ethics, Law and Social Policy (Amsterdam: Elsevier Science, 1999).   D. Weisstub (Ed.), Chair de Psychiatrie Legale et Biomdicale, Universite de Montreal, Quebec, Canada.
    Many would think that 50 years after Nuremberg and 20 years after legislation was first introduced in the USA, all the ethical concerns in medical research would have been resolved. However, as this important volume contends, the legal, ethical, philosophical and cultural issues have never been more complex. What are the ethical implications of experimentation on subjects unable to give competent consent ? What are the opportunities for social and economic fraud ? How has the pressure to compete as a company or a nation affected our view of the ethics of experimentation ? This book addresses the key implications of experimentation on humans.  For further information or to order online, visit the book's homepage at www.elsevier.com/inca/publications/store/6/0/1/0/5/1/index.htt

    Other References

 Healthy People 2000 Review 1998-2000. 246 pp. (PHS) 99-1256 go to: http://www.cdc.gov/nchs/products/pubs/pubd/hp2k/review/review.htm

For more information on the new initiative for the first decade of the new century, Healthy People 2010, go to http://web.health.gov/healthypeople.

Supplementary Resources

The Evaluation Kit, is a nine-volume set of handbooks for planning and executing evaluations of programs from Sage Publishing Co. For more information, visit  http://www.sagepub.com

Big Brothers Big Sisters of America study linking the impact of positive adult relationships with at-risk youth. Information on the study is available from their website, www.bbbsa.org

    In 1992 and 1993, some 959 boys and girls in eight states, ages 10 through 16, entered into a historic experiment. Half the children were matched with a Big Brother or Big Sister, while the other half were assigned to a waiting list, or control group. On average, the matched children met with their Big Brothers or Big Sisters about three times a month for at least a year.

Researchers found that 18 months later, the Little Brothers and Little Sisters were:

  • 46 percent less likely to begin using illegal drugs
  • 27 percent less likely to begin using alcohol
  • 53 percent less likely to skip school, and
  • 37 percent less likely to skip a class
  • more confident of their performance in schoolwork
  • less likely to hit someone, and
  • getting along better with their families.

 

Slide presentation on indicators for development communications evaluation. A major challenge in evaluation is agreement on the set of core indicators on which to collect information. Those indicators need to provide short-term feedback that predicts long term change and relates the communication intervention to that change. The Network has begun the process of applying itself to this task. 6 general indicators have been developed. 3 of these are: expanded public and private dialogue and debate; increased accuracy of the information that people share in the dialogue/debate; supported the people centrally affected by an issue[s] voicing their perspective. These provide a strong social change perspective to balance the established behavioral approaches to evaluation. Full presentation: http://www.comminit.com/power_point/measure_eval

 

Course on Ethical Issues on International Health Research
http://www.hsph.harvard.edu/bioethics/