Guideline 5: Report your results using a multilevel reporting scheme featuring written and oral reports.

If you design and carry out your evaluation study following the first four guidelines, you will have an intellectually manageable set of evidence—primarily student pretest and posttest data—bearing on a modest number of important program-relevant decisions. Your task at reporting time is to present that evidence to decision makers in a form most likely to influence the decisions they need to make.

An appropriate level of detail

Before reporting your evaluation study's results, you will typically find yourself in a dilemma over the suitable level of detail to include. To report concise results for busy decision makers, you would invariably need to leave out important information about such matters as the specific procedures used to assure student anonymity. On the other hand, if you chose to report the evaluation's procedures in full detail, the resulting report would often be so lengthy that decision makers would be put off by its size.

Fortunately, there is a way out of this bind: prepare two written reports. The first report should be very brief (no more than a few pages in length) and should hit only the high points namely, the evidence that bears most directly on the decisions at issue. This brief executive report should direct readers who wish more information to a technical report that describes the evaluation study's procedures and results in greater detail. Even with the more technical report, however, you must employ good sense regarding the level of detail acceptable to the decision makers you are attempting to serve. Too often, evaluators become caught up in the intricacies of their evaluation study's procedural nuances and tend to create excessively lengthy reports. Evaluation reports perceived as hyper-detailed will rarely be read by anyone except the evaluators who prepared them. Thus, even the technical report should be sufficiently succinct and focused so that decision makers will be inclined to use it.

In any evaluation report, try to use visual and/or graphic methods to make the results as palatable to readers as possible. Few decision makers relish the prospect of reading even three pages of single-spaced prose. Although it may be difficult, particularly in the executive report, use white space and graphic presentation techniques that stimulate the reader's interest.

Return to top

Increasingly, educational evaluators are being asked to supplement written evaluation reports with oral presentations to, for example, a district's school board or the teachers staffing the district's HIV education program. Such sessions provide you with an excellent opportunity to educate decision makers about the impact of your study's results on the decisions they face. The give-and-take discussion that often follows an evaluator's oral report is a wonderful forum for such educative efforts.

Be sure to devote enough preparation time to make your oral reports polished, professional, and decision relevant. If you are only asked to give a written report, encourage decision makers to allow a brief oral presentation highlighting the study's key results.

Return to top

Making recommendations

Another issue you are apt to face when you make your report is whether to offer recommendations to decision makers. Suppose the pretest-posttest results of a program improvement evaluation study regarding a one-week unit dealing with refusal skills indicate that the unit was particularly ineffective. Students' skills after the unit are essentially the same as before the unit. A logical recommendation would be that the unit be seriously overhauled. But should you make such a recommendation? Similarly, if your program-continuation evaluation study indicates that an HIV education program is having a decisively beneficial impact on reducing students' HIV-risk behaviors, should you recommend that the program be continued?

Evaluation specialists are divided over this issue. For some specialists, making decision-related recommendations is a logical extension of the evaluator's decision-facilitation role. Other specialists, however, regard evaluator-generated recommendations as intrusions on the decision maker's prerogatives. These individuals believe that the evaluator should supply evidence only and should offer no guidance regarding program decisions.

It is suggested, therefore, that you be guided by decision makers' expressed preferences. You will doubtless have met with decision makers during the early stages of designing your study, for you clearly need to find out what their important decision points really are. At that stage of the process, you can easily learn whether decision makers wish your report to include recommendations.

If you present an oral report, your recommendations will often be solicited even though you may have been directed to avoid such recommendations in writing. Be prepared to respond to such requests.

Return to top

Final thoughts about Guideline 5

This final guideline may appear to involve substantial effort. After all, not only are two written reports to be authored, but an oral report is to be made as well. Any effort associated with reporting an evaluation study's results, however, will usually be well worth it. What good does it do to design and carry out a first-rate evaluation study if the results make little impact on the decision makers for whom it was originally conducted?

Reporting an evaluation's results should not be an afterthought. From the earliest days of the evaluation study, you should continually think about how the study's results can be most effectively communicated.

Although Guideline 5 does not directly address the topic of making recommendations regarding the decisions at issue, you will find that if decision makers request your recommendations, they will typically be influenced by your views. If you offer recommendations without being asked, however, your advice may be seen as presumptuous and may be rejected. Be guided by the decision makers' preferences.

Return to top

Back to Booklet 1 Table of Contents