By Don Zimmerman, Ph.D., Principal, Communication Science, Inc. 

No longer can professionals produce communications without capitalizing on evaluations to help understand their audiences and send better messages to their target audiences.

Daily, your audiences face an information tsunami—they must read, listen to, watch, and view hundreds of messages.

Evaluations can help you target and improve your communications (both individual media products and media/information campaigns).

The following discussion describes formative and summative evaluation.

Formative evaluations include:

• Identifying key audiences
• Assessing audiences’ characteristics
• Testing of draft communications and campaigns

To illustrate, consider how we evaluate prototype (draft) websites. Our approach includes:

• Asking what will be the target audience’s first impression of the prototype website?
• Determining how likely users will be to continue using the website
• Identifying problems users encounter, such as
  • Are the fonts large enough, legible, and easy to read?
  • Can users find the information they seek?
  • Asking do they understand the terminology used?

Based on our findings and working collaboratively with our clients, we then recommend redesigns to improve websites.

Consider a formative evaluation example. We evaluated the distribution of a food-safety brochure in supermarkets. First, we tested brochures with high and low-risk messages and opted to use a low-risk food safety message for a field experiment. Next, we developed a kiosk displaying a colorful poster, interactive computer program, and a holder for the brochures.

Then, we placed the kiosk where shoppers exited the produce (fresh fruit & vegetable) section in two different supermarkets. During a month’s time, about 100,000 shoppers passed the kiosks, and yet, only 65 shoppers picked up a brochure. We concluded that distributing food-safety information in a supermarket may not be the most effective way of communicating food-safety information to shoppers.

In contrast to formative evaluations, summative evaluations assess the effectiveness of existing communications. While the objectives vary from project to project, we ask questions such as

• Did members of the target audience see or pay attention to the communications? o If so, do they watch, read, or listen to the message?
• Did they understand the message?
• Did it change their information levels? Behaviors? Attitudes?

Consider a summative evaluation example. A local condominium developer had slow sales. He asked us to evaluate his information communication campaign. He was spending $20,000 annually advertising his condos on one local radio station.

We surveyed 60 customers who had bought his condos, and 60 people who had bought competing condos. We asked them which radio stations if any, they listened to. Only five of the 120 respondents reported listening to any one radio station—including the station the developer was spending $20,000 annually. Surprisingly, more than 50% of our respondents reported seeing the developer’s highway billboard as they entered town.

The condominium developer was wasting advertising money buying ads on a single radio station.

Why use summative evaluations?

The lessons learned from summative evaluations can determine whether your target audience receives your messages and then identifies the strengths and weaknesses of your message(s). Lessons learned from evaluations can guide your selection and development of future communications.