INFORMATION DATA BANK Existing Information Case Studies Surveys Written Telephone Interviews Personal Group Observation Mass Media/Public Hearings/Public Forums Existing Information (This might consist of records, receipts, personnel records, changes, reports, census data,  agency and school records, state and national Web sites.)   What it Measures  Advantages Disadvantages Examples Insight into programs that cannot be observed in any other way Readily available Minimal cost Data available on a wide variety of characteristics Can be accessed on a continuing basis Descriptive data   User may need to sort, discriminate and correlate Takes time to find sources Figures may be dated or represent estimates rather than actual accounts Does not reveal individual values beliefs or reasons underlying current trends   Past plans of work are reviewed to determine program trends or changes. Census data are read to note family demographic trends. Back to top Case Studies What it Measures  Advantages Disadvantages Examples Experiences and characteristics of selected persons in a project (generally utilized with a small number of people) Procedures evolve as work progresses No confining categories or classifications Allows depth or insight into relationships and personal feelings Can be effectively used in combination with other methods, such as survey and observation Unique opportunity for in-depth study of organization, group, etc., over time Can be tailored to specific situations   Requires absolute accuracy Can be very subjective Time consuming; requires extensive amounts of data; several cases are needed for best analysis Focus is on a limited number of cases; cannot necessarily be generalized to larger community Not suitable as a method in isolation; best for a background or as a guide to further study Several observation techniques (e.g., sampling) are available, but must be learned   A few participants from each program are visited about their experience. Their colleagues are interviewed. Three families are followed for a six- month period to seek changes.   Back to top Written Surveys What it Measures  Advantages Disadvantages Examples Opinions, attitudes, beliefs, behaviors, reactions and attributes, in response to specific questions   Efficient for volume of information collected People more likely to provide frank, thoughtful, honest information, tension- free situation Gives people more time to complete the questionnaire All respondents receive exact same questions in printed form - free from bias  A sample can be used to provide much information about a population Can provide an opportunity for many people to be involved in the decision making process   Low response rate One or two follow-ups are usually needed for a good return Questionnaire must be simple and easy to understand Difficult to summarize open-ended questions Accurate mailing lists are required Overuse of this method may make some people reluctant to respond privacy, confidentiality, and anonymity must be assured Can be expensive and time consuming Scope is limited Results may be misleading if only respondents who are are interested in topic respond   Parents are asked about their needs for childcare. Childcare providers rate and rank types of training they need. Youth are asked to rate their concerns. Their responses are compared to parental responses.   Back to top Telephone Surveys What it Measures  Advantages Disadvantages Examples Opinions, attitudes, beliefs, behaviors, reactions and attributes, in response to specific questions   Response rate is generally high Less expensive to program Cost is competitive with mail survey Speed and efficient source of data Researcher can provide clarification on unclear questions Respondents are more relaxed with a stranger by telephone than face to face Interviewer can read questions from script and take notes without concern of distracting respondents Can be qualitative or quantitative Time consuming Telephone numbers are needed Proportion of unlisted numbers or households without phones may cause frame error Questions should still be simple and easy to understand (no more than five response categories) Interviewer's voice or identity may lead to biased results Respondents may feel interview is an invasion of privacy     Participants who drop out of a program are assessed for reasons. A sample of school parents are asked how they would like to be involved in schools. Welfare moms are asked to rate aspects of their work stress and childcare.   Back to top Personal Interviews What it Measures  Advantages Disadvantages Examples Person's responses, views and perceptions    Easier to reach those who are considered unreachable (e.g., the poor, homeless, high status, mobile, etc.) May be easier to reach specific individuals (e.g., community leaders, etc.) Higher response rate  More personalized approach Easier to ask open-ended questions, use probes and pick up on nonverbal cues Can be qualitative or quantitative   May be most expensive method Slowest method of data collection and analysis Responses may be less honest and thoughtful Interview's presence and characteristics may bias results Interviewer should go to location of respondent Respondents who prefer anonymity may be inhibited by personal approach May reach a smaller sample Extension responses must be sorted and coded   Program participants are interviewed about program and instructor. Fathers are asked how they are involved in their child's life. Teens are asked to voice their concerns about peer pressure.    Back to top Group Interviews What it Measures  Advantages Disadvantages Examples Group's responses, views, perceptions; often consensus   Less expensive and faster than personal interviews Personalized approach Group members stimulate each other Grouping similar people together assures comfort   Respondents who prefer anonymity may be inhibited by personal approach Input may be unbalanced because some group members dominate group members, and interviewer can bias responses Data more difficult to analyze and summarize   A focus group is conducted with members of a divorce education class to understand the effectiveness of instructor and value of content. A school staff discusses inservice needs. A community coalition assesses impact of Extension leadership on progress toward goals. Incarcerated parents discuss needs as a long-distance parent.   Back to top Observation What it Measures  Advantages Disadvantages Examples Particular physical and verbal behaviors   Setting is natural, flexible and unstructured Evaluator may make his/her identity known or remain anonymous Evaluator may actively participate or observe passively  Can be combined with other data collection methods Most useful for studying a "small unit" such as a classroom, council, etc.   The evaluator has less control over the situation in a natural environment Hawthorne effect -- if group members are aware that they are being observed, resulting behavior may affect observations Cannot be generalized to entire population unless a plan for representativeness is developed If observer chooses to be involved in the activity, he/she may lose objectivity Not realistic for use with large groups Several observation techniques (e.g., sampling) are available, but must be learned   Record how frequently 4-H campers use appropriate safety measures.   Back to top Mass Media/Public Hearings/Public Forums What it Measures  Advantages Disadvantages Examples Opinions, ideas   Allows all citizens to have an opportunity to respond and voice opinion Teleconferencing, call-in and town meeting methods are quick methods of obtaining input   The extremes of a population (those definitely "for" or "against") tend to respond Use of public television and teleconferencing is limited to those who have access to public television and a phone Public hearings are time consuming, especially for the interviewers Summary and analysis of data can be difficult Response to public hearing is affected by location, distance and date Not everyone is comfortable in this setting Large population pockets can be left out unintentionally   Use of public television to address the national issue of "youth at risk." Public hearing or community meeting concerning environmental issues or use of pesticides and water quality. Public forum to discuss demolition of community center to build a parking lot.  Community forum to identify concerns of youth.   Adapted from: The Answers to Program Evaluation Workbook, North Central Region Extension Publication # 751. January, 1997. Back to top

[ Home ] [ Introduction ] [ 1.Information Data Bank ] [ 2. Sample Data ] [ 3. Presentations ] [ Ground Control, Credits ] [ Definitions ]