Data collection methods for impact evaluation vary along a continuum. At the one end of this continuum are quantitative methods and at the other end of the continuum are Qualitative methods for data collection .
(http://www.worldbank.org/poverty/impact/methods/datacoll.htm )
Quantitative and Qualitative Data collection methods
The Quantitative data collection methods, rely on random sampling and structured data collection instruments that fit diverse experiences into predetermined response categories. They produce results that are easy to summarize, compare, and generalize.
Quantitative research is concerned with testing hypotheses derived from theory and/or being able to estimate the size of a phenomenon of interest. Depending on the research question, participants may be randomly assigned to different treatments. If this is not feasible, the researcher may collect data on participant and situational characteristics in order to statistically control for their influence on the dependent, or outcome, variable. If the intent is to generalize from the research participants to a larger population, the researcher will employ probability sampling to select participants.
Typical quantitative data gathering strategies include:
- Experiments/clinical trials.
- Observing and recording well-defined events (e.g., counting the number of patients waiting in emergency at specified times of the day).
- Obtaining relevant data from management information systems.
- Administering surveys with closed-ended questions (e.g., face-to face and telephone interviews, questionnaires etc).(http://www.achrn.org/quantitative_methods.htm)
Interviews
In Quantitative research(survey research),interviews are more structured than in Qualitative research
In a structured interview,the researcher asks a standard set of questions and nothing more.(Leedy and Ormrod, 2001)
Face -to -face interviews have a distinct advantage of enabling the researcher to establish rapport with potential participants and therefor gain their cooperation. These interviews yield highest response rates in survey research. They also allow the researcher to clarify ambiguous answers and when appropriate, seek follow-up information. Disadvantages include impractical when large samples are involved time consuming and expensive. (Leedy and Ormrod, 2001)
Telephone interviews are less time consuming and less expensive and the researcher has ready access to anyone on the planet who has a telephone. Disadvantages are that the response rate is not as high as the face-to- face interview as but considerably higher than the mailed questionnaire. The sample may be biased to the extent that people without phones are part of the population about whom the researcher wants to draw inferences.
Computer Assisted Personal Interviewing (CAPI): is a form of personal interviewing, but instead of completing a questionnaire, the interviewer brings along a laptop or hand-held computer to enter the information directly into the database. This method saves time involved in processing the data, as well as saving the interviewer from carrying around hundreds of questionnaires. However, this type of data collection method can be expensive to set up and requires that interviewers have computer and typing skills.
Questionnaires
Paper-pencil-questionnaires can be sent to a large number of people and saves the researcher time and money. People are more truthful while responding to the questionnaires regarding controversial issues in particular due to the fact that their responses are anonymous. But they also have drawbacks. Majority of the people who receive questionnaires don't return them and those who do might not be representative of the originally selected sample.(Leedy and Ormrod, 2001)
Web based questionnaires: A new and inevitably growing methodology is the use of Internet based research. This would mean receiving an e-mail on which you would click on an address that would take you to a secure web-site to fill in a questionnaire. This type of research is often quicker and less detailed. Some disadvantages of this method include the exclusion of people who do not have a computer or are unable to access a computer. Also the validity of such surveys are in question as people might be in a hurry to complete it and so might not give accurate responses. (http://www.statcan.ca/english/edu/power/ch2/methods/methods.htm)
Questionnaires often make use of Checklist and rating scales. These devices help simplify and quantify people's behaviors and attitudes. A checklist is a list of behaviors, characteristics, or other entities that te researcher is looking for. Either the researcher or survey participant simply checks whether each item on the list is observed, present or true or vice versa. A rating scale is more useful when a behavior needs to be evaluated on a continuum. They are also known as Likert scales. (Leedy and Ormrod, 2001)
Qualitative data collection methods play an important role in impact evaluation by providing information useful to understand the processes behind observed results and assess changes in people’s perceptions of their well-being. Furthermore qualitative methods can be used to improve the quality of survey-based quantitative evaluations by helping generate evaluation hypothesis; strengthening the design of survey questionnaires and expanding or clarifying quantitative evaluation findings. These methods are characterized by the following attributes:
- they tend to be open-ended and have less structured protocols (i.e., researchers may change the data collection strategy by adding, refining, or dropping techniques or informants)
- they rely more heavily on interactive interviews; respondents may be interviewed several times to follow up on a particular issue, clarify concepts or check the reliability of data
- they use triangulation to increase the credibility of their findings (i.e., researchers rely on multiple data collection methods to check the authenticity of their results)
- generally their findings are not generalizable to any specific population, rather each case study produces a single piece of evidence that can be used to seek general patterns among different studies of the same issue
Regardless of the kinds of data involved, data collection in a qualitative study takes a great deal of time. The researcher needs to record any potentially useful data thoroughly, accurately, and systematically, using field notes, sketches, audiotapes, photographs and other suitable means. The data collection methods must observe the ethical principles of research.
The qualitative methods most commonly used in evaluation can be classified in three broad categories:
- in depth interview
- observation methods
- document review
No comments:
Post a Comment