Where to get the questions from?
The importance of preliminary qualitative research.
Delia Chiaro, University of Bologna at Forlì
In questionnaire based survey research wording is all important because the way in which a question is couched affects respondents’ understanding of what is being asked of them (see “Reasons for piloting questionnaires”). However, not only is careful wording essential, but the type of question asked also needs careful planning. If we choose to adopt open-ended questions, the information we retrieve may well be of interest, but it is likely to be unfocussed, pretty chaotic and certainly of no use for statistical analysis. Thus, generally speaking, questionnaires tend to contain closed questions such as “yes/no”; “Wh-” questions followed by a list of answers from which the respondent is asked to choose the one closest to his or her opinion or attitude; statements to agree or disagree with etc. Wording and form are two crucial technicalities in questionnaire design, but let us take a step back at what happens before researchers put pen to paper and the instrument is still its ideational phase. Let us consider the actual substance of the questions to be asked. How do we decide what the questions should contain? What exactly do we ask? In other words, where do we actually get the questions from?
If we are working within the ESP paradigm then we are likely to have some kind of research question in our minds from which we begin working and we are also bound to have our own ideas about what we would like to ask. This is all well and good, but somewhat insufficient to develop a whole questionnaire which sets out to produce data that is as objective, complete and all-encompassing as possible. We also need to avoid bringing our own biases into our research. For these reasons, prior to writing a questionnaire it is standard practice to carry out a number of interviews with people in order to broaden the researcher’s ideas on the subject as well as to gain more input for what is to go into the questions. Thus, before drafting the instrument, researchers will ideally have obtained a set of questions which will be a mediation between their own ideas and fresh input from preliminary qualitative research.
Preliminary Qualitative Research
Although interviews are normally seen as instruments connected to qualitative research proper in their own right (which indeed they are) they are also frequently adopted in the groundwork that contributes to gaining insight into what actually goes into questionnaires.
Interviewing is a direct approach to obtaining information from respondents. Interviews are less structured and more intensive than standardized questionnaire-based interviews, (i.e. those which are administered by reading questions over the phone or face-to-face with the respondent). Furthermore, unlike “written” questionnaires, they are normally carried out on small samples of people and may be representative of only part of the population. Two types of interviews are used mostly: individual in-depth interviews and focus group interviews.
Individual In-Depth Interviews
These interviews are conducted face-to-face with a single respondent with the aim of exploring a subject in detail. If the respondent agrees, the interview can be recorded so that the researcher can examine and interpret what is said at a later date. In these interviews the respondent is prompted to speak freely about the subject in question. It is the task of the interviewer to create a relaxed atmosphere and without biasing responses, to encourage the respondent to speak and to get her/him back on track if s/he digresses. Such an interview will normally be between 30 minutes and 2 hours long. The interviewer must be highly skilled to be able to probe as much as possible into responses by asking questions such as “Why did you say that?” and “That’s interesting, can you tell me more?”
Advantages and Disadvantages of Individual In-Depth Interviews
A one-to-one setting can provide a comfortable atmosphere that will enable the respondent to speak freely with the interviewer. Subtleties and nuances can emerge from an individual interview which might well remain masked in a wider group setting. On the other hand, in-depth interviews are lengthy and time consuming as it would be extremely difficult for an interviewer to conduct more than a couple in a day without sacrificing quality.
Focus Group interviews
These interviews are conducted by a trained mediator among a small group of respondents who are encouraged to discuss a subject in a natural and unstructured manner. This particular interview type is especially useful for preparatory questionnaire work as free-flowing group discussion often leads to unexpected findings and ideas of which the researcher may not have thought. A focus group normally includes 8-12 members who are homogenous in terms of their socio-demographic features. It is essential that the group has adequate experience of the subject at issue. Thus, depending on the object of our research, for example, a typical focus group could be made up of young, legal translators or community interpreters working in hospital maternity departments. The interviews take place in a relaxed atmosphere with refreshments offered beforehand and readily available throughout the 1–3 hours of the interview. The discussion is usually video-taped, although often researchers also observe the group via a two-way mirror. The skill of the interviewer who acts as moderator is paramount: s/he must establish a rapport with participants, keep the discussion moving and elicit insights. Usually the moderator will play an essential role in the elaboration and interpretation of data.
Advantages and Disadvantages of Focus Group Interviews
One advantage of focus group interviews is that respondents can enlarge and refine their responses through their interaction with other group members, something which tends to naturally lead to more accurate and detailed information than what emerges from an individual interview. Furthermore, the situation and atmosphere of the group is more exciting than that of a standard interview and the “crowd” makes individuals feel more secure in expressing themselves. The moderator questions the group as a whole and not as individuals so respondents may feel uninhibited in expressing themselves.
Of course participants in focus groups may “play games” and go along with the group rather than express their own opinions, not to mention the fact that one more vocal member with strong opinions may alter the group’s expressed view substantially.
In both Translation and Interpreting Studies researchers often wear two hats in that they are often also practitioners of what they are researching. Unfortunately this does not necessarily mean that they do better research, as their experience may bias their questions and interpretation of data. A non-practitioner may be a better researcher, but is bound to know less about the topic s/he is researching due to his/her lack of personal experience. It is precisely here that preliminary research can be of great help allowing the practitioner-cum-researcher to step back and look at what s/he is doing more objectively through the eyes of the interviewee and the researcher to glean first-hand information about a world s/he has never personally experienced.
Thus the researcher’s ideas, broadened and fine-tuned by extensive preliminary investigations, can provide ample material for the questions that will make up the questionnaire. However, this stage does not substitute piloting. It goes without saying that more interviewing and more piloting will lead to more viewpoints and more fresh input. Questionnaires should undergo numerous drafts before definitive administration. Repetitive piloting with wording sometimes subtly different in each draft is essential to ensure a well-adjusted instrument. Only when all the questions “work” can administration proper be carried out.
Recommended reading
Malholtra. Naresh K. 1996. Marketing Research: an Applied Orientation. Saddle River: Prentice Hall. 166-169 & 174-178.
Tull, Donald S. and Hawkins, Del I. 1993. Marketing Research: Measurement and Method. Sixth Edition. New York: Macmillan. 411-460.
Aaker, David A.; Kumar, V. and Day, George, S. 1995. Marketing Research Fifth Edition. New York: John Wiley.176-183.