Primary data capture methods: summary of advantages and disadvantages
Type Advantage Disadvantage
Face-to-face Highest quality results
Can use stimulus material Highest response rate
Most expensive Time-consuming Clustering will occur
Telephone Fast
Unclustered
Limited stimuli Requires skilled interviewers
Drop/Collect Low cost
Fair response rate
Can only reach respondents available
Postal Convenient for respondents
Low cost Can reach most respondents
No control over the respondent Can be slow
Poor response rate
Online Fast
Low-cost Good for international
No control over the respondent Lowest response rate/device bias Gaps in coverage
Panel (all above) Can show changes over time
(advantages as method above)
High investment (disadvantages as method above)
Observation (all above)
No interviewer or interviewee bias (advantages as method above)
Ethical concerns (disadvantages as method above)
Table
4.2
Part 2 Data collection
112
Figures 4.1 and 4.2 give an outline of diff erent data capture methods. The term ‘question-
naire’ has been used to typify the research instrument, but we will discover that the situation is
more complex than this.
These diff erent modes of data capture have an infl uence on any market research study. The
researcher is driven by three main constraints: the time available, the cost, and the control.
Table 4.3 shows how the four diff erent modes match to these. Clearly, the Internet is the fastest
and cheapest, but it also has a major limitation of being unable to exert any control over the
respondent, a weakness shared with postal methods. Although the face-to-face, personal
approach is the most expensive, it does provide the research user with results of the highest
possible quality.
Similarly, Table 4.4 shows some typical response rates for the main modes of interviewing.
This is based on the author’s experience and hides many things; it is not a defi nitive indication
of response rates. There are many variables that intervene in any response rate calculation. The
most contentious fi gure is for Internet response rates: practitioners who use the technique with
specifi c populations argue that their response rates are as high as 70 per cent.
Questionnaire delivery
Questionnaire retrieval Pre-notification to respondent
By phone
In person
By post
By fax
By email or web
By advert
By leaflet drop Word of mouth
By phone
In person
By post
By fax
By email or web Other
By phone
In person
By post
By fax
By email or web Other
Figure 4.1 An outline of data capture methods
Chapter 4 Primary data
113
Table
4.3
How methods may match up to research constraints
Fieldwork time Cost Control
Personal 2–4 weeks Very high Very good
Telephone 1–2 weeks High Good
Post 3–8 weeks Medium Poor
Online 1–7 days Very low Poor
Self- completion Telephone Personal
Questioning
Figure 4.2 The main questioning options for data capture
Table
4.4
‘Common’ response rates for the main modes of interviewing
Mode of interviewing Common response rate (%) Useful length
Face-to-face 60 45 mins
Drop and collect 60 Four sides
Telephone 45 20 mins
Postal 30 Four sides
Online 10 30 questions
Part 2 Data collection
114
One solution to problems created by short ‘snapshot’ methods of data collection is the
‘panel’ concept. The panel is a set of individuals who are questioned or observed, or who
report over a period of time. Any changes can therefore be identifi ed and, if necessary,
investigated. All of the data capture methods can be applied to this method. The panel
interview off ers a unique method to assess the eff ectiveness of any marketing initiative; it is
particularly useful to ‘measure’ the eff ectiveness of communications initiatives. The nature of
the panel is explored in Chapter 8 and its applications can be seen in Chapter 14.
Interviewing
Personal interviewing
Personal interviewing is best used for respondents who need reassurance, who may not have
easy access to the phone, and who may need to be guided through showcard materials. For
many years, this has been the most common method of interviewing in terms of number of
interviews and money paid for research. It involves trained interviewers working on location,
whether in the home, on the street, or at the workplace. The advantages are those of control:
it is possible to use stimulus material such as showcards or exhibits, and there is a personal
touch that may allow an interview to be lengthy compared with other options. Disadvantages
include the expensive nature of such a labour-intensive technique. In the long run, the
process can be slow; this clearly depends on the sample size, the location, and the research
instrument used.
In the 1990s, computer-assisted personal interviewing (CAPI) was introduced, off ering
unique benefi ts of speed in data collection and cost savings in processing. It was also
possible to carry out routing and edit checks in the fi eld, essentially leading to an intelligent
questionnaire. Stimulus material could be stored on a laptop and could appear easily and
quickly as required.
Telephone interviewing
Telephone interviewing is best used for respondents who are not readily available in person
and on projects that do not have a great deal of visual stimulus material.
The telephone has passed through several phases of popularity. In 1910, there were just
122,000 telephones in the UK (Terramedia 2004). In 1912, the Post Offi ce in the UK opened
the national telephone service. We know that, in 2002 in the UK, 98 per cent of households
had a telephone (ONS 2004). In an article in the 1970s, the then-president of the Market
Research Society wrote: ‘Personal interviewing has dominated the UK scene for many years
but there are now signs that this is changing. After all, some 50 per cent of UK homes are now
equipped with a telephone!’ (Treasure 1976, p. 61). The telephone became extremely important
to researchers in the 1970s. In recent years, mobile phone use has increased signifi cantly. It is
estimated that, in 2006, mobile phone use accounted for one-third of all voice calls.
Telephone interviewing can be at a central location or decentralised; typically, a
decentralised operation is one where trained interviewers work from home. Calls can be made
Chapter 4 Primary data
115
from interviewers to respondents and this is known as ‘outbound interviewing’. ‘Inbound’ is also possible, where respondents are given a telephone number to call, at a time convenient to themselves.
The advantages come from the speed of telephone-administered surveys: they do indeed provide quick results, and this method is generally cost-eff ective. The approach is particularly good at reaching neighbourhoods that may not be easily accessed by the face- to-face interviewer. Of course, this method can also be used for international work. Telephone calls can be recorded for later analysis, but remember that this needs to be made clear to respondents before the interview starts. Disadvantages concern control, which is limited, and there is little chance to show stimulus material (adverts, etc.). Indeed, other than an attractive voice and any interest the respondent may have in the topic, there is little to keep the respondent on the phone. This is illustrated well by looking at the outcome of calls. In telephone interviewing, there are several possible outcomes: contact may be made or there may be no contact. Reasons for each are shown in Table 4.5.
The calculation of the response rate percentage can change throughout the project duration. For example, if busy respondents are not recontacted, the response rate will not rise. The response rate will certainly fall if more telephone numbers are added to the sample in an attempt to increase the number of completed returns.
In the 1980s, computer-assisted telephone interviewing (CATI) came to prominence. CATI means that a questionnaire appears onscreen and the interviewer enters responses directly into the computer. This use of a computer was restricted to the manual keyboard because the
fi rst complete mouse was yet to become a standard part of computing—the fi rst complete mouse was introduced with Apple’s LISA computer.
The advantage of CATI is that it cuts out several processes, such as handwriting, moving paper, and inputting into the computer. It also allows sample management, whereby interview quotas can be monitored across the telephone unit, avoiding over- or under-sampling. In recent years, automated diallers have been introduced, whereby respondents are called
by computer at a specifi ed time. This time may be deemed convenient for interviewers and respondents, sometimes to keep an appointment. The equipment can be confi gured to administer a questionnaire automatically, using a prerecorded voice or to be passed to an interviewer. The Market Research Society (MRS) has guidelines on the use of such diallers, because they have caused confusion among respondents.
The fi nal Research in focus at the end of this chapter brings this type of work into focus.
Table
4.5
Possible outcomes in telephone interviews
Contact made No contact
Refusal by respondent Number unobtainable
Out of quota No answer
Ineligible Busy
Completed Respondent not available
Language problem
Part 2 Data collection
116
Interviewing skills
Clearly, telephone and personal interviewing relies heavily on the interviewer. Some have
divided the needed qualities for interviewers into three: skills, attitudes, and knowledge. The
skills of interviewing include listening skills, probing, and handling respondents. An interviewer
needs to be well prepared, an excellent communicator, interested in people, able to work
with others, able to work without close supervision, and a competent administrator. The work
must be carried out without prejudice, professionally, courteously, objectively, and responsibly.
Interviewers are expected to know the industry codes of conduct and data protection
legislation. An interviewer is expected to have been briefed properly on the purpose and
methods of the project.
We know a fair amount about diff erent aspects of interviewing from various fi elds—the topics
of interrogations and confessions, legal and psychological aspects, British Court of Appeal cases,
and foreign cases of disputed confessions (see Gudjonsson 2003). Despite the superior control of
interviewers over the data collection process, there can still be problems. By acting as a conduit
for the respondent, interviewers may distort or misreport responses. This has been shown by
comparing sound recordings with completed questionnaires. The eff ect is far greater for open-
ended questions than closed questions (see Belson 1983, p. 486). It is clear that training will help
in this respect, and research agencies provide regular training and project-specifi c briefi ngs.
Sometimes, respondents do not answer; they may give incomplete answers or answers that
are not clear. ‘Probing’ is a technique used to clarify such situations; it attempts to motivate
the informant to communicate more information without introducing bias into the questions
or answers. Some common ways to stimulate respondents to provide more detail are: to
repeat the question, to pause to motivate the respondent to speak, to repeat the respondent’s
answer/last word, and to ask standard (neutral) probe questions. Probing should not be
confused with prompting. Prompts are generally showcards of brand lists (for example), which
remind the respondent in order to direct him/her to answer in a particular way. Here are some
neutral probe questions: ‘Why do you say that? Anything else? Any others? What do you mean?
Can you please explain why?’.
Self-completion methods
Self-completion methods refer to approaches where the respondent is primarily responsible
for providing responses to questions. The approach is best used for respondents who are likely
to be motivated to respond and who will need little reassurance and minimal guidance: for
example, people who are interested in the topic, such as their new car. The most familiar form
of this is the questionnaire delivered by post.
We can also consider delivery by fax, Internet, at trade fairs, as magazine inserts and hand
delivery, etc. Comments cards, for example, in hotels, are another option and ‘diaries’ are also
used. For more on comments cards and diaries, see Chapter 6.
The advantage of the self-completion method is that the respondent spends his or her time
in answering the questions; eff ectively, this is unpaid labour. There is no interviewer to expend
time and money on the exercise. It is therefore a cheap option. It might also be argued that
the respondent will give a more considered response without the interference or apparent
pressure of an interviewer.
Chapter 4 Primary data
117
The disadvantages of the self- completion mode concern response quality, response rate, and the response time. There is little control over a respondent when an interviewer is not present; therefore, if a respondent does not understand something or has general queries about the research, there is not
an easy way of fi nding out. This may lead
to a poorer quality of answer, or even a lack of reply due to apathy or putting the task off until there is more time. The overall number of replies may suff er (the response or cooperation levels may be low) or replies may be received later than expected (response time suff ers). By poor response rate, we are suggesting 30–40 per cent. The questionnaire must be short and simple and the number of returns can be slow. In order to overcome some of these problems, reminders can be introduced. The ‘Options for primary data capture’ in the Market Researcher’s Toolbox list some of these options.
Drop and collect methods
The ‘interviewer drop and interviewer collection’ approach has aroused much interest in recent years. It has been called the DCS method after the words ‘Drop-and-Collect Survey method’ (see Brown 1987) and has been put forward as a method that can achieve higher response rates than mail questionnaires (see Ibeh et al. 2004).
A variation on this is for the interviewer to ‘drop’ the questionnaire and for it to be returned
by another means. In 2001, UK census forms were returned by postal services and the information was scanned into a computer directly from the forms.
Prenotifi cation may assist in response rate and it can be communicated in person, by post,
by telephone, by advertisement, or even indirectly, by word of mouth. This then leads to many more options being available to the research designer. The ‘Options for primary data capture’
in the Market Researcher’s Toolbox list prenotifi cation options.
Postal methods
The postal service is an important way to deliver and return research documentation. The growth of postal services began with establishment of important routes to key destinations such as important parts of large towns. In the UK, the Royal Mail service was fi rst made available to the public in 1635, when postage was paid by the recipient. The postcode, in the form used today, was fi rst used in 1959 in Norwich. This was the world’s fi rst attempt to use postal address codes for sorting mail by machine. Until 1974, delivery was a fragmented aff air, although effi cient, and it was only in 1974 that postcodes were extended over all the UK. The postal method of delivering questionnaires probably reached its peak in the 1960s.
‘Probing gets where other techniques cannot . . .’.
Part 2 Data collection
118
It is also essential to monitor progress. If there is a quota to achieve, there must be some
mechanism by which to know when to stop selecting and interviewing certain people,
otherwise the study is wasting the time of both respondent and researcher. In the case of
a postal questionnaire, a simple graph is a useful device to keep track of receipts. This may
indicate which dates are the best on which to send reminders. A good guide is to send a
reminder when the graph starts a downward turn, that is when replies start to become fewer!
A well-organised project allocates identity codes to all forms, so the senders of questionnaires
that have already been returned are known. This allows reminders to be sent only to those
people who have not replied. This saves both time and money.
Online methods
Internet questionnaires are, at present, self-completed, in that respondents use their own
hands with which to respond to questions. This mainly involves the hand in conjunction
with a keyboard or with a mouse. However, it must not be forgotten that it is technically
possible for voice recognition software to be used to capture spoken responses and it is
also possible to use handwriting recognition software to capture handwritten responses. In
a similar way, it is possible for questionnaires to be ‘spoken’, in a way similar to telephone
interviewing. With the increasing use of multimedia facilities in computing, it is also possible
for an interviewer to appear ‘onscreen’, making the interview similar to personal interviewing
methods (see Table 4.6).
In the current situation, it is the self-completion questionnaire, with words and text, which
dominates the quantitative Internet research fi eld. But this does not mean that the choices are
limited. Questionnaires may be delivered as a web page, as part of an email message, or as a
combination of both methods. To date, most Internet questionnaires have been computer-
assisted self-completion interviews, abbreviated sometimes to CASI and other times to CASCI.
Other abbreviations in use include CAWI to denote computer-assisted web interviews and
CAMI to denote computer-assisted mobile phone interviewing. Note that mobile phones
are also used for traditional telephone voice interviews, in addition to SMS/text message
interviews and Internet research. With all modes, the intention is to return questionnaires to
the sender electronically, although they may also be printed and returned offl ine.
The Internet is made up of numerous services. If we consider it as the ‘information highway’,
then there are various territories, and diff erent vehicles that can take us across those territories.
Let us confi ne the debate to two territories: email and the World Wide Web. The two vehicles
relevant here are email clients such as Microsoft Offi ce Outlook, Hotmail, Yahoo!Mail,
Modes of delivery and return for electronic data capture
Delivery Return
Text on screen Mouse
Interviewer voice Keyboard
Interviewer image Handwriting recognition software
Voice (recognition software or recording)
Table
4.6
Chapter 4 Primary data
119
and web browsers such as Internet Explorer, Firefox, and Chrome. These ‘brand names’ are not fi xed—their popularity rises and falls. Some will become redundant and new names will appear, but their purposes will remain: they allow users to communicate and receive information. Internet services can be received on computers at home or in a work situation, but they can also be accessed on television sets, laptops, personal digital assistants (PDAs), and mobile phones. Email clients and web browsers can be used to receive questionnaires and also to return them to the sender. A major consideration must be the source of the sample. Web page questionnaires can be divided into three types: the ‘open-web’ type, which is part
of a website and ‘open’ to any visitor—there is no control over who visits. This type includes the ‘Banner invitation’. The ‘closed-web’ type is closed, in that respondents are invited to visit the site to complete the questionnaire and it may be protected by a password. The third type
is the ‘hidden-web’ type: this appears to a visitor when triggered by some mechanism. The trigger may be the date, the fact that the visitor has expressed a particular interest by viewing specifi c information, or it may be a simple mechanism that identifi es the visitor as the tenth
or twentieth, for example, when systematic sampling is being used. The ‘pop-up’ survey is an example of this mechanism.
Email questionnaires are of three types. The simple email is a common message, which has the appearance of a letter, with questions included in the text. The respondent simply clicks
‘reply’, keeping the message as a text message that s/he modifi es by adding answers. These answers may be text (to open-ended questions) or s/he may be asked to simply mark ‘x’ after the correct answer in closed questions. It is also possible for respondents simply to delete answers that do not apply.
The email-attachment type is a questionnaire delivered as a word-processed or spreadsheet-based attachment. The respondent must ‘open’ the attachment and follow the instructions to complete it. It will be then ‘saved’ and another email to the researcher will be necessary, with the respondent remembering to re-attach the fi le with the answers.
The third type of email questionnaire is the email URL-embedded type. The email request for participation has a website address (a URL) mentioned in the message. For email clients that support HTML, the respondent simply links to the web page. This then evokes their web browser, presenting the reader with a web-based questionnaire.
The advantages are many. The nature of Internet interviewing is that it is not intrusive. Internet surveys are quick and easy to prepare, administer, and analyse, as well as being relatively low cost. Answers are entered directly into the software, which can mean that they are received ‘live’ in ‘real time’, and therefore give the most up-to-date information possible. The Internet makes international research feasible. The cost of researching the domestic situation does not diff er from research overseas. Such research does not suff er from ‘clustering’,
Table
4.7
Internet questionnaires
Web page Email
Open-web Simple email
Closed-web Email attachment
Hidden-web Email URL embedded
Part 2 Data collection
120