Luận văn, báo cáo, luận án, đồ án, tiểu luận, đề tài khoa học, đề tài nghiên cứu, đề tài báo cáo - Báo cáo khoa học, luận văn tiến sĩ, luận văn thạc sĩ, nghiên cứu - Kinh Doanh - Business TABLE OF CONTENTS Chapter 1. INTRODUCTION.....................................................................................................1 1.1. What is Research? ............................................................................................................1 1.2. Kinds of Research Papers ................................................................................................1 1.3. The Applications of Research ..........................................................................................2 1.4. Types of Research ............................................................................................................3 1.5. Steps in Planning and Conducting Research ...................................................................4 Chapter 2. LITERATURE REVIEW ........................................................................................13 2.1. Definition of Literature Review? ...................................................................................13 2.2. Contents of a Literature Review ....................................................................................13 2.3. Steps of the Literature Review Process .........................................................................15 2.4. Citation and plagiarism ..................................................................................................18 2.5. Quoting, Paraphrasing and Summarizing ......................................................................18 Chapter 3. RESEARCH METHODS........................................................................................21 3.1. The Qualitative Method (Phương pháp nghiên cứu định tính) ......................................22 3.2. The Quantitative Methods (Phương pháp nghiên cứu định lượng) ...............................23 3.3. Descriptive methods......................................................................................................26 Chapter 4. INSTRUMENTS FOR DATA COLLECTION ......................................................31 4.1. QUESTIONNAIRES .........................................................................................................32 4.1.1. What is a Questionnaire............................................................................................32 4.1.2. Ways of Administering a Questionnaire .....................................................................33 4.1.3. Kinds of Questionnaires..............................................................................................34 4.1.4. Constructing the Questionnaire...................................................................................35 4.2. INTERVIEWS ...................................................................................................................54 4.2.1. Interview data..............................................................................................................54 4.2.2. Types of Interviews.....................................................................................................54 4.2.3. Planning and Conducting Interviews ..........................................................................55 4.3. OBSERVATION ...............................................................................................................63 4.3.1. Definition of Observation ...........................................................................................63 4.3.2. Types of Observation ..................................................................................................63 4.3.3. Situations in which observations can be made ...........................................................63 4.3.4. Conducting Observations for Research ......................................................................63 Chapter 5. ESTABLISHING THE VALIDITY AND RELIABILITY OF A RESEARCH INSTRUMENT .........................................................................................................................68 5.1. The Concept of Validity.....................................................................................................68 5.2. Types of Validity in Quantitative Research ........................................................................69 5.2.1. Face and Content Validity ..........................................................................................69 5.2.3. Construct Validity ......................................................................................................70 5.3. The Concept of Reliability ................................................................................................70 5.4. Factors Affecting the Reliability of a Research Instrument ...............................................70 5.5. Methods of Determining the Reliability of an Instrument in Quantitative Research .........71 5.5.1. External consistency procedures ...............................................................................71 5.5.2. Internal consistency procedures .................................................................................72 5.6. Validity and Reliability in Qualitative Research.................................................................72 Chapter 6. WRITING UP A RESEARCH PAPER ..................................................................75 A. HOW TO STRUCTURE A RESEARCH PAPER ..........................................................75 B. HOW TO WRITE THE STRUCTURES OF A RESEARCH PAPER............................77 6.1. Title ................................................................................................................................77 6.2. Abstract ..........................................................................................................................78 6.3. Introduction ....................................................................................................................81 6.4. Literature review ............................................................................................................81 6.5. Methodology chapter ....................................................................................................82 6.6. Results and discussion chapter.......................................................................................83 6.7. Writing the conclusion ...................................................................................................85 6.8. Referencing the research paper: the APA ......................................................................85 Course objectives By the end of this course, students should be able to: distinguish different research types; recognize the basic steps in planning and conducting a research study; identify a research area in foreign language education (FLE); formulate good research questions for a research study in FLE; develop an appropriate research design; construct some means of data collection in survey research in FLE such as the questionnaire, interview and observation; write a research proposal and research paper in FLE Students are expected to Actively prepared for the class (read and reflect on what they have read for each class meeting) Actively participate in class activities Work cooperatively and collaboratively with peers Complete all the assignments by the due date 1 Chapter 1 INTRODUCTION In this chapter >>> Identifying research areas: sources of research ideas Categorizing types of research Narrowing down the topic: focusing the question, deciding on the objectives Developing research questions: good research questions Deciding on the research design: choosing the right type of research, deciding on the data, the target population, the data collection and analysis methods Other considerations 1.1. What is Research? Nunan (1992, p. 3) defines that ‘research is a systematic process of inquiry consisting of three elements or components: (1) a question, problem or hypothesis, (2) data, (3) analysis and interpretation of data’. Kothari (2004) states that the term research refers to the systematic method consisting of enunciating the problem, formulating a hypothesis, collecting the facts or data, analysing the facts and reaching certain conclusions either in the form of solution(s) towards the concerned problem or in certain generalisations for some theoretical formulation. 1.2. Kinds of Research Papers 1.2. 1. Academic Papers Academic papers (also called scholarly papers), which are in academic journals, contain original research results or reviews existing results or show a totally new invention. 1.2. 2. Assignments and Term Papers (Bài tập lớn, tiểu luận) Written work is an integral part of a university education. Assignment and research papers perform an individual function in prompting students to think independently about issues and subjects. Students also learn how to assess, select and evaluate information from different sources and to formulate ideas. Typically, written assignments and term papers are geared to course work covered by a series of lecturers. Students are assigned a particular topic to write about or given a list of topics from which to choose. They receive instructions as to the length of the essay and the due date for submission. Some guidance may be given to students in the form of a suggested reading list. In courses that are well planned, lecturers usually inform students of the written work commitments for the full course at the beginning of the semester so that students can plan their study program effectively. The topics set for written assignments may give a useful guide to important content areas of a course of study. Assignments encourage students to read critically in a particular content area, to research for and select from available material, to organize their thoughts on a topic, and to submit to the discipline of communicating their thoughts through the presentation of evidence that they have sifted and evaluated to arrive at certain conclusions. Apart from learning the subject matter of a course, this process of independent study has considerable educational value. 2 1.2.3. Theses and Dissertations (Luận văn, Luận án) Students who enroll for an honor degree, a postgraduate diploma or a higher degree are almost invariably required to submit a thesis or dissertation. A thesis is much more than a large term paper. It normally presents the culmination of a substantial piece of original work over a period of at least a year. Some research replicates previous research with the object of testing the reported findings of that research or testing the relevance of findings of research completed in a different cultural milieu. Other research builds on existing studies to follow up new leads or to refine or qualify the findings of earlier studies. In either case, the thesis is expected to make an original contribution to knowledge. Students embarking on a thesis are usually much more responsible for selecting and delimiting an area of study than are students writing assignments or essays. However, it is not uncommon for students commencing a thesis to be given a topic or to be guided into areas that are of special interest to department members with whom they will work. Indeed, students may seek enrolment in a particular university because of the special interests of department there. 1.3. The Applications of Research The application of research can be viewed from four different perspectives: 1. the service provider; 2. the service administrator, manager andor planner; 3. the service consumer; and 4. the professional. 3 1.4. Types of Research Figure 1: Types of research Types of Research: Application Perspective Pure research involves developing and testing theories and hypotheses that are intellectually challenging to the researcher but may or may not have practical application at the present time or in the future. Thus such work often involves the testing of hypotheses containing very abstract and specialised concepts. Pure research is also concerned with the development, examination, verification and refinement of research methods, procedures, techniques and tools that form the body of research methodology. Examples of pure research include developing a sampling technique that can be applied to a particular situation; developing a methodology to assess the validity of a procedure; developing an instrument, say, to measure the stress level in people; and finding the best way of measuring people’s attitudes. The knowledge produced through pure research is sought in order to add to the existing body of knowledge of research methods. Applied research is used in most of the research in the social sciences. In other words, the research techniques, procedures and methods that form the body of research methodology are applied to the collection of information about various aspects of a situation, issue, problem or phenomenon so that the information gathered can be used in other ways – such as for policy formulation, administration and the enhancement of understanding of a phenomenon. Types of Research: Objectives Perspective Descriptive study attempts to describe systematically a situation, problem, phenomenon, service or programme, or provides information about, say, the living conditions of a community, or describes attitudes towards an issue. For example, it may attempt to describe the types of service provided by an organisation, the administrative structure of an organisation, the living conditions of Aboriginal people in the outback, the needs of a community, what it means to go through a divorce, how a child feels living in a house with domestic violence, or the attitudes of employees towards management. The main purpose of such studies is to describe what is prevalent with respect to the issueproblem under study. 4 Correlational study is to discover or establish the existence of a relationshipassociationinterdependence between two or more aspects of a situation. What is the impact of an advertising campaign on the sale of a product? What is the relationship between stressful living and the incidence of heart attack? What is the relationship between fertility and mortality? What is the relationship between technology and unemployment? What is the effect of a health service on the control of a disease, or the home environment on educational achievement? These studies examine whether there is a relationship between two or more aspects of a situation or phenomenon and, therefore, are called correlational studies. Explanatory research attempts to clarify why and how there is a relationship between two aspects of a situation or phenomenon. This type of research attempts to explain, for example, why stressful living results in heart attacks; why a decline in mortality is followed by a fertility decline; or how the home environment affects children’s level of academic achievement. Exploratory research is from the viewpoint of the objectives of a study. This is when a study is undertaken with the objective either to explore an area where little is known or to investigate the possibilities of undertaking a particular research study. When a study is carried out to determine its feasibility it is also called a feasibility study or a pilot study. It is usually carried out when a researcher wants to explore areas about which she has little or no knowledge. Exploratory studies are also conducted to develop, refine andor test measurement tools and procedures. Types of Research: Mode of Enquiry Perspective A study is classified as qualitative if the purpose of the study is primarily to describe a situation, phenomenon, problem or event; if the information is gathered through the use of variables measured on nominal or ordinal scales (qualitative measurement scales); and if the analysis is done to establish the variation in the situation, phenomenon or problem without quantifying it. The description of an observed situation, the historical enumeration of events, an account of the different opinions people have about an issue, and a description of the living conditions of a community are examples of qualitative research. On the other hand, the study is classified as quantitative if you want to quantify the variation in a phenomenon, situation, problem or issue; if information is gathered using predominantly quantitative variables; and if the analysis is geared to ascertain the magnitude of the variation. Examples of quantitative aspects of a research study are: How many people have a particular problem? How many people hold a particular attitude? 1.5. Steps in Planning and Conducting Research 5 Interpret and report FF" FF" Review"the"literature" F" F" I" III" IV" V" VI" VII" II" F" Review"concepts"and" theories" Review"previous" research"finding" Formulate" hypotheses" Analyse" data" (Test" hypotheses"if"any)" Design" research" (including" sample" design)" Collect" data" (Execution)" Figure 2: Steps in Planning and Conducting Research F: feedback (Helps in controlling the sub-system to which it is transmitted) FF: feed forward (serves the vital function of providing criteria for evaluation) Define Research Problem 6 1. Formulating the Research Problem: There are two types of research problems, viz., those which relate to states of nature and those which relate to relationships between variables. At the very outset the researcher must single out the problem he wants to study, i.e., he must decide the general area of interest or aspect of a subject-matter that he would like to inquire into. Initially the problem may be stated in a broad general way and then the ambiguities, if any, relating to the problem be resolved. Then, the feasibility of a particular solution has to be considered before a working formulation of the problem can be set up. The formulation of a general topic into a specific research problem, thus, constitutes the first step in a scientific enquiry. Essentially two steps are involved in formulating the research problem, viz., understanding the problem thoroughly, and rephrasing the same into meaningful terms from an analytical point of view. The best way of understanding the problem is to discuss it with one’s own colleagues or with those having some expertise in the matter. In an academic institution the researcher can seek the help from a guide who is usually an experienced man and has several research problems in mind. Often, the guide puts forth the problem in general terms and it is up to the researcher to narrow it down and phrase the problem in operational terms. In private business units or in governmental organizations, the problem is usually earmarked by the administrative agencies with whom the researcher can discuss as to how the problem originally came about and what considerations are involved in its possible solutions. The researcher must at the same time examine all available literature to get himself acquainted with the selected problem. He may review two types of literature—the conceptual literature concerning the concepts and theories, and the empirical literature consisting of studies made earlier which are similar to the one proposed. The basic outcome of this review will be the knowledge as to what data and other materials are available for operational purposes which will enable the researcher to specify his own research problem in a meaningful context. After this the researcher rephrases the problem into analytical or operational terms i.e., to put the problem in as specific terms as possible. This task of formulating, or defining, a research problem is a step of greatest importance in the entire research process. The problem to be investigated must be defined unambiguously for that will help discriminating relevant data from irrelevant ones. Care must, however, be taken to verify the objectivity and validity of the background facts concerning the problem. Professor W.A. Neiswanger correctly states that the statement of the objective is of basic importance because it determines the data which are to be collected, the characteristics of the data which are relevant, relations which are to be explored, the choice of techniques to be used in these explorations and the form of the final report. If there are certain pertinent terms, the same should be clearly defined along with the task of formulating the problem. In fact, formulation of the problem often follows a sequential pattern where a number of formulations are set up, each formulation more specific than the preceding one, each one phrased in more analytical terms, and each more realistic in terms of the available data and resources. 2. Extensive Literature Survey: Once the problem is formulated, a brief summary of it should be written down. It is compulsory for a research worker writing a thesis for a Ph.D. degree to write a synopsis of the topic and submit it to the necessary Committee or the Research Board for approval. At this juncture the researcher should undertake extensive literature survey connected with the problem. For this purpose, the abstracting and indexing journals and published or unpublished bibliographies are the first place to go to. Academic journals, conference proceedings, government reports, books etc., must be tapped depending on the nature of the problem. In this process, it should be remembered that one source will lead to another. The earlier studies, if any, which are similar to the study in hand should be carefully studied. A good library will be a great help to the researcher at this stage. 3. Development of Working Hypotheses: After extensive literature survey, researcher should state in clear terms the working hypothesis or hypotheses. Working hypothesis is tentative assumption made in order to draw out and test its logical or empirical consequences. As such 7 the manner in which research hypotheses are developed is particularly important since they provide the focal point for research. They also affect the manner in which tests must be conducted in the analysis of data and indirectly the quality of data which is required for the analysis. In most types of research, the development of working hypothesis plays an important role. Hypothesis should be very specific and limited to the piece of research in hand because it has to be tested. The role of the hypothesis is to guide the researcher by delimiting the area of research and to keep him on the right track. It sharpens his thinking and focuses attention on the more important facets of the problem. It also indicates the type of data required and the type of methods of data analysis to be used. How does one go about developing working hypotheses? The answer is by using the following approach: (a) Discussions with colleagues and experts about the problem, its origin and the objectives in seeking a solution; (b) Examination of data and records, if available, concerning the problem for possible trends, peculiarities and other clues; (c) Review of similar studies in the area or of the studies on similar problems; and (d) Exploratory personal investigation which involves original field interviews on a limited scale with interested parties and individuals with a view to secure greater insight into the practical aspects of the problem. Thus, working hypotheses arise as a result of a-priori thinking about the subject, examination of the available data and material including related studies and the counsel of experts and interested parties. Working hypotheses are more useful when stated in precise and clearly defined terms. It may as well be remembered that occasionally we may encounter a problem where we do not need working hypotheses, specially in the case of exploratory or formulative researches which do not aim at testing the hypothesis. But as a general rule, specification of working hypotheses in another basic step of the research process in most research problems. 4. Preparing the Research Design: The research problem having been formulated in clear cut terms, the researcher will be required to prepare a research design, i.e., he will have to state the conceptual structure within which research would be conducted. The preparation of such a design facilitates research to be as efficient as possible yielding maximal information. In other words, the function of research design is to provide for the collection of relevant evidence with minimal expenditure of effort, time and money. But how all these can be achieved depends mainly on the research purpose. Research purposes may be grouped into four categories, viz., (i) Exploration, (ii) Description, (iii) Diagnosis, and (iv) Experimentation. A flexible research design which provides opportunity for considering many different aspects of a problem is considered appropriate if the purpose of the research study is that of exploration. But when the purpose happens to be an accurate description of a situation or of an association between variables, the suitable design will be one that minimises bias and maximises the reliability of the data collected and analysed. There are several research designs, such as, experimental and non-experimental hypothesis testing. Experimental designs can be either informal designs (such as before-and-after without control, after-only with control, before-and-after with control) or formal designs (such as completely randomized design, randomized block design, Latin square design, simple and complex factorial designs), out of which the researcher must select one for his own project. The preparation of the research design, appropriate for a particular research problem, involves usually the consideration of the following: (i) the means of obtaining the information; (ii) the availability and skills of the researcher and his staff (if any); (iii) explanation of the way in which selected means of obtaining information will be organised and the reasoning leading to the selection; (iv) the time available for research; and 8 (v) the cost factor relating to research, i.e., the finance available for the purpose. 5. Determining Sample Design: All the items under consideration in any field of inquiry constitute a ‘universe’ or ‘population’. A complete enumeration of all the items in the ‘population’ is known as a census inquiry. It can be presumed that in such an inquiry when all the items are covered no element of chance is left and highest accuracy is obtained. But in practice this may not be true. Even the slightest element of bias in such an inquiry will get larger and larger as the number of observations increases. Moreover, there is no way of checking the element of bias or its extent except through a resurvey or use of sample checks. Besides, this type of inquiry involves a great deal of time, money and energy. Not only this, census inquiry is not possible in practice under many circumstances. For instance, blood testing is done only on sample basis. Hence, quite often we select only a few items from the universe for our study purposes. The items so selected constitute what is technically called a sample. The researcher must decide the way of selecting a sample or what is popularly known as the sample design. In other words, a sample design is a definite plan determined before any data are actually collected for obtaining a sample from a given population. Thus, the plan to select 12 of a city’s 200 drugstores in a certain way constitutes a sample design. Samples can be either probability samples or non-probability samples. With probability samples each element has a known probability of being included in the sample but the non-probability samples do not allow the researcher to determine this probability. Probability samples are those based on simple random sampling, systematic sampling, stratified sampling, clusterarea sampling whereas non- probability samples are those based on convenience sampling, judgement sampling and quota sampling techniques. A brief mention of the important sample designs is as follows: (i) Deliberate sampling: Deliberate sampling is also known as purposive or non-probability sampling. This sampling method involves purposive or deliberate selection of particular units of the universe for constituting a sample which represents the universe. When population elements are selected for inclusion in the sample based on the ease of access, it can be called convenience sampling. If a researcher wishes to secure data from, say, gasoline buyers, he may select a fixed number of petrol stations and may conduct interviews at these stations. This would be an example of convenience sample of gasoline buyers. At times such a procedure may give very biased results particularly when the population is not homogeneous. On the other hand, in judgement sampling the researcher’s judgement is used for selecting items which he considers as representative of the population. For example, a judgement sample of college students might be taken to secure reactions to a new method of teaching. Judgement sampling is used quite frequently in qualitative research where the desire happens to be to develop hypotheses rather than to generalise to larger populations. (ii) Simple random sampling: This type of sampling is also known as chance sampling or probability sampling where each and every item in the population has an equal chance of inclusion in the sample and each one of the possible samples, in case of finite universe, has the same probability of being selected. For example, if we have to select a sample of 300 items from a universe of 15,000 items, then we can put the names or numbers of all the 15,000 items on slips of paper and conduct a lottery. Using the random number tables is another method of random sampling. To select the sample, each item is assigned a number from 1 to 15,000. Then, 300 five digit random numbers are selected from the table. To do this we select some random starting point and then a systematic pattern is used in proceeding through the table. We might start in the 4th row, second column and proceed down the column to the bottom of the table and then move to the top of the next column to the right. When a number exceeds the limit of the numbers in the frame, in our case over 15,000, it is simply passed over and the next number selected that does fall within the relevant range. Since the numbers were placed in the table in a completely random fashion, the resulting sample is random. This procedure gives each item an equal probability of being selected. In case of infinite population, the selection of each item in a random sample is controlled by the same probability and that successive selections are independent of one another. 9 (iii) Systematic sampling: In some instances the most practical way of sampling is to select every 15th name on a list, every 10th house on one side of a street and so on. Sampling of this type is known as systematic sampling. An element of randomness is usually introduced into this kind of sampling by using random numbers to pick up the unit with which to start. This procedure is useful when sampling frame is available in the form of a list. In such a design the selection process starts by picking some random point in the list and then every nth element is selected until the desired number is secured. (iv) Stratified sampling: If the population from which a sample is to be drawn does not constitute a homogeneous group, then stratified sampling technique is applied so as to obtain a representative sample. In this technique, the population is stratified into a number of non- overlapping subpopulations or strata and sample items are selected from each stratum. If the items selected from each stratum is based on simple random sampling the entire procedure, first stratification and then simple random sampling, is known as stratified random sampling. (v) Quota sampling: In stratified sampling the cost of taking random samples from individual strata is often so expensive that interviewers are simply given quota to be filled from different strata, the actual selection of items for sample being left to the interviewer’s judgement. This is called quota sampling. The size of the quota for each stratum is generally proportionate to the size of that stratum in the population. Quota sampling is thus an important form of non-probability sampling. Quota samples generally happen to be judgement samples rather than random samples. (vi) Cluster sampling and area sampling: Cluster sampling involves grouping the population and then selecting the groups or the clusters rather than individual elements for inclusion in the sample. Suppose some departmental store wishes to sample its credit card holders. It has issued its cards to 15,000 customers. The sample size is to be kept say 450. For cluster sampling this list of 15,000 card holders could be formed into 100 clusters of 150 card holders each. Three clusters might then be selected for the sample randomly. The sample size must often be larger than the simple random sample to ensure the same level of accuracy because is cluster sampling procedural potential for order bias and other sources of error is usually accentuated. The clustering approach can, however, make the sampling procedure relatively easier and increase the efficiency of field work, specially in the case of personal interviews. Area sampling is quite close to cluster sampling and is often talked about when the total geographical area of interest happens to be big one. Under area sampling we first divide the total area into a number of smaller non-overlapping areas, generally called geographical clusters, then a number of these smaller areas are randomly selected, and all units in these small areas are included in the sample. Area sampling is specially helpful where we do not have the list of the population concerned. It also makes the field interviewing more efficient since interviewer can do many interviews at each location. (vii) Multi-stage sampling: This is a further development of the idea of cluster sampling. This technique is meant for big inquiries extending to a considerably large geographical area like an entire country. Under multi-stage sampling the first stage may be to select large primary sampling units such as states, then districts, then towns and finally certain families within towns. If the technique of random-sampling is applied at all stages, the sampling procedure is described as multi-stage random sampling. (viii) Sequential sampling: This is somewhat a complex sample design where the ultimate size of the sample is not fixed in advance but is determined according to mathematical decisions on the basis of information yielded as survey progresses. This design is usually adopted under acceptance sampling plan in the context of statistical quality control. In practice, several of the methods of sampling described above may well be used in the same study in which case it can be called mixed sampling. It may be pointed out here that normally one should resort to random sampling so that bias can be eliminated and sampling error can be estimated. But purposive sampling is considered desirable when the universe happens to be small and a known characteristic of it is to be studied intensively. Also, there are 10 conditions under which sample designs other than random sampling may be considered better for reasons like convenience and low costs. The sample design to be used must be decided by the researcher taking into consideration the nature of the inquiry and other related factors. 6. Collecting the Data: In dealing with any real life problem it is often found that data at hand are inadequate, and hence, it becomes necessary to collect data that are appropriate. There are several ways of collecting the appropriate data which differ considerably in context of money costs, time and other resources at the disposal of the researcher. Primary data can be collected either through experiment or through survey. If the researcher conducts an experiment, he observes some quantitative measurements, or the data, with the help of which he examines the truth contained in his hypothesis. But in the case of a survey, data can be collected by any one or more of the following ways: (i) By observation: This method implies the collection of information by way of investigator’s own observation, without interviewing the respondents. The information obtained relates to what is currently happening and is not complicated by either the past behaviour or future intentions or attitudes of respondents. This method is no doubt an expensive method and the information provided by this method is also very limited. As such this method is not suitable in inquiries where large samples are concerned. (ii) Through personal interview: The investigator follows a rigid procedure and seeks answers to a set of pre-conceived questions through personal interviews. This method of collecting data is usually carried out in a structured way where output depends upon the ability of the interviewer to a large extent. (iii) Through telephone interviews: This method of collecting information involves contacting the respondents on telephone itself. This is not a very widely used method but it plays an important role in industrial surveys in developed regions, particularly, when the survey has to be accomplished in a very limited time. (iv) By mailing of questionnaires: The researcher and the respondents do come in contact with each other if this method of survey is adopted. Questionnaires are mailed to the respondents with a request to return after completing the same. It is the most extensively used method in various economic and business surveys. Before applying this method, usually a Pilot Study for testing the questionnaire is conduced which reveals the weaknesses, if any, of the questionnaire. Questionnaire to be used must be prepared very carefully so that it may prove to be effective in collecting the relevant information. Through schedules: Under this method the enumerators are appointed and given training. They are provided with schedules containing relevant questions. These enumerators go to respondents with these schedules. Data are collected by filling up the schedules by enumerators on the basis of replies given by respondents. Much depends upon the capability of enumerators so far as this method is concerned. Some occasional field checks on the work of the enumerators may ensure sincere work. The researcher should select one of these methods of collecting the data taking into consideration the nature of investigation, objective and scope of the inquiry, finanical resources, available time and the desired degree of accuracy. Though he should pay attention to all these factors but much depends upon the ability and experience of the researcher. In this context Dr A.L. Bowley very aptly remarks that in collection of statistical data commonsense is the chief requisite and experience the chief teacher. 7. Execution of the Project: Execution of the project is a very important step in the research process. If the execution of the project proceeds on correct lines, the data to be collected would be adequate and dependable. The researcher should see that the project is executed in a systematic manner and in time. If the survey is to be conducted by means of structured questionnaires, data can be readily machine-processed. In such a situation, questions as well as the possible answers may be coded. If the data are to be collected through interviewers, arrangements should be made for proper selection and training of the interviewers. The training may be given with the help of instruction manuals which explain clearly the job of 11 the interviewers at each step. Occasional field checks should be made to ensure that the interviewers are doing their assigned job sincerely and efficiently. A careful watch should be kept for unanticipated factors in order to keep the survey as much realistic as possible. This, in other words, means that steps should be taken to ensure that the survey is under statistical control so that the collected information is in accordance with the pre-defined standard of accuracy. If some of the respondents do not cooperate, some suitable methods should be designed to tackle this problem. One method of dealing with the non-response problem is to make a list of the non-respondents and take a small sub-sample of them, and then with the help of experts vigorous efforts can be made for securing response. 8. Analysis of Data: After the data have been collected, the researcher turns to the task of analysing them. The analysis of data requires a number of closely related operations such as establishment of categories, the application of these categories to raw data through coding, tabulation and then drawing statistical inferences. The unwieldy data should necessarily be condensed into a few manageable groups and tables for further analysis. Thus, researcher should classify the raw data into some purposeful and usable categories. Coding operation is usually done at this stage through which the categories of data are transformed into symbols that may be tabulated and counted. Editing is the procedure that improves the quality of the data for coding. With coding the stage is ready for tabulation. Tabulation is a part of the technical procedure wherein the classified data are put in the form of tables. The mechanical devices can be made use of at this juncture. A great deal of data, specially in large inquiries, is tabulated by computers. Computers not only save time but also make it possible to study large number of variables affecting a problem simultaneously. Analysis work after tabulation is generally based on the computation of various percentages, coefficients, etc., by applying various well defined statistical formulae. In the process of analysis, relationships or differences supporting or conflicting with original or new hypotheses should be subjected to tests of significance to determine with what validity data can be said to indicate any conclusion(s). For instance, if there are two samples of weekly wages, each sample being drawn from factories in different parts of the same city, giving two different mean values, then our problem may be whether the two mean values are significantly different or the difference is just a matter of chance. Through the use of statistical tests we can establish whether such a difference is a real one or is the result of random fluctuations. If the difference happens to be real, the inference will be that the two samples come from different universes and if the difference is due to chance, the conclusion would be that the two samples belong to the same universe. Similarly, the technique of analysis of variance can help us in analysing whether three or more varieties of seeds grown on certain fields yield significantly different results or not. In brief, the researcher can analyse the collected data with the help of various statistical measures. 9. Hypothesis-testing: After analysing the data as stated above, the researcher is in a position to test the hypotheses, if any, he had formulated earlier. Do the facts support the hypotheses or they happen to be contrary? This is the usual question which should be answered while testing hypotheses. Various tests, such as Chi square test, t-test, F-test, have been developed by statisticians for the purpose. The hypotheses may be tested through the use of one or more of such tests, depending upon the nature and object of research inquiry. Hypothesis-testing will result in either accepting the hypothesis or in rejecting it. If the researcher had no hypotheses to start with, generalisations established on the basis of data may be stated as hypotheses to be tested by subsequent researches in times to come. 10. Generalisations and Interpretation: If a hypothesis is tested and upheld several times, it may be possible for the researcher to arrive at generalisation, i.e., to build a theory. As a matter of fact, the real value of research lies in its ability to arrive at certain generalisations. If the researcher had no hypothesis to start with, he might seek to explain his findings on the basis of some theory. It is known as interpretation. The process of interpretation may quite often trigger off new questions which in turn may lead to further researches. 12 11. Preparation of the Report or the Thesis: Finally, the researcher has to prepare the report of what has been done by him. Writing of report must be done with great care keeping in view the following: 1. The layout of the report should be as follows: (i) the preliminary pages; (ii) the main text, and (iii) the end matter. In its preliminary pages the report should carry title and date followed by acknowledgements and foreword. Then there should be a table of contents followed by a list of tables and list of graphs and charts, if any, given in the report. The main text of the report should have the following parts: (a) Introduction: It should contain a clear statement of the objective of the research and an explanation of the methodology adopted in accomplishing the research. The scope of the study along with various limitations should as well be stated in this part. (b) Summary of findings: After introduction there would appear a statement of findings and recommendations in non-technical language. If the findings are extensive, they should be summarised. (c) Main report: The main body of the report should be presented in logical sequence and broken-down into readily identifiable sections. (d) Conclusion: Towards the end of the main text, researcher should again put down the results of his research clearly and precisely. In fact, it is the final summing up. At the end of the report, appendices should be enlisted in respect of all technical data. Bibliography, i.e., list of books, journals, reports, etc., consulted, should also be given in the end. Index should also be given specially in a published research report. 2. Report should be written in a concise and objective style in simple language avoiding vague expressions such as ‘it seems,’ ‘there may be’, and the like. 3. Charts and illustrations in the main report should be used only if they present the information more clearly and forcibly. 4. Calculated ‘confidence limits’ must be mentioned and the various constraints experienced in conducting research operations may as well be stated. 13 Chapter 2 LITERATURE REVIEW In this chapter >>> Identifying role of related literature Carrying out steps of the literature review process Applying search strategies to find EFL sources Outlining a literature review 2.1. Definition of Literature Review? A literature review discusses published information in a particular subject area, and sometimes information in a particular subject area within a certain time period. Literature reviews provide a solid background for a research paper''''s investigation. The purpose of reviewing existing information is to find what is already known about the topic. This part should answer four questions: 1. What is the present state of knowledge regarding the topic under consideration? 2. How are the studies related to the one being proposed? 3. What is the quality of the studies reviewed? 4. How will the proposed study contribute to the existing literature? 2.2. Contents of a Literature Review A literature review may be: a single entity - an end in itself (essay) A preface to and rationale for engaging in research (thesis chapter) Some points to remember when writing a literature review: it is not a descriptive list of the information gathered it is not a summary of one piece of literature after another the review must be defined by a guiding concept (e.g. essay question, research objective, etc.) your purpose is to convey to the reader what knowledge and ideas have been established on a topic - what are the strengths and weaknesses organise the information gathered into sections that present themes do not attempt to list all published material, but rather synthesise and evaluate the literature according to your guiding concept A literature review should contain: an introduction, a middle (body) and conclusion. 14 In the Introduction define the topic, providing an appropriate context for reviewing the literature establish writer‘s reasons (point of view) for reviewing the literature explain the organisation of the review (sequence) state what literature is and is not included (scope) Example: Pedagogic approaches are typically informed by both a theory of language and a theory of language learning (Richards Rogers, 1986). For example, audiolingualism (Lado, 1964) (phương pháp nghe khẩu ngữ), was informed by a structuralist model of language and by behaviourist learning theory. In this review, however, we will focus on the underlying theories of language learning. We will discuss three general approaches to the teaching of a second foreign language and identify the learning theories that underpin them. The three approaches are (a) the oral-situational approach (lối tiếp cận bằng lời theo tình huống), (b) the notional- functional approach and (tiếp cận theo hướng ý niệm-chức năng) (c) the task-based approach (tiếp cận dựa trên nhiệm vụ). These approaches have been chosen because they are ‘mainstream’ and thus probably reflect the current practice of language pedagogy in New Zealand. There are other approaches, e.g. various humanistic approaches (Moskowitz, 1978), content-based language teaching (Brinton, Snow, Wesche, 1989) and the lexical approach (Lewis, 1993), which figure in the literature on language pedagogy and also draw heavily on theories of language learning but which have not figured widely in school-based language teaching. As it is unlikely that these alternative approaches will drive language curricula in New Zealand, they are not considered here. In the Body group the literature according to common themes provide insight into relationship between central topic and a larger area (i.e. discipline) proceed from the general, wider view of the research under review to the specific problem Example: Outline of the body: The Oral-Situational Approach The Notional-Functional Approach The Task-Based Approach Example: The Oral-Situational Approach The oral-situational approach was developed by British applied linguists as an alternative approach to the audiolingual approach promoted in the United States. It resembles the audiolingual approach in being based on a structural syllabus (i.e. a specification of the linguistic structures to be taught) but differs from it in that it emphasizes the meanings realised by the different structures, not just their forms, and also the importance of situational teaching structures (i.e. identifying situational contexts for practising the structures). This approach was dominant in British- influenced teaching contexts from the sixties onwards. It still underlies many textbooks used to teach languages today (Terrell, 2002). 15 In the Conclusion summarise major contributions of the literature evaluate the current "state of the art” literature reviewed point out major flaws, or gaps in research outline issues pertinent to future study (Source: httpwww.wisc.eduwritingHandbookReviewofliterature.html) Example: Early research (e.g. Scherer Wertheimer, 1964) investigating the relationship between teaching and learning sought to compare the relative effectiveness of different pedagogic approaches by examining learning outcomes. However, the results proved inconclusive (Allwright, 1988). This led researchers to focus attention on the actual classroom processes that arise in language instruction (i.e. the teacher and learner behaviours) and to examine the impact that specific processes (e.g. the types of questioning used by the teacher) have on language learning. 2.3. Steps of the Literature Review Process 2.3.1. Step 1: Identifying Key Terms Before you choose the key words for your literature search, here are some things to consider: Need to Have a Precise Topic It is essential that one defines a research question or topic very carefully. For example, it should not be too far-reaching. Limitations of Study In specifying precisely one''''s research topic, one is also specifying appropriate limitations on the research. Limiting, for example, by time, personnel, gender, age, location, nationality etc. results in a more focused and meaningful topic. Scope of the Literature Review It is also important to determine the precise scope of the literature review. For example, What exactly will you cover in your review? How comprehensive will it be? How long? About how many citations will you use? Are you focusing on methodological approaches; on theoretical issues; on qualitative or quantitative research? Will you broaden your search to seek literature in related disciplines? Will you confine your reviewed material to English language only or will you include research in other languages too? Keywords are the words you will use to search electronic catalogs, databases and the Internet. You can miss lots of useful information if you don''''t use keywords effectively, because computers will only search on the terms you provide. You should think about: synonyms (e.g. mobile telephones, cell phones) pluralsingular forms (e.g. women, woman) spelling variations (e.g. globalisation, globalization) variations of a root word (e.g. feminism, feminist, feminine) acronyms (e.g. non-governmental organisation, NGO) regional historic variations (e.g. Native American or Indian) broader and narrower terms (e.g. drugs - cannabis, heroin, cocaine) 16 Before you start searching, a good strategy is to brainstorm any keywords you can think of that are related to the key concepts in your research question. Then as you search, note any useful alternative words that occur during your searches and add them to your keyword list. 2.3.2. Step 2: Locate literature Overview Use primary and secondary sources. In general, a “primary source” is research reported by the researcher that conducted the study. A “secondary source” is research that summarizes or reports findings that come from primary sources. Search different types of literature: summaries, encyclopaedias, dictionaries and glossaries of terms, handbooks, statistical indexes, reviews and syntheses, books, journals, indexed publications, electronic sources, abstract series, and databases. Places to search for reading material: Scholarly online indexing and abstracting databases - search individual indexes relevant to your topic, such as PsycINFO for psychology, ERIC for education, efl...
INTRODUCTION
What is Research?
Nunan (1992, p 3) defines that ‘research is a systematic process of inquiry consisting of three elements or components: (1) a question, problem or hypothesis, (2) data, (3) analysis and interpretation of data’
Kothari (2004) states that the term research refers to the systematic method consisting of enunciating the problem, formulating a hypothesis, collecting the facts or data, analysing the facts and reaching certain conclusions either in the form of solution(s) towards the concerned problem or in certain generalisations for some theoretical formulation.
Kinds of Research Papers
Academic papers (also called scholarly papers), which are in academic journals, contain original research results or reviews existing results or show a totally new invention
1.2 2 Assignments and Term Papers (Bài tập lớn, tiểu luận)
Written work is an integral part of a university education Assignment and research papers perform an individual function in prompting students to think independently about issues and subjects Students also learn how to assess, select and evaluate information from different sources and to formulate ideas
Typically, written assignments and term papers are geared to course work covered by a series of lecturers Students are assigned a particular topic to write about or given a list of topics from which to choose They receive instructions as to the length of the essay and the due date for submission Some guidance may be given to students in the form of a suggested reading list In courses that are well planned, lecturers usually inform students of the written work commitments for the full course at the beginning of the semester so that students can plan their study program effectively
The topics set for written assignments may give a useful guide to important content areas of a course of study Assignments encourage students to read critically in a particular content area, to research for and select from available material, to organize their thoughts on a topic, and to submit to the discipline of communicating their thoughts through the presentation of evidence that they have sifted and evaluated to arrive at certain conclusions Apart from learning the subject matter of a course, this process of independent study has considerable educational value
1.2.3 Theses and Dissertations (Luận văn, Luận án)
Students who enroll for an honor degree, a postgraduate diploma or a higher degree are almost invariably required to submit a thesis or dissertation
A thesis is much more than a large term paper It normally presents the culmination of a substantial piece of original work over a period of at least a year Some research replicates previous research with the object of testing the reported findings of that research or testing the relevance of findings of research completed in a different cultural milieu Other research builds on existing studies to follow up new leads or to refine or qualify the findings of earlier studies In either case, the thesis is expected to make an original contribution to knowledge
Students embarking on a thesis are usually much more responsible for selecting and delimiting an area of study than are students writing assignments or essays However, it is not uncommon for students commencing a thesis to be given a topic or to be guided into areas that are of special interest to department members with whom they will work Indeed, students may seek enrolment in a particular university because of the special interests of department there.
The Applications of Research
The application of research can be viewed from four different perspectives:
2.! the service administrator, manager and/or planner;
Types of Research
Types of Research: Application Perspective
Pure research involves developing and testing theories and hypotheses that are intellectually challenging to the researcher but may or may not have practical application at the present time or in the future Thus such work often involves the testing of hypotheses containing very abstract and specialised concepts
Pure research is also concerned with the development, examination, verification and refinement of research methods, procedures, techniques and tools that form the body of research methodology Examples of pure research include developing a sampling technique that can be applied to a particular situation; developing a methodology to assess the validity of a procedure; developing an instrument, say, to measure the stress level in people; and finding the best way of measuring people’s attitudes The knowledge produced through pure research is sought in order to add to the existing body of knowledge of research methods
Applied research is used in most of the research in the social sciences In other words, the research techniques, procedures and methods that form the body of research methodology are applied to the collection of information about various aspects of a situation, issue, problem or phenomenon so that the information gathered can be used in other ways – such as for policy formulation, administration and the enhancement of understanding of a phenomenon
Types of Research: Objectives Perspective
Descriptive study attempts to describe systematically a situation, problem, phenomenon, service or programme, or provides information about, say, the living conditions of a community, or describes attitudes towards an issue For example, it may attempt to describe the types of service provided by an organisation, the administrative structure of an organisation, the living conditions of Aboriginal people in the outback, the needs of a community, what it means to go through a divorce, how a child feels living in a house with domestic violence, or the attitudes of employees towards management The main purpose of such studies is to describe what is prevalent with respect to the issue/problem under study
Correlational study is to discover or establish the existence of a relationship/association/interdependence between two or more aspects of a situation What is the impact of an advertising campaign on the sale of a product? What is the relationship between stressful living and the incidence of heart attack? What is the relationship between fertility and mortality? What is the relationship between technology and unemployment? What is the effect of a health service on the control of a disease, or the home environment on educational achievement? These studies examine whether there is a relationship between two or more aspects of a situation or phenomenon and, therefore, are called correlational studies
Explanatory research attempts to clarify why and how there is a relationship between two aspects of a situation or phenomenon This type of research attempts to explain, for example, why stressful living results in heart attacks; why a decline in mortality is followed by a fertility decline; or how the home environment affects children’s level of academic achievement
Exploratory research is from the viewpoint of the objectives of a study This is when a study is undertaken with the objective either to explore an area where little is known or to investigate the possibilities of undertaking a particular research study When a study is carried out to determine its feasibility it is also called a feasibility study or a pilot study
It is usually carried out when a researcher wants to explore areas about which s/he has little or no knowledge Exploratory studies are also conducted to develop, refine and/or test measurement tools and procedures
Types of Research: Mode of Enquiry Perspective
A study is classified as qualitative if the purpose of the study is primarily to describe a situation, phenomenon, problem or event; if the information is gathered through the use of variables measured on nominal or ordinal scales (qualitative measurement scales); and if the analysis is done to establish the variation in the situation, phenomenon or problem without quantifying it The description of an observed situation, the historical enumeration of events, an account of the different opinions people have about an issue, and a description of the living conditions of a community are examples of qualitative research
On the other hand, the study is classified as quantitative if you want to quantify the variation in a phenomenon, situation, problem or issue; if information is gathered using predominantly quantitative variables; and if the analysis is geared to ascertain the magnitude of the variation Examples of quantitative aspects of a research study are: How many people have a particular problem? How many people hold a particular attitude?
Steps in Planning and Conducting Research
Analyse" data" (Test" hypotheses"if"any)"
Figure 2: Steps in Planning and Conducting Research
F: feedback (Helps in controlling the sub-system to which it is transmitted)
FF: feed forward (serves the vital function of providing criteria for evaluation)
1 Formulating the Research Problem: There are two types of research problems, viz., those which relate to states of nature and those which relate to relationships between variables At the very outset the researcher must single out the problem he wants to study, i.e., he must decide the general area of interest or aspect of a subject-matter that he would like to inquire into Initially the problem may be stated in a broad general way and then the ambiguities, if any, relating to the problem be resolved Then, the feasibility of a particular solution has to be considered before a working formulation of the problem can be set up The formulation of a general topic into a specific research problem, thus, constitutes the first step in a scientific enquiry Essentially two steps are involved in formulating the research problem, viz., understanding the problem thoroughly, and rephrasing the same into meaningful terms from an analytical point of view
The best way of understanding the problem is to discuss it with one’s own colleagues or with those having some expertise in the matter In an academic institution the researcher can seek the help from a guide who is usually an experienced man and has several research problems in mind Often, the guide puts forth the problem in general terms and it is up to the researcher to narrow it down and phrase the problem in operational terms In private business units or in governmental organizations, the problem is usually earmarked by the administrative agencies with whom the researcher can discuss as to how the problem originally came about and what considerations are involved in its possible solutions
The researcher must at the same time examine all available literature to get himself acquainted with the selected problem He may review two types of literature—the conceptual literature concerning the concepts and theories, and the empirical literature consisting of studies made earlier which are similar to the one proposed The basic outcome of this review will be the knowledge as to what data and other materials are available for operational purposes which will enable the researcher to specify his own research problem in a meaningful context After this the researcher rephrases the problem into analytical or operational terms i.e., to put the problem in as specific terms as possible This task of formulating, or defining, a research problem is a step of greatest importance in the entire research process The problem to be investigated must be defined unambiguously for that will help discriminating relevant data from irrelevant ones Care must, however, be taken to verify the objectivity and validity of the background facts concerning the problem Professor W.A Neiswanger correctly states that the statement of the objective is of basic importance because it determines the data which are to be collected, the characteristics of the data which are relevant, relations which are to be explored, the choice of techniques to be used in these explorations and the form of the final report If there are certain pertinent terms, the same should be clearly defined along with the task of formulating the problem In fact, formulation of the problem often follows a sequential pattern where a number of formulations are set up, each formulation more specific than the preceding one, each one phrased in more analytical terms, and each more realistic in terms of the available data and resources
2 Extensive Literature Survey: Once the problem is formulated, a brief summary of it should be written down It is compulsory for a research worker writing a thesis for a Ph.D degree to write a synopsis of the topic and submit it to the necessary Committee or the Research Board for approval At this juncture the researcher should undertake extensive literature survey connected with the problem For this purpose, the abstracting and indexing journals and published or unpublished bibliographies are the first place to go to Academic journals, conference proceedings, government reports, books etc., must be tapped depending on the nature of the problem In this process, it should be remembered that one source will lead to another The earlier studies, if any, which are similar to the study in hand should be carefully studied A good library will be a great help to the researcher at this stage
3 Development of Working Hypotheses: After extensive literature survey, researcher should state in clear terms the working hypothesis or hypotheses Working hypothesis is tentative assumption made in order to draw out and test its logical or empirical consequences As such
7 the manner in which research hypotheses are developed is particularly important since they provide the focal point for research They also affect the manner in which tests must be conducted in the analysis of data and indirectly the quality of data which is required for the analysis In most types of research, the development of working hypothesis plays an important role Hypothesis should be very specific and limited to the piece of research in hand because it has to be tested The role of the hypothesis is to guide the researcher by delimiting the area of research and to keep him on the right track It sharpens his thinking and focuses attention on the more important facets of the problem It also indicates the type of data required and the type of methods of data analysis to be used
How does one go about developing working hypotheses? The answer is by using the following approach:
(a) Discussions with colleagues and experts about the problem, its origin and the objectives in seeking a solution;
(b) Examination of data and records, if available, concerning the problem for possible trends, peculiarities and other clues;
(c) Review of similar studies in the area or of the studies on similar problems; and
(d) Exploratory personal investigation which involves original field interviews on a limited scale with interested parties and individuals with a view to secure greater insight into the practical aspects of the problem
Thus, working hypotheses arise as a result of a-priori thinking about the subject, examination of the available data and material including related studies and the counsel of experts and interested parties Working hypotheses are more useful when stated in precise and clearly defined terms It may as well be remembered that occasionally we may encounter a problem where we do not need working hypotheses, specially in the case of exploratory or formulative researches which do not aim at testing the hypothesis But as a general rule, specification of working hypotheses in another basic step of the research process in most research problems
4 Preparing the Research Design: The research problem having been formulated in clear cut terms, the researcher will be required to prepare a research design, i.e., he will have to state the conceptual structure within which research would be conducted The preparation of such a design facilitates research to be as efficient as possible yielding maximal information
In other words, the function of research design is to provide for the collection of relevant evidence with minimal expenditure of effort, time and money But how all these can be achieved depends mainly on the research purpose Research purposes may be grouped into four categories, viz., (i) Exploration, (ii) Description,
(iii)! Diagnosis, and (iv) Experimentation A flexible research design which provides opportunity for considering many different aspects of a problem is considered appropriate if the purpose of the research study is that of exploration But when the purpose happens to be an accurate description of a situation or of an association between variables, the suitable design will be one that minimises bias and maximises the reliability of the data collected and analysed There are several research designs, such as, experimental and non-experimental hypothesis testing Experimental designs can be either informal designs (such as before-and-after without control, after-only with control, before-and-after with control) or formal designs (such as completely randomized design, randomized block design, Latin square design, simple and complex factorial designs), out of which the researcher must select one for his own project The preparation of the research design, appropriate for a particular research problem, involves usually the consideration of the following:
(i) the means of obtaining the information;
(ii) the availability and skills of the researcher and his staff (if any);
(iii) explanation of the way in which selected means of obtaining information will be organised and the reasoning leading to the selection;
(iv) the time available for research; and
(v) the cost factor relating to research, i.e., the finance available for the purpose
5 Determining Sample Design: All the items under consideration in any field of inquiry constitute a ‘universe’ or ‘population’ A complete enumeration of all the items in the
‘population’ is known as a census inquiry It can be presumed that in such an inquiry when all the items are covered no element of chance is left and highest accuracy is obtained But in practice this may not be true Even the slightest element of bias in such an inquiry will get larger and larger as the number of observations increases Moreover, there is no way of checking the element of bias or its extent except through a resurvey or use of sample checks Besides, this type of inquiry involves a great deal of time, money and energy Not only this, census inquiry is not possible in practice under many circumstances For instance, blood testing is done only on sample basis Hence, quite often we select only a few items from the universe for our study purposes The items so selected constitute what is technically called a sample
The researcher must decide the way of selecting a sample or what is popularly known as the sample design In other words, a sample design is a definite plan determined before any data are actually collected for obtaining a sample from a given population Thus, the plan to select
12 of a city’s 200 drugstores in a certain way constitutes a sample design Samples can be either probability samples or non-probability samples With probability samples each element has a known probability of being included in the sample but the non-probability samples do not allow the researcher to determine this probability Probability samples are those based on simple random sampling, systematic sampling, stratified sampling, cluster/area sampling whereas non- probability samples are those based on convenience sampling, judgement sampling and quota sampling techniques A brief mention of the important sample designs is as follows:
(i) Deliberate sampling: Deliberate sampling is also known as purposive or non-probability sampling This sampling method involves purposive or deliberate selection of particular units of the universe for constituting a sample which represents the universe When population elements are selected for inclusion in the sample based on the ease of access, it can be called convenience sampling If a researcher wishes to secure data from, say, gasoline buyers, he may select a fixed number of petrol stations and may conduct interviews at these stations This would be an example of convenience sample of gasoline buyers At times such a procedure may give very biased results particularly when the population is not homogeneous
On the other hand, in judgement sampling the researcher’s judgement is used for selecting items which he considers as representative of the population For example, a judgement sample of college students might be taken to secure reactions to a new method of teaching Judgement sampling is used quite frequently in qualitative research where the desire happens to be to develop hypotheses rather than to generalise to larger populations
LITERATURE REVIEW
Definition of Literature Review?
A literature review discusses published information in a particular subject area, and sometimes information in a particular subject area within a certain time period Literature reviews provide a solid background for a research paper's investigation
The purpose of reviewing existing information is to find what is already known about the topic This part should answer four questions:
1 What is the present state of knowledge regarding the topic under consideration?
2 How are the studies related to the one being proposed?
3 What is the quality of the studies reviewed?
4 How will the proposed study contribute to the existing literature?
Contents of a Literature Review
* a single entity - an end in itself (essay)
* A preface to and rationale for engaging in research (thesis chapter)
Some points to remember when writing a literature review:
* it is not a descriptive list of the information gathered
* it is not a summary of one piece of literature after another
* the review must be defined by a guiding concept (e.g essay question, research objective, etc.)
* your purpose is to convey to the reader what knowledge and ideas have been established on a topic - what are the strengths and weaknesses
* organise the information gathered into sections that present themes
* do not attempt to list all published material, but rather synthesise and evaluate the literature according to your guiding concept
A literature review should contain: an introduction, a middle (body) and conclusion
•! define the topic, providing an appropriate context for reviewing the literature
•! establish writer‘s reasons (point of view) for reviewing the literature
•! explain the organisation of the review (sequence)
•! state what literature is and is not included (scope)
Pedagogic approaches are typically informed by both a theory of language and a theory of language learning (Richards & Rogers, 1986) For example, audiolingualism (Lado, 1964) (phương pháp nghe khẩu ngữ), was informed by a structuralist model of language and by behaviourist learning theory In this review, however, we will focus on the underlying theories of language learning
We will discuss three general approaches to the teaching of a second/ foreign language and identify the learning theories that underpin them The three approaches are (a) the oral-situational approach (lối tiếp cận bằng lời theo tình huống), (b) the notional- functional approach and (tiếp cận theo hướng ý niệm-chức năng) (c) the task-based approach (tiếp cận dựa trên nhiệm vụ) These approaches have been chosen because they are ‘mainstream’ and thus probably reflect the current practice of language pedagogy in New Zealand
There are other approaches, e.g various humanistic approaches (Moskowitz, 1978), content-based language teaching (Brinton, Snow, & Wesche, 1989) and the lexical approach (Lewis, 1993), which figure in the literature on language pedagogy and also draw heavily on theories of language learning but which have not figured widely in school-based language teaching As it is unlikely that these alternative approaches will drive language curricula in New Zealand, they are not considered here
* group the literature according to common themes
* provide insight into relationship between central topic and a larger area (i.e discipline)
* proceed from the general, wider view of the research under review to the specific problem
The oral-situational approach was developed by British applied linguists as an alternative approach to the audiolingual approach promoted in the United States It resembles the audiolingual approach in being based on a structural syllabus (i.e a specification of the linguistic structures to be taught) but differs from it in that it emphasizes the meanings realised by the different structures, not just their forms, and also the importance of situational teaching structures (i.e identifying situational contexts for practising the structures)
This approach was dominant in British- influenced teaching contexts from the sixties onwards It still underlies many textbooks used to teach languages today (Terrell, 2002)
* summarise major contributions of the literature
* evaluate the current "state of the art” literature reviewed
* point out major flaws, or gaps in research
* outline issues pertinent to future study
(Source: http//www.wisc.edu/writing/Handbook/Reviewofliterature.html)
Early research (e.g Scherer & Wertheimer, 1964) investigating the relationship between teaching and learning sought to compare the relative effectiveness of different pedagogic approaches by examining learning outcomes However, the results proved inconclusive (Allwright, 1988) This led researchers to focus attention on the actual classroom processes that arise in language instruction (i.e the teacher and learner behaviours) and to examine the impact that specific processes (e.g the types of questioning used by the teacher) have on language learning.
Steps of the Literature Review Process
Before you choose the key words for your literature search, here are some things to consider:
Need to Have a Precise Topic
It is essential that one defines a research question or topic very carefully For example, it should not be too far-reaching
In specifying precisely one's research topic, one is also specifying appropriate limitations on the research Limiting, for example, by time, personnel, gender, age, location, nationality etc results in a more focused and meaningful topic
Scope of the Literature Review
It is also important to determine the precise scope of the literature review For example,
What exactly will you cover in your review?
How comprehensive will it be?
How long? About how many citations will you use?
Are you focusing on methodological approaches; on theoretical issues; on qualitative or quantitative research?
Will you broaden your search to seek literature in related disciplines?
Will you confine your reviewed material to English language only or will you include research in other languages too?
Keywords are the words you will use to search electronic catalogs, databases and the Internet
You can miss lots of useful information if you don't use keywords effectively, because computers will only search on the terms you provide
•! synonyms (e.g mobile telephones, cell phones)
•! plural/singular forms (e.g women, woman)
•! variations of a root word (e.g feminism, feminist, feminine)
•! acronyms (e.g non-governmental organisation, NGO)
•! regional & historic variations (e.g Native American or Indian)
•! broader and narrower terms (e.g drugs - cannabis, heroin, cocaine)
Before you start searching, a good strategy is to brainstorm any keywords you can think of that are related to the key concepts in your research question Then as you search, note any useful alternative words that occur during your searches and add them to your keyword list
• Use primary and secondary sources In general, a “primary source” is research reported by the researcher that conducted the study A “secondary source” is research that summarizes or reports findings that come from primary sources
• Search different types of literature: summaries, encyclopaedias, dictionaries and glossaries of terms, handbooks, statistical indexes, reviews and syntheses, books, journals, indexed publications, electronic sources, abstract series, and databases
Places to search for reading material:
•! Scholarly online indexing and abstracting databases - search individual indexes relevant to your topic, such as PsycINFO for psychology, ERIC for education, efl-resource.com for English as a Foreign Language
•! Literature reviews may already exist on some aspect of your topic Search online databases carefully to find literature reviews Some databases enable you to limit your search to literature reviews, or use literature review or review as a keyword
•! Key articles, which are very relevant to the topic, may have bibliographies or reference lists which often lead to other relevant items These can also be a good guideline about how much you have read on your topic
2.3.3 Step 3: Critically evaluate and select literature
• Rely on primary sources, including journal articles published in peer-reviewed journals
• Prioritize your search: first look for peer reviewed, scholarly journal articles, then, non- refereed articles, then books, then conference papers, dissertations
Once you have some articles that look good, read the abstracts to get an idea of what they say You may want to skim over the best ones, especially if they have good literature reviews themselves That will give you an idea how literature reviews are written on this topic!
Now, think about how the ideas in the articles you have might be organized One of the purposes of the literature review is to provide an overview and synthesis of information; grouping similar articles gives you a framework for your overview
It is usually wise to move from broad to narrow Provide your reader with the most general information first, then building toward the specifics of your research concerns
There are many different approaches to how to organize your literature review, depending on what the literature looks like Think about what the articles you have are talking about Do they group themselves naturally to you? Some examples of ways to organize a literature review include:
CHRONOLOGICAL: This is a common approach, especially for topics that have been talked about for a long time and have changed over their history Organize it in stages of how the topic has changed: the first definitions of it, then major time periods of change as researchers talked about it, then how it is thought about today
COMPARISON TO PRESENT HYPOTHESIS: If your literature review is part of an empirical article or meta-analysis, where you intend to present a hypothesis and come to a conclusion, you can organize the literature review to show the articles that share or support your hypothesis, and those that disagree with it This gives a chance to show the strengths of the supporting research, discuss any validity/methodology issues with past research that disagrees with your findings, and explain how the past research leads up to and supports yours
BROAD-TO-SPECIFIC: Another approach is to start with a section on the general type of issue you're reviewing, then narrow down to increasingly specific issues in the literature until you reach the articles that are most specifically similar to your research question, thesis statement, hypothesis, or proposal This can be a good way to introduce a lot of background and related facets of your topic when there is not much directly on your topic but you are tying together many related, broader articles
MAJOR MODELS or MAJOR THEORIES: When there are multiple models or prominent theories, it is a good idea to outline the theories or models that are applied the most in your articles That way you can group the articles you read by the theoretical framework that each prefers, to get a good overview of the prominent approaches to your concept
PROMINENT AUTHORS: If a certain researcher started a field, and there are several famous people who developed it more, a good approach can be grouping the famous author/researchers and what each is known to have said about the topic You can then organize other authors into groups by which famous authors' ideas they are following With this organization it can help to look at the citations your articles list in them, to see if there is one author that appears over and over
CONTRASTING SCHOOLS OF THOUGHT: If you find a dominant argument comes up in your research, with researchers taking two sides and talking about how the other is wrong, you may want to group your literature review by those schools of thought and contrast the differences in their approaches and ideas
PROBLEM->SOLUTION: This approach groups quotations from articles first that introduce and describe the problem or problems being addressed in your research Then group articles by types of solutions that are proposed in the articles
Citation and plagiarism
When writing up your research findings, it is very important to cite anything that is not your own original work, using a combination of in-text citations and a reference list at the end of the piece The reference list should include all the sources that you consulted in relation to a particular piece of work Citations should be accurate to enable the reader to locate the item easily Citations also need to be presented in a consistent and appropriate style
If you do not acknowledge the work of others in your research, your readers will assume that it is your own work Plagiarism occurs when you present work containing ideas or quotations from other authors, without acknowledging the original authors You will never be criticised for referring to the work of others in your work; in fact this will be seen as a strength in your research rather than a weakness
Plagiarism is akin to theft, and is considered by all to be a serious offence If proven, plagiarism results in disciplinary action against the guilty party Most universities have policies that define and help students to avoid committing plagiarism
To avoid accidental plagiarism conduct your research carefully and record your notes meticulously Summarise findings in your own words If you want to use another person’s original work, whether this is words, diagrams, images or data, record the details of the work accurately and immediately.
Quoting, Paraphrasing and Summarizing
What are the differences among quoting, paraphrasing, and summarizing?
These three ways of incorporating other writers' work into your own writing differ according to the closeness of your writing to the source writing
Quotations must be identical to the original, using a narrow segment of the source They must match the source document word for word and must be attributed to the original author
Paraphrasing involves putting a passage from source material into your own words A paraphrase must also be attributed to the original source Paraphrased material is usually shorter than the original passage, taking a somewhat broader segment of the source and condensing it slightly
Summarizing involves putting the main idea(s) into your own words, including only the main point(s) Once again, it is necessary to attribute summarized ideas to the original source Summaries are significantly shorter than the original and take a broad overview of the source material
Why use quotations, paraphrases, and summaries?
Quotations, paraphrases, and summaries serve many purposes You might use them to:
•! Provide support for claims or add credibility to your writing
•! Refer to work that leads up to the work you are now doing
•! Give examples of several points of view on a subject
•! Call attention to a position that you wish to agree or disagree with
•! Highlight a particularly striking phrase, sentence, or passage by quoting the original
•! Distance yourself from the original by quoting it in order to cue readers that the words are not your own
•! Expand the breadth or depth of your writing
Writers frequently intertwine summaries, paraphrases, and quotations As part of a summary of an article, a chapter, or a book, a writer might include paraphrases of various key points blended with quotations of striking or suggestive phrases as in the following example:
In his famous and influential work The Interpretation of Dreams, Sigmund Freud argues that dreams are the "royal road to the unconscious" (page #), expressing in coded imagery the dreamer's unfulfilled wishes through a process known as the "dream-work" (page #) According to Freud, actual but unacceptable desires are censored internally and subjected to coding through layers of condensation and displacement before emerging in a kind of rebus puzzle in the dream itself (page #)
How to use quotations, paraphrases, and summaries
It might be helpful to follow these steps:
•! Read the entire text, noting the key points and main ideas
•! Summarize in your own words what the single main idea of the essay is
•! Paraphrase important supporting points that come up in the essay
•! Consider any words, phrases, or brief passages that you believe should be quoted directly
There are several ways to integrate quotations into your text Often, a short quotation works well when integrated into a sentence Longer quotations can stand alone Remember that quoting should be done only sparingly; be sure that you have a good reason to include a direct quotation when you decide to do so You'll find guidelines for citing sources and punctuating citations at our documentation guide pages
(https://owl.english.purdue.edu/owl/resource/563/01/)
2.6 Example of a good and bad review
Neurnan (2000, p.113) provides an example of a good and bad review:
Sexual harassment has many consequences Adams, Koltke, and Padgitt (1983) found that some women students said they avoided taking a class or working with certain professors because of the risk of harassment They also found that men and women students reacted differently Their research was a survey of 1,000 men and women graduate and undergraduate students Benson and Thomson's study in Social Problems (1982) lists many problems created by sexual harassment In their excellent book, The Lecherous Professor,
Dziech and Weiner (1990) give a long list of difficulties that victims have suffered
The victims of sexual harassment suffer a range of consequences, from lowered self-esteem and loss of self-confidence to withdrawal from social interaction, changed career goals, and depression (Adams, Kottke, and Padgitt, 1983; Benson and Thomson, 1982; Dziech and Weiner, 1990) For example, Adams, Kotlke, and Padgitt (1983) noted that 13 percent of women students said they avoided taking a class or working with certain professors because of the risk of harassment
Online resource: Giáo trình điện tử: Phương pháp nghiên cứu tài liệu trong nghiên cứu khoa học: http://www.khoahocviet.info/meresci/vi/meresci00a.html
RESEARCH METHODS
The Qualitative Method (Phương pháp nghiên cứu định tính)
Qualitative research places stress on the validity of multiple meaning structures and holistic analysis, as opposed to the criteria of reliability which can be statistically compartmentalized (khoanh vùng) in quantitative research
Qualitative forms of investigation tend to be based upon recognition of the importance of the subjective, experiential life-world of human beings Perceptions and interpretations of reality are linked with these meaning structures Thus, the ‘reality’ of a given educational setting may be seen, not as a fixed and stable entity, but as a type of variable that might be discerned only through an analysis of these multiple forms of understanding Qualitative methodologies provide avenues that can lead to the discovery of these deeper levels of meaning
Qualitative methods are essentially concerned with processes rather than consequences, with wholeness rather than independent variables, and with meanings rather than behavioral statistics
Several inter-related approaches are used in qualitative educational research such as action research, case studies, and ethnography These approaches tend to be characterized by being context specific, collaborative and interventionist Ethnography provides the major guidelines in methodology for qualitative research
Because of the subjective nature of qualitative data and its origin in single context, it is difficult to apply conventional standards of reliability and validity Contexts, situations, events, conditions and interactions cannot be replicated to any extent nor can generalizations be made to a wider context than the one studied with any confidence Basically, the richness, individuality and subjective nature of a participant’s perspective and understanding is not amendable to the usual scientific criteria This does not, however, make such understandings any less real or valid for that participant, and their explanatory function for that person’s behavior is highly predictive
One of the major limitations of qualitative research and evaluation is the time required for data collection, analysis and interpretation There is a critical need for the researcher to spend a considerable amount of time in the research setting in order to examine, holistically and aggregately, the interactions, reactions and activities
In addition, Parlett (1975) notes that because of the intimacy of participant-observer relationships within the setting, there is no doubt that the researcher’s mere presence will have profound reactive effects on the subjects of the study Concomitantly, the promise of anonymity, which often serves as the basis for trust, in concert with the requirement of authenticity, makes the qualitative evaluator’s task particularly difficult in terms of the preparation and presentation
23 of results Possible bias, from the viewpoints of both researcher and participants must also be identified and elucidated
Rist (1975) also notes that: The variable of time may be viewed as a handicap; should one attempt to replicate findings accumulated for nearly three years, a comparable length of time would again be necessary for the adequate observation of a similar group Similarly, there is no guarantee that the replication could be one of an identical social context Perhaps the most that is feasible is to achieve a high degree of similarity and to recognize that absolute reproduction is impossible (p94)
The qualitative mode of inquiry is characterized by methodological eclecticism, a hypothesis-free orientation and an implicit acceptance of the natural scheme of things Because of the need for the researcher to maintain close association with both participants and activities within the setting, the researcher gains an insider’s view of the field This proximity to the field often allows the evaluator to see and document the qualities of educational interaction too often missed by the scientific, more positivistic inquiries Such propinquity can reveal subtleties and complexities that could go undetected through the use of more standardized measures
Qualitative descriptions can play the important role of suggesting possible relationship(s), causes, effects, and even dynamic processes in school settings Qualitative methods can highlight subtleties in pupil behavior and response, illuminate reasons for action and provide in-depth information on teacher interpretations and teaching style
Since qualitative reports are not presented as statistical summations, but rather in a more descriptive, narrative style, this type of research might be of particular benefit to practitioner Qualitative approaches certainly do not provide easy quick answers to the complex issues that front the field of education But they do offer a viable alternative.
The Quantitative Methods (Phương pháp nghiên cứu định lượng)
In the social sciences, quantitative research refers to the systematic empirical investigation of social phenomena via statistical, mathematical or computational techniques The objective of quantitative research is to develop and employ mathematical models, theories and/or hypotheses pertaining to phenomena The process of measurement is central to quantitative research because it provides the fundamental connection between empirical observation and mathematical expression of quantitative relationships
Quantitative research is used widely in social sciences such as psychology, economics, sociology, and political science, and less frequently in anthropology and history Research in mathematical sciences such as physics is also 'quantitative' by definition, though this use of the term differs in context In the social sciences, the term relates to empirical methods, originating in both philosophical positivism and the history of statistics, which contrast qualitative research methods
Qualitative methods produce information only on the particular cases studied, and any more general conclusions are only hypotheses Quantitative methods can be used to verify, which of such hypotheses are true
The epistemological underpinnings of the quantitative motif ‘hold that there exist definable “social facts” (Rist, 1979) Quantitative research is typified by experimental studies in science- based disciplines where findings are usually expressed in numerical form Research in the quantitative tradition follows a linear sequence as in Figure 3
Figure 3 Linear Sequence in Quantitative Research
A major element of a research study is analyzing the data by statistical techniques
Quantitative research is generally made using scientific methods, which can include:
•! The generation of models, theories and hypotheses
•! The development of instruments and methods for measurement
•! Experimental control and manipulation of variables
•! Modeling and analysis of data
In the social sciences particularly, quantitative research is often contrasted with qualitative research which is the examination, analysis and interpretation of observations for the purpose of discovering underlying meanings and patterns of relationships, including classifications of types of phenomena and entities, in a manner that does not involve mathematical models Approaches to quantitative psychology were first modelled on quantitative approaches in the physical sciences by Gustav Fechner in his work on psychophysics, which built on the work of Ernst Heinrich Weber Although a distinction is commonly drawn between qualitative and quantitative aspects of scientific investigation, it has been argued that the two go hand in hand For example, based on analysis of the history of science, Kuhn (1961, p 162) concludes that “large amounts of qualitative work have usually been prerequisite to fruitful quantification in the physical sciences” Qualitative research is often used to gain a general sense of phenomena and to form theories that can be tested using further quantitative research For instance, in the social sciences qualitative research methods are often used to gain better understanding of such things as intentionality (from the speech response of the researchee) and meaning (why did this person/group say something and what did it mean to them?)(Kieron Yeoman)
Formulate hypotheses Design the study
Although quantitative investigation of the world has existed since people first began to record events or objects that had been counted, the modern idea of quantitative processes have their roots in Auguste Comte's positivist framework
Statistics is the most widely used branch of mathematics in quantitative research outside of the physical sciences, and also finds applications within the physical sciences, such as in statistical mechanics Statistical methods are used extensively within fields such as economics, social sciences and biology Quantitative research using statistical methods starts with the collection of data, based on the hypothesis or theory Usually a big sample of data is collected - this would require verification, validation and recording before the analysis can take place Software packages such as SPSS and R are typically used for this purpose Causal relationships are studied by manipulating factors thought to influence the phenomena of interest while controlling other variables relevant to the experimental outcomes In the field of health, for example, researchers might measure and study the relationship between dietary intake and measurable physiological effects such as weight loss, controlling for other key variables such as exercise Quantitatively based opinion surveys are widely used in the media, with statistics such as the proportion of respondents in favor of a position commonly reported In opinion surveys, respondents are asked a set of structured questions and their responses are tabulated In the field of climate science, researchers compile and compare statistics such as temperature or atmospheric concentrations of carbon dioxide
Empirical relationships and associations are also frequently studied by using some form of General linear model, non-linear model, or by using factor analysis A fundamental principle in quantitative research is that correlation does not imply causation This principle follows from the fact that it is always possible a spurious relationship exists for variables between which covariance is found in some degree Associations may be examined between any combination of continuous and categorical variables using methods of statistics
Views regarding the role of measurement in quantitative research are somewhat divergent Measurement is often regarded as being only a means by which observations are expressed numerically in order to investigate causal relations or associations However, it has been argued that measurement often plays a more important role in quantitative research For example, Kuhn argued that within quantitative research, the results that are shown can prove to be strange This is because accepting a theory based on results of quantitative data could prove to be a natural phenomenon He argued that such abnormalities are interesting when done during the process of obtaining data, as seen below:
When measurement departs from theory, it is likely to yield mere numbers, and their very neutrality makes them particularly sterile as a source of remedial suggestions But numbers register the departure from theory with an authority and finesse that no qualitative technique can duplicate, and that departure is often enough to start a search (Kuhn, 1961, p 180)
In classical physics, the theory and definitions which underpin measurement are generally deterministic in nature In contrast, probabilistic measurement models known as the Rasch model and Item response theory models are generally employed in the social sciences Psychometrics is the field of study concerned with the theory and technique for measuring social and psychological attributes and phenomena This field is central to much quantitative research that is undertaken within the social sciences
Quantitative research may involve the use of proxies as stand-ins for other quantities that cannot be directly measured Tree-ring width, for example, is considered a reliable proxy of ambient environmental conditions such as the warmth of growing seasons or amount of rainfall Although scientists cannot directly measure the temperature of past years, tree-ring width and other climate proxies have been used to provide a semi-quantitative record of average temperature in the Northern Hemisphere back to 1000 A.D When used in this way, the proxy record (tree ring width, say) only reconstructs a certain amount of the variance of the original record The proxy may be calibrated (for example, during the period of the instrumental record) to determine how much variation is captured, including whether both short and long term variation is revealed In the case of tree-ring width, different species in different places may show more or less sensitivity to, say, rainfall or temperature: when reconstructing a temperature record there is considerable skill in selecting proxies that are well correlated with the desired variable
In most physical and biological sciences, the use of either quantitative or qualitative methods is uncontroversial, and each is used when appropriate In the social sciences, particularly in sociology, social anthropology and psychology, the use of one or other type of method can be a matter of controversy and even ideology, with particular schools of thought within each discipline favouring one type of method and pouring scorn on to the other The majority tendency throughout the history of social science, however, is to use eclectic approaches Quantitative methods might be used with a global qualitative frame Qualitative methods might be used to understand the meaning of the conclusions produced by quantitative methods Using quantitative methods, it is possible to give precise and testable expression to qualitative ideas This combination of quantitative and qualitative data gathering is often referred to as mixed-methods research.
Descriptive methods
Descriptive research methods are pretty much as they sound — they describe situations They do not make accurate predictions, and they do not determine cause and effect
There are three main types of descriptive methods: observational methods, case-study methods and survey methods This article will briefly describe each of these methods, their advantages, and their drawbacks This may help you better understand research findings, whether reported in the mainstream media, or when reading a research study on your own
With the observational method (sometimes referred to as field observation) animal and human behavior is closely observed There are two main categories of the observational method — naturalistic observation and laboratory observation
The biggest advantage of the naturalistic method of research is that researchers view participants in their natural environments This leads to greater ecological validity than laboratory observation, proponents say
Ecological validity refers to the extent to which research can be used in real-life situations
Proponents of laboratory observation often suggest that due to more control in the laboratory, the results found when using laboratory observation are more meaningful than those obtained with naturalistic observation
Laboratory observations are usually less time-consuming and cheaper than naturalistic observations Of course, both naturalistic and laboratory observation are important in regard to the advancement of scientific knowledge
Case study research involves an in-depth study of an individual or group of indviduals Case studies often lead to testable hypotheses and allow us to study rare phenomena Case studies should not be used to determine cause and effect, and they have limited use for making accurate predictions
There are two serious problems with case studies — expectancy effects and atypical individuals Expectancy effects include the experimenter’s underlying biases that might affect the actions taken while conducting research These biases can lead to misrepresenting participants’ descriptions Describing atypical individuals may lead to poor generalizations and detract from external validity
In survey method research, participants answer questions administered through interviews or questionnaires After participants answer the questions, researchers describe the responses given In order for the survey to be both reliable and valid it is important that the questions are constructed properly Questions should be written so they are clear and easy to comprehend
Another consideration when designing questions is whether to include open-ended, closed- ended, partially open-ended, or rating-scale questions (for a detailed discussion refer to Jackson, 2009) Advantages and disadvantages can be found with each type:
Open-ended questions allow for a greater variety of responses from participants but are difficult to analyze statistically because the data must be coded or reduced in some manner Closed- ended questions are easy to analyze statistically, but they seriously limit the responses that participants can give Many researchers prefer to use a Likert-type scale because it’s very easy to analyze statistically (Jackson, 2009, p 89)
In addition to the methods listed above some individuals also include qualitative (as a distinct method) and archival methods when discussing descriptive research methods
It is important to emphasize that descriptive research methods can only describe a set of observations or the data collected It cannot draw conclusions from that data about which way the relationship goes — Does A cause B, or does B cause A?
Unfortunately, in many studies published today, researchers forget this fundamental limitation of their research and suggest their data can actually demonstrate or “suggest” causal relationships Nothing could be further from the truth
Jackson, S.L (2009) Research Methods and Statistics: A Critical Thinking Approach 3rd edition Belmont, CA: Wadsworth
Task : Look at the following studies Comment on their components: research questions, methods You may wish to suggest some improvement
Title: "Techniques in the while-reading stage to develop the reading skills for the 11th form students in Hanoi”
1 What techniques are being used in the while-reading stage for the 11th form students in
2 What are advantages and disadvantages of using these techniques?
3 What techniques should be used in the while-reading stage to develop the reading skills for the 11th form students in Hanoi?
Research design: survey, classroom observation
Participants: 100 11th form students and 20 English teachers at three schools in Hanoi
* Student survey questionnaires seek information on students’ attitudes towards learning reading skill, their purpose of learning reading, their difficulties in learning reading and solutions, their evaluation of reading materials in textbook, frequency with which different types of reading activities are used in class, and work arrangement in reading lessons
Teacher survey questionnaires seek information on teachers’ ratings of the importance of reading skills, their evaluation of reading materials in textbook, difficulties their students have when reading, reading strategies they train the students, and frequencies with which they use different reading activities, and work arrangement in reading lessons
Title: A study on the exploitation of authentic materials in teaching reading skills to 1 st year students of English department, QN university
1 What are authentic materials and the reasons for using them to teach foreign languages in general and to teach reading skills in particular?
2 What are the effective ways of exploiting authentic materials in teaching reading skills?
3 Are all of these above really effective and practical?
Participants: 1st year English major students and their teachers, high school students and their teachers (number is not reported)
Instruments: questionnaires for students, interviews with teachers
* Questionnaires for high school students seek information on their attitudes towards types of reading materials used in class, frequencies with which authentic materials are used, students’ attitudes towards authentic materials, and teachers’ methods of teaching reading
* 1st Questionnaires for 1st year students seek information on types of reading materials used in class, frequencies with which authentic materials are used, their attitudes towards authentic materials
2nd Questionnaire for 1st year students (conducted 1 year after) seek information on
29 frequencies with which authentic materials are used in class, purpose of using authentic materials, students’ method of reading authentic materials, students’ evaluation of the effectiveness and difficulty of authentic materials, and students’ preferred ways of using authentic materials
Interviews with high school teachers seek information on their attitudes towards authentic materials, the possibility to use authentic materials for their students, their perceived difficulties if using authentic materials, and their methods of adapting and designing tasks with authentic materials if they have a chance to use this kind of materials
INSTRUMENTS FOR DATA COLLECTION
QUESTIONNAIRES
* advantages and disadvantages: what to use for and when to use
•! considerations in constructing a questionnaire: considerations of length and layout, writing good items, and piloting
•! considerations in administering a questionnaire: sampling, ways of administration, and strategies to increase response rate and response quality
A questionnaire is a written list of questions, the answers to which are recorded by respondents In a questionnaire respondents read the questions, interpret what is expected and then write down the answers The only difference between an interview schedule and a questionnaire is that in the former it is the interviewer who asks the questions (and if necessary, explains them) and records the respondent’s replies on an interview schedule, and in the latter replies are recorded by the respondents themselves This distinction is important in accounting for the respective strengths and weaknesses of the two methods
In the case of a questionnaire, as there is no one to explain the meaning of questions to respondents, it is important that the questions are clear and easy to understand Also, the layout of a questionnaire should be such that it is easy to read and pleasant to the eye, and the sequence of questions should be easy to follow A questionnaire should be developed in an interactive style This means respondents should feel as if someone is talking to them In a questionnaire, a sensitive question or a question that respondents may feel hesitant about answering should be prefaced by an interactive statement explaining the relevance of the question It is a good idea to use a different font for these statements to distinguish them from the actual questions
A questionnaire can be administered in different ways
The mailed questionnaire – The most common approach to collecting information is to send the questionnaire to prospective respondents by mail Obviously this approach presupposes that you have access to their addresses Usually it is a good idea to send a prepaid, self-addressed envelope with the questionnaire as this might increase the response rate A mailed questionnaire must be accompanied by a covering letter (see below for details) One of the major problems with this method is the low response rate In the case of an extremely low response rate, the findings have very limited applicability to the population studied
Collective administration – One of the best ways of administering a questionnaire is to obtain a captive audience such as students in a classroom, people attending a function, participants in a programme or people assembled in one place This ensures a very high response rate as you will find few people refuse to participate in your study Also, as you have personal contact with the study population, you can explain the purpose, relevance and importance of the study and can clarify any questions that respondents may have The author’s advice is that if you have a captive audience for your study, don’t miss the opportunity – it is the quickest way of collecting data, ensures a very high response rate and saves you moneyon postage.
Administration in a public place – Sometimes you can administer a questionnaire in a public place such as a shopping centre, health centre, hospital, school or pub Of course this depends upon the type of study population you are looking for and where it is likely to be found Usually the purpose of the study is explained to potential respondents as they approach and their participation in the study is requested Apart from being slightly more time consuming, this method has all the advantages of administering a questionnaire collectively
According to Brown (2001, p 6), “questionnaires are any written instruments that present respondents with a series of questions or statements to which they are to react either by writing out their answers or selecting from among existing answers”
Broadly speaking, questionnaires can yield three types of data about the respondent: factual, behavioural, and attitudinal
Factual questions (also called “classification” questions or “subject descriptors”) are used to find out about who the respondents are They typically cover demographic characteristics (e.g age, gender, and race), residential location, marital and socioeconomic status, level of education, religion, occupation, as well as any other background information that may be relevant to interpreting the findings of the survey Such additional data in L2 studies often include facts about the learners’ language learning history, amount of time spent in an L2 environment, level of parents’ L2 proficiency, or the L2 course book used
Behavioural questions are used to find out what the respondents are doing or have done in the past They typically ask about people's actions, life‘-styles, habits, and personal history Perhaps the most well-known questions of this type in L2 studies are the items in language learning strategy inventories that ask about the frequency one has used a particular strategy in the past
Attitudinal questions are used to find out what people think This is a broad category concerns attitudes, opinions, beliefs, interests, and values These five interrelated terms are not always distinguished or defined very clearly in the literature a Attitudes concern evaluative responses to a particular target (eg people, institution, situation, and so on) They are deeply embedded in the human mind, and are very often not the product of rational deliberation of facts —they can be rooted back in our past or modelled by certain
35 significant people around us For this reason, they are rather pervasive and resistant to change b Opinions are just as subjective as attitudes, but they are perceived as being more factually based and more changeable People are always aware of their opinions but they may not be fully conscious of their attitudes (Aiken, 1996) c Beliefs have a stronger factual support than opinions and often concern the question as to whether something is true, false, or "right" d Interests are preferences for particular activities e Values on the one hand concern preferences for "life goals” and "ways of life” (eg Christian values); on the other hand they are also used to describe the utility, importance, or worth attached to particular activities, concepts, or objects (e.g., instrumental/ utilitarian value of L2 proficiency)
What kind of data can we collect through questionnaire?
The main attraction of questionnaires is their unprecedented efficiency in terms of (a) researcher time, (b) researcher effort, and (c) financial resources
By administering a questionnaire to a group of people, one can collect a huge amount of information in less than an hour, and the personal investment required will be a fraction of what would have been needed for, say, interviewing the same number of people Furthermore, if the questionnaire is well constructed, processing the data can also be fast and relatively straightforward, especially by using some modem computer software These cost benefit considerations are very important, particularly for all those who are doing research in addition to having a full-time job (Gillham, 2000)
Cost effectiveness is not the only advantage of questionnaires They are also very versatile, which means that they can be used successfully with a variety of people in a variety of situations targeting a variety of topics As a result, the vast majority of research projects in the behavioural and social sciences involve at one stage or another collecting some sort of questionnaire data
"Questionnaires can be designed to minimize, but not eliminate, dishonest, and careless reporting." (Aiken, 1997, p.58)
Constructing a good questionnaire involves a series of steps and procedures, including:
* Deciding on the general features of the questionnaire, such as the length, the format, and the main parts
* Writing effective items/ questions and drawing up an item pool
* Selecting and sequencing the items
* Writing appropriate instructions and examples
* Piloting the questionnaire and conducting item analysis
First of all, we need to specify the maximum length of time that the completion of the
36 questionnaire could take; then we need to consider general format characteristics; and finally we need to think about the issue of anonymity, particularly if we are going to target sensitive/ confidential
When we design a questionnaire, the general temptation is always to cover too much ground by asking everything that might turn out to be interesting This must be resisted: in questionnaire design less is often more because long questionnaires can become counter-productive
How long is the optimal length? It depends on how important the topic of the questionnaire is for the respondent If we feel very strongly about something, we are usually willing to spend several hours answering questions However, most questionnaires in the L2 field concern topics that have a low salience from the respondents’ perspective, and in such cases the optimal length is rather short Perhaps only in exceptional cases should a questionnaire:
*Take more than 30 minutes to complete
A questionnaire that is longer and requires more time than that to complete may be considered too much of an imposition
INTERVIEWS
* Ways to use interview data in your paper
Interviewing is a commonly used method of collecting information from people In many walks of life we collect information through different forms of interaction with others There are many definitions of interviews According to Monette et al (1986: 156), ‘an interview involves an interviewer reading questions to respondents and recording their answers’ According to Burns (1997: 329), ‘an interview is a verbal interchange, often face to face, though the telephone may be used, in which an interviewer tries to elicit information, beliefs or opinions from another person’ Any person-to-person interaction, either face to face or otherwise, between two or more individuals with a specific purpose in mind is called an interview
When interviewing a respondent, you, as a researcher, have the freedom to decide the format and content of questions to be asked of your respondents, select the wording of your questions, decide the way you want to ask them and choose the order in which they are to be asked This process of asking questions can be either very flexible, where you as the interviewer have the freedom to think about and formulate questions as they come to your mind around the issue being investigated, or inflexible, where you have to keep strictly to the questions decided beforehand – including their wording, sequence and the manner in which they are asked Interviews are classified into different categories according to this degree of flexibility as in figure
The strength of unstructured interviews is the almost complete freedom they provide in terms of content and structure You are free to order these in whatever sequence you wish You also have complete freedom in terms of the wording you use and the way you explain questions to your respondents You may formulate questions and raise issues on the spur of the moment, depending upon what occurs to you in the context of the discussion
Unstructured interviews are prevalent in both quantitative and qualitative research The difference is in how information obtained through them in response to your questions is likely to be used In quantitative research you develop response categorisations from responses which are then coded and quantified In qualitative research the responses are used as descriptors, often in verbatim form, and can be integrated with your arguments, flow of writing and sequence of logic
In a structured interview the researcher asks a predetermined set of questions, using the same wording and order of questions as specified in the interview schedule An interview schedule is a written list of questions, open ended or closed, prepared for use by an interviewer in a person-to-person interaction (this may be face to face, by telephone or by other electronic media) Note that an interview schedule is a research tool/instrument for collecting data, whereas interviewing is a method of data collection
One of the main advantages of the structured interview is that it provides uniform information, which assures the comparability of data Structured interviewing requires fewer interviewing skills than does unstructured interviewing
There are numerous practical suggestions in the literature for planning and conducting interviews, ranging from the sensible to the self-evidence The following procedures are recommended by Cohen and Manion (1985), Bell (1987), and Spradley (1979)
Once the research objectives have been established, the researcher has to translate these into interview questions Cohen and Manion (1985, p.305) recommend that the variables under investigation be written down by name in order to facilitate this process At this stage, the question format and response mode need to be considered These will, of course, vary according to the nature of the variables under investigation, the type of subjects, the resources available to the researcher, and so on However, regardless of the issues, the researcher will still need to decide on the type of questions to be used (open-ended versus closed, direct or indirect, etc.) and in what form the responses are to be collected and analysed
Because of the potential problems in the use of the interview that we have already identified, it is very important that interview questions are piloted with a small sample of subjects before being used This gives the researcher the opportunity to find out if the questions are yielding the kind of data required and to eliminate any questions which may be ambiguous or confusing to the interviewee
Bell (1987) points out that efforts should be made to rescue a representative sample, even in a small scale study This will involve selecting appropriate proportions of subgroups of the
56 population, using whatever variables you have determined to be important In the selection process, it may be necessary to negotiate access to informants or data collection sites with individuals or institutions
The structure of the interview itself will depend on the extent to which the sequence of questions is fixed However, it is possible to delineate a number of key elements which may be present These are as follows:
Briefing and explanation: Before the interview begins, the researcher explains the nature of the research and the purpose of the interview to the interviewee and answers any question that he or she may have This includes telling the interviewee how the data are to be used If the data are to be recorded and, in particular, if the data are to be made available to other people, the interviewee's permission must be sought
Questioning: In the less structured ethnographic interview a range of question types may be used In order to encourage the respondent to recount his or her experiences, opinions, and so on, the researcher may also use a variety of other strategies These are outlined by Spradley (1979, p.67)
Walker (1985) makes a number of practical suggestions about the actual conduct of the interview He considers first of all the physical positioning of the interviewer and interviewee, suggesting that sitting side-by-side can often result in a more productive interview than sitting face-to-face (sitting side-by-side can convey the message that the interaction is meant to be cooperative rather than confrontational) The researcher must also decide how the interview is going to be recorded While tape-recording is the obvious choice for someone collecting linguistic data, it is not the only option The other option is for the interviewer to take notes and to use these to reconstruct the interview at a later date Walker points out that tape-recording and note-taking are not simply alternative data collection techniques, but represent quite different ways of going about doing research Some of the relative strengths and weaknesses of the two recording devices are set out in the table below
Tape-recording * Preserves actual language
* Data can be reanalyzed after the event
* Presence of machine off putting
* Core issues masked by irrelevancies
Note-taking *Central issues/ facts recorded
* Off-record statements not recorded
*Actual linguistic data not recorded
* Encoding may interfere with interview
*Status of data may be questioned
Interviews are particularly useful for getting the story behind a participant's experiences The interviewer can pursue in-depth information around a topic Interviews may be useful as follow- up to certain respondents to questionnaires, e.g., to further investigate their responses Usually open-ended questions are asked during interviews
OBSERVATION
* How to conduct the Observation
Observation is one way to collect primary data Observation is a purposeful, systematic and selective way of watching and listening to an interaction or phenomenon as it takes place There are many situations in which observation is the most appropriate method of data collection; for example, when you want to learn about the interaction in a group, study the dietary patterns of a population, ascertain the functions performed by a worker, or study the behaviour or personality traits of an individual It is also appropriate in situations where full and/or accurate information cannot be elicited by questioning, because respondents either are not co-operative or are unaware of the answers because it is difficult for them to detach themselves from the interaction In summary, when you are more interested in the behaviour than in the perceptions of individuals, or when subjects are so involved in the interaction that they are unable to provide objective information about it, observation is the best approach to collect the required information
There are two types of observation:
Participant Observation is when you, as a researcher, participate in the activities of the group being observed in the same manner as its members, with or without their knowing that they are being observed For example, you might want to examine the reactions of the general population towards people in wheelchairs You can study their reactions by sitting in a wheelchair yourself Or you might want to study the life of prisoners and pretend to be a prisoner in order to do this
Non-participant Observation, on the other hand, is when you, as a researcher, do not get involved in the activities of the group but remain a passive observer, watching and listening to its activities and drawing conclusions from this For example, you might want to study the functions carried out by nurses in a hospital As an observer, you could watch, follow and record the activities as they are performed After making a number of observations, conclusions could be drawn about the functions nurses carry out in the hospital Any occupational group in any setting can be observed in the same manner
4.3.3 Situations in which observations can be made
Observations can be made under two conditions:
Observing a group in its natural operation rather than intervening in its activities is classified as observation under natural conditions Introducing a stimulus to the group for it to react to and observing the reaction is called controlled observation
Determine what you want to observe and why Are looking to see how students respond to a new environment? How customers interact with employees? How bosses interact with subordinates?
When conducting observations, you are trying to learn habits, patterns, behaviors, reactions, and general information about people in a particular environment to better understand what they do and, hopefully eventually, why they do it (though observations alone often won’t tell you the
Identifying the theory that drives the inquiry
The qualitative observational researcher must determine what underlying theory or model should inform the research This may mean replicating or building on an earlier study, or it may mean formulating a new model or theory by which to conduct the study Either way, the theory or model chosen will help the researcher determine how to structure the study
Determine whether you are conducting an informal or a formal observation (see explanations to the far right.) Knowing your objective, determine if there are specific questions you have or if you are going in completely open-minded What you hope to learn will help you know what specifically to look for Be prepared when entering an observation space by having a sound understanding of the type of information you are trying learn
Unlike most scientific research methods, qualitative observational inquiry does not require the researcher to define a precise set of issues in the initial phases; these issues often emerge from the study over time While some qualitative inquiries may begin with a set of questions, it is common for theories about group behavior and interactions to emerge as a result of the observer's exploratory work (emergent design) And, those theories may identify relevant questions for further research
The goal of qualitative observational research is to define and answer a specific research problem or question, but this problem or question may or may not be defined at the time when the researcher first begins the study Some researchers like to enter the field with a specific research problem already in mind While such researchers still want to let events unfold as freely as possible once in the field, they believe that by defining the research problem in advance they are better able to observe the study group or culture and identify specific patterns of behavior Other qualitative observational researchers like to enter the field first and let the research questions or problems identify themselves These researchers believe that entering the field with a specifically defined research question may bias their observations, and they may fail to notice relationships or behavior patterns that are important in understanding the study group or culture Whatever approach is taken in determining the research question, the observer does need to be clear about the purpose, scope, and focus of the study and identify the subjects and the context in which they will be studied
To make observations most effective, it’s important that you minimize or eliminate any disruptive or unfamiliar devices into the environment you wish to observe For example, it is often least effective to videorecord observations in situations where the people being observed know they are being filmed (but it’s usually unethical to film without telling them Note-taking is the most common method, though in some public spaces you can take photographs, audio recordings, and other methods
Visit the space you are hoping to get information from Be as unobtrusive as possible, taking notes, photographs, audio, and film, only where it is allowed, you have permission, and it makes sense for the research without disrupting the environment If you are doing formal observations, will you need to code certain behaviors, actions, words, visuals, and other observed data
Separate the difference between what you observed (which are factual behaviors) and why what you observed happened Typically, to make some sense of your observed data, you will need to
65 interview people in the environment you are observing, either during the observation itself, or afterwards Make connections between interactions, responses, behaviors, and other phenomena
There are many ways of recording observations The selection of a method of recording depends upon the purpose of the observation The way an observation is recorded also determines whether it is a quantitative or qualitative study Narrative and descriptive recording is mainly used in qualitative research but if you are doing a quantitative study you would record an observation in categorical form or on a numerical scale Keep in mind that each method of recording an observation has its advantages and disadvantages:
Narrative recording – In this form of recording the researcher records a description of the interaction in his/her own words Such a type of recording clearly falls in the domain of qualitative research Usually, a researcher makes brief notes while observing the interaction and then soon after completing the observation makes detailed notes in narrative form In addition, some researchers may interpret the interaction and draw conclusions from it The biggest advantage of narrative recording is that it provides a deeper insight into the interaction However, a disadvantage is that an observer may be biased in his/her observation and, therefore, the interpretations and conclusions drawn from the observation may also be biased In addition, interpretations and conclusions drawn are bound to be subjective reflecting the researcher’s perspectives Also, if a researcher’s attention is on observing, s/he might forget to record an important piece of interaction and, obviously, in the process of recording, part of the interaction may be missed Hence, there is always the possibility of incomplete recording and/or observation In addition, when there are different observers the comparability of narrative recording can be a problem
Using scales – At times some observers may prefer to develop a scale in order to rate various aspects of the interaction or phenomenon The recording is done on a scale developed by the observer/researcher A scale may be one-, two- or three-directional, depending upon the purpose of the observation For example, in the scale in Figure 9.2 – designed to record the nature of the interaction within a group – there are three directions: positive, negative and neutral
ESTABLISHING THE VALIDITY AND RELIABILITY OF A RESEARCH
The Concept of Validity
To examine the concept of validity, let us take a very simple example Suppose you have designed a study to ascertain the health needs of a community In doing so, you have developed an interview schedule Further suppose that most of the questions in the interview schedule relate to the attitude of the study population towards the health services being provided to them Note that your aim was to find out about health needs but the interview schedule is finding out what attitudes respondents have to the health services; thus, the instrument is not measuring what it was designed to measure The author has come across many similar examples among students and less skilled researchers
In terms of measurement procedures, therefore, validity is the ability of an instrument to measure what it is designed to measure: ‘Validity is defined as the degree to which the researcher has measured what he has set out to measure’ (Smith 1991: 106) According to Kerlinger, ‘The commonest definition of validity is epitomised by the question: Are we measuring what we think we are measuring?’ (1973: 457) Babbie writes, ‘validity refers to the extent to which an empirical measure adequately reflects the real meaning of the concept under consideration’ (1989: 133) These definitions raise two key questions:
- Who decides whether an instrument is measuring what it is supposed to measure?
- How can it be established that an instrument is measuring what it is supposed to measure
Obviously the answer to the first question is the person who designed the study, the readership of the report and experts in the field The second question is extremely important
On what basis do you (as a researcher), a reader as a consumer or an expert make this judgement? In the social sciences there appear to be two approaches to establishing the validity of a research instrument These approaches are based upon either logic that underpins the construction of the research tool or statistical evidence that is gathered using information generated through the use of the instrument Establishing validity through logic implies justification of each question in relation to the objectives of the study, whereas the statistical procedures provide hard evidence by way of calculating the coefficient of correlations between the questions and the outcome variables
Establishing a logical link between the questions and the objectives is both simple and difficult It is simple in the sense that you may find it easy to see a link for yourself, and difficult because your justification may lack the backing of experts and the statistical evidence to convince others Establishing a logical link between questions and objectives is easier when the questions relate to tangible matters
For example, if you want to find out about age, income, height or weight, it is relatively
69 easy to establish the validity of the questions, but to establish whether a set of questions is measuring, say, the effectiveness of a programme, the attitudes of a group of people towards an issue, or the extent of satisfaction of a group of consumers with the service provided by an organisation is more difficult When a less tangible concept is involved, such as effectiveness, attitude or satisfaction, you need to ask several questions in order to cover different aspects of the concept and demonstrate that the questions asked are actually measuring it Validity in such situations becomes more difficult to establish, and especially in qualitative research where you are mostly exploring feelings, experiences, perceptions, motivations or stories
It is important to remember that the concept of validity is pertinent only to a particular instrument and it is an ideal state that you as a researcher aim to achieve.
Types of Validity in Quantitative Research
There are three types of validity in quantitative research:
The judgement that an instrument is measuring what it is supposed to is primarily based upon the logical link between the questions and the objectives of the study Hence, one of the main advantages of this type of validity is that it is easy to apply Each question or item on the research instrument must have a logical link with an objective Establishment of this link is called face validity It is equally important that the items and questions cover the full range of the issue or attitude being measured Assessment of the items of an instrument in this respect is called content validity In addition, the coverage of the issue or attitude should be balanced; that is, each aspect should have similar and adequate representation in the questions or items Content validity is also judged on the basis of the extent to which statements or questions represent the issue they are supposed to measure, as judged by you as a researcher, your readership and experts in the field Although it is easy to present logical arguments to establish validity, there are certain problems:
- The judgement is based upon subjective logic; hence, no definite conclusions can be drawn Different people may have different opinions about the face and content validity of an instrument
- The extent to which questions reflect the objectives of a study may differ If the researcher substitutes one question for another, the magnitude of the link may be altered Hence, the validity or its extent may vary with the questions selected for an instrument
‘In situations where a scale is developed as an indicator of some observable criterion, the scale’s validity can be investigated by seeing how good an indicator it is’ (Moser & Kalton 1989: 356)
Suppose you develop an instrument to determine the suitability of applicants for a profession The instrument’s validity might be determined by comparing it with another assessment, for example by a psychologist, or with a future observation of how well these applicants have done in the job If both assessments are similar, the instrument used to make the assessment at the time of selection is assumed to have higher validity These types of comparisons establish two types of validity: predictive validity and concurrent validity Predictive validity is judged by the degree to which an instrument can forecast an outcome Concurrent validity is judged by how well an instrument compares with a second assessment concurrently done: ‘It is usually possible to express predictive validity in terms of the correlation coefficient between the predicted status and the criterion Such a coefficient is called a validity coefficient’ (Burns 1997: 220)
Construct validity is a more sophisticated technique for establishing the validity of an instrument It is based upon statistical procedures It is determined by ascertaining the contribution of each construct to the total variance observed in a phenomenon
Suppose you are interested in carrying out a study to find the degree of job satisfaction among the employees of an organisation You consider status, the nature of the job and remuneration as the three most important factors indicative of job satisfaction, and construct questions to ascertain the degree to which people consider each factor important for job satisfaction After the pre-test or data analysis you use statistical procedures to establish the contribution of each construct (status, the nature of the job and remuneration) to the total variance (job satisfaction) The contribution of these factors to the total variance is an indication of the degree of validity of the instrument The greater the variance attributable to the constructs, the higher the validity of the instrument
One of the main disadvantages of construct validity is that you need to know about the required statistical procedures.
The Concept of Reliability
The concept of reliability in relation to a research instrument has a similar meaning: if a research tool is consistent and stable, hence predictable and accurate, it is said to be reliable The greater the degree of consistency and stability in an instrument, the greater its reliability Therefore, ‘a scale or test is reliable to the extent that repeat measurements made by it under constant conditions will give the same result’ (Moser & Kalton 1989: 353)
The concept of reliability can be looked at from two sides:
1.! How reliable is an instrument?
The first question focuses on the ability of an instrument to produce consistent measurements When you collect the same set of information more than once using the same instrument and get the same or similar results under the same or similar conditions, an instrument is considered to be reliable
The second question focuses on the degree of inconsistency in the measurements made by an instrument- that is, the extent of difference in the measurements when you collect the same set of information more than once, using the same instrument under the same or similar conditions Hence, the degree of inconsistency in the different measurements is an indication of the extent of its inaccuracy This ‘error’ is a reflection of an instrument’s unreliability Therefore, reliability is the degree of accuracy or precision in the measurements made by a research instrument The lower the degree of ‘error’ in an instrument, the higher the reliability
Let us take an example Suppose you develop a questionnaire to ascertain the prevalence of domestic violence in a community You administer this questionnaire and find that domestic violence is prevalent in, say, 5 per cent of households If you follow this with another survey using the same questionnaire on the same population under the same conditions, and discover that the prevalence of domestic violence is, say, 15 per cent, the questionnaire has not given a comparable result, which may mean it is unreliable The less the difference between the two sets of results, the higher the reliability of the instrument.
Factors Affecting the Reliability of a Research Instrument
In the social sciences it is impossible to have a research tool which is 100 per cent accurate, not only because a research instrument cannot be so, but also because it is impossible to control the factors affecting reliability Some of these factors are:
The wording of questions – A slight ambiguity in the wording of questions or statements can affect the reliability of a research instrument as respondents may interpret the questions differently at different times, resulting in different responses
The physical setting – In the case of an instrument being used in an interview, any change in the physical setting at the time of the repeat interview may affect the responses given by a respondent, which may affect reliability
The respondent’s mood – A change in a respondent’s mood when responding to questions or writing answers in a questionnaire can change and may affect the reliability of that instrument The interviewer’s mood – As the mood of a respondent could change from one interview to another so could the mood, motivation and interaction of the interviewer, which could affect the responses given by respondents thereby affecting the reliability of the research instrument
The nature of interaction – In an interview situation, the interaction between the interviewer and the interviewee can affect responses significantly During the repeat interview the responses given may be different due to a change in interaction, which could affect reliability
The regression effect of an instrument – When a research instrument is used to measure attitudes towards an issue, some respondents, after having expressed their opinion, may feel that they have been either too negative or too positive towards the issue The second time they may express their opinion differently, thereby affecting reliability.
Methods of Determining the Reliability of an Instrument in Quantitative Research
There are a number of ways of determining the reliability of an instrument and these can be classified as either external or internal consistency procedures
External consistency procedures compare findings from two independent processes of data collection with each other as a means of verifying the reliability of the measure The two methods of doing this are as follows:
1 Test/retest – This is a commonly used method for establishing the reliability of a research tool In the test/retest (repeatability test) an instrument is administered once, and then again, under the same or similar conditions The ratio between the test and retest scores (or any other finding, for example the prevalence of domestic violence, a disease or incidence of an illness) is an indication of the reliability of the instrument – the greater the value of the ratio, the higher the reliability of the instrument As an equation, (test score)/(retest) = 1 or (test score) – (retest) = 0
A ratio of 1 shows 100 per cent reliability (no difference between test and retest) and any deviation from it indicates less reliability – the less the value of this ratio, the less the reliability of the instrument Expressed in another way, zero difference between the test and retest scores is an indication of 100 per cent reliability The greater the difference between scores or findings obtained from the two tests, the greater the unreliability of the instrument The main advantage of the test/retest procedure is that it permits the instrument to be compared with itself, thus avoiding the sort of problems that could arise with the use of another instrument
The main disadvantage of this method is that a respondent may recall the responses that s/he gave in the first round, which in turn may affect the reliability of the instrument Where an instrument is reactive in nature (when an instrument educates the respondent with respect to what the researcher is trying to find out) this method will not provide an accurate assessment of its reliability One of the ways of overcoming this problem is to increase the time span between the two tests, but this may affect reliability for other reasons, such as the maturation of respondents and the impossibility of achieving conditions similar to those under which the questionnaire was first administered
2 Parallel forms of the same test – In this procedure you construct two instruments that are intended to measure the same phenomenon The two instruments are then administered to two similar populations The results obtained from one test are compared with those obtained from the other If they are similar, it is assumed that the instrument is reliable
The main advantage of this procedure is that it does not suffer from the problem of recall found in the test/retest procedure Also, a time lapse between the two tests is not required A disadvantage is that you need to construct two instruments instead of one Moreover, it is extremely difficult to construct two instruments that are comparable in their measurement of a phenomenon It is equally difficult to achieve comparability in the two population groups and in the two conditions under which the tests are administered
The idea behind internal consistency procedures is that items or questions measuring the same phenomenon, if they are reliable indicators, should produce similar results irrespective of their number in an instrument Even if you randomly select a few items or questions out of the total pool to test the reliability of an instrument, each segment of questions thus constructed should reflect reliability more or less to the same extent It is based upon the logic that if each item or question is an indicator of some aspect of a phenomenon, each segment constructed will still reflect different aspects of the phenomenon even though it is based upon fewer items/questions Hence, even if we reduce the number of items or questions, as long as they reflect some aspect of a phenomenon, a lesser number of items can provide an indication of the reliability of an instrument The internal consistency procedure is based upon this logic The following method is commonly used for measuring the reliability of an instrument in this way:
The split-half technique – This technique is designed to correlate half of the items with the other half and is appropriate for instruments that are designed to measure attitudes towards an issue or phenomenon The questions or statements are divided in half in such a way that any two questions or statements intended to measure the same aspect fall into different halves The scores obtained by administering the two halves are correlated Reliability is calculated by using the product moment correlation (a statistical procedure) between scores obtained from the two halves Because the product moment correlation is calculated on the basis of only half the instrument, it needs to be corrected to assess reliability for the whole This is known as stepped-up reliability The stepped- up reliability for the whole instrument is calculated by a formula called the Spearman–Brown formula (a statistical procedure).
Validity and Reliability in Qualitative Research
One of the areas of difference between quantitative and qualitative research is in the use of and the importance given to the concepts of validity and reliability The debate centres on whether or not, given the framework of qualitative research, these concepts can or even should be applied in qualitative research As you know, validity in the broader sense refers to the ability of a research instrument to demonstrate that it is finding out what you designed it to and reliability refers to consistency in its findings when used repeatedly In qualitative research, as answers to research questions are explored through multiple methods and procedures which are both flexible and evolving, to ensure standardisation of research tools as well as the processes becomes difficult As a newcomer to research you may wonder how these concepts can be applied in qualitative research when it does not use standardised and structured methods and procedures which are the bases of testing validity and reliability as defined in quantitative research You may ask how you can ascertain the ability of an instrument to measure what it is expected to and how consistent it is when the data collection questions are neither fixed nor structured
However, there are some attempts to define and establish validity and reliability in qualitative research In a chapter entitled ‘Competing paradigms in qualitative research’ (pp
105–117) in the Handbook of Qualitative Research, edited by Denzin and Lincoln (1994), Guba and Lincoln have suggested a framework of four criteria as a part of the constructivism paradigm paralleling ‘validity’ and ‘reliability’ in quantitative research According to them, there are two sets of criteria ‘for judging the goodness or quality of an inquiry in constructivism paradigm’ (1994: 114) These are: ‘trustworthiness’ and ‘authenticity’
According to Guba and Lincoln, trustworthiness in a qualitative study is determined by four indicators – credibility, transferability, dependability and confirmability – and it is these four indicators that reflect validity and reliability in qualitative research ‘The trustworthiness criteria of credibility (paralleling internal validity), transferability (paralleling external validity), dependability (paralleling reliability), and confirmability (paralleling objectivity)’, according to Guba and Lincoln (1994: 114) closely relates to the concepts of validity and reliability
Trochim and Donnelly (2007) compare the criteria proposed by Guba and Lincoln in the following table with validity and reliability as defined in quantitative research:
Traditional criteria for judging quantitative research
Alternative criteria for judging qualitative research
+ Credibility – According to Trochim and Donnelly (2007: 149), ‘credibility involves establishing that the results of qualitative research are credible or believable from the perspective of the participant in the research’ As qualitative research studies explore perceptions, experiences, feelings and beliefs of the people, it is believed that the respondents are the best judge to determine whether or not the research findings have been able to reflect their opinions and feelings accurately Hence, credibility, which is synonymous to validity in quantitative research, is judged by the extent of respondent concordance whereby you take your findings to those who participated in your research for confirmation, congruence, validation and approval The higher the outcome of these, the higher the validity of the study
+ Transferability – This ‘refers to the degree to which the results of qualitative research can be generalized or transferred to other contexts or settings’ (2007: 149) Though it is very difficult to establish transferability primarily because of the approach you adopt in qualitative research, to some extent this can be achieved if you extensively and thoroughly describe the process you adopted for others to follow and replicate
+ Dependability – In the framework suggested by Guba and Lincoln this is very similar to the concept of reliability in quantitative research: ‘It is concerned with whether we would obtain the same results if we could observe the same thing twice’ (Trochim and Donnelly 2007: 149) Again, as qualitative research advocates flexibility and freedom, it may be difficult to establish unless you keep an extensive and detailed record of the process for others to replicate to ascertain the level of dependability
+ Confirmability – This ‘refers to the degree to which the results could be confirmed or corroborated by others’ (2007: 149) Confirmability is also similar to reliability in quantitative research It is only possible if both researchers follow the process in an identical manner for the results to be compared
To the author’s mind, to some extent, it is possible to establish the ‘validity’ and
‘reliability’ of the findings in qualitative research in the form of the model suggested by Guba and Lincoln, but its success is mostly dependent upon the identical replication of the process and methods for data collection which may not be easy to achieve in qualitative research
1 Explore how the concepts of reliability and validity are applicable to research in your academic field or profession
2 Consider what strategies or procedures you could put in place to limit the affect on reliability of the following factors:
- regression wording of questions; effect of an instrument
WRITING UP A RESEARCH PAPER
HOW TO STRUCTURE A RESEARCH PAPER
The process of writing up your research can be greatly facilitated by working out a structure for the report before beginning Below is an example of how a research report is structured
Title: The effect of instruction on the development of pragmatic competence in the EFL context: A study based on suggestions
Aims of the study and Research Questions
Chapter 1 Introduction: Pragmatics and language learning
Chapter 2 Literature review: Developing pragmatics in instructional context
Chapter 3 Classroom research on pragmatics
Data collection procedures and instructional sessions
Coding and analysis of data
It can be seen that the paper falls naturally into several sections It begins with an introduction, purpose, and rationale, including an indication of the organization of the study This is followed by several chapters summarizing the literature review The study itself is then described, and the results are presented and discussed The paper concludes with a discussion of the theoretical and practical implications of the research, the limitations of the research, and suggestions for further studies While research papers will vary, the above sample is provided because it contains all of the essential elements of a good report
Note: The expected format of a research paper may vary from institution to institution, so make sure you check with your institution before writing up your research paper
The purpose(s) of the different parts of a research report is summarized in the following
Table 13: Overview of the structure of your thesis
Section Purpose of the Section
Abstract To briefly introduce the reader to the aims of the study, the methodology, results and findings
To state a clear overall purpose for the study, often framed in a discussion of the need the research is satisfying
To define the research question(s) of the study
To give a very brief background of relevant theory and practice for your topic
Literature Review To demonstrate that you are aware of key theories and research in this area
To summarize what conclusions have been reached in the research literature and whether different writers agree or not
To highlight main issues and controversies around the problem
To show the gaps in the literature and how your research therefore fits in
Methodology To demonstrate that you are aware of the research methods used to study this topic
To justify the research method and approach you have taken for your study
To show how and why you chose your research participants, and to describe the participants and the context in more detail
To explain and justify the method of data collection and
Results To present the findings of your research in an orderly manner, using headings planned in your methodology or headings arising from patterns found in the research
Discussion To comment on the trends/findings and show your understanding of what your data suggests
To compare your findings with what the literature says and with what you may have predicted or what you set out to find out more about
To highlight anything unexpected that came up
Conclusion To sum up your findings and highlight the significance of the outcomes of your study
To outline any implications or recommendations indicated by the findings
To discuss the limitations of your study and indicate where further research is needed
References To list alphabetically all the reference materials that have been cited in the text of the report
Appendices To present relevant details such as letters (e.g regarding the ethics & conduct of the project) to participants and organisations
To present details of questionnaires, surveys and other relevant instruments that you developed for the purpose of the study
To present relevant documents, e.g reports/ policy/ historical documents.
HOW TO WRITE THE STRUCTURES OF A RESEARCH PAPER
+ Three kinds of main titles:
- Declarative title: These titles include what papers say (their main conclusions), not just what they cover, for instance: "Adipose gene expression prior to weight loss can differentiate and weakly predict dietary responders"
- Descriptive or neutral titles: A descriptive title only describes the subject of the paper and does not reveal the main outcome or conclusion For example: "A worldwide phylogeography for the human X chromosome"
- Interrogative (question) titles: These titles indicate the subject of the paper in the form of a question that appeal to the curiosity of readers (Gustavii 2008) "Are there rearrangement hotspots in the human genome?"
+ How to write a good title:
- The title should be started with a keyword E.g: Osteoporosis: effect of dietary calcium intake
- The title must reflect the main contents of the research E.g Zinc supplementation for growth in preterm infants: a randomized controlled trial
- The title should be brief and specific
- The title should contain new elements of the research
- No abbreviations are used in the title
- The title must not be vague
- The title should not be long (less than 20 words)
- The title should not be a declaration
- Avoid the unnecessary words such as On the, Investigation…
An abstract is a summary of a body of information An abstract expresses the main claim and argument of a paper In most disciplines, it never includes bibliographic citations An abstract concisely highlights or reviews the major points covered along with the content and scope of the writing
An abstract can also be a useful tool for writers to check that they have a clear grasp of their thesis and argument It the writer can state the thesis and argument clearly in a few sentences— and in such a way that someone who doesn‘t know the subject will still be able to understand the main idea-then the writer knows she has a good grasp of the ideas she is trying to express An abstract says everything of central importance in a way that gives the reader a clear overview of what is contained in the essay
6.2.1 Essential elements of the abstract are
* Background: A simple opening sentence or two placing the work in context
* Aims: One or two sentences giving the purpose of the work
* Method(s): One or two sentences explaining what was done (described at length only if it is unusual)
* Results: One or two sentences indicating the main findings (absolutely essential)
* Conclusions: One sentence giving the most important consequence of the work (telling what the results mean)
6.2.2 Questions that an abstract answers
* Why did you do this study or project?
* What did you do, and how?
* What do your findings mean?
If the paper is about a new method the last two questions might be changed to:
* What are the advantages of the method?
* How well does it work? after the details of your paper may be forgotten
An abstract should include the few things you would like your reader to remember long after the details of your paper may be forgotten
* Well developed paragraphs are unified, coherent, concise, and able to stand alone
* Uses an introduction/ body/ conclusion structure which presents the report's purpose, results, conclusions, and recommendations in that order
* Follows strictly the chronology of the report
* Provides logical connections (or transitions) between the information included
* Adds no new information, but simply summarizes the report
* Is understandable to a wide audience
* Uses passive verbs to downplay the author and emphasize the information
6.2.4 Steps to writing effective abstracts
Reread the research report with the goal of abstracting in mind Look specifically for these main parts of the report: purpose, methods, scope, results, conclusions, and recommendation
After you've finished rereading the report, write a rough draft without looking back at what you‘re abstracting Don't merely copy key sentences from the report: you'll put in too much or too little information Don’t rely on the way material was phrased in the report: summarize information in a new way
* Do not commence with "this paper .", "this report ." or similar It is better to write about the research than about the paper
39.!Do not explain the sections or parts of the paper
40.!Avoid sentences that end in " is described", " .is reported", " .is analyzed" or similar
41.!Do not begin sentences with "it is suggested that .” "it is believed that .", “it is felt that "or similar In every case, the four words can be omitted without damaging the essential message
42.!Do not repeat or rephrase the title
43.!Do not refer in the abstract to information that is not in the document
44.!lf possible, avoid trade names, acronyms, abbreviations, or symbols You would need to explain them, and that takes too much room
The abstract should be about the research, not about the act of writing
6.2.6 Where to find examples of abstracts:
* The best source of example abstracts is journal articles Go to the library and look at biology journals, or look at electronic journals on the web
45.!Read the abstract; read the article Pick the best ones, the examples where the abstract makes the article easier to read, and figure out how they do it
46.!Not everyone writes good abstracts, even in refereed journals, but the more abstracts you read, the easier it is to spot the good ones
(Source: _http:// www.olemiss.edu/ depts/writing_center/grabstract.html)
This study examines how Australian learners of Indonesian perform requests in everyday situations compared to Indonesian native speakers
THE DATA WERE COLLECTED BY MEANS OF
INTERACTIVE ROLE PLAY Results showed that both groups of subjects favour the same request type: query preparatory However, learner subjects used a different modal verb to perform their requests, used Want statements and hint statements more than native Indonesians, and used elided imperatives less often The findings emphasise the potential importance of positive pragmatic transfer, of the negative effect of formal instruction, and of learners’ concern for clarity The study also has developmental implications for L2 pragmatics Most broadly, it supports the claim of Bialystok
(1993) that the main task for adult learners of L2 pragmatics is to gain control over knowledge rather than knowledge itself It also suggests that Want statements and hint statements will be used in inverse proportion to language proficiency, that competence in aspects of requesting linked to conversational management will develop in tandem with linguistic proficiency, and that a Ll- shaped curve of development is likely in the acquisition of request strategies realized by very short forms
(Hassal, T (2003) Requests by Australian learners of Indonesian
For many years, English has proved its importance in peoples life and work As an international language, it is widely used in all
fields of activity throughout the world In Vietnam, English is taught as an important part and compulsory subject in most secondary schools English teaching and the teaching of reading comprehension, however, still leaves some shortcomings, especially at the time when communication is largely considered as the main purpose of language teaching and learning THIS
SITUATION INSPIRES THE WRITING OF THIS STUDY IN
AN ATTEMPT TO SEARCH FOR SOME APPROPRIATE
TECHNIQUES IN TEACHING READING COMPREHENSION
TO DEVELOP STUDENTS’ READING SKILLS IN THE
WHILE-READING STAGE The study consists of the following parts: Introduction will present the rationale the aims, the scope, the methods, and the research questions of the study Chapter 1 provides theoretical overview of the background concerning reading comprehension and teaching reading organization of comprehension Chapter 2 is mentioning the present situation of teaching the research and learning reading in English of the 11th form students in Hanoi It is completed by analyzing statistics of
This is an overview of organization of the research report, which should not be included in the abstract This abstract does not provide information on the methods, the
81 the two survey questionnaires conducted among students and teachers at the three upper-secondary schools: Lomonosov, Viet
Duc, and Thang Long and some class observations at these schools Chapter 3 presents some techniques which are mainly used in the while-reading stage They are categorized according to different purposes in reading a text, i.e reading for general ideas, reading for specific information and reading for further understanding Chapter 4 is a sample lesson plan applying techniques which have been presented in the previous chapter
Conclusion is the last part which summarises the obtained results of the study It also concludes suggestions for further study results, and the implications of the study
The purpose of the introduction is to frame the paper for its readers It should provide:
- a brief description of the topic including any background that might be necessary for the general reader to understand the research area
- a statement as to why the topic is worth researching or why it could be found interesting
- a statement of the research objectives or hypothesis
- a statement of the method(s) and the scope of study
- an overview of the rest of the paper
- Organize your ideas, making one major point with each paragraph
- Present background information only as needed in order support a position The reader does not want to read everything you know about a subject
- State the hypothesis/ objective precisely - do not oversimplify
- As always, pay attention to spelling, clarity and appropriateness of sentences and phrases
Ask yourself questions like these:
1 What is the specific problem, or question that my literature review helps to define?
2 What type of literature review am I conducting? Am l looking at issues of theory?
3 What is the scope of my literature review? What types of publications am I using (e.g., journals, books, government documents, popular media)?
4 How good was my information seeking? Has my search been wide enough to ensure I've found all the relevant material? Has it been narrow enough to exclude irrelevant material? Is the number of sources I've used appropriate for the length of my paper?
5 Have I critically analysed the literature I use? Instead of just listing and summarizing items, do I assess them, discussing strengths and weaknesses?
6 Have I cited and discussed studies contrary to my perspective?
7 Will the reader find my literature review relevant, appropriate, and useful?
Ask yourself questions like these about each book or article you include:
1 Has the author formulated a problem/ issue?
2 Is it clearly defined? Is its significance (scope, severity, relevance) clearly established?
3 Could the problem have been approached more effectively from another perspective?
4 What is the author's research orientation (erg, interpretive, critical science, combination)?
5 What is the author's theoretical framework (e.g., psychological, developmental, feminist)?
6 What is the relationship between the theoretical and research perspectives?
7 Has the author evaluated the literature relevant to the problem/ issue? Does the author include literature taking positions she or he does not agree with?
8 In a research study, how good are the basic components of the study design (e.g., population, intervention, and outcome)? How accurate and valid are the measurements? Is the analysis of the data accurate and relevant to the research question? Are the conclusions validly based upon the data and analysis?
9 In material written for a popular readership, does the author use appeals to emotion, one- sided examples, or rhetorically-charged language and tone? Is there an objective basis to the reasoning, or is the author merely "proving" what he or she already believes?
10 How does the author structure the argument? Can you "deconstruct" the flow of the argument to see whether or where it breaks down logically (e.g., in establishing cause-effect relationships)?