Part II :Critical Review of QUALITATIVE and QUANTITATIVE approaches to research in Applied Linguistics in TESOL

Advantages and limitations of Quantitative Approach

          It is assumed that excessive empathy and interpretation bias are likely to be melted into the data subjectively collected from the world of the participant unless the research is objectively, systematically and reliably experimented. In this sense, Burns (2000:9) argues that the quantitative approach is founded on the assertion that precision, control and reliability (repeatability) achieved through statistical analysis such as sampling and design and quantitative and reliable measurement provide answers which have a much firmer basis than the lay person’s common sense or intuition or opinion considered to be salient factors in the qualitative approach. Quantitative research also generates reliable population based and gereralizable data and is well suited to establishing cause-and-effect relationships.

          However, there are limitations in the approach as human beings are very complex subjects and cannot easily be analyzed in ways that suit the physical sciences. It is also worth remembering that the scientific approach can never be completely objective because subjectivity is involved in both the choice of a problem considered worth investigation and the way results are interpreted.

 

Advantages and limitations of Qualitative Approach

           Qualitative research generates rich and detailed data that contribute to in-depth understanding of the context as the emphasis in this approach is upon description, uncovering patterns in the data, giving voice to the participants, and maintaining flexibility as the research project develops.

          However, there are limits to the qualitative approach. Adequate validity and reliability will be difficult to achieve and may leave researchers open to criticism. One of strengths in the qualitative approach, which is a narrow scope providing great detail about a specific interaction or situation, is, on the other hand, taken into account as one of its limitations, because of a lack of generalizability (how applicable or true the conclusions would be to different situations or individuals). In addition, the time required for data collection, analysis and interpretation is considered as another problem.

How limitations of each approach can be addressed

          As briefly discussed above, each approach has its strengths and limitations. Then, some questions may arise: Are there any ways to reduce limitations? If so, in which way can it be attempted? It can be attempted to incorporate elements of both to ensure that their studies are as accurate and thorough as possible rather than to discount either approach for its drawbacks. In other words, both approaches can be used in conjunction with each other. For example, interviews considered as qualitative techniques may be structured and analyzed in a quantitative manner, as when numeric data is collected or when non-numeric answers are categorized and coded in numeric form. Additionally, triangulation, a commonly used technique through a variety of research methods can be another effective way to reinforce drawbacks or each approach. For example, once statistical data about responses is gathered by questionnaire (normally quantitative), interviewing selected members of the sample used in the questionnaire to back up the result from the previous method and research in more depth. In this way, advantages of quantitative approach, which lie in generalizability and those of qualitative approach, which lie in validity are likely to be well combined.

 

Reliability and Validity

        Reliability and validity are likely to always be issues to be taken into consideration to pursue valid, accurate and generalizable research. In other words, the issues of them determine what level of confidence we may have in any research.

 

1) Reliability of each approach

        As mentioned early, the quantitative approach is considered the scientific approach on a basis of objectivity, stability, systemicity and precision proven by statistical data. In this sense, the same result is likely to be gained consistently under the same conditions. This is referred to as reliability and is regarded salient in the quantitative approach. Babbie (1998) says that reliability refers to whether a particular research technique will yield the same results if applied repeatedly to the same object. More standardized methods yield more reliable data. Quantitative research is considered as inherently more reliable than qualitative research because of the nature of the techniques used in data collection and analysis. However, it can be submitted this is too general a statement when applied to second language acquisition. For example, research data on English learning processes from a survey of ESL students in Australia cannot be simply applied or compared to a survey of EFL students in Korea due to the differences of language environment. This may rather provide misleading results. In this sense, reliability is likely to be differently interpreted according to a variety of factors such as environment, culture, value, custom, belief and thoughts, which is more likely to be taken into account in the qualitative approach There are four general classes of reliability estimates in the quantitative approach, which include the test-retest, parallel forms, internal consistency and split-half methods, each of which estimates reliability in a different way.                     

        In the meantime, reliability in the qualitative approach is not regarded as important as one in the quantitative approach in that research is not an entirely precise science when human interactions are studied. Sometimes, the only measuring device available is the researcher's own observations of human interaction or human reaction to varying stimuli, so results may be unreliable and multiple interpretations are possible. In this regard, qualitative researchers are more concerned with what actually occurs in the setting they are studying, rather than any recurrence across different observations. However, qualitative data can be more reliable when it accurately (objectively) records events and information.

According to Burns(2000), reliability can be enhanced in these ways:

- Clear statements of reason for research and main question to be addressed

- Clear statements about the researcher’s perspectives on the questions, their assumptions and biases.

- Clear statements about data-gathering procedures.

 

2) Validity of each approach

 While reliability is concerned with the accuracy of the actual measuring instrument or procedure, validity is concerned with the study's success at measuring what the researchers intend to measure. Burns (2000:418) claims that two questions are likely to be asked to demonstrate the necessity of establishing validity considering the relationships between scientific explanations of the world and its actual conditions:

1)                    To what extent are the abstract constructs and postulates generated, refined or tested by scientific researchers applicable across groups? This addresses the issue of external validity.

2)                    Do scientific researchers actually observe or measure what they think they are observing and measuring? This is the problem of internal validity.

 

Researchers should be concerned with both external and internal validity. External validity refers to the extent to which the results of a study can be generalized across variations in people, settings, treatments, outcomes, and times. External validity is more likely to be treated importantly in quantitative approach, while qualitative approach concerns more about its overall validity. However, qualitative findings can be best generalizable when it comes to the development of theories.

Internal validity is regarding causal relationships, that is, it questions whether the outcomes or findings obtained in your study are from the result of the experimental treatment or other factors. There is no doubt that causality is surely established for the test to be valid and useful within quantitative research and it is gained by attempting to isolate the factors under investigation away from any confusing variables. On the other hand, qualitative research validity also partially requires internal validity but is not centrally concerned with issues of cause and effect. Burns (2000:476) asserts that ‘.. it is not feasible to try an measure congruence between the data collected and some notion of reality… What seems true may be more important than what is true.’

Maxwell (1992) comments that although quantitative researchers are likely to address threats to validity through such techniques as random selection of participants and the use of controls, qualitative researchers are more likely to address validity throughout the data collection and analysis processes. As qualitative researchers review more cases, seeking common themes and patterns and testing emerging hypotheses, they are in essence working to ensure validity. According to Burns (2000), triangulation, the use of different strategies to approach the same topic of investigation, is commonly used technique to enhance the internal validity.

 

Methodology

          Burns(2000:391) summarises the different methods used by qualitative and quantitative researchers in the following table.

Qualitative	Quantitative
* Data collection using participant observation, unstructured interviews * Conclude with hypothesis and grounded theory * Emergence and portrayal * Inductive and naturalistic * Data analysis by themes from informants’ descriptions * Data reported in language of informant * Descriptive write-up	* Testing and measuring * Commences with hypothesis and theory * Manipulation and control * Deductive and experimental * Statistical analysis * Statistical reporting * Abstract impersonal write-up

Some other factors such as assumptions, purpose and role of researcher also need to be taken into account to select the appropriate research method for a given study. For example, ethnographic study, phenomenological study and case study research, conducted by qualitative methods, are likely to be major types of qualitative research, whereas, nonexperimental, quasi-experimental and experimental research, conducted by quantitative methods, are considered major types of quantitative research.
       In total, all research (whether quantitative or qualitative) is based on some underlying assumptions about what constitutes valid research and which research methods are appropriate.

 

Research Questions
            Research starts with a problem and/or a problematic situation. The important thing is to state one’s problem clearly and fully. Knowing exactly what one is trying to find out is crucial, which can lead researchers either into the quantitative or qualitative direction.

 

Three Quantitative Research Questions
1. What is the most difficult part of English grammar for Korean students to learn?
: This research question is more likely to be done by a survey or a questionnaire method in the quantitative approach. The results can be obtained by Korean ESL students randomly selected. The more participants there are for the research the more concise one can expect the results to be. No subjectivity seems to be required to answer this research question only, but objectivity, precision, control and reliability for more accurate statistical data are more likely to be employed. However, this question can be possibly conducted by the qualitative approach for rich information such as students’ learning style and approaches to study.

2. Does teaching students to use successful L2 reading strategies help them improve their reading comprehension?
: The experimental method, one of quantitative methods, is likely to be conducted by assessing that one variable causes another

3. What is the relationship between vocabulary and reading achievement in the primary grades?
: The correlation method, seeking to ascertain the relationship between variables, will be appropriate for this research question. According to Johnson (1992), the correlation coefficient is a quantitative measure that represents the degree of relationship between two variables.

 

Three Qualitative Research Questions
1. Why do Korean ESL students have difficulty in learning articles in English grammar?
: Action research will be appropriate for this research in that it is essentially practical and problem solving in nature with teacher as researcher. Besides, a problem is identified in situation and the idea is to continually evaluate and improve teaching practice while investigating a problem.

2. How do Korean students feel about learning English only in English?
: This research question cannot be simply measured by statistical numbers but is likely to be conducted by observations and interviews. Action research is also possibly done to seek more precise and in-depth understanding that can provide researchers with reasons and ideas for a better teaching approach.

3. Why did English education based on grammar-translation in Korea fail in terms of improvement of speaking proficiency?
: This research question is asking an exploratory and historical question so historical method will be appropriate. The purpose of this type of research is not only for collecting data related to past occurrences for the purpose of describing causes and effects, or trends of those events but also for helping explain current events and changes and even to anticipate future ones.

          The aforementioned research questions are categorized into and dealt by two different approaches, the quantitative approach and the qualitative approach, according to the characteristics of each question. However, some questions are possibly conducted by a variety of methods and a combination of the two approaches. 

 

Conclusion

Avoiding biased and distorted research from subjectivity is pursued by the objective, scientific and statistical quantitative approach. However, considering the fact that all types of research are done on phenomena taken place among people and by people, ultimate objectivity cannot be solely expected. In SLA educational settings, for example, it is assumed that interaction between teachers and students takes place naturally and generates issues that require research in search of a better understanding of difficulties acquiring a second language and improving teaching methods. In fact, it surely all depends on characteristics of the research question when it comes to choosing one appropriate research method. However, in consideration of the fact that both approach are not mutually exclusive as mentioned early, making best of advantages and reducing limitations of each approach can surely be well combined to improve more reliability and validity for any research.