Discussion: Designing Mixed Methods Research

2 INTERNATIONAL JOURNAL OF MULTIPLE RESEARCH APPROACHES Volume 3, Issue 1, April 2009

Mixed methods research is a rapidly emergingresearch paradigm and, although various sources are available to assist the novice researcher in terms of books (e.g. Creswell & Plano Clark 2007; Greene 2007; Johnson & Christensen 2008; Onwuegbuzie, Collins, Leech & Slate [2009]; Ridenour & Newman 2008; Teddlie & Tashakkori 2009), methodological articles (e.g. Johnson & Onwuegbuzie 2004; Onwueg- buzie & Johnson 2006), and journal editorials (e.g. Tashakkori & Creswell 2007), it might be quite daunting for the novice researcher to stay abreast of the emerging trends in the field of mixed methods research. Therefore, our goal in writing this article is to present ten points that a novice researcher should be cognizant of when designing a mixed methods study in accordance to the following three phases: research formulation,

research planning, and research implementation. Additionally, we present rationales for why these points are important, and a brief description of selective typologies that novice researchers might access when conducting mixed methods research.

RESEARCH FORMULATION PHASE

1. Importance of a definition

Individuals who share a profession develop and use a professional language or lexicon. An impor- tant component of a lexicon is definitions. Shared definitions provide precision when researchers are communicating to an audience and collaborating with peers when designing a study or program of research. Because mixed methods research is an emerging paradigm, ‘new’ definitions also are emerging. In this article, we use the term ‘mixed

Copyright © eContent Management Pty Ltd. International Journal of Multiple Research Approaches (2009) 3: 2–7.

INTRODUCTION Ten points about mixed methods research to be considered by the

novice researcher

KATHLEEN MT COLLINS Associate Professor, Department of Curriculum & Studies, College of Education and Health Professions, University of Arkansas at Fayetteville AR, USA

ALICIA O’CATHAIN Senior Research Fellow, Medical Care Research Unit, ScHARR, University of Sheffield, UK

ABSTRACT Our goal in writing this article is to present ten points that a novice researcher should be cognizant of when formulating, planning, and implementing a mixed methods study. We provide rationales for why these points are important and a brief description of selective typologies that novice researchers might access to address these points when conducting mixed methods research.

Keywords: mixed methods design, research planning, research formulation

methods research’ to be consistent with the title of this special issue; however, other terms such as mixed research and integrative methods also are used by researchers when conducting this form of research (Onwuegbuzie & Johnson 2006; Teddlie & Tashakkori 2009). Mixed methods studies have been defined as studies that ‘combine qualitative and quantitative approaches into the research methodology of a single study or multi-phased study’ (Tashakkori & Teddlie 1998: 17-18) and ‘as a research design in which QUAL [i.e. qualitative] and QUAN [i.e. quantitative] approaches are used in types of questions, research methods, data col- lection and analysis procedures, and /or inferences’ (Tashakkori & Teddlie 2003: 711). Mixed meth- ods designs have been defined as designs which include at least one quantitative method (designed to collect numbers) and one qualitative method (designed to collect words), where neither type is linked to a particular inquiry paradigm (Greene, Caracelli & Graham 1989). Similarly, mixed methods have been defined as quantitative and qualitative data collection, data analysis and the mixing of quantitative and qualitative approaches within a single study, with data integrated at some stage (Creswell & Plano Clark 2007; Creswell, Plano Clark, Gutmann & Hanson 2003). A defi- nition is useful to the novice researcher because it can be used to facilitate his or her awareness and understanding of mixed methods research as a research paradigm distinct from other monomethod approaches (quantitative and quali- tative) and provide useful terminology for report- ing research findings across various venues (e.g. conference presentations, technical and govern- ment reports, and published articles).

2. Importance of a mental model for mixing A researcher approaches a mixed methods investi- gation by initiating and completing a series of steps focused on delineating the process of mixing in a study or a program of research. Collins, Onwuegbuzie and Sutton (2006) have conceptu- alized the mixed methods research process as com-

prising 13 distinct steps (cf. Onwuegbuzie and Leech, this issue, for an identification of each step). Although similar to the research process in general, the intent of this 13-step process is to facilitate novice researcher decisions pertaining to the process of mixing at each step and it represents a recursive process. Additionally, it is important that the novice researcher recognize that decisions made at each of these steps are shaped by the researcher’s mental model (Greene 2007). Greene (2007) conceptualizes a mental model ‘as a com- plex, multifaceted lens through which a social inquirer perceives and makes sense of the social world’ ….and it is the ‘inquirer’s’ mental models that importantly frame and guide social inquiry’ (p.13). A mental model consists of the researcher’s personal assumptions, experiences, values, and beliefs about what constitutes an effective mode of inquiry (Greene 2007). Therefore, it is important that the novice researcher is cognizant of his or her mental model and also aware of the degree that this model shapes his or her interpretation of what constitutes rigor within an investigation. Discussion: Designing Mixed Methods Research

3. Utilizing typologies of designs Although we advocate that the novice researcher access typologies of designs when conducting mixed methods research, we also caution the novice researcher to be aware that typologies do not offer a panacea. Indeed, given the breath of mixed meth- ods studies, typologies have been criticized because they cannot address sufficiently the wide range of mixed methods designs implemented in various fields (Maxwell & Loomis 2003); in some cases, typologies delineate only minimally the informa- tion required by the researcher, or give inconsistent information, or present overly complex informa- tion (Leech & Onwuegbuzie 2009). However, we agree with Teddlie and Tashakkori (2006) who note that, although typologies are not exhaustive, they can provide to researchers distinct guidelines that serve to differentiate mixed methods as a research paradigm from other paradigms, namely quantitative and qualitative, thereby legitimating mixed methods research as a unique research para-

3

Introduction – Ten points about mixed methods research to be considered by the novice researcher

Volume 3, Issue 1, April 2009 INTERNATIONAL JOURNAL OF MULTIPLE RESEARCH APPROACHES

digm (Johnson & Onwuegbuzie 2004). Typologies also provide to researchers an organizational struc- ture to design and to implement studies, and a lexi- con to utilize when interpreting and disseminating information (Teddlie & Tashakkori 2009). Lastly, typologies facilitate learning by providing to researchers opportunities to compare and to con- trast various typologies, consequently expanding their levels of understanding of the mixed methods research process (Teddlie & Tashakkori 2006). In our later points we recommend selective typologies that the novice researcher might find useful when conducting mixed methods research. Discussion: Designing Mixed Methods Research

4. Selecting the reason, rationale, and purpose for mixing

The decisions pertaining to the reason, the ration- ale, and the purpose for mixing serve to differenti- ate the mixed methods research process from other research processes and it is these decisions that lead the novice researcher to develop the study’s research question(s) (Teddlie & Tashakkori 2009). Teddlie and Tashakkori (2009) have developed a typology comprising the following three general categories for identifying various reasons for con- ducting mixed methods research: (a) personal rea- sons for conducting the study, (b) reasons associated with advancing knowledge, and (c) soci- etal reasons associated with improving or empow- ering society, institutions, and oppressed groups. Adhering to this three-component process leads the novice researcher to develop research objectives followed by the development of research question(s) and hypotheses. It is the research ques- tion that drives the methods that will be imple- mented in the study (Tashakkori & Teddlie 1998). The typology developed by Greene et al. (1989) offers the novice researcher five options for deter- mining the purpose for mixing approaches: trian- gulation (i.e. comparison of findings derived from different methods to interpret the phenomenon); complementarity (different methods utilized to assess various dimensions of the phenomena); development (methods implemented sequentially, thereby allowing results of one method [e.g. quali-

tative] to inform development of the other method [e.g. quantitative]); expansion (different methods utilized to measure different phenomena); and ini- tiation (to address the goal of divergence, different methods used to assess various dimensions of the phenomena of interest). These five purposes relate to the data analysis step of the mixed methods research process. Collins, Onwuegbuzie and Sut- ton (2006) developed a typology that presents the novice researcher with four rationales for mixing and 65 purposes that are applicable to multiple steps of the mixed methods research process.

5. Determining the research question Teddlie and Tashakkori (2009) define mixed meth- ods research questions as questions ‘concerned with the unknown aspects of a phenomena and are answered with information that is presented in both narrative and numerical forms’ (p.129). They recommend developing one mixed methods ques- tion that serves as an overarching question and this question can be extended into qualitative and quantitative sub-questions. Formulating one over- arching question provides a justification for mixing and guides the novice researcher’s processes of mix- ing methods and integrating findings. Creswell and Plano Clark (2007) offer an alternative approach to developing research questions. They advocate separate quantitative and qualitative ques- tions, followed by development of a mixed meth- ods question framing integration of the findings from both phases of the study. Onwuegbuzie and Leech (2006) provide to novice researchers specific examples of how to write quantitative, qualitative, and mixed methods research questions and they also provide a framework connecting research questions to various data analytical techniques. Discussion: Designing Mixed Methods Research

RESEARCH PLANNING PHASE

6. Selecting a mixed methods research design

The novice researcher can select a preexistent mixed methods design or develop a specific design to address the study’s particular research

4

Kathleen MT Collins and Alicia O’Cathain

INTERNATIONAL JOURNAL OF MULTIPLE RESEARCH APPROACHES Volume 3, Issue 1, April 2009

objectives, purposes and research questions. Ted- dlie and Tashakkori (2006, 2009) provide to the novice researcher a typology called the Methods- Strands Matrix. This matrix facilitates a novice researcher’s decisions by presenting design options that are organized by: (a) choosing the type of approach that will be

utilized in the study (i.e. monomethod [qual- itative or quantitative approach used across all stages of the study] or mixed methods [qualitative and quantitative approaches mixed across the stages of the study); and

(b) selecting the number of strands or phases that will be implemented in the study (Ted- dlie & Tashakkori 2009). Discussion: Designing Mixed Methods Research

The two types of mixed methods designs are designs with one strand (monostrand) and designs with more than one strand (multistrand). When utilizing mixed methods multistrand designs, the novice researcher can select from the following five families of mixed methods research designs: paral- lel, sequential, conversion, multilevel, and fully integrated (Teddlie & Tashakkori 2009). The Methods-Strands Matrix also can guide the novice researcher in the process of deciding the stage(s) that mixing will occur (i.e. conceptualization, experimental, inferential stages) within the study (Ridenour & Newman 2008; Teddlie & Tashakkori 2009). Additionally, the novice researcher also can access other typologies to guide the design process (e.g. Creswell et al. 2003; John- son & Onwuegbuzie 2004; Leech & Onwueg- buzie 2009; Maxwell & Loomis 2003).

7. Determining the sampling design The researcher’s choice of a sampling design impacts the legitimation of the researcher’s infer- ences and the appropriate generalization of results (Collins, Onwuegbuzie & Jiao 2006, 2007; Onwuegbuzie & Collins 2007; Teddlie & Yu 2007). A sampling scheme represents the strategies used by the novice researcher to select the unit of analysis (individuals, cases, groups, contexts in terms of settings and events) accompanied by a decision pertaining to the size of the sample (the

number of units chosen for the study). When con- ducting mixed methods research, a novice researcher’s sampling decisions must pertain to both the quantitative and qualitative phases of the study. The typology developed by Teddlie and Yu (2007) presents to the novice researcher sampling schemes that are categorized into four types: probability, purposive sampling, convenience sampling, and mixed methods sampling. Relevant to mixed meth- ods sampling, four schemes are introduced, namely: basic sampling strategies, sequential sampling, con- current sampling, and multilevel sampling. An alternative typology developed by Onwuegbuzie and Collins (2007) presents to the novice researchers sampling schemes and sample size guidelines appropriate for the quantitative and qualitative of the mixed methods study. Additional- ly, they present a matrix comprising two dimen- sions. Dimension one matches the time orientation of the quantitative and qualitative phases of the study (the novice researcher’s decision to administer the quantitative and qualitative phase at the same or approximately the same point in time) or sequen- tially (one phase is initiated and informs delivery of the second phase) to the purpose of mixing (cf. Greene et al. 1989). Dimension two provides guidelines to identify clearly the relationship of the study’s participants in the quantitative and qualita- tive samples, namely: identical (the same individu- als participate in both phases); parallel (different individuals participate in the phases but are drawn from the same population); nested (individuals for one phase represent a subset of individuals who par- ticipated in the other phase); multilevel (different individuals participate in the phases and represent different levels of the population as exemplified by selecting corporate personnel as one sample versus consumers of a product as the other sample). Discussion: Designing Mixed Methods Research

RESEARCH IMPLEMENTATION PHASE

8. Collecting data

Johnson and Turner (2003) note that the investi- gator’s selection of data methods reflects the fun- damental principle of mixed methods research,

5

Introduction – Ten points about mixed methods research to be considered by the novice researcher

Volume 3, Issue 1, April 2009 INTERNATIONAL JOURNAL OF MULTIPLE RESEARCH APPROACHES

such that the ‘methods should be mixed in a way that has complementary strengths and non overlap- ping weaknesses’ (Johnson & Turner 2003: 299, italics in original). Adherence to this principle enables the novice researcher to collect strategical- ly multiple forms of evidence, such that the com- bination of methods presents convergent and divergent evidence, subsequently strengthening the findings of the mixed methods study (Johnson & Turner 2003; Johnson & Christensen 2008). Johnson and Turner (2003) present a two-dimen- sional matrix outlining data collection techniques to allow the novice researcher to engage in two forms of mixing: intramethod mixing (employing a single method that includes quantitative and qualitative components [e.g. open-closed items on a single questionnaire] and intermethod mixing (mixing two or more methods [e.g. questionnaire, interview and observation]). The first dimension of the matrix is to select a research approach (i.e. pure quantitative, pure qualitative or mixed) and the second is to select a method of data collection (questionnaire, interviews, focus groups, tests, observations secondary data [archival data]).

9. Conducting data analysis A mixed methods analysis entails the use of quali- tative and quantitative analytical techniques that are implemented either concurrently (at the same time or in a relatively close time frame) or sequen- tially (one form of analysis is conducted first and it informs the other type of analysis) from which interpretations are made in a parallel or an integra- tive or an iterative manner (Onwuegbuzie & Ted- dlie 2003; Teddlie & Tashakkori 2009). When analyzing data, a novice researcher can utilize a combination of the following seven phases of the mixed methods analysis process: (a) data reduction (e.g. quantitative data are analyzed using descrip- tive statistics and qualitative data are categorized as descriptive themes; (b) data display (e.g. data per- taining to both strands are organized and presented visually in graphs and matrices); (c) data transfor- mation (quantitative data converted into narrative codes [qualitized] that can be analyzed using quali-

tative techniques and qualitative data converted into numerical codes [quantitized] and analyzed using quantitative techniques); (d) data correlation (correlating quantitative data with qualitized data or vice versa; (e) data consolidation (different data types merged into one data set); (f ) data compari- son (comparing data from two different sources); and (g) data integration (integrating quantitative and qualitative data into one coherent whole that will be analyzed and interpreted simultaneously as a single data set or two data sets [quantitative and qualitative] to be analyzed separately by the researcher) (Onwuegbuzie & Teddlie 2003). Discussion: Designing Mixed Methods Research

10. Legitimating Inferences and formulating generalizations

Data validation refers to the implementation of appropriate steps or procedures to assure legiti- mation (Onwuegbuzie & Johnson 2006) by establishing a process to examine ‘inference qual- ity’ (Teddlie & Tashakkori 2003) in terms of the design quality and the interpretive rigor of the study’s outcomes, and, thereby leading the novice researcher to formulate appropriate generaliza- tions termed ‘inference transferability’ by Teddlie and Tashakkori (2003). Legitimation also has been defined as a recursive process in which the novice researcher evaluates the quality of the inferences drawn from the quantitative and qual- itative phases at each stage of the study and/or across a program of research (Onwuegbuzie & Johnson 2006). Subsequently, the decisions per- taining to both the quantitative and qualitative phases of the study impact the novice researcher’s ability to draw appropriate inferences and gener- alizations. Onwuegbuzie and Johnson’s (2006) legitimation model and Dellinger and Leech’s validation framework (2007) offer the novice researcher two alternatives to evaluate inferences on the basis of the study’s findings.

CONCLUSION In conclusion, we hope that consideration of the ten points above and the accompanying typolo- gies will facilitate the novice researcher’s efforts to

6

Kathleen MT Collins and Alicia O’Cathain

INTERNATIONAL JOURNAL OF MULTIPLE RESEARCH APPROACHES Volume 3, Issue 1, April 2009

formulate, plan, and implement both successful and rigorous mixed methods research.

References Collins KMT, Onwuegbuzie AJ and Jiao QG

(2006) Prevalence of mixed methods sampling in social science research. Evaluation and Research in Education, 19(2): 267-291.

Collins KMT, Onwuegbuzie AJ and Jiao, QG (2007) A mixed methods investigation of mixed methods sampling designs in social and health science research. Journal of Mixed Methods Research, 1(3): 267-294.

Collins KMT, Onwuegbuzie AJ and Sutton IL (2006) A model incorporating the rationale and purpose for conducting mixed methods research in special education and beyond. Learning Disabilities: A Contemporary Journal, 4: 67-100.

Creswell JW and Plano Clark VL (2007) Designing and conducting mixed methods research. Thousand Oaks, CA: Sage.

Creswell JW, Plano Clark VL, Gutmann M and Hanson W (2003) Advanced mixed methods research designs. In A Tashakkori & C Teddlie (Eds) Handbook of mixed methods in social and behavioral research (pp. 209-240) Thousand Oaks, CA: Sage.

Dellinger AB and Leech NL (2007) Toward a unified validation framework in mixed methods research. Journal of Mixed Methods Research, 1: 359-375.

Greene JC (2007) Mixed methods in social inquiry. San Francisco, CA: Jossey Bass.

Greene JC, Caracelli VJ and Graham WF (1989) Toward a conceptual framework for mixed- method evaluation designs. Educational Evaluation and Policy Analysis, 11: 255– 274.

Johnson B and Christensen L (2008) Educational research quantitative, qualitative, and mixed approaches (3rd ed) Thousand Oaks, CA: Sage

Johnson RB and Onwuegbuzie AJ (2004) Mixed methods research: A research paradigm whose time has come. Educational Researcher, 33(7): 14–26.

Johnson RB and Turner LA (2003) Data collection strategies in mixed methods research. In A Tashakkori & C Teddlie (Eds) Handbook of mixed methods in social and behavioral research (pp 297–319) Thousand Oaks, CA: Sage.

Leech NL and Onwuegbuzie AJ (2009) A typology of mixed methods research designs. Quality & Quantity International Journal of Methodology, 43: 265-275.

Maxwell JA and Loomis DM (2003) Mixed methods design: An alternative approach. In A

Tashakkori & C Teddlie (Eds) Handbook of mixed methods in social and behavioral research (pp. 241–272) Thousand Oaks, CA: Sage.

Onwuegbuzie AJ and Collins KMT (2007). A typol- ogy of mixed methods sampling designs in social science research. The Qualitative Report, 12(2). Retrieved February 21 2009, from http://www. nova.edu/ssss/QR/QR12-2/onwuegbuzie2.pdf

Onwuegbuzie AJ, Collins KMT, Leech NL and Slate JR (2009) Mixed research: A step-by-step guide. New York: Taylor & Francis, in press.

Onwuegbuzie AJ and Johnson RB (2006) The validity issues in mixed research. Research in Schools, 13(1): 48-63.

Onwuegbuzie AJ and Leech NL (2006) Linking research questions to mixed methods data analysis procedures. The Qualitative Report, 11(3). Retrieved February 23, 2009 from http://www. nova.edu/ssss/QR/QR11-3/onwuegbuzie.pdf

Onwuegbuzie AJ and Teddlie C (2003) A framework for analyzing data in mixed methods research. In A Tashakkori & C Teddlie (Eds) Handbook of mixed methods in social and behavioral research (pp. 351-383) Thousand Oaks, CA: Sage.

Ridenour CS and Newman I (2008) Mixed methods research: Exploring the interactive continuum. Carbondale: Southern Illinois University Press.

Tashakkori A and Creswell JW (2007) Exploring the nature of research questions in mixed methods research. Journal of Mixed Methods Research, 1: 207-211.

Tashakkori A and Teddlie C (1998) Mixed method- ology: Combining qualitative and quantitative approaches. Thousand Oaks, CA: Sage

Tashakkori A and Teddlie C (2003) Handbook of mixed methods in social and behavioral research. Thousand Oaks, CA: Sage.

Teddlie C and Tashakkori A (2003) Major issues and controversies in the issue of mixed methods in the social and behavioral sciences. In A Tashakkori & C Teddlie (Eds) Handbook of mixed methods in social and behavioral research (pp.3-50) Thousand Oaks, CA: Sage.

Teddlie C and Tashakkori A (2006) A general typology of research designs featuring mixed methods. Research in the Schools, 13: 12-28.

Teddlie C and Tashakkori A (2009) Foundations of mixed methods research: Integrating quantitative and qualitative approaches in the social and behavioral sciences. Thousand Oaks, CA: Sage.

Teddlie C and Yu F (2007) Mixed methods sampling: A typology with examples. Journal of Mixed Methods Research, 1: 77-100.

7

Introduction – Ten points about mixed methods research to be considered by the novice researcher

Volume 3, Issue 1, April 2009 INTERNATIONAL JOURNAL OF MULTIPLE RESEARCH APPROACHES

EDUCATIONAL RESEARCHER14

The purposes of this article are to position mixed methods research

(mixed research is a synonym) as the natural complement to tradi-

tional qualitative and quantitative research, to present pragmatism

as offering an attractive philosophical partner for mixed methods re-

search, and to provide a framework for designing and conducting

mixed methods research. In doing this, we briefly review the para-

digm “wars” and incompatibility thesis, we show some commonali-

ties between quantitative and qualitative research, we explain the

tenets of pragmatism, we explain the fundamental principle of mixed

research and how to apply it, we provide specific sets of designs for

the two major types of mixed methods research (mixed-model de-

signs and mixed-method designs), and, finally, we explain mixed meth-

ods research as following (recursively) an eight-step process. A key

feature of mixed methods research is its methodological pluralism

or eclecticism, which frequently results in superior research (com-

pared to monomethod research). Mixed methods research will be

successful as more investigators study and help advance its concepts

and as they regularly practice it.

F or more than a century, the advocates of quantitative and qualitative research paradigms have engaged in ardent dis- pute.1 From these debates, purists have emerged on both

sides (cf. Campbell & Stanley, 1963; Lincoln & Guba, 1985).2

Quantitative purists (Ayer, 1959; Maxwell & Delaney, 2004; Popper, 1959; Schrag, 1992) articulate assumptions that are con- sistent with what is commonly called a positivist philosophy.3, 4

That is, quantitative purists believe that social observations should be treated as entities in much the same way that physical scientists treat physical phenomena. Further, they contend that the observer is separate from the entities that are subject to ob- servation. Quantitative purists maintain that social science inquiry should be objective. That is, time- and context-free gen- eralizations (Nagel, 1986) are desirable and possible, and real causes of social scientific outcomes can be determined reliably and validly. According to this school of thought, educational re- searchers should eliminate their biases, remain emotionally de- tached and uninvolved with the objects of study, and test or empirically justify their stated hypotheses. These researchers have traditionally called for rhetorical neutrality, involving a formal

Mixed Methods Research: A Research Paradigm Whose Time Has Come by R. Burke Johnson and Anthony J. Onwuegbuzie

writing style using the impersonal passive voice and technical ter- minology, in which establishing and describing social laws is the major focus (Tashakkori & Teddlie, 1998).

Qualitative purists (also called constructivists and interpretivists) reject what they call positivism. They argue for the superiority of constructivism, idealism, relativism, humanism, hermeneutics, and, sometimes, postmodernism (Guba & Lincoln, 1989; Lincoln & Guba, 2000; Schwandt, 2000; Smith, 1983, 1984). These purists contend that multiple-constructed realities abound, that time- and context-free generalizations are neither desirable nor possible, that research is value-bound, that it is impossible to dif- ferentiate fully causes and effects, that logic flows from specific to general (e.g., explanations are generated inductively from the data), and that knower and known cannot be separated because the subjective knower is the only source of reality (Guba, 1990). Qualitative purists also are characterized by a dislike of a de- tached and passive style of writing, preferring, instead, detailed, rich, and thick (empathic) description, written directly and some- what informally. Discussion: Designing Mixed Methods Research

Both sets of purists view their paradigms as the ideal for re- search, and, implicitly if not explicitly, they advocate the in- compatibility thesis (Howe, 1988), which posits that qualitative and quantitative research paradigms, including their associated methods, cannot and should not be mixed. The quantitative versus qualitative debate has been so divisive that some gradu- ate students who graduate from educational institutions with an aspiration to gain employment in the world of academia or re- search are left with the impression that they have to pledge alle- giance to one research school of thought or the other. Guba (a leading qualitative purist) clearly represented the purist position when he contended that “accommodation between paradigms is impossible . . . we are led to vastly diverse, disparate, and to- tally antithetical ends” (Guba, 1990, p. 81). A disturbing fea- ture of the paradigm wars has been the relentless focus on the differences between the two orientations. Indeed, the two dom- inant research paradigms have resulted in two research cultures, “one professing the superiority of ‘deep, rich observational data’ and the other the virtues of ‘hard, generalizable’ . . . data” (Sieber, 1973, p. 1335).

Our purpose in writing this article is to present mixed meth- ods research as the third research paradigm in educational re- search.5 We hope the field will move beyond quantitative versus qualitative research arguments because, as recognized by mixed methods research, both quantitative and qualitative research are important and useful. The goal of mixed methods research is not to replace either of these approaches but rather to draw from theEducational Researcher, Vol. 33, No. 7, pp. 14–26

15OCTOBER 2004

strengths and minimize the weaknesses of both in single research studies and across studies. If you visualize a continuum with qualitative research anchored at one pole and quantitative re- search anchored at the other, mixed methods research covers the large set of points in the middle area. If one prefers to think cat- egorically, mixed methods research sits in a new third chair, with qualitative research sitting on the left side and quantitative re- search sitting on the right side. Discussion: Designing Mixed Methods Research

Mixed methods research offers great promise for practicing researchers who would like to see methodologists describe and develop techniques that are closer to what researchers actually use in practice. Mixed methods research as the third research paradigm can also help bridge the schism between quantitative and qualitative research (Onwuegbuzie & Leech, 2004a). Meth- odological work on the mixed methods research paradigm can be seen in several recent books (Brewer & Hunter, 1989; Creswell, 2003; Greene, Caracelli, & Graham, 1989; Johnson & Christensen, 2004; Newman & Benz, 1998; Reichardt & Rallis, 1994; Tashakkori & Teddlie, 1998, 2003). Much work remains to be undertaken in the area of mixed methods research regarding its philosophical positions, designs, data analysis, va- lidity strategies, mixing and integration procedures, and ratio- nales, among other things. We will try to clarify the most important issues in the remainder of this article.

Commonalities Among the Traditional Paradigms

Although there are many important paradigmatic differences be- tween qualitative and quantitative research (which have been fre- quently written about in the Educational Researcher and other places), there are some similarities between the various approaches that are sometimes overlooked. For example, both quantitative and qualitative researchers use empirical observations to address research questions. Sechrest and Sidani (1995, p. 78) point out that both methodologies “describe their data, construct explana- tory arguments from their data, and speculate about why the outcomes they observed happened as they did.” Additionally, both sets of researchers incorporate safeguards into their inquiries in order to minimize confirmation bias and other sources of in- validity (or lack of trustworthiness) that have the potential to exist in every research study (Sandelowski, 1986).

Regardless of paradigmatic orientation, all research in the so- cial sciences represents an attempt to provide warranted assertions about human beings (or specific groups of human beings) and the environments in which they live and evolve (Biesta & Burbules, 2003). In the social and behavioral sciences, this goal of under- standing leads to the examination of many different phenomena, including holistic phenomena such as intentions, experiences, at- titudes, and culture, as well as more reductive phenomena such as macromolecules, nerve cells, micro-level homunculi, and bio- chemical computational systems (de Jong, 2003). There is room in ontology for mental and social reality as well as the more micro and more clearly material reality. Although certain methodolo- gies tend to be associated with one particular research tradition, Dzurec and Abraham (1993, p. 75) suggest that “the objectives, scope, and nature of inquiry are consistent across methods and across paradigms.” We contend that researchers and research methodologists need to be asking when each research approach

is most helpful and when and how they should be mixed or com- bined in their research studies.

We contend that epistemological and methodological pluralism should be promoted in educational research so that researchers are informed about epistemological and methodological possibilities and, ultimately, so that we are able to conduct more effective re- search. Today’s research world is becoming increasingly inter- disciplinary, complex, and dynamic; therefore, many researchers need to complement one method with another, and all researchers need a solid understanding of multiple methods used by other scholars to facilitate communication, to promote collaboration, and to provide superior research. Taking a non-purist or com- patibilist or mixed position allows researchers to mix and match design components that offer the best chance of answering their specific research questions. Although many research procedures or methods typically have been linked to certain paradigms, this linkage between research paradigm and research methods is nei- ther sacrosanct nor necessary (Howe, 1988, 1992). For example, qualitative researchers should be free to use quantitative meth- ods, and quantitative researchers should be free to use qualitative methods. Also, research in a content domain that is dominated by one method often can be better informed by the use of mul- tiple methods (e.g., to give a read on methods-induced bias, for corroboration, for complimentarity, for expansion; see Greene et al., 1989). We contend that epistemological and paradigmatic ecumenicalism is within reach in the research paradigm of mixed methods research. Discussion: Designing Mixed Methods Research