Literature Reviewand Focusing the ResearchWhen asked, Why do a literature review?, a somewhat cynical answer may have popped into some of your minds: “Why do a literature review? It is required for my research class,” or “I have to do a thesis or dissertation.” Then, again, someof you may have more socially redeeming motivations, such as wanting to change theworld or improve your practice of a profession.Literature reviews are important as research tools, especially in emerging areas,with populations that typically yield small samples (e.g., special education researchoften does), or in areas that represent value-laden positions adopted by advocacygroups. Literature reviews are also valuable in light of the knowledge explosion and theconsequent impossibility of reading everything. Therefore, it is good that someone doesliterature reviews.A few defnitions will make your progress through this chapter more enjoyable:Preliminary sources: Databases that contain information about research articles thatare published on the topic of interest to you.Secondary sources: Literature reviews that are published on your topic of interestconsisting of a synthesis and analysis of previous research published on that topic.Primary empirical research: Reports of studies that are conducted by the researcher(s)that include a description of the methods, sampling and data collection strategies,and data analysis and results.390 ReseaRch and evaluation in education and PsychologyReasons for Doing Literature ReviewsThere are two major reasons for conducting a literature review: to conduct primaryresearch oneself (or as a part of a team) or as an end in itself.Literature Reviews for Planning Primary ResearchAlmost every primary research study begins with a review of the literature. Thepurpose of the literature review section of a research article is to provide the reader withan overall framework for where this piece of work fts in the “big picture” of what isknown about a topic from previous research. Thus, the literature review serves to explainthe topic of the research and to build a rationale for the problem that is studied and theneed for additional research. Boote and Beile (2005) eloquently explain the purpose of aliterature review in planning primary research:As the foundation of any research project, the literature review should accomplishseveral important objectives. It sets the broad context of the study, clearly demarcateswhat is and what is not within the scope of the investigation, and justifes thosedecisions. It also situates an existing literature in a broader scholarly and historicalcontext. It should not only report the claims made in the existing literature but alsoexamine critically the research methods used to better understand whether theclaims are warranted. Such an examination of the literature enables the author todistinguish what has been learned and accomplished in the area of study and whatstill needs to be learned and accomplished. Moreover, this type of review allowsthe author not only to summarize the existing literature but also to synthesize it ina way that permits a new perspective. Thus a good literature review is the basis ofboth theoretical and methodological sophistication, thereby improving the qualityand usefulness of subsequent research. (p. 4)Researchers use the literature review to identify a rationale for the need for theirown study. Some of the specifc rationales for your research that might emerge from yourliterature review include the following:1. You may fnd a lack of consistency in reported results across the studies you havechosen to review and undertake research to explore the basis of the inconsistency. Forexample, Berliner et al. (2008) noted inconsistencies in research on high school dropouts;they suggested that the problem might be that researchers were not differentiatingbetween high school dropouts who reenrolled and those who did not.2. You may have uncovered a ﬂaw in previous research based on its design, datacollection instruments, sampling, or interpretation. For example, Borman et al. (2007)reviewed research on the Success for All literacy program and found that no randomizedcontrol studies had been conducted on its effectiveness. The quasi-experimental designsfrom past research left the fndings open to possible criticism based on uncontrolledextraneous variables.3. Research may have been conducted on a different population than the one in whichyou are interested, thus justifying your work with the different population. For example,Schirmer and McGough (2005) reviewed research literature on reading development andreading instruction and found that there was a lack of research of this type on studentswho are deaf. Therefore, they proposed a need for research on reading instruction thatliterature Review and Focusing the Research 91has been found to be effective with hearing students to be conducted with deaf students.Another justifcation for the conduct of research with deaf students when the previousresearch is based on hearing children might be to devise a very different innovativemethod of reading instruction that is based on sign language and deaf culture.4. You may document an ongoing educational or psychological problem andpropose studying the effect of an innovative intervention to try to correct that problem.For example, Burnard (2008) wanted to explore innovative pedagogical practices toengage students who were facing challenges stemming from poverty, class, race, religion,linguistic and cultural heritage, or gender. In particular, she was interested in how musicteachers engaged students who were disaffected.5. Uncertainty about the interpretation of previous studies’ fndings may justifyfurther research. For example, prior research with people with schizophrenia indicatedthat participants sometimes continued to feel bewildered about their condition andtreatment, even after meeting with a health care professional. Schneider et al. (2004)undertook a study from the perspective of people with mental illness to determine whatcontributed to their perceptions of effective and ineffective relations with professionals.As mentioned previously, a literature review can be used at the beginning of thestudy to explain what is known about your topic and provide a rationale for the studyyou are planning. In addition, the literature review can be used to help in the design ofthe study by providing guidance as to appropriate sample size or identifying promisingdata collection practices or instruments that can be used in your study. Familiarity withthe literature is useful for both quantitative and qualitative studies no matter what theresearcher’s paradigm. Everyone who prepares a literature review should do so with acritical eye: What are the strengths and weaknesses of the prior research? What is missingfrom the formal body of scholarly literature that might be necessary in order to formulatean appropriate research focus and method of investigation?When your purpose is to plan your own research study, the number of studies that youactually cite in your literature review may be fairly limited because of space limitations (forauthors who publish in journals) or because the review is considered a learning activity (inyour own course work). Typically, primary research articles published in journals contain20 to 30 references to primary research. The number of citations may be quite limited for acourse activity or more extensive if you are preparing a proposal for a thesis or dissertation.The exact number varies, depending on the purpose of the literature review and the extantliterature. The primary criterion for inclusion should be centrality to your topic, withinwhatever constraints are imposed by instructors, advisers, or publishers.Use of the literature review to plan and conduct a study requires that you criticallyevaluate the research that you read. This critical analysis can form the basis for yourrationale or for your choice of data collection procedures. Criteria for evaluating primaryresearch studies are provided at the end of each chapter.Review of Literature as an End in ItselfThe review of literature can be seen as an end in itself, either to inform practice or toprovide a comprehensive understanding about what is known about a topic. The processfor conducting this type of literature review varies, depending on your purpose. If yourpurpose is to improve your professional practice, you will want to base your literaturereview on the problem you encountered in your profession. Therefore, when you lookto the literature for a solution, you may rely on other people’s literature reviews, or you92 ReseaRch and evaluation in education and Psychologymay seek out primary research reports until you fnd one that seems to ft your situation.For example, Mayo (2007) reviewed literature from the LGBTQ community with a specifcfocus on the act of “coming out” as it is researched in schools from the perspective ofobstacles that the youth encounter, as well as in terms of the agency and resiliencydemonstrated by some youth. Mayo uses the literature review to suggest promisingstrategies for school leaders, youth, and researchers to make progress on this issue.When a literature review is conducted to provide a comprehensive understandingof what is known about a topic, the process is much longer. For example, Mckinleyet al. (2007) included over 300 references in their literature review of race as a constructin educational research, examining such topics as the meaning of equity, inequality,whiteness, and race as social constructs, and implications of desegregation and placementin special education for members of racial minority groups. Gadsden (2008) includedalmost 200 references in her review of arts education in order to examine the changingplace of the arts in education through a lens of power, culture, and representation. Shedraws conclusions for researchers and educators in terms of future directions suggestedby the current body of scholarly knowledge in this area.Extending Your Thinking:Literature Review Uses• When writing a literature review for the purposes of planning a research study,what are some of the uses that the literature review can serve for you?• Why is a literature review especially important in areas that (a) are emerging,(b) typically have small samples (e.g., special education research), or (c) representvalue-laden positions adopted by advocacy groups (e.g., gender differences)?• Students receive different kinds of advice as to how much literature to reviewand at what stage of the research process this should occur. What is your reaction tothe following pieces of advice:When you have enough sense of the conversation to argue persuasivelythat the target for your proposed study is sound, and that the methods ofinquiry are correct, you know enough for the purpose of the proposal.(Locke, Spirduso, & Silverman, 1993, p. 68)J. M. Morse (1994) recommends reading in the general area of the inquiryonce a topic has been selected:At this stage, the researcher should become familiar with the literature,with what has been done generally in the area, and with the “state of theart.” He or she should develop a comfortable knowledge base withoutspending an extraordinary amount of time on minute details or chasingobscure references. (p. 221)literature Review and Focusing the Research 93The Search ProcessNo matter what the reason for the literature review or the paradigm within whichthe researcher is working, many aspects of the literature review process are thesame. A general outline for conducting a literature review is provided in Box 3.1.Some of the differences in the process that emanate from paradigm choice includethe following:1. With the postpositivist paradigm, the researcher who plans to conduct experimentalresearch needs to be able to develop a hypothesis (a best guess as to the outcome of theplanned research) based on previous research. Quantitative researchers examine researchin order to build a knowledge base of a topic that is suffcient to develop a hypothesisthat can be tested and to beneft from guidance in terms of methodology found in priorstudies.2. With a constructivist orientation, the researcher should have a good understandingof previous research but remain open to possible emerging hypotheses that wouldrequire examination of additional literature during the study (Marshall & Rossman, 2006).Qualitative researchers (Corbin & Strauss, 2008) note that a literature review can be usefulin order to decide on a research topic, to formulate a research plan, and to enhance theresearcher’s awareness of subtleties uncovered in previous research. They do cautionboth novice and experienced researchers to be careful so that their perceptions of theirfndings emanate from their own data and not on expectations generated by readingextant literature.3. In addition to review of scholarly literature, researchers working within thetransformative paradigm should consult with persons who have experienced oppressionand seek out literature that represents their viewpoints (Mertens, 2009). In order to dothis, researchers need to develop an understanding of themselves as individuals withpotential biases, as well as understand themselves in terms of their relationships withthe community of interest. Hence, transformative researchers are more inclined to workwith community members to develop the focus of the research, rather than rely solely onextant literature.Extending Your Thinking:Literature Reviews and Qualitative ResearchWhen conducting qualitative research, some texts advise against conducting a comprehensive literature review because it may bias the researcher to see “what otherssay they saw” instead of looking with fresh eyes. What do you think?94 ReseaRch and evaluation in education and Psychology1. Identify a research topic.2. Review secondary sources to get an overview of the topic: For example, look atthe Review of Educational Research, Harvard Educational Review, PsychologicalBulletin, Review of Research in Education, or the Annual Review of Psychology.3. Develop a search strategy and use appropriate preliminary sources and primaryresearch journals (see Boxes 3.2, 3.4, and 3.6), check the references at the end ofrelevant research publications, access personal networks, and/or build relationships with appropriate community representatives.4. Conduct the search and select specifc articles to review.5. Obtain full text references (e.g., journal articles or books).6. Read articles and prepare bibliographic information and notes on each article.7. Evaluate the research reports.8. Synthesize your fndings.9. Use the literature review to gain a conceptual framework and to formulate researchquestions, hypotheses, or both.Box 3.1 Steps in the Literature Review ProcessIn the following sections that describe the steps in the literature review process, thecommonalities in the search process are described, along with recognition of appropriatecaveats that differentiate work within alternative paradigms.Step 1: Identify Research TopicA few pieces of advice should guide (novice) researchers as they begin their literaturereview process. They should be ﬂexible in their conceptualization of the research problembeing investigated, and they should begin with a broad idea and be prepared to narrowit down as they progress through the search. Sometimes, students choose topics forresearch that turn out to be not very researchable (in that no one else has conceptualizedthe problem quite that way), and as they begin reading and seeing what is available,their ideas change as to what they want to investigate. Also, if the topic defnition istoo narrow, it may not be possible to identify any previous research that addressed thatspecifc topic. Therefore, be ﬂexible and start broadly. In my experience with studentswho are beginning a literature review, their topics shift as they become more familiar withthe topic. Some students write me desperate e-mails explaining that they want to changetheir topics and they hope that this is OK. In most cases, I write them back to assure themthat this is a normal part of an evolutionary process of developing the topic. (Only rarelydo I think, what in the world is that student thinking!)Sources of Research TopicsA research topic can emerge from a wide variety of sources, including the researcher’sinterests, knowledge of social conditions, observations of educational and psychologicalliterature Review and Focusing the Research 95problems, challenges that arise in one’s professional practice, readings in other courses,talking to other researchers, and the availability of funds to conduct research on a specifctopic (sponsored research). Any of these is appropriate as a source to help identify theprimary research topic. For researchers interested in conducting a comprehensive reviewof literature for its own sake, another criterion must be met: They must study topics thatappear in the literature.For sponsored research, the researcher needs to clarify with the funding agencywhat the research problem is (Mertens, 2009). Often, students can apply for funding tosupport their own research, usually with a faculty sponsor. When applying for funds, it isimportant to know what the agency is interested in sponsoring and to tailor one’s researchinterests to match those of the agency. Other students might work as research assistants tofaculty members who have received fnancial support from an outside agency.Scholars working in the transformative paradigm have been instrumental in stimulatingresearch on a variety of topics that had previously received little attention, such as spousalabuse, sexual abuse, sexual harassment, homophobia, unpaid labor, and motherhood andchild care. For transformative research, S. Harding (1993) recommends beginning withmarginalized lives. To defne the research problem, the researcher might want to involvepersons affected by the research through informal or formal means such as focus groups(Mertens, 2009). The following quotation from Schneider et al.’s (2004) study of peoplewith mental illness illustrates the transformative effect of engaging participants in theprocess of developing a research topic.There was also a real transformation in group members’ sense of themselves aspeople who could accomplish something. They had all been subjects in manyresearch projects and, at the beginning of the project, could not conceive ofthemselves as people who could do research. By the end of the project, they hadtaken on a sense of themselves as researchers. They saw that they could articulateproblems, come up with ways to investigate the problems, and produce solutions.This experience increased their awareness of themselves as people with resourcesand strengths who could make a signifcant contribution to society. (p. 575)Extending Your Thinking:Selecting Your Research Topic and SettingStudents of research are sometimes given conﬂicting advice about the topic andsite for their own research. The following quotations exemplify such conﬂicts.Where do you stand on these two issues (i.e., choice of a research topic andsetting) and why?The key to selecting a qualitative research topic is to identify somethingthat will hold one’s interest over time. New investigators can best identifysuch a topic by reﬂecting on what is a real personal interest to them.(J. M. Morse, 1994, p. 220)Using. . . personal experiences as the impetus for a research study is notwrong, but it is best if the researcher is aware of his or her possible(Continued)96 ReseaRch and evaluation in education and PsychologyStep 2: Review Secondary Sources to Get an OverviewA good literature review written by someone else can provide you with an overviewof what is known about your chosen topic. Specifc places that you can look for literaturereviews include journals that typically publish literature reviews, such as the Review ofEducational Research, Harvard Educational Review, and the Psychological Bulletin, andbooks that contain literature reviews, such as the following:• Review of Research in Education: This series is published annually by the AmericanEducational Research Association. Each volume contains a series of chapters on varioustopics, such as implications for socially just education rooted in discipline-specifc areassuch as literacy and science for diverse groups of students (Vol. 31, Parker, 2007). Whatis counted as knowledge is examined in Volume 32 of this series from the perspectiveof discipline (arts, English, foreign languages, history, literacy, mathematics, and science)with specifc focus on assessment of English Language Learners, and implications acrosscultural, linguistic, and social class lines (G. J. Kelly, Luke, & Green, 2008).• Yearbook of the National Society for the Study of Education: The NSSE yearbookwas published as both a book and a journal until 2005; since then, it has been availableonly as a journal subscription. Two volumes are published on a specifc topic annually.Recent topics include examining the reasons for education in the contemporary world,with emphasis on democracy, globalization, and culture (Coulter, Weins, & Fenstermacher,2008). The three yearbook volumes that preceded Coulter et al.’s are Moss (2007), anedited volume on evidence and decision making; Smolin, Lawless, and Burbules (2007),on information and communication technology; and Ball (2006), on achieving equity andexcellence as a way to realize the potential of the Brown v. Board of Education decision.(Continued)motives for conducting the study, as such experiences may give thestudy a particular bias. Of even more concern is the possibility thatthe researcher, when meeting and interviewing participants who have had thesame experience, may have many unresolved feelings emerge and may beemotionally unable to continue with the study. (J. M. Morse, 1994, p. 221)One common instance of the problem of investigator biography occurswhen graduate students . . . design a study that requires them to return to thecontext of public education and play the role of unbiased spectator….Thisparticular problem is diffcult to overcome and is precisely why it sometimesis best to select problems in a context with which the investigator has hadlittle previous experience. (Locke et al., 1993, p. 114)When injustice persists with no evidence of unhappiness, rebellion, oroffcial grievance, we need to study the reasons why. . . . Faculty, staff,and students in the feminist and African-American communities haveargued…that the absence of grievance substantiates the very depth ofand terror imposed by harassment. Feminist research must get behind“evidence” that suggests all is well. (M. Fine, 1992, p. 23)literature Review and Focusing the Research 97• The Annual Review of Psychology contains literature reviews on topics of interestin psychology and education, such as counseling or learning theory.• Research in Race and Ethnic Relations is published annually to address racerelations and minority and ethnic group research.• Other handbooks have been published on specifc topics:Banks, J. A., & McGee-Banks, C. A. (Eds.). (2003). Handbook of research on multiculturaleducation (2nd ed.). San Francisco: Jossey-Bass.Bursztyn, A. (2006). Praeger handbook of special education. Westport, CT: Greenwood.Kitayama, S., & Cohen, D. (Eds.). (2007). Handbook of cultural psychology. New York:Guilford.Klein, S. (2007). Handbook for achieving gender equity through education (2nd ed.).Mahwah, NJ: Lawrence Erlbaum.Ponterotto, J. G., Casas, J. M., Suzuki, L. A., & Alexander, C. M. (Eds.). (2001). Handbook ofracial/ethnic minority counseling research (2nd ed.). Thousand Oaks, CA: Sage.Richardson, V. (Ed.). (2001). Handbook of research on teaching. Washington, DC: AmericanEducational Research Association.Step 3: Develop a Search StrategyFour paths for search strategies are described in this section: (a) identify preliminarysources, (b) identify primary research journals, (c) access personal networks, and(d) involve community members. These are explained below. Decide which are the beststrategies for you to follow in your search process, and remember, stay ﬂexible.Identify Preliminary Sources1Preliminary sources include databases and indexes that contain a compilation ofbibliographic information, abstracts, and sometimes full text articles for a wide range oftopics and are accessible in print form, on compact disks (CD-ROM), or through onlineservices. Examples of the most frequently used preliminary sources are listed in Box 3.2.Additional abstract and index services that specifcally target marginalized groups includeAfrican Urban and Regional Science Index, Women’s Studies Abstracts, and Women’sStudies Index.• ERIC: The Educational Resources Information Center (ERIC) contains morethan 1.2 million abstracts of journal articles and research reports on educationrelated topics. The database can be searched by going to www.eric.ed.gov, orby using ERIC in commercial databases provided in many libraries. Most ERICdocuments are available electronically, in print, or on microfche in libraries. Manynon-journal materials are available, at no charge, as PDF documents or via links topublisher Web sites. Check with your local library (academic, public, etc.) to fndout if they can provide journal articles or documents that are not available online. Ifthe library cannot do this for you, print copies of journal articles can be purchasedthrough such article reprint services as Ingenta (www.ingentaconnect.com).(Continued)Box 3.2 Most Frequently Used Preliminary Sources98 ReseaRch and evaluation in education and Psychology• ProQuest® Education Journals: The database includes more than 750 journalsin primary, secondary, and university-level education. More than 600 of these titlesinclude full texts of the articles. This and similar ProQuest products are availableat many libraries.• JSTOR: This is a database of academic journals, monographs, and otheracademic papers from multiple disciplines, including the social sciences,humanities, and the sciences. It is available from libraries that subscribe to theservice. Individuals in the United States can subscribe for a modest amount; JSTORmade special arrangements for individuals in Africa to have access to this databasefor free (www.jstor.org).• PsycINFO: This is a product of the American Psychological Association (APA)that contains indexes and abstracts from 1,300 journals, as well as books andbook chapters related to psychology. Both members and nonmembers can searchthe database and purchase articles. Information about that is at http://psycnet.apa.org/index.cfm?fa=main.landing. As with other databases, you can check on itsavailability at your local library.• PsycARTICLES: This is another product of the APA, but it includes the full textarticles of 42 journals related to psychology that APA publishes. Information aboutthis database can be found at http://psycnet.apa.org/index.cfm?fa=main.landing. Thedatabase can be searched by APA members and by nonmembers (for a small fee).Box 3.2 (Continued)World Wide Web (WWW) sites are easily and pervasively available to assist you in yourliterature searching. There are many search sites on the Web, and new ones appear withsome regularity. In October 2008, the Consumer Search Web site (www.consumersearch.com) listed the top two choices for search engines as Yahoo! and Google.2 These twosearch engines were recognized because of the size of their databases, ability to searchHTML and PDF fles, accuracy in results, and advanced searching power using Booleanlogic. The search process on Web sites generally employs Boolean logic (explained laterin this chapter) but can differ a bit from site to site. Because this is such a dynamic area,it is best to check PC Magazine or some other computer source to fnd out what sites arerecommended and to determine appropriate search strategies for those sites. One wordof caution: The WWW sites do not have a peer review system to screen what is accepted(as most professional journals do); therefore, scholars raise questions about the qualityof information available from those sources. In addition, the Web sites are not designedto contain information specifcally about research in education and psychology as are theother databases described in this chapter.The computerized databases are a tremendous resource for the researcher in theliterature review phase of a project. A researcher can identify thousands of references byonly a few keystrokes on the computer. Because of the tremendous coverage providedby the databases, the researcher should plan to include a search of appropriate databasesin the literature review process. Box 3.3 provides an example of how one researcherdescribed his method of searching the literature.literature Review and Focusing the Research 99One important limitation should be noted about the available databases. You canget out of them only what was put into them. In other words, the databases are selectiveabout the journals they include. For example, many of the best-known feminist journalsare not included in the databases. A survey of 17 feminist journals indicated that only 6are included in ERIC3 (see Box 3.4). Some of the feminist journals listed in Box 3.4 mightnot be considered appropriate for inclusion in ERIC because their content is not directlyrelated to education or psychology. However, readers who are not familiar with feministjournals might fnd the list helpful as a way of broadening their resource base. For example,Hypatia publishes mainly philosophical work and thus would be of interest to those whowant to delve more deeply into that aspect of feminist theory. Camera Obscura publisheswork that could be of interest to scholars in educational media or social learning theory(e.g., the study of the power of media to shape cultural expectations by gender).The studies were gathered from a search completed in June of 2006 using theEducational Resources Information Center (ERIC), Academic Search, and Education Full Text databases, as well as the library book database at the University of Wisconsin, Madison. Initially, the search terms “high stakes testing” and“state-mandated testing” were used to identify potential studies for use in myqualitative meta-synthesis.—Au, 2007, p. 259Box 3.3 Description of Literature Review Search MethodCamera Obscura A journal of feminist perspectives in flm, television, andvisual media.Feminist Studies The frst feminist academic journal (started in 1972) isbased at the University of Maryland. It publishes an interdisciplinary bodyof feminist knowledge and theory that makes visible assumptions aboutgender and sexual identity and the experiences and contributions of womenacross a wide range of difference.Feminist Teachera Since 1984, Feminist Teacher has published discussionsabout how to fght sexism, racism, homophobia, and other forms ofoppression in classrooms and in educational institutions. A peer-reviewedjournal, it provides a forum for interrogations of cultural assumptionsand discussions of such topics as multiculturalism, interdisciplinarity, anddistance education within a feminist context.Gender and Educationa Published in England, Gender and Education is aninternational forum for discussion of multidisciplinary educational researchand ideas that focus on gender as a category of analysis.(Continued)Box 3.4 Feminist Journals100 ReseaRch and evaluation in education and PsychologyGender & History An international journal for research and writing on thehistory of gender relations, sexuality, and the semiotics of gender in a widegeographical and chronological scope. Gender & History examines changing conceptions of gender and maps the dialogue between femininities,masculinities, and their historical contexts.Gender & Society This journal emphasizes theory and research from microand macrostructural perspectives. It aims to advance both the study ofgender and feminist scholarship.Genders Based at the University of Colorado, Genders publishes innovativework about gender and sexuality in relation to social, political, artistic, andeconomic concerns.Hypatia This journal publishes scholarly research at the intersection ofphilosophy and women’s studies.Initiativesa Published by the National Association for Women in Education,this journal covers topics of interest to women in all aspects of highereducation since 1937.Journal of Women’s History An international journal that covers new researchon women’s history, it includes scholarship about women in all time periodsthat is broadly representative of national, racial, ethnic, religious, and sexualgrouping.Meridians A feminist, interdisciplinary journal with the goal of providing aforum for scholarship and creative work by and about women of color inU.S. and international contexts.NWSA Journal An offcial publication of the National Women’s StudiesAssociation (NWSA), it publishes up-to-date interdisciplinary, multiculturalfeminist scholarship linking feminist theory with teaching and activism.Psychology of Women Quarterlya A scientifc journal that reports empiricalresearch and critical reviews, theoretical articles, brief reports, and invitedbook reviews related to the psychology of women and gender.Sex Roles: A Journal of Researcha Sex Roles publishes original researcharticles and theoretical papers concerned with the underlying processes andconsequences of gender role socialization, perceptions, and attitudes.Signs: Journal of Women in Culture and Societya An international journal inwomen’s studies that publishes articles from a wide range of disciplines ina variety of voices—articles engaging gender, race, culture, class, sexuality,and/or nation.Women’s Studies International Forum The goal of this journal is to aidthe distribution and exchange of feminist research in the multidisciplinary,international area of women’s studies and in feminist research in otherdisciplines.Women’s Studies Quarterly This journal focuses on teaching in women’sstudies. Thematic issues include such features as course syllabi, discussionsof strategies for teaching, and bibliographies.a. Journals are indexed in ERIC.Box 3.4 (Continued)literature Review and Focusing the Research 101Identify Primary Research JournalsAdditional primary research articles can be identifed by examining the reference listsfound at the end of relevant journal articles or books. You can also go directly to journalsthat you know publish articles related to your topic. This is especially important in lightof the selectivity of the databases discussed in the previous section. Researchers who areworking from a transformative paradigm should be aware of the journals that deal withissues specifc to marginalized groups, such as those in Box 3.4 for feminists, as well asjournals such as Latin American Perspectives, Hispanic Journal of Behavioral Sciences,Journal of Multicultural Counseling and Development, Journal of Negro Education,and Journal of Black Studies. Disability, Handicap & Society is a journal that frequentlyfocuses on the transformative paradigm in research with people with disabilities, andThe Counseling Psychologist (2008) devoted an entire issue to the topic of multiculturalcounseling for psychologists and educators (Vol. 36, No. 2). A more extensive list ofspecial education journals can be found in Box 3.5.American Annals of the DeafAmerican Journal on Mental RetardationAnnals of DyslexiaAustralasian Journal of Special EducationBehavioral DisordersBritish Journal of Special EducationCareer Development for Exceptional IndividualsEducation and Training in Mental RetardationExceptional ChildrenExceptionality: A Research JournalInternational Journal of Disability, Development and EducationJournal of Autism and Developmental DisordersJournal of Deaf Studies and Deaf EducationJournal of Early InterventionJournal of Learning DisabilitiesJournal of Special EducationJournal of Speech and Hearing ResearchJournal of the Association for Persons With Severe HandicapsLearning Disability QuarterlyMental RetardationRemedial and Special EducationResearch in Developmental DisabilitiesVolta Review (deafness)Box 3.5 Selected Journals Containing Special Education Resource Information102 ReseaRch and evaluation in education and PsychologyPersonal NetworkingAdditional resources can be found by talking to people who are doing work in areasrelated to your interest. This can include people at your own institution or those you meetthrough professional associations, such as the American Educational Research Association,the American Evaluation Association, the American Psychological Association, the Councilfor Exceptional Children, or the National Association of the Deaf. Talking to people whohave completed related work can reveal sources that you were unaware of, such asunpublished research reports, and provide you with leads from work that is in progressfor that researcher.Two examples of well planned and documented searches are provided in Boxes 3.6and 3.7. As a researcher, it is always a good idea to carefully document your search strategy.In this way you can backtrack to helpful strategies if you need additional information andindicate to the reader how thorough you were in your search process.The primary purpose of the accommodations research conducted over the past 3years has been to determine the effect of accommodations use on the large-scale testscores of students with disabilities.MethodFour major databases were searched to identify research on test accommodationspublished from 1999 through 2001: ERIC, PsycINFO, Educational Abstracts, and Digital Dissertations. Research papers were also obtained at major conferences. Additionalresources for identifying research included• Behavioral Research and Teaching at the University of Oregon (brt.uoregon.edu/)• Education Policy Analysis Archives (epaa.asu.edu)• National Center for Research on Evaluation, Standards, and Student Testing(www.cse.ucla.edu/)• Wisconsin Center for Educational Research (www.wcer.wisc.edu/testacc/)Several search terms were used. The terms were varied systematically toensure the identifcation of all research on changes in testing, published from 1999through 2001. Search terms included• accommodation• test adaptation• test changes• test modifcations• test accommodations• state testing accommodations• standards-based testing accommodations• large-scale testing accommodationsBox 3.6 Method Used in Literature Review: Example 1literature Review and Focusing the Research 103A decision was made to limit the selection of publications to empiricalresearch. Included within this realm are studies with samples consisting ofpreschool, kindergarten through high school, and postsecondary students. Thefocus of the empirical research was not limited only to large-scale testing, butalso included studies that incorporated intelligence tests and curriculum-basedmeasures (CBM). We decided to focus on testing accommodations as opposedto instructional accommodations, although there is some overlap between thesepurposes in the literature. We did not include any conceptual or opinion piecesin this analysis.SOURCE: S. Thompson, Blount, & Thurlow (2002, pp. 2–3).In selecting research studies for inclusion in this synthesis, a systematic review of therelevant literature was conducted according to the following parameters:1. Studies with direct relevance to the topic, i.e., those involving ELLs inscience education and those addressing the intersection between science educationand English language acquisition. To the extent that language and culture areinterrelated (“languaculture” according to Agar, 1996), this review includes studiesexamining cultural beliefs and practices that ELLs bring to the science classroom.2. Studies published from 1982 through 2004. The landmark for scienceeducation reform was the release of the Science for All Americans document(American Association for the Advancement of Science, 1989). The periodbetween 1982 and 2004 spans the years leading up to the release of this document(1982–1989) and more than a decade afterward (1990–2004).3. Studies conducted within the United States and abroad, but limited tothose published in English and focusing on settings where English is the mainmedium of science education.4. Studies focusing on science education at the elementary and secondarylevels, K–12. Studies involving postsecondary or adult learners are not included.5. Empirical studies from different methodological traditions, including(a) experimental and quasi-experimental studies; (b) correlational studies; (c) surveys;(d) descriptive studies; (e) interpretative, ethnographic, qualitative, or case studies;(f) impact studies of large-scale intervention projects; and (g) demographics orlarge-scale achievement data.6. Literature reviews and conceptual pieces.Within these parameters, the process of gathering studies from the various sourceswas carried out as follows. First, a search of the ERIC database was conducted using(Continued)Box 3.7 Literature Review in Science Education With EnglishLanguage Learners (ELLs): Example 2104 ReseaRch and evaluation in education and PsychologyInvolvement of Community MembersThe combination of self-knowledge with cultural knowledge and skills ineffective partnering facilitates the development of the research or evaluation focus andidentifcation of questions, development of interventions, and making decisions aboutdesign, measures, samples, data collection, analysis, interpretation, and use that are inkeeping with the philosophical assumptions of the transformative paradigm (Mertens,2009). Following proper channels to enter a community is important, and strategiesfor doing this will vary by context. Some Native American Indian communities havedeveloped specifc protocols for anyone who wants to conduct research in theircommunities (LaFrance & Crazy Bull, 2009); Maori people from New Zealand havealso developed similar protocols for research in their community (Cram, 2009). Deafresearchers have adapted the Maori Terms of Reference to suggest a protocol forresearch in that community (Harris, Holmes, & Mertens, 2009). These protocols willbe discussed in more depth in the chapter on sampling. However, it is importantfor researchers to know how to enter a community with respect, to communicatetheir intentions to members of the community in the appropriate way, and to makeclear what benefts will accrue to themselves and to the community. Schneideret al.’s (2004) method of involving people with mental illness is one example of howcommunity members can be involved in the decision process about what topics tostudy in the research.the terms “science education” and “school” combined with the following keywords:“bilingual,” “limited English profcient (LEP),” “English Language Learner (ELL),” “English to Speakers of Other Languages (ESOL),” “English as Second Language (ESL),”“equity,” “diversity,” “minority,” “culture,” “language,” “multicultural,” “at-risk,” “race,”“immigrant/immigration,” and “urban education.”Second, selected journals were reviewed manually, including the journals supported by the American Educational Research Association (American EducationalResearch Journal, Educational Researcher, Review of Educational Research, andReview of Research in Education), as well as other well-known journals focusingon science education (Journal of Research in Science Teaching and Science Education) and bilingual/TESOL education (TESOL Quarterly and Bilingual ResearchJournal).From the sources named above, only peer-reviewed journal articles wereincluded. Among these, articles, empirical studies, literature reviews, and conceptualpieces were included. Empirical studies were used to report research results, whereasliterature reviews and conceptual pieces were used to frame key issues.Neither practitioner-oriented articles (e.g., teaching suggestions or descriptionsof instructional programs, materials, or lesson plans), nor opinion or advocacy piecesunsupported by empirical evidence were included.SOURCE: O. Lee, (2005, p. 495).Box 3.7 (Continued)literature Review and Focusing the Research 105Step 4: Conduct the SearchIn conducting the search, you should make a plan to search preliminary sources,check the table of contents, abstracts, and lists of references in primary research journals,access your personal network, and involve community members as appropriate. Theremainder of this section focuses on the search strategy as it applies to accessingpreliminary sources.Prepare to Search Preliminary SourcesSelect the preliminary sources that you think contain the best information on yourtopic (see Box 3.2). Then identify key terms that will help you locate the literatureincluded in the database of choice. One way that researchers select key terms is to fndone primary research article that is “exactly” on target and identify the terms used todescribe that article.A search strategy based on using the ERIC online system is used to illustrate thisprocess. The search strategy is similar when using other databases and indexes, suchas PsycARTICLES and PsycINFO. Most databases give you many choices for searching,such as title, author, abstract, subject, or full text. The title, author, and abstract choicesare fairly self-explanatory. Author and title are not usually used in the beginning of aliterature review because you usually are not seeking a specifc article during the earlystages of searching. The subject choice needs a bit of explanation. Subject words are thosethat were used by the people who work for the database to categorize that item. Thesewords are contained in a thesaurus, usually available in the online system. Each item inthe database has a feld associated with it that contains subject words that an indexerselected, and that is the feld that is searched when you choose a subject word strategy.Full text searchers, on the other hand, allow researchers to choose words that reﬂect theirown vocabulary in the description of the topic. Full Text means the computer searchesfor these terms using a “free text” strategy; that is, it searches anywhere in the documentfor the words that you enter. Advantages and disadvantages accrue to whichever searchstrategy is chosen.The easiest way to start is to use a key word strategy to determine if the words thatyou think are appropriate produce references that match your conceptualization of theproblem. For example, for the topic of sexual abuse of deaf students, I started in ERICusing sex abuse deaf as key words. The computer said there were no articles availablethat combined those three terms. I took a few minutes to read the directions in ERICand found that I could use a multifeld search strategy, separating the terms. So I usedsex? AND abuse AND deaf?. (There is a good reason, explained later, for the inclusionof the ? and the word and in this search specifcation.) This resulted in 19 entries. Oneof the entries was Black and Glickman (2006), “Demographics, Psychiatric Diagnoses,and Other Characteristics of North American Deaf and Hard-of-Hearing Inpatients” (seeBox 3.8).If you have diffculty fnding references using your own key word vocabulary, checka thesaurus of terms to determine how the indexers might have conceptualized yourtopic. Use of subject descriptors can be helpful in narrowing down a search, as long asthe descriptors are defned in a way that is compatible with your topic. They can alsobe helpful in broadening a search by suggesting other terms that could prove fruitful insearching.106 ReseaRch and evaluation in education and PsychologyBlack, Patricia A. Glickman, Neil S. Demographics, Psychiatric Diagnoses, andOther Characteristics of North American Deaf and Hard-of-Hearing Inpatients [JournalArticles. Reports—Research] Journal of Deaf Studies and Deaf Education. v11 n3p303–321 2006 AN: EJ738331Abstract: This study examined demographic and clinical data from a specialty deaf inpatient unit so as to better understand characteristics of severely and chronically mentally ill deaf people. The study compares deaf and hearing psychiatric inpatients ondemographic variables, psychiatric discharge diagnoses, a language assessment measure, a cognitive ability measure, and a measure of psychosocial functioning and riskof harm to self and others. Overall, fndings indicate a broader range of diagnoses thanin past studies with posttraumatic stress disorder being the most common diagnosis.Compared with hearing patients in the same hospital, deaf patients were less likelyto be diagnosed with a psychotic or substance abuse disorder and more likely to bediagnosed with a mood, anxiety, personality, or developmental disorder. Psychosocialfunctioning of the deaf patients was generally similar to hearing psychiatric patients.Deaf patients presented signifcantly higher risks than hearing patients in areas of selfharm and risk of sexual offending. Cognitive scores show that both the deaf and hearing inpatient population is skewed toward persons who are lower functioning. Anadditional surprising fnding was that 75% of deaf individuals fell into the nonﬂuentrange of communication in American Sign Language. (Author)Box 3.8 Journal Citation Entry From ERICNow, why include a ? in the search terms, and what is the importance of the and inthe list? You can refne your search in the following ways:1. Truncate the terms you use. This has the effect of broadening the search toinclude any terms that begin with the letters that you enter, no matter how they end.In ERIC (the Educational Resources Information Center), the truncating symbol is a ?Therefore, entering sex? would include sex, sexual, sexes, and so on, and deaf? wouldinclude deaf, deafness, deafened, and so on.2. Use Boolean or positional operators to combine terms. Boolean logic allows youto use the words and, or, not, and nor (one but not both words are in a record). Thusasking for sex? and abuse and deaf? yields references in which the three terms appearin the same record. The or operator yields references that have either or both words inthe same record. So, I could have asked for sex? abuse or child abuse and deaf?. Thiswould have given me all the records that contain sex? abuse or child abuse and deafness.In addition, I could have broadened my search by including deaf? or hearing-imp?. Thiswould have resulted in all references that had either deaf, hearing-impaired, hearingimpaired, or hearing impairment in their records.Positional operators include same, with, adj, and near, and they limit retrieval byspecifying how near key words must be to each other. Same means that both words mustbe in the same feld of the same record; with means both words are in the same sectionof the same feld of the same record; adj requires that the words must be next to oneanother (adjacent) in the order specifed; and near fnds references in which words arenext to one another in any order (e.g., sex abuse or abuse sex).literature Review and Focusing the Research 1073. There are other ways to limit the search, such as by year of publication or limitingthe feld that is searched (e.g., title only). Certain stop words are not allowed to be usedas key words (e.g., about, all, its), but all of these things can be learned by reading theonline instructions. As you get into using a database, it is always a good idea to read theonline instructions to see what can be accomplished and how.4. Obviously, the search process can be broadened by inclusion of additionaldatabases or indexes. For example, when I searched PsycARTICLES using the samedescriptors (i.e., sex? and abuse and deaf?), I identifed 57 additional references that didnot overlap with those found in ERIC.A fnal word of advice: Cultivate a good relationship with your librarian. I invite theresearch librarian to my research classes to make a presentation on databases, searchstrategies, and documentation of fndings from the search. Students report that visiting theresearch librarian is extremely helpful.Interpret What You SeeYou can locate at least two types of documents in ERIC: journal articles and otherreferences that are available through ERIC. Journals may be obvious because theyhave the name of the journal, but if you are not sure look for an EJ code in the ERICcitation. Other references are noted with the abbreviation ED (education document) andare typically presentations made at professional meetings, curriculum guides, researchreports, or other similar materials. An example of an ERIC full text abstract for a journalarticle was presented in Box 3.8 and one for an ERIC document is in Box 3.9.Box 3.9 ED Document Citation From ERIC Accession NumberED477969AuthorKarcher, Michael J.TitleThe Hemingway: Measure of Adolescent Connectedness—Validation Studies.Page Count59Peer ReviewedNoDate ofPublication2001ERIC SubjectHeadings*Adolescent DevelopmentAdolescentsAge DifferencesBehavior ProblemsDelinquency*Interpersonal Relationship*Psychometrics*Social EnvironmentSubstance AbuseTest Validity (Continued)108 ReseaRch and evaluation in education and PsychologySelect TitlesMost databases provide an abstract of the articles listed. By scanning these abstracts,you can make a decision as to the worth of obtaining the complete article. Advancesin technology now also make it possible to view many full text articles while you areengaged in the search process. Hence, researchers are faced with a bit of a paradoxconcerning the amount of time it takes to do a literature review. If you only have theabstract, you read it quickly and make a determination if it is what you want. If you thinkit is and full text is not available, then you need to go to the library or order the articleBox 3.9 (Continued) Identifers*Social Connectedness.AbstractThis investigation reports the development of a measure ofadolescent connectedness and estimates of its psychometricproperties. A measure was developed to assess the ecologicaland developmental dimensions of adolescent connectedness,defned as adolescents’ caring for and involvement in specifcrelationships and contexts within their social ecology.Exploratory and confrmatory factor analyses in studies oneand two yielded theoretically consistent factor solutions. Thesemodels were cross-validated in studies three and four with threegeographically and ethnically diverse adolescent samples totaling1454 adolescents. The measure of adolescent connectednessdemonstrated satisfactory inter-item and test-retest reliability andconvergent validity across samples. Consistent with social controland problem-behavior therapy, two higher order factors emergedacross all of these samples: conventional vs. unconventionalconnectedness. These two dimensions of connectedness werefound to differentially explain substance use for delinquent andnon-delinquent adolescents. Using this ecological assessment,adolescent connectedness appears to differ as a function of age,sex, and problem-behavior status; varies across relationshipsand contexts; reﬂects either conventional or unconventionalbehaviors and attitudes; and can explain engagement in risktaking behaviors. (Contains 62 references and 6 tables.) (Author)NotesPaper presented at the Annual Conference of the AmericanPsychological Association (109th, San Francisco, CA, August24–28, 2001).Level ofAvailability1Publication TypeInformation Analyses. Reports—Research. Speeches/MeetingPapers.LanguageEnglishEntry Month200402 literature Review and Focusing the Research 109through interlibrary loan or some other mechanism. If the full text is available, you mayfnd yourself (like me) reading many articles because they are interesting and then youwonder how the day is done and you have not made the progress that you expected.Intellectual curiosity is good; focus is also good.Step 5: Obtain Full Text ResourcesAs mentioned previously, many journal articles and books are now available onlinein full text versions. If you cannot obtain the article in this manner, then it would begood to check the list of holdings at your library. If the journal you seek is held byyour library, you are in luck: Go to the shelves (or the librarian at the help desk) andread the article. However, if your library does not have the item, you may avail yourselfof an interlibrary loan service. If you provide complete bibliographic information, thelibrarian can determine which other library has the article and make a request to haveit sent to you. There is often a small charge for this service. In some libraries, obtaininga copy of the article is available by an online request as you are doing your search.The computer may ask you if you want to order the document, and then it will tellyou how much it costs to obtain. You have the option of agreeing to pay the cost, and,if you agree, the library that holds the reference is electronically contacted and askedto transmit the article to your library. Amazing! (The researcher’s equivalent to HomeShopping Network.)If you have chosen to review an ED document from an ERIC search, that documentmay also be available in full text online. However, if it is not, then the document shouldbe available for your review on microfche in the library. The microfche are organized inascending order according to their ED numbers, so they are usually easy to fnd.Extending Your Thinking:Primary and Secondary SourcesWhat is the difference between a primary source and a secondary source? When andwhy would you choose to use one or the other? Have you been able to locate a secondary source on your topic of interest?What search strategies have you found to be particularly effective in locatingresearch on your topic of interest? Have you used networking? Professional associations? Why would these be important resources? What computerized databases haveyou used? Do you feel comfortable in using the computer to search for articles ofinterest for your research topic? Have you sought out journals or other sources ofinformation (e.g., direct dialogue with individuals who are experiencing oppression)that represent the “transformative perspective” in research?H. M. Cooper and Hedges (1994) recommend that researchers limit their literature review efforts to “mainstream” journals on the grounds that these represent the“cream of the crop” of research efforts. Transformative researchers might contendthat this would result in a bias because the viewpoints of oppressed people might notbe represented in those journals. Where do you stand on this issue?110 ReseaRch and evaluation in education and PsychologyStep 6: Read and Prepare BibliographicInformation and NotesOnce you have the article in hand, read the document to determine if it is reallywhat you want. If you decide that it is relevant to your topic, you will want to recordbibliographic information and notes on each article. This can be done electronically ormanually, using old-fashioned note cards.Bibliographic InformationIf you are searching such databases as ERIC or PsycARTICLES, you can use a new digitalresource called RefWorks to electronically save the bibliographic information about all thereferences that you select. When you are ready, RefWorks will print out a reference list in APAformat (or the format that you select). That is not all: If the article is available in full text, youcan save it in RefWorks with the bibliographic information. If you do not have access to thiselectronic resource, then you can save the bibliographic information on note cards or in aword processing document. The important thing is to make sure you get ALL the informationyou need when you are working with the document so you do not have to try to fnd it laterwhen you are writing up your literature review. Words of wisdom—painfully learned.The most important thing to remember in recording bibliographic information is tobe complete and accurate. Some of the problems associated with recording bibliographicinformation have been reduced because of the ability to print such information directly fromthe computer screen. However, if you have hundreds of printouts, you may want to recordthe information on index cards or in some other easily retrievable electronic format. (I donot always have a computer with me when I want to record bibliographic information, soindex cards are handy. My secret at the moment: I cut and paste from the full text documentsand then type up the references later; next time I write a book, I plan to use RefWorks.)Although several options are available for the format of recording bibliographicinformation, the most common style for education and psychology is based on theAmerican Psychological Association’s (2001) Publication Manual (5th ed.). This is thebasic format for a journal citation:Author’s Last Name, Initials. (date). Title of journal article. Title of Journal, volumenumber(issue number), page numbers.For example:Sullivan, P. M. (1992). The effects of psychotherapy on behavior problems of sexuallyabused deaf children. Child Abuse and Neglect: The International Journal, 16(2),297–307.Book:Author’s Last Name, Initials. (date). Title of book. Place of publication: Publisher.For example:Mertens, D. M. (2009). Transformative research and evaluation. New York: Guilford.Book chapter:Author’s Last Name, Initials. (date of publication). Title of chapter. In Name of Editor (Ed.),Name of book (page numbers of chapter). Place of publication: Publisher.literature Review and Focusing the Research 111For example:LaFrance, J., & Crazy Bull, C. (2009). Researching ourselves back to life: Taking controlof the research agenda in Indian Country. In D. M. Mertens & P. Ginsberg (Eds.),Handbook of social research ethics (pp. 135–149). Thousand Oaks, CA: Sage.There are differences in citation style associated with different types of documents(e.g., books, chapters in books, government reports, etc.), so you are advised to obtain acopy of the APA Publication Manual to guide the compilation of bibliographic information.In addition, APA has added a great deal of information about how to handle Web-basedinformation. Some of that is reﬂected in the Publication Manual, but you can go to theAPA’s Web site (www.apa.org) for an update on changes that they recommend and clickon their publications link.Notes on Each StudyExactly what notes to write for each study varies greatly and depends on thenature of the study, the purpose of the review, and the intended use of the data. If theresearcher intends to conduct a comprehensive literature review of studies that reporttheir results in statistical form, the use of coding forms and computerized databases isrecommended.For empirical research studies, the following outline can be helpful:1. Area of interest; literature cited; need addressed; theoretical framework; researchquestions/hypothesis2. Paradigm of researcher(s)3. Design, includinga. Specifc research approachb. Sampling strategyc. Characteristics of participantsd. Data collection instruments and procedures4. Data analysis strategy5. Results6. Conclusions7. Your own evaluation (including strengths and weaknesses and ideas for yourown research, such as promising methodological or conceptual suggestions).The evaluation of research reports is Step 7 (discussed below). Once you haveevaluated the research report, you should return to your note cards or fles and enter yourown assessment of the strengths and weaknesses of the research.Step 7: Evaluate the Research ReportsYou will be learning how to evaluate research as you progress through this text. Alisting of critical analysis questions for evaluating primary research is provided at the endof each chapter. The questions are organized according to the sections of the researchreport (e.g., introduction, method, etc.), with additional specifc questions relevant toeach approach to research (e.g., experimental, quasi-experimental, etc.).112 ReseaRch and evaluation in education and PsychologyStep 8: Synthesize the StudiesBefore you actually begin the synthesis of the research, there are a few things to keepin mind. Organization is a plus. If you can develop a ﬂexible framework for organizingthe studies as you fnd them, it will be easier for you to approach the synthesis stage.I say ﬂexible because the framework might add, delete, or redefne categories as youmove through the review process. For example, for the revisions to the third edition ofthis book, I had categories for relevant studies I found in the early stages that related toeach chapter title. As I found studies, I saved them into those fles. As I began writing,I moved some of the fles to more differentiated categories; for example, what startedas the Chapter 1 introduction became paradigms, and paradigms became postpositivist,constructivist, transformative, and pragmatic. So, as I approached writing I had studies thatwere relevant to each part of the chapter. Of course, you sometimes run into problems inthat one study may have applicability for more than one topic, but no one said this wouldbe simple. Some of my students like to keep tables of the studies they fnd and thenorganize the studies in tables under headings that make sense for their writing; some liketo keep note cards on their studies that they can rearrange manually. Other students liketo use a qualitative software package to save the articles as “data fles” that can be codedand searched when it is time to write. This approach has merit but demands technicalskills with the software and diligence to do the coding. However, once you get to thesearch and retrieve part, it may all seem worth it.An example of a thematic organizational approach can be seen in Billingsley’s (2004,p. 39) study of teacher attrition and retention in special education. She addressed fourmajor themes: teacher characteristics and personal factors, teacher qualifcations, workenvironments, and teachers’ affective reactions to work. She included a critical analysis ofdefnitions, concepts, and methods used to study special education attrition. In anotherexample, Au (2007) included the following themes in his literature review about high stakestesting: Subject matter content, pedagogy, and structure of knowledge. Au offers someimportant insights into the ﬂexible and evolving nature of coding systems. He writes,The full elaboration of my coding template evolved during the course of theresearch. For instance, it has been widely asserted over the past 20-plus years thathigh-stakes tests cause a narrowing or contraction of nontested subject areas. Iwas aware of research substantiating this assertion prior to beginning the templateanalysis and thus assumed that I would need to code the studies that reported thetheme of contraction of subject matter content. (p. 259)Two main options exist for the synthesis of research studies: narrative and statisticalmethods. The choice of the type of synthesis depends on the type of extant researchliterature on a topic and on the purpose of the researcher. In this chapter, I focus on thenarrative approach to synthesizing literature. The statistical approach (meta-analysis) isexplained in Chapter 13.Narrative SynthesisThe narrative approach to literature synthesis is most commonly used in primaryresearch studies. It is appropriate for studies that use a qualitative design as well asfor quantitative studies. In a narrative synthesis, the writer must organize the studies ina conceptually logical order and provide suffcient detail about the studies to supportrelevant critical analysis of them. The amount of detail provided (as well as the numberof studies cited) will be inﬂuenced by the purpose of the literature review:literature Review and Focusing the Research 1131. Typically, the literature review section of a journal article includes a limitednumber of references that are selected on the basis of relevancy to the problem at hand,presenting a balanced picture, and establishing a rationale for the reported research.2. A literature review for a research proposal is usually more extensive. If theresearch proposal is for a thesis or dissertation, it is expected to be quite comprehensivein most universities.If you organized your literature into meaningful categories as you collected it, thenthis makes your writing easier. Provide an overview of your topic and describe the methodsyou used to search the literature. Then provide an advance organizer for the reader of thesubtopics that you will address. For each study make a determination if it is important toreport details of its strengths and weaknesses in order to establish the overall picture ofknowledge in the feld or to provide support for your choice of methods. It is possible toexplain several studies in detail and then cite other studies that agree or disagree with thefndings of those studies, rather than a detailed critique of every study in your literaturereview. Sometimes literature reviews include a separate section on the proposed study’stheoretical framework based on prior research. The literature review should lead to astatement of the need and purpose for the study, research questions, and hypotheses.Step 9: Use the Literature ReviewThe narrative or statistical synthesis serves as a basis for the literature section of aresearch proposal or report. The Appendix contains an outline for a research proposalfor a thesis or dissertation. It is important for the proposal writer to realize that eachinstitution and sponsoring agency has its own requirements for proposal writing, so it isbest to check with those sources before proceeding with writing. Proposal writers mustalso realize that in this synthesis of research they are “selling” their ideas to a researchcommittee, institutional review board, or funding agency. So above all, make it clear whythe research is important (based on what is known from the extant literature).Conceptual Framework and Program TheoryIn some ways, the conceptual framework is like the chicken-or-the-egg controversy.A researcher’s original conceptual framework inﬂuences the planning and conductingof the literature review. However, if a researcher keeps an open mind throughout theliterature review process, a more sophisticated and (often greatly) modifed conceptualframework should emerge. Table 3.1 displays the inﬂuence of the theoretical frameworkon the choice of research questions and its implications for action. On the basis of workby Villegas (1991) on theoretical frameworks used to explain differential achievementby ethnic minority students, four different research questions are used to illustrate thispoint. The IQ defcit theory and the cultural defcit theory reﬂect a theoretical stance thatsuggests the problem is either “in the child” or “in the cultural group from which the childcomes.” The cultural difference theory reﬂects the constructivist paradigm, and the powerinequity theory reﬂects the transformative paradigm.These various explanations for poor academic achievement by ethnic minority childrenexemplify alternative theories that might be held by the researcher or by the researchsponsor or participants. Researchers must be aware of their own personal theoretical baseas well as that of the sponsors and the participants. For example, J. E. Davis (1992) notedthat research on African American families often depicts them as deviant, pathologicalsocial organizations unable to fulfll the major responsibilities of socializing their membersfor productive roles in society (the defcit model). The conclusion based on this model,then, is that this undersocialization leads to negative outcomes, such as low academic114 ReseaRch and evaluation in education and Psychologyachievement, juvenile delinquency, drug abuse, and teenage pregnancy. This conclusion isreached by ignoring the data that “inform us of the unique and often precarious position ofAfrican Americans” (J. E. Davis, 1992, p. 59). More than one third of the African Americanpopulation in the United States lives at or near the poverty level. It is the economic conditionand its implications (e.g., inadequate housing and food, poor sanitation, overcrowding) thatbring about negative consequences, such as poor health, family violence, and delinquency.Ladson-Billings (2006) presents data that suggest that conditions have not improved forAfrican American students since Davis wrote her work in 1992. Ladson-Billings suggests thatthere is a more insidious reason that underlies both economic and education deprivation:racism. Thus, the use of a theoretical framework that starts with the marginalized livesallows researchers to understand the experiences of oppressed groups.In the past, much of educational and psychological research on racial or ethnicminorities, women, people with disabilities, and other marginalized groups derived froma defcit perspective that located the problem in individuals and focused on the negativereasons that they did not achieve or perform certain functions or activities. More recently,researchers have shifted to a social-cultural perspective that focuses on the dynamicinteraction between the individual and environment over the life span (Seelman, 2000;A. T. Wilson, 2001). This focus on strengths and modifcations of contextual factors hasemerged under a variety of names such as positive psychology (Aspinwall & Staudinger,2003; S. J. Lopez & Snyder, 2003; Seligman & Csikszentmihalyi, 2000) and resilience theory(J. H. Brown, D’Emidio-Caston, & Benard, 2001; R. Cooper, 2000). Such a theoreticalframework has led to reframing research questions to focus on strengths. For example,What are the positive aspects of parenting a deaf child? (Szarkowski, 2002)What are the variables that contribute to successful transition from high school tocollege for deaf African American students? (Williamson, 2002) Table 3.1Inﬂuences of Diﬀerent Theoretical Frameworks on Research TheorySample Research QuestionRecommendations for ActionIQ defcit theoryCultural defcit theory(sociocultural defcits inhome life)Cultural difference theoryPower inequities (school failureis rooted in a struggle forpower; schools play a role in thepreservation of the socioeconomicorder)Are minorities genetically inferiorto White students?Is there a higher rate of singleparent families among minorities?How do Black and White parentscompare in discipline techniques?What is the nature of language useat home and at school in terms ofasking and answering questions orin seeking help?How can we teach minoritystudents so they do not continueto be oppressed?Remedial education, but theproblem is really “in” the child.Remedial education, but theproblem is really “in” the family.Build on students’ priorexperiences; increase theirlanguage use structures.Explicitly teach minority childrenthe means to access power,including linguistic forms andways of talking, writing, andinteracting. Teach them to valueethnic characteristics and that theculture of the dominant group isnot necessarily superior. literature Review and Focusing the Research 115J. M. Morse (2003) also notes that the theoretical framework in qualitative research isused to focus the inquiry and give it boundaries rather than to serve as the guide for datacollection analysis. Deductive analysis based on a static theoretical framework violatesthe assumption of constructivist qualitative inquiry. The theoretical framework should beviewed as a conceptual template with which to compare and contrast results, not seen asestablishing a priori categories for data collection and analysis.Research Questions and HypothesesThe literature review serves as a foundation for forming research questions. Hedrick,Bickman, and Rog (1993) suggest that the research questions operationalize the objectivesof the proposed research. They focus the research hypotheses and clarify what informationneeds to be collected from what sources under what conditions.Framing the research questions can be a diffcult task for beginning researchers.Hedrick et al. (1993) present a taxonomy for categorizing research questions that includesfour categories of questions: descriptive, normative, correlational, and impact. Each isbrieﬂy discussed in the following paragraphs.Descriptive research questions are designed to produce information about what is orhas been happening in relation to the target of the research. For example, the researchermight want to describe certain characteristics of the participants in an intervention.Alternatively, the researcher might be interested in describing the prevalence of aparticular disability within an identifed domain (e.g., What is the prevalence of mentalretardation in Black middle school children?).Normative research questions go beyond description and require that the informationgenerated in response to the descriptive research question be compared with somestandard or expected observation. For example, in special education, there are minimumrequirements regarding most aspects of the service delivery system. A normative researchquestion might ask, Were individual education plans (IEPs) in place before the placementwas made, in accordance with the minimal service delivery requirements?Correlative research questions are used to identify relationships to enable theexplanation of phenomena. As Hedrick et al. (1993) point out, data derived in response tosuch questions indicate the strength and direction of a relationship between two or morevariables, not causality. For example, the special education researcher might ask, What isthe relationship between the size of family and the presence of emotional disturbance insiblings? If a strong, positive relationship is found, this would not lead to the conclusionthat large families cause emotional disturbance in siblings. Such a relational fnding wouldsuggest the need for further study to uncover the causal relationships.Impact research questions represent the last category offered in the Hedrick et al.(1993) taxonomy. Here, the researcher’s aim is to identify effects, to establish causallinks between an independent variable (the intervention) and a dependent variable(the anticipated change). According to Hedrick et al.’s framework, the researchermight investigate two types of effects: simple and relative. Research on the impact ofan intervention (literacy intervention) on a behavior (reading) is one example of animpact study. The research question might ask, do students who participate in the literacyintervention perform better on end-of-year assessments in reading than students who donot participate? If the researchers choose (and this is good practice) to explore the impactof the intervention on other related outcomes (e.g., math, self-confdence), additionalquestions could address relative effects.Impact questions can then be reformulated and stated as hypotheses. A hypothesisis an “if . . . , then . . .” statement. For example, a hypothesis might state this: “If studentsare exposed to a particular intervention, they will behave in a certain, predictable manner.”A sample hypothesis for the literacy study cited above might read this way: “If students116 ReseaRch and evaluation in education and Psychologyparticipate in the literacy intervention, then their scores on the end-of-year readingassessments will be higher than the scores for students who do not participate.” This is knownas a directional hypothesis because it is stated in the direction of the expected outcome. Aresearcher could choose to state a null hypothesis—that is, a statement that did not specifythe expected direction of the outcome. The previous hypothesis could be restated as a nullhypothesis: “There will be no difference in end of year reading assessments for studentswho do participate in the literacy intervention as compared to those who do not.”In summary, the literature review serves many purposes. It establishes a historicalperspective on the intended research, provides a vision of the need for additional research,and enables the researcher to develop a conceptual framework for the research. Thisframework allows the researcher to generate research questions and hypotheses to guidethe design and conduct of the research. In qualitative research, typically, the researcherwill refne, modify, add, and even discard questions throughout the progress of the study(J. M. Morse, 1994). Therefore, qualitative researchers are advised to begin with broaderquestions that can be modifed in response to discoveries made during the study. Nomatter which research paradigm or approach is used, the literature review is an essentialingredient in the research process.Critical Analysis of Literature ReviewsThe criteria for critically analyzing literature reviews depends (again) on the nature ofthe review being analyzed. A literature review that serves as an introduction to a primaryresearch study reported in a journal would be subject to a different type of scrutiny thanwould a comprehensive literature review on a topic. Nevertheless, a framework initiatedby Hart (1999) and extended by Boote and Beile (2005) provides a way to assess thequality of a literature review. (Their rubric of these categories is included as Table 3.2.)Boote and Beile included fve categories for their framework.• Coverage refers to the adequacy of the coverage of the topic, as well as makingexplicit criteria for exclusion and inclusion of studies for the review. Does the reviewerinclude relevant works and exclude irrelevant ones? Writing a dissertation does not meanciting every study ever written on your topic. Coverage should be judged in terms ofcomprehensiveness, breadth, exclusion, relevance, currency, availability, and authority(Bruce, 2001). Researchers can bias the results of a literature review by excluding datathat is methodologically questionable, based on their own personal, subjective judgment(Ogawa & Malen, 1991). Or they may present conclusions that are more frm and clearcut than is justifed because of the exclusion of studies with “murky” results. Without aclear specifcation of the method used to search for research and of the criteria used forinclusion or exclusion, it is diffcult to judge the quality of a review. Au’s (2007) review ofhigh stakes testing literature provides an example of decision rules for inclusion/exclusionwhen a large pool of studies is initially identifed. He narrowed the studies to those(a) based on original, scholarly research,(b) using qualitative methods,(c) taking place in the United States, and(d) specifcally addressing the relationship between high-stakes tests and eithercurriculum or instruction or both (p. 259).NOTE: The column-head numbers represent scores for rating dissertation literature reviews on 3-point and 4-point scales (endnote 4 explains our choice of the two types of scales). Adaptedfrome Doing a Literature Review: Releasing the Social Science Research Imagination (p. 27) by Christopher Hart. 1999, London, SAGE Publications. Copyright 1999 by SAGE Publications.Adapted with permission. CategoryCriterion12341. Coveragea. Justifed criteria for inclusion andexclusion from reviewDid not discuss the criteriainclusion or exclusionDiscussed the literature includedand excludedJustifed inclusion andexclusion of literature2. Synthesisb. Distinguished what has been done inthe feld from what needs to be donec. Placed the topic or problem in thebroader scholarly literatured. Placed the research in the historicalcontext of the felde. Acquired and enhanced the subjectvocabularyf. Articulated important variables andphenomena relevant to the topicg. Synthesized and gained a newperspective on the literatureDid not distinguish what hasand has not been doneTopic not placed in broaderscholarly literatureHistory of topic not discussedKey vocabulary notdiscussedKey variables andphenomena not discussedAccepted literature at facevalueDiscussed what has and has notbeen doneSome discussion of broaderscholarly literatureSome mention of history of topicKey vocabulary defnedReviewed relationships amongkey variables and phenomenaSome critique of literatureCritically examined the state ofthe feldTopic clearly situated inbroader scholarly literatureCritically examined history oftopicDiscussed and resolvedambiguities in defnitionsNoted ambiguities inliterature and proposed newrelationshipsOffered new perspective3. Methodologyh. Identifed the main methodologies andresearch techniques that have beenused in the feld and their advantagesand disadvantagesi. Related ideas and theories in the feldto research methodologiesResearch methods notdiscussedResearch methods notdiscussedSome discussion of researchmethods used to produce claimsSome discussion ofappropriateness of researchmethods to warrant claimsCritiqued research methodsCritiqued appropriateness ofresearch methods to warrantclaimsIntroducednew methodsto addressproblems withpredominantmethods4. Signifcancej. Rationalized the practical signifcanceof the research problemk. Rationalized the scholarly signifcanceof the research problemPractical signifcance ofresearch not discussedScholarly signifcance ofresearch not discussedPractical signifcance discussedScholarly signifcance discussedCritiqued practical signifcanceof researchCritiqued scholarly signifcanceof research5. Rhetoricl. Was written with a coherent, clearstructure that supported the reviewPoorly conceptualized,haphazardSome coherent structureWell developed, coherent 117Table 3.2 Literature Review Scoring Rubric118 ReseaRch and evaluation in education and PsychologyAu (2007) excluded studies that examined therelationship between high-stakes testing and retention, studies that focus on therole of high-stakes testing and access to teacher education programs (e.g., PraxisII), studies that focus on the tests themselves (e.g., discourse analyses of the actualtest content), and policy studies that use qualitative methods to compare pressuresbetween states. In addition, because of their ambiguous and complicated positions inschool hierarchies, studies that focus on student teachers are also excluded. (p. 259)• Synthesis is the second category, and it refers to how well the author summarized,analyzed, and synthesized the selected literature on a topic. The criteria include how wellthe author(a) distinguished what has been done in the feld from what needs to be done,(b) placed the topic or problem in the broader scholarly literature,(c) placed the research in the historical context of the feld,(d) acquired and enhanced the subject vocabulary,(e) articulated important variables and phenomena relevant to the topic, and(f) synthesized and gained a new perspective on the literature. (Boote & Beile,2005, p. 7)To satisfy these criteria, the writer needs to identify tensions and inconsistencies inthe literature, provide clarity discussing the strengths and weaknesses of the individualstudies as factors that inﬂuence the interpretation of their results, and use the extantknowledge base to suggest directions and topics for additional empirical investigations.• Methodology as a criterion for judging a literature review refers to the author’saccurate inclusion of details about method that have relevance for identifcation ofmethodologies and research techniques, and their strengths and weaknesses, anddiscussion of the relationship between theories and ideas in the feld to the researchmethodologies (Boote & Beile, 2005).Literature reviews should not be simple summaries of fndings of previous research;they should be critical analyses of previous research. In order to critically analyze thestrengths and weaknesses of prior research, several skills are necessary. One is the abilityto accurately identify the methodologies; a second is the ability to identify strengths andweaknesses in the methodologies and how they impact the interpretation of results.Unless you have had prior experience critiquing research, you are probably wonderinghow you can do this type of critical analysis. You can continue through this book, and bythe end you’ll be able to critically analyze the major approaches to research in educationand psychology. At the present moment, we will focus on your being able to criticallyanalyze the literature review section of a research study, and then you can add to yourskill set as you progress through subsequent chapters. What you can do for the momentis know that when you take notes about your studies, you want to include informationabout the methodology, not just the results.• Signifcance is the fourth category and includes establishing both the practicaland the scholarly signifcance of the research problem (Boote & Beile, 2005). While someresearch studies will focus more or less on one of these aspects, it is useful to provideimplications for both the practical and scholarly signifcance of research.• Rhetoric is the fnal category and it refers to the writers’ ability to organize andwrite cogently about the literature in such a way that they can articulate and support theirclaims about the knowledge in the feld (Boote & Beile, 2005).literature Review and Focusing the Research 119Extending Your Thinking:Choices in Literature ReviewsHow much and what kind of information should you include about each study inyour literature review? About your literature review search method?What are your rules for inclusion and exclusion of studies in your literature review?Several caveats are in order at this point:• The points made in this chapter are relevant to almost all types of educationaland psychological research, whether it is a single empirical study, cyclical study, master’sthesis, or doctoral dissertation.• The act of reviewing the literature does not just occur in the beginning of aresearch study and is not completed once the introduction to the article or the proposalfor the research is completed. Literature review should be an ongoing process, and theresults of that review should be integrated into the body of the report at appropriatepoints, but especially in the discussion and conclusions sections.• The researcher should be aware of potential biases in literature reviews. Thereis a greater tendency for research with statistically signifcant results (i.e., those showinggroup differences larger than chance) to be published. Research studies that show nodifferences either are not submitted by the authors or are rejected more frequently byjournal editors (Begg, 1994; P. B. Campbell, 1989). Campbell suggested that this publicationbias leads to an exaggerated concept of differences between males and females. Beggrecommended tracking down (or determining if authors of literature reviews trackeddown) unpublished studies on the topic to correct for this bias. However, Begg alsocautioned that the quality of the unpublished data may be suspect because they havenot been through a review process. For this reason, he recommended a conservativeinterpretation of literature review results (especially meta-analyses).• Matt and Cook (1994) focus on threats to inference from research synthesesbased on the quality (or lack thereof) of the primary research studies included in thereview. They point out weaknesses commonly found in quantitative research studiesthat could be used in a statistical synthesis of previous research fndings. In assessingthe conclusions reached in any literature review, the reader should be cognizant of thequality of the studies included.Questions for Critically Analyzing Literature ReviewsThe following questions can be used to determine if a literature review is satisfactory. In preparingyour answers to these questions, cite evidence in the article to support your answers.1. The purpose of the literature review is to place the current research into the “bigpicture” of what is known and not known about a specifc topic. What is the big pictureinto which this study fts? What is the central topic? How is the researcher conceptualizingthe problem?120 ReseaRch and evaluation in education and Psychology2. What is the nature of the literature cited?a. Is the review current, using research that is recent enough to have applicabilityto the proposed research?b. Is the review based predominately on primary research rather than onsecondary or opinion pieces?c. Does the review provide a critical analysis of existing literature, recognizingthe strengths and weaknesses of previous research? Or, is the review just asummary of prior research?d. Is the literature review well balanced, presenting evidence on both (or all)sides of the issue?3. Is the review free from the biases of the reviewer? Is there any evidence in termsof emotional language, institutional affliation, funding source, and so on to suggest thatthe reviewer might be biased?4. To what extent does the review establish a need for the study? What is theauthor’s rationale for why this study is needed? What do we know? What do we need toknow? Why is this study important (practically and in terms of scholarship)?5. What is the theoretical framework and what are the research questions? Does thereview provide enough information to support the researcher’s theoretical framework andresearch questions posed?6. Does the review provide suffcient information to guide the research procedures,including the identifcation of subject participants, selection of data collection and analysisprocesses, and use of appropriate reporting strategies? After you read the review andyou see what research questions and methods are used, do you think they are logicallyconnected? Does what the researchers do in terms of method make sense in terms of whatis presented in the literature review?7. Are sources cited inclusive of “marginalized” voices? Are citations made thatreference viewpoints of those with the least power?To really have a basis for critically analyzing research, it is helpful to have broadexperience with different types of research as well as with a number of studies thatrepresent the same research approach. Of course, such breadth and depth takes time toachieve. Nevertheless, a long journey begins with a single step. Throughout this text, youwill be encouraged to identify full text research articles that relate to your area of interestand to critically analyze those studies. The ability to critically analyze research is also askill that becomes more holistic with experience.When you are in the beginning stages of learning critical analysis, it is helpful to lookat each section of the research study. So, in this chapter, we focus on the introductorysection that includes the literature review and research problem, hypothesis, questions, orobjectives. Later, you will be able to look at other aspects of the article, such as how theauthor handled certain aspects of data collection, analysis, credibility building, or ethics.You can then do comparisons across studies on these dimensions, analyzing how andwhy texts differ, how they relate to theoretical readings, whether the authors are justifedin their methods or presentations, and how they can help you in your own decisionsabout research. With each research article that you review, you will increase your abilityto determine the quality of the author’s work and the validity of the fndings.4literature Review and Focusing the Research 121Summary of Chapter 3: Literature Reviewand Focusing the ResearchA review of scholarly literature provides information that can be used to investigatea topic of importance to learn what is known about that topic for its own sake (i.e.,to improve teaching or therapeutic practices) or as a basis for designing a researchstudy. The formulation of a research topic is enabled by reading about research that hasalready been conducted because the reader can fgure out what is already known aswell as become acquainted with the strengths and weaknesses of methods used in priorresearch. Multiple sources exist for the conduct of literature reviews, including secondarysources that provide an overview of past research and primary sources that report originalresearch. Primary sources can be identifed through several different electronic means thatare described in this chapter. Persons conducting literature reviews can summarize theirresults in narrative form or a quantitative form known as meta-analysis that is describedin more detail in Chapter 13. A literature review is used to develop research questionsof different types, such as descriptive, correlational, or interventionist. Researchers canalso beneft by looking outside of published scholarly research to community members toprovide a different perspective on what needs to be studied and how it should be studied.You are now ready to consider which specifc research approach is appropriate to answerthe research questions.Notes1. I want to acknowledge the contribution of Gallaudet’s research librarian, Jane Rutherford,for her many years of support for the students in my courses and for keeping me up-to-date onresources available from the library.2. Search engine is the term used in the technology literature for search sites.3. This is twice as many as were included in the frst edition of this book published in 1998.4. I am indebted to the comments of an anonymous reviewer for this framing of criticalanalysis.Extending Your Thinking:Critically Analyzing Literature Reviews• Locate several empirical research studies. Identify the following features of thestudies: (a) the paradigm that the researchers used, (b) the research problem, (c) thetheoretical framework that underlies the study, and (d) the research questions orhypothesis.• Using the questions at the end of Chapter 3 for critically analyzing literaturereviews, critique literature reviews in several different literature studies, identifyingtheir strengths and weaknesses and supporting your claims with evidence from thearticles.Quantitative research is rooted in the postpositivist paradigm,which holds that the purpose of research is to develop confdence thata particular knowledge claim about an educational or psychologicalphenomenon is true or false by collecting evidence in the form ofobjective observations of relevant phenomena (Gall, Gall, & Borg, 2007).Research design can be defned as a process of creating an empiricaltest to support or refute a knowledge claim. Two tests of knowledgeclaims exist in the postpositivist paradigm: (a) Is the knowledgeclaim true in this situation (does it have internal validity)? (b) Is theknowledge claim true in other situations (does it have external validityor generalizability)?Knowledge claims concerning internal validity require some complexthinking. Suppose you have a frst grader who generally refuses to sit inhis seat and pushes so hard on the paper when he tries to write that thepaper tears. When he does sit in his seat, he bangs his head on the desk. Itdoes not matter whether you are the parent, teacher, counselor, or schooladministrator, you want to be able to identify the variables that cause thebehavior and fgure out a treatment that allows this child to learn andthrive. You might formulate a wide variety of hypotheses as to why thesebehaviors are occurring. You might speak with other staﬀ members andfnd out that they have observed similar behaviors with other children. Youmight consult the literature and fnd that such behaviors could result froma lack of self-confdence in a school setting or from frustration associatedwith a learning disability or a developmental delay. The recommendedcourses of action could be to try to build the child’s self-confdence, tochange teaching strategies to address the learning disability, or to lessenthe demands on the child until maturation occurs.If you are operating in the postpositivist paradigm, you mightdesign a research study in which you decide to administer a selectedtreatment (e.g., a program designed to build self-confdence) to onegroup of children, and another similar group of children would not getthe treatment. Suppose that the group of children who received thetreatment improved their behavior more than the other group. Howcan you claim that it was your “treatment” that caused the observedchange in behavior? For the researcher to make a knowledge claimthat this treatment caused this eﬀect, certain tests of internal validitymust be met. These tests of internal validity are the subject of a majorportion of this chapter.Most quantitative research is of two types: studies aimed atdiscovering causal (or correlational) relationships and descriptive studiesthat use quantitative data to describe a phenomenon. Six approachesto undertaking quantitative research are explained in this text: singlegroup, experimental, and quasi-experimental designs (Chapter 4); causalcomparative and correlational research (Chapter 5); survey methods(Chapter 6); and single-case research (Chapter 7).In This Chapter♦ The importance ofexperimental design in thepostpositivist paradigm isdiscussed.♦ Independent and dependentvariables, experimental andcontrol groups, randomassignment, and internal andexternal validity are defnedand illustrated.♦ Threats to internal andexternal validity, along withways to minimize thesethreats, are explained.♦ Research designsare diagrammed andexplained for single-group,experimental, and quasiexperimental studies.♦ Other design issues arediscussed, such as the type oftreatment variable, orderingeﬀects, and matchingsubjects.♦ Complex issues concerningexperimental research in thetransformative paradigm arediscussed.
- Assignment status: Already Solved By Our Experts
- (USA, AUS, UK & CA PhD. Writers)
- CLICK HERE TO GET A PROFESSIONAL WRITER TO WORK ON THIS PAPER AND OTHER SIMILAR PAPERS, GET A NON PLAGIARIZED PAPER FROM OUR EXPERTS