When social science researchers engage in fieldwork projects, there is usually a set of prescribed of protocols that guide the parameters of their research. Some of these protocols and principles are grounded in legal issues, such as Institutional Review Board (IRB) guidelines, while others are guided more by informal or formally-codified “best practices” intended to help ensure access as well as convey the positionality of the researcher to the reader.
The integrity of such practices in field research is even more paramount when the population researched is of a different background than that of the researcher in some significant way. Some examples of this might be families from lower-income backgrounds, racial, linguistic, gender, sexual, religious, or ethnic minorities, children, persons with disabilities, and persons with medical conditions or risky behaviors who may not wish to self-identify.
When research is conducted for the purposes of program evaluation or compliance monitoring, however, these practices are often reduced to issues of quality control and access to data. This can result in culturally disconnected- if not insensitive- research. Without the same standards of sensitivity to culture and context as academic research, even the crispest research design can be compromised.
Access
Effective evaluations or monitoring projects will inevitably require interviews. Of foremost importance in the exercise of obtaining quality data from the interviews is gaining access to participants in your evaluation study. When entering an unfamiliar community, it is important to do some historical research first. Some questions to consider at the outset of a monitoring or evaluation project include:
- What is the history of the community into which you plan to enter?
- Have there been significant demographic shifts in the community over time? Is the community undergoing demographic change now?
- How do changes in the population over time affect perceptions of various groups within the community? Who feels welcome? Who feels unwelcome?
- What languages are spoken in the community? Do you have access to a person within the community that can open doors of access for you?
- What are the relevant community organizations working with your population of interest?
- How might broader sociopolitical trends or attitudes toward your population of interest affect their willingness to self-identify and participate in the research? How do you ease those concerns?
- What are the politics of the community, both within and without your population of interest?
Based on questions like these, your research team may identify a list of utmost priorities prior to engaging in any sampling for interviews, such as announcing your presence to residents, declaring your purpose, and identifying key allies within the community who can help you gain access to members of your intended communities. Such allies may include school administrators, teachers, social workers, churches, community organizations, public partners, or residents who trust your organization and can vouch for your integrity and discretion.
Interviews
Once you gain that consent to interview, you have to decide whether your interview goals are best achieved via focus group, individual interviews, or some combination. There are other questions to consider as well: How long do you want your participants to stay? What times of day or locations work best for your participants? Who could benefit from assistance with transportation or some form of reimbursement for their time? Will a translator need to be present? The answers to these questions can often be culturally mediated, and it is important to make sure your limitations do not result in the exclusion of significant segments of your population. Most of these issues apply equally to surveys and interviews.
Again, while some of these notions are congruent with best practices in academic research, it is also important to note that the answers to these questions will vary immensely as a function of the community being interviewed. Some people will be much more forthcoming in a group interview setting because they are more comfortable around peers. Others may not wish to provide critical feedback in front of others and would be more candid in a one-on-one setting in a comfortable location. If you want the highest quality responses in your program evaluation, you want to make the participants as comfortable in providing feedback as possible.
Analysis
Data analysis is another key area in which cultural competence is paramount. Assuming you’ve gained access to a sufficient sample for interviews and surveys, and assuming you’ve been able to conduct most or all of that effectively, implicit bias can still sour all that hard work by clouding what you or your team can observe in the data. Having a team trained in culturally responsive fieldwork can help minimize that risk.
The people who analyze and code your interviews should be familiar with the cultures of those within those same communities that were interviewed. If the interviews were conducted in another language, the analyst should be an accomplished translator. If the interviews contain names of local landmarks, parks, places of interest or prominent local persons, someone familiar with the area and those places and people of interest should be the ones to go back and verify the transcriptions. Competence in community slang or other cultural totems (music, dance, festivals, holidays, traditions) are also important forms of cultural capital that interview analysts should possess.
Interpretation
Measures of triangulation that are traditionally used in field research can be especially important when researching communities whose members have different backgrounds than those of the researchers. Verifying trends in interview results with community groups, key participants, and allies within the community all become important to avoid misquotation, misinterpretation, and over/understatement. Member checking, when possible, can also ensure quality interpretation.
Triangulation is particularly important when both qualitative and quantitative measures were used to ensure that the perspectives expressed via quantitative measures are supported and conveyed via interview or focus group data. This both helps ensure that the interviews are representative of the diverse views of the communities surveyed, and also provides opportunities for additional significant themes and viewpoints not intentionally surveyed to come to light and provide a more holistic interpretation of all data points.
Conclusion
Cultural competence is an often unspoken, underrated, but essential component of culturally responsive field research. While program evaluation and monitoring can often be seen from the outside as dry, disconnected work entrenched in endless analysis of mundane numbers, effective evaluation programs are tremendously proactive, sensitive, and empathetic regarding the concerns and needs of diverse communities. Failure to recognize how differences among communities influence strategies for access, data gathering, analysis and interpretation can lead to incomplete, inaccurate or wholly invalid research. This disservices the communities and service providers who truly wish to do their best by those communities by offering needed and relevant programs. The work of good evaluations must be culturally competent and responsive.
Evan M. Johnston is a doctoral student at NYU Steinhardt's Teaching and Learning Department and a Graduate Research Assistant in the Center for Research and Evaluation at Metro Center. Follow him @evanmjohnston on Twitter.