Engaging Communities as Experts throughout the Assessment Process

Best practices in assessment call for thoughtful, authentic collaboration with the community at every step of the process, from project design to dissemination of findings. However, it can be all too easy to send out a survey or host a focus group and call it community engagement. That approach may elicit feedback from community members, but it does not truly engage them in a process in a meaningful way. While often time-consuming, a more thorough and authentic process leads to richer data, increased capacity of all partners, and stronger relationships with communities.

This post answers the questions:

  1. How can the community be engaged at each of the three phases of the assessment process (design and planning; data collection; and analysis and interpretation)?
  2. Why is community engagement important at each of these phases?

While this post provides an overview of community engagement in the assessment process, this approach should be seen as a loose framework rather than step-by-step instructions. Each community is unique, and engagement efforts should acknowledge this and be tailored to best fit the needs and preferences of a community.

Assessment Design and Planning

Community engagement is not a step in conducting an assessment; it is a critical element to the entire assessment process, beginning with the planning of the assessment. Community expertise is required to appropriately design an assessment, and there are no greater community experts than its members. One way to engage community in the beginning of an assessment is by establishing and/or collaborating with an Advisory Group.

An Advisory Group is made up of community leaders that are members of and/or work with the communities of interest, as identified by the assessment’s sponsoring organization, and its partners. Members could include residents who are formal and informal leaders in their community (e.g., head of a tenants’ organization, informal leader in the church, mosque, etc.) as well as providers and staff from community organizations that work closely with community and/or are community members themselves (e.g., social service providers, community health workers). Advisory Group members are often experts on a community (for example, cultures, language, norms, practices) and do not necessarily need to be experts in an assessment’s topic of focus (for example, a specific disease or condition). The appropriate types of Advisory Group members vary by community and the focus of the assessment.

Engaging community Advisory Groups during the early stages of an assessment supports four fundamental factors of success:

  1. Define the community,
  2. Create a relevant assessment approach,
  3. Ensure cultural relevance of tools, and
  4. Build trust and shared understanding.

Define the community. At the beginning of the assessment, members of the community Advisory Group can be engaged to define the community. The definition of community will inform outreach to community members, identification of secondary data, and framing for qualitative data collection. Self-defining by a community helps us better understand who and what makes up a community rather than simply looking at how a community is defined by external sources. In an assessment examining cancer attitudes and practices among foreign-born Asians, the HRiA team identified Burmese as one of the targeted ethnic groups for data collection. When the targeted communities were reviewed by the project’s Advisory Group, a member who worked closely with southeast Asian populations noted that this would be too broad a definition and the assessment should instead target Karen and Karenni ethnic groups. This change lead to data collection being done in the appropriate languages and a better targeted assessment.

Create a relevant assessment approach. In addition to defining the community, an Advisory Group can be instrumental in creating the overarching approach to an assessment and coming up with guiding questions that are relevant and impactful for their communities. While researchers may come in with an idea of the question(s) they’d like to answer through an assessment, those may not be the questions that are most important to a community itself. Asking an Advisory Group what questions they want answered through an assessment process can be an important step towards shifting power to the community, as well as increasing the likelihood that the findings from the assessment will be actionable for the community.

Ensure cultural relevance of tools. Engaging a community Advisory Group as part of an assessment is also valuable for the process of designing surveys or qualitative data collection instruments that best answer the guiding research questions the Advisory Group has helped to create. Advisory Group members should be engaged through the design and review of data collection tools by informing the questions being asked, identifying areas of potential cross-cultural confusion, and addressing any linguistic or cultural translation nuances. For example, concepts and words that are considered commonplace in the dominant culture or language may not exist in other cultures or languages. Direct translation of words into another language may lead to confusion. An example of this comes from HRiA’s work looking at attitudes and practices about cervical cancer screenings among foreign-born Muslim women. In an initial draft of educational materials developed as part of the assessment, a direct translation into Arabic erroneously translated eggs (reproductive) as eggs (food). A member of the Advisory Group, engaged in reviewing assessment deliverables, identified the correct translation, avoiding potential problems in later assessment steps and beyond.

Advisory Group input can also assist researchers in phrasing questions that are respectful of cultural beliefs about health, community, or modesty. A community-leader Advisory Group will raise these considerations and more during the design process and will help to solve problems from the beginning so that the assessment doesn’t find itself lost in translation in later stages.

Build trust and shared understanding. By engaging members of the community in the planning process, an assessment also builds early community buy-in. Having existing relationships and community support for the assessment helps facilitate later activities like data collection and analysis. Involving community input from the beginning also helps to ensure that the assessment is asking the right questions of the right people in the right way. Communities are experts on themselves – let their expertise help create a better designed assessment that will lead to richer findings.

Further, Advisory Groups can help provide a deeper understanding of not only the challenges that a community faces, but also its assets and strengths that can be explored during the data collection phase and later leveraged for intervention.  In previous assessment work, Advisory Group members have encouraged HRiA to explore formal and informal methods of communication that community members already use for sharing information about health and medical care. These networks have later been capitalized on to pilot test tailored education materials developed as part of the assessment.

Data Collection

HRiA’s preferred method of engagement in support of data collection is partnering with community-based organizations. Partnering with community-based, religious, or other organizations that work with members of the assessment’s target communities is a powerful, authentic method of engagement that helps ensure that data is accurate and complete and the process of collection effective, efficient, and thoughtful. The four strategies for engaging communities in data collectors are: 1) Engage community members as data collectors, 2) Engage in community-centered qualitative data collection, 3) Meet them where they are, 4) Conduct key informant interviews with community leaders.

Engage community members as data collectors. Professional researchers will encounter numerous challenges when conducting data collection for a community health needs assessments. The invisible barrier between the researchers and community members, regardless of who they are, can hinder the process. A more effective way to collect data is to engage community members as data collectors. This approach also builds capacity among community members themselves. With community members conducting data collection activities, individuals may feel they can speak more freely than they might in a group facilitated by a perceived outsider. In a large collaborative assessment involving hospitals, non-profit organizations, community development corporations, and the health department, engaging community members in the data collection process was foundational. In this effort, HRiA provided multiple trainings to residents and staff from community-based organizations for them to build their capacity and feel comfortable as survey administrators and interview facilitators for the assessment.     Having them in this role was critical as they were trusted community sources and knew residents’ preferred languages. This role is also especially important for assessments covering more sensitive, personal topics, such as those around physical health or mental health. It’s no longer a research group—it’s a community conversation in which researchers can participate.

Engage in community-centered data collection. Researchers can partner with community-based organizations, religious organizations, and other programs or organizations that work with members of the assessment’s target communities, to organize and facilitate data collection (e.g. through community member focus groups or survey distribution).  In addition to continuously building buy-in and activating the community’s social capital, working with community-based organizations helps ensure that community voice is heard in the data. Centering community voice in the data collection process enables researchers to hear about lived experiences firsthand and strengthens assessment findings. Hearing directly from communities about their perspectives leads to richer and more nuanced data. Additionally, for many communities, secondary data is limited (if it exists at all) and rarely tells the whole story. Therefore, community-centered data collection may fill in gaps in existing data and/or provide context for the “why” behind the quantitative data (“the numbers”) that do exist. Engaging communities in data collection in a meaningful way can answer the question of “why” needs, barriers, or disparities persist; knowing “the why” will help to create a more appropriate outreach or intervention plan going forward.

Meet them where they are. Expecting community members to travel to the researchers (for example, hosting focus groups only held at and staffed by the lead research organization) is likely to result in poor community representation and incomplete, biased data. It is important to go where residents already are, to places that community members trust to have their best interest in mind. One way to do this is to partner with community organizations who are, as one Advisory Group member for an HRiA assessment called them, “the gatekeepers to the community.” Obtaining the buy-in of these organizations makes community members feel more comfortable participating in data collection, whether it’s participating in a focus group or filling out a survey. These organizations can also help the assessment engage harder to reach communities, such as linguistically isolated communities or recent immigrants.

Conduct key informant interviews with community leaders. In addition to engaging community members as data collectors, researchers should also consider interviewing community leaders as well. Community leaders can often speak from two perspectives, their own and the communities, giving research both a micro- and macro-view. Leaders can also often speak to both the community and the system and see both sides of an interaction.

Data Analysis, Interpretation, and Results

Effective community engagement continues beyond the completion of data collection. This can be done through sharing early results with community stakeholders to validate findings, obtain candid feedback, and adjust the interpretation of data. Based on our previous framework, two groups that should be engaged in this process are 1) the Advisory Group and 2) any partnerships with community-based organizations. Additionally, we recommend that researchers explicitly and vulnerably ask these two groups where the assessment fell short.

Share data with the Advisory Group. The presentation of data back to the assessment’s Advisory Group provides guidance on interpretation, dissemination of findings, and planning for next steps.  Researchers may bring their own experiences or biases as lenses to the interpretation of the results, which may be different than those of the community. Engagement at this phase ensures a more accurate interpretation of findings, from ensuring that any quotes presented are framed accurately to advising on whether key takeaways resonate with the community’s experience.

Share data with partnering community-based organizations. Community engagement at this phase can also occur by going back to community-based organizations or groups that participated in data collection to share the findings and ask for input on results and next steps. This closes the loop on community engagement and helps to build lasting relationships with communities.

Ask (and be willing to hear) what the assessment got wrong. Communities are experts on their lives and experiences and their expertise should be valued and trusted. This includes times when the assessment gets something wrong in its approach or analysis. Researchers should consider explicitly asking “What did we get wrong?” when sharing findings to make the audience more comfortable offering this kind of feedback. That feedback should then be considered when planning next steps and incorporated into any final deliverables.

Key Concepts for Community Engagement

In addition to the framework described above, there are several key concepts and strategies that are critical to success throughout and beyond the assessment process. Ongoing community engagement should be done with the following in mind:

Research with, not on, communities. The community is a partner in the assessment and should be treated accordingly. Ongoing communication about process, requests for input, and dissemination of findings should be key components of community engagement in any assessment. If possible, offer partnering community-based organizations and community leaders shared ownership over data to use for their own purposes.

Reduce the burden of participation. Engaging in an assessment project as a community partner can be burdensome, and activities should be designed to reduce burden as much as possible. This includes, but is not limited to, the following:

  • Conduct activities in the native language(s). If this is not possible, provide real-time translation.
  • Provide culturally appropriate food during activities
  • Offer access to supporting resources such as childcare services and transportation
  • Hold activities on days and times that are most convenient to the community; offer varying times to ensure maximum participation

Additionally, whenever possible, offer communities financial compensation for their time spent participating in advisory Groups, focus groups, or reviewing and revising deliverables. They are taking time out of their lives for the assessment, providing valuable insight and expertise. Whether or not financial compensation is possible, offer to share data back with communities to use for their own assessment and planning purposes.

Build capacity through the assessment process. Community engagement should be approached as a learning opportunity for the researchers and organizations leading the assessment. Thoughtful engagement throughout the process is an opportunity for researchers to learn about and partner with communities that they are not a part of. Community members can also benefit from the process, as it can be done in a way to intentionally build the capacity of a community to assess and advocate for itself.

True community engagement takes time. Authentic community engagement, as outlined above, is a lengthy yet worthy cause. When assessments are designed, implemented, and analyzed with the community as a partner, the data and findings are often richer and more informative. Additionally, developing trusting, bi-directional relationships between researchers and communities can build a foundation for ongoing collaboration.

Next Steps