Earlier this week I opened an email from someone I didn’t know asking me to take a survey about equity in education. I closed the email without taking the survey. I’m tired of taking surveys, especially from people I don’t know. I’m saying no more taking surveys unless I know the person sending it, or if there is a prize or compensation attached – I can be bought, often for the price of a taco. I realize I’m being a survey hypocrite since my organization just ran a large successful survey project. I know the value of a good survey and the data generated can be powerful. But I’m tired of being over-surveyed by askholes (people who ask questions and don’t listen to the answers); so many of the surveys are well intention but poorly designed or executed the data becomes meaningless and waste time, resources, and burns goodwill.
Surveys do not Substitute for Community Engagement
Often times organizations produce a survey “to hear” from the community. Just this week, I’ve gotten four request to complete surveys. All of these well-intention surveys are asking questions about topics important to them, but I don’t know what I’m gaining or what communities of color are gaining by filling out the survey. Is launching a survey a way of ‘checking the box’ and saying the community had a say in the project? Will things change because of the survey results? Is the survey collection method right for the community being asked? Asking without first listening is fakequity; good surveys are driven by the community and the community has ownership over the process and data.
How to do better surveys
A good survey design produces good survey results; vice versa crappy design produces crappy results, and fakequity in the design produces fakequity in results. Designing a survey rooted in the principles of equity will produce more equitable results. This isn’t hard, but it takes intentional thought and work.
One of the first steps is to look at who has a say in the survey design. The project should be centered in communities of color, which means you are doing the upfront work of gaining trust and buy-in, communities of color have a say on what is included and counted, and communities have control of the data. Inclusiveness is a key step in engaging the community.
My organization just completed a big survey project. Our design team was made up of a diverse group of stakeholders. We were thoughtful in putting together the team to include a mix of people, including parents of color, school partners, and community based organizations. We also invited partners to expand the table as we went along. This team guided the survey design process. They had a say and ownership around what data was captured, how questions were written and translated, and how surveys were disseminated and collected.
On the flip fakequity side, I recently reviewed survey data from a process put together by a small insular team. The return rate from people of color was so low it wasn’t a representative sample and the results were invalid for communities of color, essentially wasting the time of the people of color who took the survey.
Survey Delivery Methods – SurveyMonkey FTW (for the win), WRONG
Online surveys have become the default method of collecting surveys. Online methods definitely have their perks: cost effective, easy to put together, eliminates the need for data entry, and they are easy to disseminate. However, we need to ask is the ease of use still the right way to get results? One of the biggest limitations with online surveys is in-person engagement is lost. Online surveys are a one-way communication stream out then in, they don’t allow for dialogue to happen while taking the survey. Many people of color, especially non- or limited-English residents, benefit from having a translator and/or cultural broker who can explain the survey questions to gather the feedback.
My colleague Jondou shared a story about working on a survey translation into Somali. His Somali colleague looked at the survey and said “Somalis don’t talk like this. We are more direct and we ask open ended questions. You took an American survey and translated it into Somali, that doesn’t make it a Somali survey.” He also shared a similar story out of the Chinese community. Jondou was orally translating a survey and the survey taker was getting agitated. After a few questions he paused and said (in Chinese) “I know this isn’t how Chinese people talk, these are American questions,” after acknowledging the
questions were biased and not culturally appropriate they were able to continue. These types of dialogues and exchanges can’t happen over a computer screen; they happen in person and where the experience is validated.
Paper surveys are much more adaptable for focus groups, interviews, and guided conversations around survey collection. Paper also wins when you have little children who want to ‘help’ you complete a survey – the picture is of a survey completed by a preschooler; can’t get that experience over a computer screen. Community engagement starts young.
Invest in People for Better Results
Racial equity work happens best when there is a relational component. When we invest in people we get better results. As you design surveys think about the people part of the project, what is the experience you want to give people? Don’t focus on the data, focus on people of color.
Quick tips:
Design team: Ensure you have diverse voices helping to write the questions.
Translation: Translate your survey to make it language accessible, and ensure the translation is appropriate and high quality (have a second person proof the translation). We also have to acknowledge having it translated is at best a minimum step, it is still an American/dominant culture survey and may need cultural brokers to help explain the nuances of the survey.
Allow for multiple ways to take a survey: Invest in multiple ways to take the survey, online, paper, focus groups, interviews, etc.
I have more to write about around survey designs and how to keep the experience positive for communities and people of color, but I’ll save the rest for future posts. Feel free to post questions or email us if you have thoughts or questions for a future post on surveys, fakequity@gmail.com.
Posted by Erin Okuno
Wednesday was GiveBIG day in Seattle and about 50 other cities across America, also known as Give Local America! GiveBIG is a big day for a lot of nonprofits in the Seattle and Western Washington region. The Seattle Foundation, which runs the day, set a goal of having $20-million raised on the day, much of it driven by an incentive pool of funds – every gift given through the Foundation’s website will be ‘stretched’ by a pool of money the foundation raised. Hooray, for $20-million flowing to nonprofits.
First, we need to pay tribute to Prince who died on Thursday. As Gen Xer/Millennial, Prince defined the music for many of us and our peers. Prince was unapologetic in fighting the call to assimilate and conform to music industry norms. This quote Heidi heard on the radio sums it up well “[Prince] resisted the pressure to do music like everyone else and that was freedom.” This article,
In the spirit of Monday’s tax filing deadline, this week members of the Fakequity team and a few members of our extended Fakequity 
It’s April 1. No joke and no joking about budgeting season budgeting season at my organization. Last year I put together my organization’s budget based on a lot of assumptions, sort of like April Fools but more serious. I was new to the job and our work had shifted so much the previous year’s budget was a decent frame, but didn’t provide the clarity needed to build a beautiful budget. This year’s budgeting process will be much more reflective of our racial equity lens and goals. Where we place and spend our money says a lot about our priorities and commitments. Martin Luther King Jr. said “budgets are moral documents,” and our budgets should reflect our commitments and accountability to our priorities.
Grouping Asian with White people shifts the burden of closing gaps to Asians rather than identifying factors that continue to marginalize Asian communities. When we look at disaggregated data, such as this chart from the Washington State Commission on Asian Pacific American Affairs (CAPAA) we can clearly see not all Asians are academically achieving at high rates. If Asians were grouped with Whites this data would be invisible and we would continue to wonder and at worst blame Asians for not academically achieving. With this data we can begin to look more closely at the systems holding Asians back.
Chinese restaurants or a sweet from the Japanese, or Filipino bakeries. I love seeing the new affordable housing going up a half-mile from my house. I loved hearing from a colleague how a neighboring community is embracing a tent city that is moving in. At their community meeting instead of “we don’t want homeless here,” or “we’re worried about crime, trash, intoxication and drugs” as happened in other neighborhoods, the community’s biggest concern was “How do we get the [tent city] residents hot water? They need hot water!” Taken all together these are the things that define the character of my community. While it is true my neighborhood probably has a higher crime rate, higher poverty, and lower education and health outcomes, do they define us – no. We don’t excuse the crime and we work hard to build stronger communities rooted and glued by culture, diversity, and relationships.
