By Erin Okuno

We get them in our inboxes, surveys. They are the love/hate way of gathering information – easy, affordable, and we can say we’ve heard back from the community. Yet the cavalier attitude of easy and cheap surveys needs to stop. I was trained during my college days to think very carefully about survey design and to never put out a survey without doing a practice survey first. As the saying goes: junk questions, junk data back. Unfortunately, we don’t get too many chances to go out into the community to really collect good data, and you better do it right the first time. Getting it right means you also better include a question about race:

Without a question asking about the race of the survey taker, you can’t disaggregate your data. If you can’t disaggregate your data, you can’t tell who you’re NOT hearing from. If you’re not hearing from people and communities of color, what is

Excuses

A few months ago a state government agency launched a statewide survey to ask people to give input on budgeting priorities. It was heralded as an innovative way to influence the budgeting process which had become rote. The survey was posted online, the organization did their due diligence of having it translated into a few different languages. Since it was a pretty big deal to ask for public budgeting input the survey gained media attention. Out of curiosity I opened the survey link and scrolled through the list of options and dutifully clicked what I thought the priorities should be. Towards the end of the survey I saw they were collecting demographic information. There was an emphasis on making sure they were getting statewide representation since asked about what area of the state people lived, I can’t remember but there might have been a question about rural, suburban, and urban school districts. I think there was a question about how people identified by role—parent, teacher/educator, leadership, etc. Yet there was no question about race.

When I reached out to the organization’s leadership to ask why they didn’t include a question about race I got back a convoluted answer. It said in part: according to national rhetoric there is believed to be lower participation rates if the government is thought to be involved in collecting personal data, thus they wanted to avoid any appearance of collecting personal data. I rolled my eyes when I read that line. Excuses like this are how racism self-corrects to protect itself; underhanded ways of keeping racism alive. This organization has a public commitment to equity and they forgot to actualize it and live it, however I bet this same leader wouldn’t hesitate to use the word equity when talking to the media. This org is aren’t alone in putting out race-blind surveys, just this week a peer organization to them also put out a race-blind survey. Their staff must have traded notes since their excuse was similar.

Why We Ask About Race

It is important to ask about race in surveys because we need to track survey returns to figure out who we are and aren’t hearing from. If we want to make life better for people of color, whether by closing achievement gaps in education or infant mortality gaps, we need to hear from those most impacted by the problems. If I want to close an infant mortality gap I shouldn’t be asking or listening to white people; white people aren’t as negatively impacted by infant mortality as Native Americans/Indigenous and Black/African American people. People of color, especially people farthest from racial justice, must have a say in solution finding and their voices need to rise about the noise and din of a crowded data field. A race-blind/neutral survey that doesn’t allow for the disaggregation of survey results will distort the data in favor of white people.

Disaggregate

A better survey design allows survey collectors to disaggregate the survey returns by race. This disaggregation is important for multiple reasons. First, as you’re collecting surveys you can gauge who you aren’t hearing from. If you’re not hearing from a certain demographic you can double down on outreach and hopefully nimbly adjust to seek more input from whoever is missing. I once ran a family engagement survey and noticed we were missing input from East Africans. There are a lot of East Africans in our survey catchment area but they weren’t being reached by our traditional survey collection methods. Mid-way through our survey collection I reached out to a Somali colleague and hired her to help me with survey collection. Had we not been tracking our returns by race there is a high probability we wouldn’t have had any surveys from East African families in our survey pool.

It is also important to disaggregate the survey results. Race-neutral surveys don’t allow you to pull out the results of people of color. When we disaggregate we can also target resources with more precision and adaptively. Such as the data for one community of color might show different trends and the solutions should be different. Such as an intervention for a white student is probably not the right approach for a first generation Hmong student. A race-blind survey doesn’t allow us to get to this level of specificity.

What to Do

If you must put out a survey, please at a minimum include an optional question about race. If you don’t you’re squandering a chance to do something more meaningful with the results. If you do collect race data, then use it! Do the harder work of disaggregating your data and work with communities of color to make sense of the data. And really you should be doing this before you even write your survey, communities of color should be the ones writing the survey, but we’ve already blogged about that and will probably write about that again some other time.

Thank you to our Patreon subscribers who help to keep the blog going: Aimie, Ali, Annie, Brooke, C+C, Carrie, Chelsea, Clarissa, Edith, Elizabeth, Evan, Heidi, Janis, Jennifer, Jillian, Julie Anne, Kathryn, Kari, Kelli, Kirsten, Kristy, Laurel, Lori, Matthew, McKenzie, Michael, Megan, Miriam, Molly, Sarah, Selina. If I missed anyone my apologies and thank you for your support.

If you subscribe to the blog, thank you. Please check fakequity.com for the most up to date version of the post. We often make grammatical and stylistic corrections after the first publishing which shows up in your inbox. If you would like to subscribe there is a sign-up box on the right sidebar (desktop version). Support the blog by becoming a Patreon supporter.