How Do You Know What You Don’t Know?
As researchers, we are used to asking questions, it is the very essence of our job. The problem is: how do we know we are asking the right questions? Because if we aren’t asking the right question, then we won’t get the right answer – or indeed any answer.
Recent experiences in my house buying journey brought home this conundrum (no pun intended!) How do you know what you don’t know? How many questions do you ask to ensure you’ve covered all the bases? And even then, will that get you where you need to be?
I have many examples of my question-asking failures to draw upon. There’s the dryer for which I had to refuse delivery because I didn’t ask the right question about the location and length of my gas connector. This is despite having a printed photo of my laundry area, detailed with all the key dimensions, and consulting with a sales associate when making the in-store purchase. I had all bases covered, right?
Then there’s the color that the waterproofing people painted the replacement steel edge. To be honest, it didn’t even register to me that it would need to be painted, so I didn’t ask the question. The result – I now need to sand and repaint said steel edge.
My favorite example, however, is the most stressful one. In preparation for the closing day, I phoned my bank to confirm their wire transfer requirements and processing time. I was assured it would be a same-day transfer and I simply needed to provide details of the receiving account. I hung up content that I’d done my due diligence.
I didn’t ask the all-important question; ‘can I transfer to a third-party account? I incorrectly assumed that a wire could be sent to any account that I specified. Fast-forward to 5 am on the morning the purchase was due to be completed, I call to execute the wire, and the bombshell hits – I can only transfer to myself. I’m glad to say everything worked out, but it was an exceedingly tense 72 hours.
All these adventures, shall we call them, have given me food for thought in how to be a better researcher. How to get answers to the questions that we don’t know we should be asking. The importance of second-guessing the assumptions that we are making, and taking time to consider those assumptions that we aren’t aware we are making.
This is a core benefit of qualitative research. We can have open-ended conversations with people, be part of their lives for an extended period, and uncover their understanding and meanings. Qualitative research is for when we know that we don’t know. Of course, we still need to take the time to think about the unknown unknowns.
Quantitative research, on the other hand, is when we know but don’t know to what extent. That said, I’ll suggest that we shouldn’t ignore the possibility that there are things we don’t know when asking our survey questions. There is an argument for having a generic ‘is there anything else you’d like to tell us’ open-end question at the end of every survey. Because it might just shed light on some of the things we don’t know.