Writing a good discussion guide
Writing a good discussion guide for qualitative interviewing is something of an art form. They’re always a contested space - some people want them to be well defined, other people want them loose. Both of these impulses have merits, and both have problems. A good guide will aim to balance these competing needs.
A further complication is that a discussion guide has different uses. Obviously it maps out the space of an interview for those participating, but it often also serves to map out what an interview will cover for clients and other stakeholders. This diversity of audiences tends to exaggerate the definition/flexibility fault line.
I believe that the interview should be the primary focus of a discussion guide. It should be seen as a tool designed for the intense needs of practitioners who need to quickly scan for prompts and act as a checklist that necessary topics have been addressed. And it should do this without forcing a stilted structure on conversation. In other words, it needs to be able to be used in a non-linear way. Respondents can and will steer conversations in often productive directions that could not be anticipated prior to the conversation and a tool that hinders this is unlikely to be able to be successfully used in an interview setting.
However creating some structure with which to frame each interview in a project can be immensely productive once analysis begins. Having a data grid to populate for each case allows for easy comparison and allows data to be filtered to the most relevant points. Useful when analysis time is in short supply.
So what can we do to craft a practical and functional document that meets as many of these competing needs as possible? I’d like to set out several tricks that I’ve learnt or seen other people practice as a way of fostering a discussion. Please jump in in the comments with things that you’ve tried and found to work well.
In the past, I have tried using a small (A5) sheet of paper with simple topics printed on it. This acts as a constant checklist to what needs discussing - it’s easy to scan in the middle of a conversation without breaking the flow of conversation. However making sure that you have enough material for each topic relies on quite agile interviewing and makes it difficult to highlight specific data points that must be collected.
More common is a list of questions in verbose format, for example: “Are there particular times that are more difficult during your journey to work?” The danger of this is that either you follow the list slavishly, losing productive spontaneity, or you lose track of what’s in the guide, since it’s difficult to scan and formulate agile prompts when the guide is dense and pre-formatted in a way that the respondent may not follow.
I want to suggest some ways that we can bring these two impulses (structure/flexibility) together for the best of both worlds.
When it comes to the practicality of conducting qualitative interviews, I can’t recommend Grant McCracken’s The Long Interview highly enough. It’s pragmatic yet thoughtful and best of all short and to the point. A key idea I’ve taken from this book is the idea of floating and prompted questions.
McCracken advocates the use of non-directive, “grand-tour” questions that prompt the respondent to frame the question in their own terms and take you on a “grand tour” of the way they understand the topic. These are the short section titles I used in my A5 topic guide example, framed in verbose question format for the benefit of casual readers. McCracken explains that these questions can be elongated with minimal direction from the interviewer through the use of “floating prompts”, raised eyebrows to indicate interest or confusion, repeating phrases used by the respondent for clarification and such like.
The openness of these questions is important, it’s the reason for the impulse to conduct interviews without questionnaires. How a respondent prioritises their response, whilst not immediately available as gospel truth, is an important guide to what they consider valuable even when it’s what they think the interviewer wants to hear.
This strategy may produce discussion of all the topics required of the interview, in which case the remainder of the topic guide can function as our checklist above - no need to repeat questions that have been answered in the same phrasing as the topic guide. Time is better used for probing and exploring in more depth the most interesting aspects of what a respondent has said and obtaining some early validation of what we understand about what they’ve said.
However the discussion can also veer wildly off course, or run it’s own course without fully exploring what we need to explore. In this situation, McCracken says we need to turn to planned prompts that tease out further aspects of the topic. These could be contrast questions such as “How does x relate to y?”, categorical questions defining key characteristics of topics, example questions in which the respondent is asked to describe an incident related to the topic or stimulus questions in which the respondent is given some activity or material with which to frame their response. All these tactics should be used once “grand tour” testimony has produced as much as it is able to. Remember that how a respondent tells their story is data too. Every prompt tactic we use comes at the expense of understanding more about their story so we should be careful how we use them.
So how to put these ideas into practice?
I have taken to framing each topic as a “grand-tour” question with instructions to the interviewer to use floating prompts. There is a single such question per topic area, followed by several short planned prompts. Breaking up the floating/planned prompt dynamic per topic gives a better opportunity to actually use the topic guide as an active tool in the interview itself.
I tend to assign a rough time guide to each section so that I can keep track of the amount of time I’ve devoted to a particular topic and make sure we can cover everything in the allotted time. Interview length tends to depend on the topic and the amount of time the respondent can devote to it. Paradoxically, topics that you might think would be more difficult for the respondent are often the ones that they can comfortably discuss for long periods. I’ve known difficult health related interviews to run for over 3 hours because the respondent appreciated the chance to reflect on a situation that had a big impact on their lives. Conversely, a dry and technical subject may cover all the required topics in less than an hour. All depends on the amount of understanding required around each topic before you are able to move on.
An interesting idea I’ve been trying out recently came from my colleague Leeor Levy. I noticed her preparing for an interview by highlighting the key idea in each verbose question - effectively filtering the amount of information requiring visual processing during a scan of the topic guide during discussion. This is a fantastic way of keeping the guide active and relevant - the flexibility impulse - and I’ve taken to rewriting verbose questions to make this key idea compact and concise and then pre-highlighting them. For example the question, “Are there particular times that are more difficult during your journey to work?” can just as easily be written “Are there any particularly difficult times during your journey to work?” With this phrasing, it’s easy to highlight “particularly difficult times”. The rest of the verbose question gives helpful context when it’s needed but can easily fall away. These highlights make for a useful, visual checklist that can be used in the interview context - my key criteria.
I hope some of these ideas can help the next time you need to plan a qualitative interview. The advantages of some element of structure, particularly when analysis time is short and topics need to be addressed succinctly, are manifold, however should not come at the expense of creating a useful tool for the messy process of co-creating a purposeful conversation with a person whose own way of seeing and interacting with the world is, in the end, why we have come to speak to them.
What are your tricks and tactics for writing topic guides? Please contribute your experiences and discuss in the comments!