In a previous Incubator post, our colleagues pointed out that leading effective focus groups for implementation research requires clarity about the critical topics to explore. Our experience conducting the focus groups for our evaluation of PowerTeaching illustrates how we targeted a few key topics while at the same time encouraging conversational flow. We prepared a framework in advance to help us analyze the information we gathered during the sessions.

Identifying the Key Questions

Overall, our implementation research was asking whether the program, a version of “cooperative learning,” had been implemented as designed, and what kinds of relevant activities were occurring in schools that did not receive the program.

In practice, cooperative learning may be simply “group work” that involves seating students in groups and asking them to work together, sometimes assigning a unique role and responsibilities to each student in the group. Yet cooperative learning as defined by the developers of PowerTeaching — the Success for All Foundation — specifically means that groups exhibit team interdependence. The key elements are team recognition, individual accountability, and equal opportunities for success: Teachers give student teams recognition for good collaboration and academic work, and, critically, they use random reporting, calling randomly on team members. This method gives the team incentives to collaborate to make sure that every member is involved and ready to respond.

We knew that PowerTeaching was complex and equal to more than the sum of its parts; surveying teachers about their implementation of the parts would not yield a full picture of how the program had been implemented. We needed to dig deeply into such questions as:

• How were teachers defining cooperative learning?

• How were teachers using cooperative learning, and why?

• Did teachers at the same school use cooperative learning in the same way?

• How did students respond to and engage with cooperative learning?

• How did teachers’ implementation of PowerTeaching differ from typical group work?

We wanted to learn about how cooperative learning actually occurred in classrooms, but the project budget did not allow for direct classroom observation in multiple schools at multiple times. We examined the project’s logic model and the options for data collection and determined that focus groups were essential to inform us about what cooperative learning looked like across the schools in the sample.

Focusing the Focus Group

In developing the focus group protocol, we wanted to zero in on the extent to which teachers using cooperative learning were facilitating team interdependence. But by its nature, cooperative learning is a very complex practice with overlapping parts. It is hard to have a conversation about one of those parts without also talking about another part. So we needed a protocol that allowed the discussion to move between topical buckets, rather than a protocol that sounded like a survey and forced the discussion to move in a line from one topic to the next.

We developed a one-page protocol with several layers (see figure below), beginning with an icebreaker question about classroom layout meant to get at whether teachers grouped their students, followed by questions and probes related to the three big themes — team recognition, individual accountability, and equal opportunities for success. Our overall research question appears near the bottom of the figure, and the protocol ends with a question that asks teachers to share their personal opinions about their experiences implementing the program.

Creating and using this figure helped us in several ways. First, while developing it, we were challenged to articulate the central questions that needed to be addressed during the sessions. This helped us conceptualize how cooperative learning might manifest in different schools. Having this exhibit in our minds during the focus group sessions also changed the way that we conducted the groups: We were better able to navigate the protocol, using comments from participants, than we had been in our prior work when we had lists of questions. And the protocol enabled us to focus on depth, not breadth, in collecting information. Participants did most of the talking and the conversation did not feel contrived, but we could bring the focus back to the key questions when needed, and we had a sense throughout of where information was richest and where it was leanest.

Analyzing Responses to Inform the Central Questions

The protocol fed into our coding, starting with the three “parent codes” for team recognition, individual accountability, and equal opportunities for success. We then created “child codes” based on themes that emerged from a first round of reviewing transcripts. For example, for the equal opportunities for success parent code, we created child codes that captured whether the teacher discussed (a) creating mixed ability groups, (b) setting team goals, and (c) assigning team roles as part of the effort to create equal opportunities for success. We then used a weight system to flag whether the teacher’s description of these themes was positive (weight of 2), neutral (weight of 1), or negative (weight of zero). Our coding team had three coders, each specializing in one parent code and gaining expertise in that part of the implementation story.

When Is a One-Page Protocol for Focus Groups Appropriate?

Using this kind of one-page protocol is unlikely to work well when focus group facilitators are inexperienced or do not have a deep understanding of the implementation context or of the study priorities. And it’s not appropriate if the focus group sessions are a last-resort source of information for questions better answered with surveys or other data. This kind of approach works best when the focus groups are part of a broader information collection strategy for implementation research and when the sessions can inform a particular aspect of the intervention in depth.

Suggested citation for this post:

Rappaport, Shelley, and Kelly Granito. 2018. “Focusing Focus Groups: A 'One-Page Protocol' Approach.” Implementation Research Incubator (blog), November. https://www.mdrc.org/publication/focusing-focus-groups.