Navigating a sea of words: An assessment of academic conversation
Wednesday, July 27, 2016
The use of structured academic conversation in the classroom is becoming more sophisticated. Jeff Zwiers and Kenji Hakuta of Stanford University continue to do much work through books and online courses to develop skills in this area. However, many teachers are still unsure about how to assess the quality of the conversations.
How do we know the students are talking about germane content, addressing the questions presented, and using the academic language skills targeted for that lesson? As with any lesson, we need assessment to know whether re-teaching is required, or if we can move on to new skills and content.
Assessment of academic conversation is separate from assessment of the content. Content assessment can be done in a number of ways, including portfolio production, projects, and pencil and paper tests. Each can be scored with check sheets, rubrics or answer keys.
Academic oral language, on the other hand, has to heard. It is transient and elusive — once a thing is said, it’s gone, although it’s effect can be lingering and powerful. We need ways to capture what is said (content) and how it is said (language).
As mentioned, there are other ways to assess the content besides the oral language used to discuss it. But the only way to assess the oral language itself is by being there when it happens. How do we "be there" for numerous simultaneous conversations?
In an ideal conversation activity, the room will be a wall of sound, with many animated voices speaking at once. Depending on the skill and age of the students, that animation will probably last between 1 to 5 minutes before conversation shifts off-topic. A teacher who just sits at the front of the room while this is happening has no chance of making any kind of useful assessment of any of the conversations.
What Do You Want to Know and Why?
It’s probably a combination of two things. If you are using your state’s version of ELD Standards, and you have included one or more of them in the lesson, you’ll want to know if students are meeting them.
If you have specified certain language functions such as persuasion, elaboration, citing evidence or synthesizing in the lesson, you’ll want to know how proficient the students are at those functions. Usually teachers provide language frames, or models of specific wording, that match up with the targeted functions. Students can be heard using the frames, or not.
Teacher Observation and Tracking
Once the targets are set, the skills and content taught, and the conversation activity is launched, the fun begins. You can set sail into the sea of talk to see what new lands you can find. What islands of brilliance; what empty waters of silence; what dangerous shoals of language misuse?
If you are the only teacher in the room, you have two choices: mingle among the whole group, or focus on one conversation at a time. Either way, you will need a recording device. I’ll talk about electronic recording a bit later.
For now, a simple checklist will do. Set up the checklist with short phrases indicating the specific language you want to hear that will show you the skills are being used or standards are being met. Every time you hear one of them used in a conversation, make a tally mark.
You can stay with the same team throughout the lesson, or move to another team at the next chunk (this is recommended). It is very difficult with this method to assess individual student accurately. It is best used as an overall picture of the class.
Observer's Paradox and the Hawthorne Effect
The problem with the above method is that your physical proximity can influence the result. The observer's paradox is that we want to know how the students are talking when we aren’t there but we can only find out how they’re talking by being there. The Hawthorne effect, or observer effect, refers to the tendency of subjects under study to enhance their behavior or performance simply because they’re being studied.
Some students will perk up and make extra use of the frames when the teacher hovers into view. Of course, we want our students to perform beautifully even when we aren’t there, because we won’t be most of the time.
The upside is that the Hawthorne effect shows that we all have a "passing gear” of enhanced performance that we can tap under the right conditions. Can we structure in enough practice that the enhanced level becomes the ordinary, making room for a new, higher enhanced level? That’s the plan behind sports and music practice — do enough of it that the difficult level becomes the new normal.
Recording and Transcripts
If you want minimize the observer effect and get more accurate data on a pair of students, recording them with your phone or other device is the way to go.
After a few minutes, they will most likely forget the device is there. You will hear natural pauses, repetitions, subject-verb mismatches, as well as the use or lack of use of the functions and skills. Make a transcript of the recording and you will have a goldmine of information for adjusting instruction for those students. However, it is time consuming and not practical for the whole classroom use unless you have someone to help with it.
I find this method particularly useful for assessing Long-Term English Learners (LTELs) in order to identify what’s keeping them from going that last mile to fluency.
Student Observation and Tracking
A third way to quantify conversation is to put your labor force to work. Upper elementary and secondary students can learn to do this, and may even like it.
The pros of this method are that you will get a lot of information in a short time, and the student who is doing the listening and observing will learn much about the language skills. The cons are that you are using a semi-skilled labor force. The students are just learning the skills themselves, so they will not be able to be as accurate in their assessments as you would. As with the first method, this is not useful or fair for individual grading, but it can give you a mountain of information on trends of improvement over time for the whole class.
In academic conversation, students are ideally assigned in pairs. They discuss the question or topic using the assigned language function and frames.
Figure 1. Three-way student assessment plan
In student observation assessment, a third student is added to the pair. At each chunk or talking break, two students discuss, and one student assesses the language of only one student (Figure 1). It’s hard enough to track just one student, never mind two. Remember, these are all beginners at this, even if they have more developed language.
At the next chunk, rotate. A different pair discusses, and one of the previous talkers is now assessing. The first assessor is now a talker.
Figure 2. Four-way student assessment plan
If there are four students in the group, the principle is the same, but the picture is different (Figure 2). In the first chunk, student 1 talks with student 2. Student 3 assesses 1, and 4 assesses 2. In the next chunk, 3 talks with 4. 1 assesses 3, 2 assesses 4.
Students use a shortened checklist called a Conversation Counter. The teacher fills in one to four language functions (talk moves), depending on the age and ability. I recommend starting with one. Only use this strategy after students are familiar with the conversation skills. It adds a layer of challenge at first, but after several times, it becomes routine.
Quantitative and qualitative assessment of conversation is possible with the right planning and tools. You can do assessment for learning by performing it yourself or with the help of a colleague. You can use assessment as learning by teaching students how to identify and track components of academic language in their own conversations.
- Breaking down barriers to make career and technical pathways accessible for everyone
- 8 exercises for strengthening your business writing
- The importance of guided practice in the classroom
- Millions of high school students set for success: Celebrating Career and Technical Education Month
- How often and why college students are dropping out
- You can’t be what you can’t see
- To fight crime, engage kids in quality after-school programs
- Grouping students: Heterogeneous, homogeneous and random structures
- Tips for hiring, onboarding and training employees remotely
- Tap into board talent with a survey
- Getting grounded: Implications for business
- 5 ways to show your employees you care
- COVID-19 and the power of the collective
See your work in future editions
Your content, Your Expertise,
Your Industry Needs YOUR Expert Voice & We've got the platform you needFind Out How