Data Driven?

I am interviewing a teaching candidate.  The candidate is telling me that he is extremely “data-driven”  and I grit my teeth. This phrase drives me batty and fills my head with images of teachers as number-crunchers more worried about the spreadsheet cell turning green than the student the cell represents.

The candidate continues “when I say data I mean everything, every shred of information I can gather from the students…the assessments, the conversations the students have, her facial expressions during seminar.”  I perk back up because this definition of data is infinitely richer and fosters the kind of intentional student driven decisions that effective teachers and leaders make.

The candidate’s definition of data makes me reflect on a recent blogpost from Elliot Witney (principal of KIPP Academy in Houston and chief academic officer of KIPP Houston):

Imagine being able to transcribe every experience from the moment a child wakes up to the moment she drifts off to sleep.  If you could capture the nuances of a child’s academic experience – the teacher’s eyes that prodded her to stick with a challenging problem, the pats on the back or encouraging thumbs up, the whispers under the breath by friends, fake friends, or supposed enemies – and then cut that script into different layers.

What percentage of the time did that child spend actively engaged?  Reviewing prior material?  Checked out, imagining castles and fairies?

What percentage of the time was that child asked a “what?” question?  A “why?” question?  A “what other options would you have considered if this weren’t your first thought?” question?

What percentage of teacher responses encouraged risk-taking and courageous, bold thinking?  Creative thinking?

Read more at…http://kipphoustonacademics.wordpress.com/2010/04/27/script-of-a-child/

Elliot’s script idea speaks to a struggle for teachers and leaders alike.  We make thousands of small daily decisions that drive or halt learning, build or break relationships and we make those decisions based on our observations of a small slice of the child’s experiences.  Our “data” is incomplete because we can’t get a global feel of the child’s experience.

One relatively simple way we tried to develop this script at GCP was having some college interns follow individual students for a day and take literal notes on the student’s day.  The  results were both amazing and predictable.   Despite tremendous effort on our part to create cultural and academic consistency there were big gaps between a student’s experience form one classroom to the next.  In one class students were asked two questions during the period vs. hearing twenty three declarative sentences.  The next period students were asked twenty-three questions.  One student raised his hand fifty times in a day and was called on twice.  By the end of the day students had spent much longer in transitions than we ever imagined.

What was the value of this exercise?  It didn’t yield any big a-ha about the most effective instruction but it did force us to experience school the way our kids do.  Their day is a quilt  not a fabric square, a novel not an individual story.

What  if every teacher and leader spent a day following a student?  Would this investment of time and energy (sitting quietly and observing is surprisingly exhausting for people used to being active) lead to better decisions and more intentional design of the students’ day?

What other ways could we collect about our schools that would help us see what is happening from the student’s eyes?  A few ideas I would love to try…

Pull a student from each grade level at the end of every day and interview them.  Maybe two simple questions like “What did you learn today?”  “Why did you learn it?”  Record the interviews on a flipcam and post them for the staff to watch.

Teach the students to use excel and have a student job in every class to be tracking types of participation.  Would we start to see which students are remarkably good at raising their hand only to read directions or answer an easy question?  Would our estimate of student:teacher talk match the reality?

If you are a teacher or leader who uses paychecks, digitize them so teachers can spend more time analyzing the potentially rich information they reveal about how and why our kids succeed or struggle behaviorally.  What other “data” gives us insight into the development of student character and work habits?  Rumor has it that KIPP NYC is doing some work in this area.

What “data” have you collected in your classroom or school that changed the way you operated?

A few links and related readings.

This recent NYT article about a certain group of tech savvy, slightly obsessive people track data on their individual behavior didn’t get me to start tracking my own caffeine consumption or sleep habits but it did inspire some of the thinking above.  http://www.nytimes.com/2010/05/02/magazine/02self-measurement-t.html?ref=magazine

Collecting student achievement data is one thing.  Using it to improve performance is another.  If you want to read about how to implement quality interim assessment and error analysis read Paul Bambrick Santoyo’s Driven by Data.  I overcame my bias about the title because the book is such a sharp exploration of how to shift our thinking from “What is the teacher doing?” to “What are the students learning?”  The book also includes a ton  of practical, no need to re-invent the wheel resources.

Advertisements

6 Responses to “Data Driven?”

  1. jsmith6 Says:

    I’ve recently been exposed to using Flander’s Interaction Analysis, which is time consuming for sure, but allows you to break down the entirety of your lesson into broad categories, and then more specific categories within these. I’ve been using it on my own instruction, but if an entire staff was trained, you could also do these for each other.

    You watch or listen to a recording of a lesson, and every three seconds you put a tally in the category that is happening at that time. Every time something new happens you also make a tally. Before using I’d highly recommend practicing on small chunks of other’s lessons and comparing results, to make sure you’re comfortable with what actions fall within which categories. Even more helpful when done as a whole staff.

    The categories are:

    INDIRECT TEACHER INFLUENCE
    *Accepts Feelings
    *Praises or Encourages
    *Accepts or Uses Ideas of a Student
    *Asks Questions (seperated again into Open-ended vs. Restricted Response)

    DIRECT TEACHER INFLUENCE
    *Lectures
    *Gives Directions
    *Criticizes or Justifies Authority (negative evaluation of student contributions; self-reference to teacher’s authority)

    STUDENT TALK
    *Student talk response
    *Student talk initiation

    SILENCE OR CONFUSION
    *Periods of more than 3 seconds
    (I don’t particularly love this category, because often independent work would be categorized as silence, when in fact I would consider it student cognitive engagement).

    I’ve also simplified it before into three broader categories- Indirect Teacher, Direct Teacher and Student Cognitive Engagement, which would encompass group work or independent work where students are engaged in content. It all depends on what you’re hoping to get out of the data.

  2. mrdolan Says:

    Super stuff. I actually think the biggest benefit might not be the data as much as how this structures staff conversation and observation. Have you seen the ECOVE software? It has a bunch of tools for tracking some of what you describe.

  3. Mike G Says:

    Caleb, how worried are u about teachers drowning in data?

    • mrdolan Says:

      Huge worry-have you read How We Decide? One of several ideas that sticks is that our rational brain isn’t very useful after we have 3-5 pieces of information in our prefrontal cortext.

      Check out the chocolate cake experiment http://scienceblogs.com/cortex/2008/03/glucose_selfcontrol_and_the_pr_1.php

      If it’s true that we can only keep a few pieces of data in our head at any one time and still make decent decisions then we need to teach teachers how to focus on the data that matters most and we need to ensure that the data is a mix of the quantitative and qualitative. For instance:

      How to create clearer less sprawling spreadsheets and rubrics

      How to be stronger listeners who can pick up the key messages in a student’s answer.

      How to trim and edit assessments

      There’s a question for leaders that gets glossed over in my response. If you have to administer a variety of state and national tests, plus your interims, plus regular classroom assessments your teachers are already in the middle of the Pacific on an inner tube. At this very moment I’m enamored with Bambrick’s point that teachers shouldn’t spend much time looking at the state/national data and should be guided more by error analysis of individual questions on the interims.

      • Eddie Says:

        Superb inoaimotrfn here, ol’e chap; keep burning the midnight oil.

      • http://goanalyze.info/1000.menu Says:

        (Oui mais je vous avais dit de pas aller faire du bazar dans des commentaires d’il y a 12 ans, résultat, j’ai pas tout lu, alors la proposition BITPU, ben je la découvre l? . Nan mais)(et vous vexez pas mais je préfère birnu)

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: