Sarah Seleznyov

Unlike in medicine, education and research have always had an uneasy relationship. In 2013, Goldacre described how education lags behind other professions in its systematic use of evidence.  You wouldn’t want to be treated by a doctor who wasn’t using the latest medical research, so why would you let your children be taught by a teacher who wasn’t using the latest education research? He also called for more scientific studies of education interventions and the wider argument this article engendered led eventually to the establishment of the Education Endowment Foundation’s Teacher Toolkit, which aims to make accessible a range of evidence in easily digestible formats to teachers, and is now used by the majority of schools in the UK. As well as making existing evidence available and accessible, the EEF have begun to accumulate a body of new knowledge about ‘what works’, by funding a series of Randomised Control Trials (RCTs) which seek to measure the impact of interventions and to share these findings with teachers, in a format that is easy to understand and use.

I’m all for research evidence getting into the hands of teachers, and all of this seemed like a good idea. Teachers can’t access published research journals without paying, and it’s very time-consuming to have to read across several research studies in order to make sense of them as a body of evidence.  Making research summaries publicly available and writing them in plain English is one of the most powerful and useful pieces of work the EEF have undertaken. If this was the main output of the EEF, I would have no concerns about its impact on the teaching profession.

However, I do worry about the EEF’s obsession with RCTs and with the unproblematised representation of the findings of these RCTs on the EEF website. The RCTs the EEF funds are often short-term, for example one or two years in length, and we know that some educational interventions take time to have a significant impact on pupil learning, especially when they are professional development programme that disrupt teachers’ habitual classroom habits: we know that a dip in performance often precedes a subsequent lift when a change is implemented. We also know that early changes to pupil learning behaviours are often predictors of later academic gains, and often the only impact we see of a teaching and learning intervention delivered over a short time frame.

The RCTs focus almost exclusively on impact in terms of achievement in (usually core) subject areas.  This is a very limited way of measuring learning: most of us working in schools would say that learning is so much greater than test results, and so much more important. All the EEF’s research projects have a process evaluation, and for me, these are often way more telling than the headline effect size. These tell us what teachers had noticed about learning, and the impact they felt the intervention would have on their practice in the long term. Why is this data less valuable than pupil test data?  Like Biesta (2010), I question whether the EEF’s ‘what works’ model of evidence-engagement serves to undermine the value of teachers’ professional judgment or what might be called ‘practice expertise’, since it is based on narrow conceptions of teaching and school improvement. If I am teaching in a small rural PRU, with a particular cohort of pupils, the intervention that the EEF says has a zero or a negative impact might be the best intervention for me to put in place with my pupils.

So as school leaders, what should we do when reading EEF research evidence?

  1. Give equal weight to both effect sizes and process evaluation outcomes when reading the EEF website
  2. Rely more on the literature reviews that EEF publishes than the single studies
  3. Educate your staff so that they understand how to read the evidence, and how to be critical about its relevance to their practice.

And finally, get teachers to use evidence to carry out their own classroom research. Godfrey (2017) distinguishes between three approaches to engagement with research evidence for teachers:

  • ‘evidence-based practice’, a passive process in which teaching approaches are based on evidence about ‘what works’ that has been produced by academics;
  • ‘evidence-informed practice’, whereby teachers actively combine evidence from academic research, practitioner enquiry, such as lesson study or action research, and other school-level data;
  • and, ‘research-informed practice’, whereby teachers engage in and with academic and practitioner forms of research, using evidence from both to make changes to their practices.

It is ‘research-informed practice’ that really makes the difference to teaching and learning. Schools who engage in research-informed practice have been shown to benefit in several different ways: increased professionalism (Furlong, 2014), improved attitudes to learning and renewed practice (Cain, 2015; Greany and Maxwell, 2017), improved pupil outcomes (Cordingley, 2015), and school and system performance (Mincu 2013; Supowitz, 2015).

Let’s not allow ourselves to be led by the evidence, let’s shape it ourselves and make it our own.

Read more

Schools have rightly moved away from lesson observation gradings, a damaging system created by Ofsted and whose legacy persists.  As Chris Watkins rightly notes, we want lesson observations to be about learning and improving, not proving and performance:

The legacy of lesson gradings is the tendency of teachers to try to ‘perform’ when being observed. And when teachers are ‘performing’ for the observer, one of two things will happen: either the observer will witness an all-singing, all-dancing lesson that is in no way reflective of that teacher’s day to day practice, meaning that any feedback the observer gives is not useful in terms of improving regular class teaching. Or the teacher’s attempt to teach an all-singing, all-dancing lesson means they lose focus on the learning they are trying to achieve and again, the lesson is therefore a disaster and the feedback the observer can provide is not relevant to the teacher’s daily practice.

In fact, even if the observer does manage to see a lesson that is broadly reflective of the teacher’s actual practice, there is no real evidence that the feedback the observer gives makes a difference to classroom practice. A Sutton Trust exploration of what supports teacher development stated that the evidence of impact of lesson observation on student outcomes is ‘generally limited’.

But even if it doesn’t work, there is surely an obligation on senior leaders to observe teachers in their schools in order to have a handle on teacher quality? Here again however, the evidence is dubious.

‘Hill, Charalambous and Kraft (2012) estimated that using observations of practice to produce ratings of teacher quality with a reliability of 0.9 would require seeing a teacher teaching five different classes and having each lesson observed by six independent observers.’ RSA, Licensed to Create

So what should we look at? The same Sutton Trust report evaluates some familiar sets of data.  Progress measures are noted to be notoriously unreliable since even tiny changes in measures can dramatically change judgments, the effects of the ‘best’ teachers are long and not short-term (i.e next year’s teacher will benefit), there is a strong chance of measurement error, and a ‘cohort effect’.  Reviewing work outcomes or planning is reliable only when raters follow a specific protocol for evaluating them.  There is one measure that does provide a reliable evaluation: ‘Student ratings are valid, reliable, cost-effective, related to future achievement, valuable for teacher formative feedback and require minimal training.’ The report states that the only disadvantages is the need to consider how best to manage the feedback process in relation to the pupils’ ages.

Given that none of these ways of judging teaching can claim to be fully accurate, how should a school leader proceed? The Sutton Trust advises triangulation: look across more than one set of data before coming to conclusions about a teacher’s performance.

And yet it is this advice that has given us punitive performance management systems which have significantly contributed to teacher workload and stress. Many schools conduct frequent book looks, half termly assessment weeks in which children are tested, termly lesson observations and more regular learning walks or drop ins. Are these systems really built around the needs of the majority of teachers in our schools? Or are they built around the need to ‘catch out’ the weaker members of staff, whilst maintaining a veneer of all teachers being treated equally?

We would never build our class teaching around the needs of the lowest achievers; instead, we take extra care to monitor their progress and put in place an additional intervention to support their learning.

What would a performance management system look like, if it was built around the learning needs of the majority of teaching staff, rather than the minority? Might it encourage self-assessment of strengths and needs, and self-identification of suitable professional development. Surely it would involve evidence, but this might be evidence that is used to corroborate a self-assessment, rather than to judge. A group of experienced headteachers I met in March explained how the removal of regular lesson observations had had a surprising impact: teachers started asking senior leaders to observe them as they focused on developing a particular aspect of their practice, and often repeatedly.

It is possible to develop performance management systems which enable teachers to focus on learning and improving, instead of proving and performance. And such systems are predicated on a belief not only that the majority of teachers do know what they need to do to improve learning in their own classrooms, but that they are also motivated to make this happen, without the need for constant checking and feedback from senior leaders.

Read more

The new Education Endowment Foundation (EEF) guide to implementation makes sobering reading for those of us who have led change projects in schools. Its messages are clear and fly in the face of what is often expected by the DFE and school leaders. Implementation of any new practice requires a considerable build up, in order to create a school culture of trust and risk taking that is conducive to implementation, and in order to think through in detail the implementation proposal, both in terms of its content and its practicalities. Key to this is the notion that strict fidelity is not always possible, nor is it desirable. Teachers must make sense of the proposed change in the context of their own pupils and their own classrooms, so a ‘tight but loose’ approach is much more likely to achieve success

Once this ground work has been laid, school leaders should not simply assume that up front professional development will enable the change in practice to be adopted by all school staff: follow up coaching and mentoring will be required, for both individuals and teams, tailored to their needs and based on the implementation data school leaders are regularly gathering. Scaling up from an early small-scale implementation requires a completely new implementation plan. And in order for such implementation projects to be truly successful, school leaders should assume that two to four years will be needed.

Where the report is less helpful is in helping school leaders ensure that implementation is of the highest quality. Here, Hall’s paper on change processes is more helpful. His paper analyses the reasons why teachers do and don’t change their practice in line with leaders’ expectations. He identifies four types of responses to any proposed change in practice:

  1. The teacher makes no changes to their practice, as the new practice was already familiar and part of their usual teaching repertoire
  2. The teacher changes their practice in line with what was asked
  3. The teacher thinks they have made the desired change to their practice but when the school leader looks at it, the new practice is very far from what was desired
  4. The teacher deliberately does not make the desired change to their practice

Obviously responses 1 and 2 pose no problems at all for the school leader. With regards to response 3, Hall offers a specific tool that implementers can develop collaboratively with staff, called an Implementation Configuration.  This simple grid describes different stages the teacher’s practice might go through in order to reach the desired practice. Here is an example from an Implementation Configuration developed with teachers around pupil self- and peer-assessment:

The grid enables teachers to fully understand what the desired new practice looks like, to self-assess and to make plans for their own progress.  It can also support the work of middle and senior leaders who are coaching and mentoring colleagues to successfully implement the new practice.

And Viviane Robinson’s new book Reduce Change to Increase Improvement, a brilliant guide for school leaders wanting to successfully implement change, offers clear guidance to the school leader trying to tackle teacher response number 4. Leaders need to explore teachers’ ‘theory of action’: the beliefs that underpin their actions, and that have led them to resist implementation.  It is only by recognising and acknowledging these beliefs, that leaders can begin to support teachers to change their practices and make a difference to pupils’ learning. Here, trust is key: teachers must trust leaders enough to feel they can be honest about their feelings, and leaders must be prepared to really listen to teachers’ concerns, and take them seriously. Here again then, a supportive school culture makes a big difference to successful implementation.

Read more

We are delighted to announce the appointment of Sarah Seleznyov as Director of Southwark Teaching School Alliance. Here, in her own words, she shares a bit about herself and why she’s excited be joining us on 1 March:

“I began my career around 25 years ago, in a London classroom very much like the ones in Southwark. It was a challenge meeting the needs of a diverse range of children, with different home languages, different family contexts, and many living in very challenging circumstances. I loved it, and I stayed, and I haven’t left London schools since.

“I am passionate about the right of every pupil to enjoy their education and to be successful. I believe that with the right support, every pupil can grow into an adult who is able to make informed choices about the kind of life they want to have. I have pursued this aim throughout my teaching career, a career which has led me from the classroom, to school leadership, to school improvement work, to working for a national literacy charity, to a mathematics research project with King’s College University, and finally to my role at UCL Institute of Education. Throughout this journey, I have met many practitioners and leaders who are as passionate as I am about this goal, including those I have met as part of my recent work with Southwark schools.

“For this reason, I am excited to be taking up the post of Director of the Southwark Teaching School Alliance. This is an opportunity for me to spend more time in some excellent schools working in challenging circumstances, more time in the classroom with highly skilled and committed practitioners, and more time with pupils who are going places and have so much to offer to future society.

“Schools are increasingly under pressure to do more with less, and quicker. For me, key to working within this ever-changing, challenging context and to making a real difference to the lives of London pupils is collaboration: working towards a shared vision of how learning could be and should be. In such an environment, schools need to hold on to what they believe, making use of evidence about what works to generate home grown solutions to their own unique problems of practice. I believe this intention should act as a road map for the future work of the alliance.

“I am keen for the alliance to continue to grow and reach out to new partners, the early years, primary, secondary and special schools within Southwark, and schools and alliances beyond Southwark. I want to support practitioners within the alliance schools to continue to develop their skills as leaders of learning, both within their own schools and across schools. I want our pupils, and pupils beyond our schools to benefit directly from the professional development we offer, and to go beyond what they thought was possible.

“I look forward to meeting you all and listening to your goals and dreams for the alliance as it moves into its next phase.”

Read more