Research impact: the importance of effective research management

Tunnel Image

Striving to achieve impact from academic research may appear to be a wholly good thing, but how research leaders and managers interpret what impact means is vital to ensuring they can generate outstanding impact. Catherine O’Connell, lecturer, Liverpool Hope University, asks whether current formulaic interpretations of research impact have a limited, rather than enabling, effect on higher education institutions.

The introduction of research impact to the Research Excellence Framework 2014 prompted significant consternation in the academic community at the time. For some it was perceived as an erosion of academic freedom and increased government control of research agendas. However, a cautious optimism has been evident in some quarters on the potential to broaden the concept of research excellence in a constructive way. The Million+ group, in its response to the Stern Review consultation, indicated that the process would be ‘valuable even if no funding decisions were associated’. The importance of this policy formulation is emphasised by newer universities in recognising and supporting a broader range of applied and translational research, and increased attention paid to delivering on universities’ civic duties. Even more telling is the recently announced decision by Hefce to increase the weighting of impact from 20% in 2014 to 25% in 2021.

So, how do higher education leaders and managers make strategic decisions on how to identify, nurture and select impact examples from research? And how could the Leadership Foundation contribute?

Tunnel vision

Research conducted by Watermeyer & Hedgcoe mid-way through the last REF cycle (2009-2014) highlighted local responses to impact policy in research-intensive institutions. They observed a tendency to frame impact around the activities of individual (and commonly senior-level) academics which can affect the level of resources and support available for earlier career academics in impact-related activities. Watermeyer also identified a tendency among academics to interpret impact in relation to interactions with government, reflecting ‘a rather one-dimensional form of impact as emergent from interactions with a singular research beneficiary/user’. For example policymakers, who are only one specific type of beneficiary from academic research, overlooking industry, public, private and voluntary sectors, as well as the higher education sector itself.

Panoramic view

What is needed is a more critical debate around what impact means, how it can be supported and how broadening the definitions, mechanisms and support for impact planning can ultimately enhance research impact for public and institutional good. The Leadership Foundation has published research on a data mining exercise of impact from REF case studies on leadership, governance and management, which has informed the development of a toolkit. The toolkit aims to assist research leaders, managers and individual researchers to develop an embedded and strategic approach to research impact, covering a broad spectrum of impact areas to enable different staff groups and stakeholders to coordinate their approach to impact. One such tool (Tool 8, Anticipating the horizon of possible benefits) stimulates thinking about where impact might occur across a whole spectrum of categories – from Culture to Policy, and Technology to Environment – and uses prompt questions to discern the nature of such potential impact.

Analysis of the 46 impact case studies in my area of higher education -focused educational research demonstrated greater diversity of impact activities than anticipated by earlier research. There is only limited information in the public domain on those studies which achieved 3 and 4* but, of the case studies where grading can be determined, several reflected pedagogic research and impact strategies aimed at broader policy communities.

Why the narrow face?

To understand this better, I have interviewed academics who have suggested that instead of a broad interpretation of impact, a more formulaic response to impact was described in many cases which seemed to prioritise particular forms of research (based on prevailing hierarchies of research reputation) and effectively narrow the parameters of national REF impact policy:

My university wants impact supported only by 3 or 4 star research – I think that’s a mistake. 

Several interviewees, in senior academic positions, reflected on the advice they were inclined to pass on to early career researchers; effectively to advise against pursuing particular forms of research, such as research conducted with policy communities rather than policy makers. The disenchantment expressed was troubling and reflected largely negative experiences of local management of REF impact policy:

So we’ve got this mad game playing now where you start to decide what is and what isn’t impact in quite draconian ways…  so they’re already starting to be shaped up and crafted, and then anything else that’s outside those case studies, whether it has impact or not, it doesn’t really matter because they’re not important…

Impact leadership to enable

Impact brings a new element to research evaluation policy that gives conceptual and managerial space for interpretation. The Leadership Foundation 2014 report ‘Academic leadership: changing conceptions, identities and experiences in UK higher education’ emphasises the qualities of management and leadership sought by the academic community: of enabling environments which can nurture the next generation of researchers; fostering academic citizenship and reflecting different ways of making a contribution. Potentially, impact can value a broader range of research activities and give recognition to different types of research contribution. In the Research Leader’s Impact Toolkit, emphasis is placed upon the importance of understanding context, engagement and collaboration at an institutional and research team level.

However, the formulaic and normative interpretations of REF impact policy identified in several organisational contexts suggest that impact policy is having a limiting rather than enabling effect. There are clear challenges but also opportunities at organisational level in responding to this indicator constructively and in defining institutional policy responses which foster inclusion rather than exclusion within the academic community.

As highlighted in my study, in some organisations impact policy is being interpreted in ways that resonate with, and build upon, academics’ research practices in departmental and faculty contexts:

I actually became quite a fan. I thought it was something significant and important and it brought to light some of the research people were doing that was having really significant effect in people’s lives […]  That kind of research hasn’t necessarily been valued.

Having a critical debate to develop and define local impact policies and practices, from the strategic to the operational, can be an important first step in this journey – and resources like the Research Leader’s Impact Toolkit will be a valuable companion.

Catherine O’Connell is a Lecturer in Education Studies at the Centre for Education and Policy Analysis, Liverpool Hope University.

Find out more about our Research Leader’s Impact Toolkit: www.lfhe.ac.uk/RIT

Download Dr Elizabeth Morrow’s report on The Impact of Higher Education Leadership, Governance and Management Research: Mining the 2014 Research Excellence Framework Impact Case Studies: www.lfhe.ac.uk/Morrow5.2

Do university leaders really understand how they are creating value in their universities?

In advance of the Leadership Foundation’s announcement of the universities that will be taking part in the new Integrated Thinking and Report project, Kim Ansell, managing consultant, sets out why the time has never been so important for getting to grips with creating and communicating value. 

Setting out long-term plans in the face of sector-wide turbulence is a challenge for every executive team and it is clear that given very recent government policy commitments, many of the assumptions underpinning institutional strategies have to be revisited, even if they were written as recently as a year ago.

These strategies are often being pursued in isolation without a real understanding of the wider risks and potential to destroy rather than create value. At a time when universities are facing ever more scrutiny and public accountability some have started to embrace a new way of thinking and reporting based on the Integrated Reporting Framework. This framework goes further than simply describing an organisation’s financial performance and among other things describes its contribution to society, the environment and its own community and stakeholders.

One of the challenges that universities face is summed up in a recent article on Media FHE by Professor Nick Talbot, deputy vice-chancellor for research and impact at Exeter University, on how higher education can defend itself from critics.  He asserts that, “It is almost as if there were two higher education sectors – the ‘schooly’ bit and the ‘researchy’ bit, which exist as separate islands”.

Although written in the context of the Research Excellence Framework (REF), this remark makes the case perfectly for integrated thinking, and the Leadership Foundation believes that if universities can create a joined-up approach to thinking and reporting, aligned to clear business and value creation models, they will more successfully navigate the journey ahead and overcome barriers to sustainable strategic planning.

In particular, the need to be able to communicate to all stakeholders (from employees and governors to students and strategic partners) that the institution is creating value in the short, medium and long term is critical to leadership success.

With this in mind we are planning to unpack the concept of Integrated Reporting and facilitate the journey; to think about the full range of resources (or capitals), for example:

  • Intellectual Capital
  • Human Capital
  • Social and Relationship Capital
  • Financial Capital
  • Environmental Capital

Understanding the trade-off between such resources when making strategic decisions is something which could be more successful and provide more sustainable outcomes if done in the context of an informed and disciplined approach to integrated thinking.

University leaders need information that assists them in making sense of a complex world and the direction of travel their institution is likely to take. There have been a plethora of articles recently, referring to changing business and financial models in universities, public value and value for money. Equally there continues to be events, conferences and articles ‘talking’ about institutional strategy, but the Leadership Foundation aims to take the talking one step further: to test and evaluate what works and to provide some real examples of how universities can mobilise integrated thinking and reporting to drive value creation and to implement a sustainable and  successful strategy. With the support of an expert steering group, and building on previous work done by British Universities Finance Directors Group, we will consider what universities need to do

In this new work we aim to tackle sensitive issues with objectivity and equip institutions and their governing bodies with the skills, knowledge and insights to be able to evidence and report on holistic value creation and contribution, and provide necessary assurance on this key topic.

The time has never been better for universities to look to integrated thinking and integrated reporting to help them on their journey through these turbulent times in UK higher education.

For more on the Leadership Foundation’s Integrating Thinking and Reporting project visit www.lfhe.ac.uk/IntegratedThinking 

Leading People is Leading Diversity

‘Reality is diverse; therefore a true reflection of reality includes diversity.’  Nancy Kline

Shirley Wardell, programme director of our research leadership development programmes discusses the importance of encouraging diverse thinking and insight into the valuable skills every leader should prioritise.

I have come to think of the skills leaders need to understand the diversity issues as mainstream leadership skills.  To my mind managing people is managing diversity. Diversity goes beyond minority groups and the obvious power imbalances.  Diversity extends to the subtle depth of how we think, which has a direct impact on how well we perform in our jobs.

Diversity grows when people have the ability to hear, openly, what everybody thinks.  Having practised that skill, with people we believe are similar to us, we may be better prepared to listen to those we assume are more different to us.  The charming surprise is; that as Maya Angelou says, ‘We are more similar than we are different.’ Once we have accepted that we are more likely to be similar in a broad way, appreciating the specific differences seems to be the key.  So how can we be sure that we are able to allow, or even encourage, different ways of thinking?

I choose the Thinking Environment® to help me, and my clients, to create the conditions for diverse thinking to flourish. When you run an event in a Thinking Environment®; everyone has a turn. That means; you go round the group and ask everyone what they think.  Sometimes people tell me it takes too long, but they are really stumped when I ask them who they would leave out of the round.

In an event such as this no-one interrupts and participant say; ‘If I don’t interrupt, I might forget my idea?’ And again, they look a bit blank when I ask, ‘What if the person you interrupt forgets theirs?’ Giving turns, not interrupting, appreciating each other, asking how to make things better and a positive philosophy are a few of the ways to get everyone involved in a productive way.

The Thinking Environment® has ten components; however there are a few principles that sum it up for me:

  • The way we listen to someone has an impact on the quality of their thinking.  If we are able focus on them, stop judging and create a time and space for them; the quality of their thinking improves.  At a recent workshop I asked how it feels to be listened to really well and people said they felt valued, important, as if their ideas matter, that they have a contribution to make, happy, it improved their self esteem, relaxed and intelligent.  Well, if all those things can be achieved by, ‘just listening’ we should perhaps put listening at the top of the leadership skills list.
  • When you think on behalf of someone else you are disempowering them.  When you think your ideas are better, or you are simply too busy for them to find their own answer, you are stopping them from thinking and therefore stopping them from learning and growing.  Being able to develop staff has become one of the most valuable assets to Institutions and leaders who can do this will have the evidence of their success in their research output.
  • A positive philosophy is required to help people perform well.  Our expectations will have an impact on the outcomes.  Those expectations include what I expect from the person and what my prejudices are about that person. I need to be able to see there are numerous and unknown possibilities yet to be achieved for every individual.
  • We also need to examine our assumptions about the world.  What we expect to be possible in this office, this organisation, this market, this country and this world; will have an impact on our own and our team’s thinking.  Leadership training needs to explore the assumptions we make and the impact that has on performance; and then show how to, pragmatically, choose assumptions that will help us perform better.

Research Team Leadership and Leading Research Leaders are run in a Thinking Environment® and include many of the reliable principles and actions that help research leaders to think. They are then able to pass that favour on to their teams and collaborators.

The Thinking Environment® was developed by Nancy Kline of Time to Think

Find out more about Shirley Wardell by visiting our website www.lfhe.ac.uk/resprog

Leadership Foundation Research Impact – Working for Wales

science-banner-3

Professor Fiona Ross reflects on the impact that our stimulus paper on research funding in Wales has had on prompting the Welsh government to make positive steps towards supporting and encouraging research in the nation.

Last year Peter Halligan and Louise Bright (2015) published a paper on the Case for Growing STEMM Research Capacity in Wales. Theirs is a story of research funding for Science Technology Engineering Mathematics and Medicine (STEMM) in Wales. It provides a powerful review and explanation for what appeared to be Wales’s poor comparative research performance and productivity compared to Scotland. We published it in our stimulus paper series and this blog reflects on its impact.

The Leadership Foundation’s stimulus paper series is designed to support thought leadership and to provide the sector with an opportunity to challenge established perceptions and discuss them from a new position. Independent from disciplinary lobbying, government policy making and mission group, over the years the Leadership Foundation has offered an alternative space for incubating ideas on leadership, challenging the status quo in leadership, publish and disseminate for greater impact through its network of member institutions. We thought this was the ideal vehicle for this research.

Halligan and Bright’s paper is a detailed and longitudinal policy analysis of comparative data on research funding in Wales. It is not for the faint hearted and does not leave a stone unturned! It lays out the drivers behind the Welsh Office, and subsequent Welsh Government’s focus on an input target of total research council income. It argues that policy reliance on securing Wales’s UK share of Research Council funding had contributed to a misleading and reputational damaging perception of the Welsh university research base. To derive a more complete picture of the STEMM shortfall in Wales, Halligan and Bright calculated the total number of STEMM academic researchers in the four UK nations. Using Wales’s population share of total UK academics engaged in research they found that the academic research workforce was some 0.5% below Wales’s population standard share. Despite this discrepancy, the evidence shows that relatively low levels of Research Council income have nevertheless been effectively translated into high impact research.

Halligan and Bright’s paper concludes that the critical problem lay not in the quality of the science being done in Wales, but rather with the inadequate size of the science base and the number of researchers in STEMM. So what has been the impact from the paper and what happened? The Welsh Government listened, the Chief Scientific Advisor (CSA) acted swiftly though her Ser Cymru programme to deliver an ambitious strategy to increase research capacity in science to enhance economic growth.  To achieve this, the CSA brought together a number of initiatives involving COFUND funding from the EU Horizon 2020 and the European Regional Development Fund in association with Welsh Universities. This amounted to over £50M to support a capacity building programme to fund over 100 new fellowships in science. This is providing support for large scale doctoral training schemes, postdoctoral rising stars and promising research leaders and support for scientists (particularly women) returning to their fields after a long absence.

I am often asked about Leadership Foundation research outputs and what difference they make? The honest answer is it varies. Sometimes we hit on a winner, like Halligan and Bright. But impact does not happen by accident. It is a complex process. Here it took compelling evidence supporting the case for change, authors who were both authoritative and influential, a receptive policy context and respectful and longstanding relationships between government and academic institutions. Our analysis of LF impact shows the secret is about the quality of commissioning, and being able to anticipate the “burning platform” issues, working hard with authors to ensure quality and using the LF network to provide a conduit for dissemination and exchange of ideas. It has worked for Wales.

Professor Fiona Ross is Director of Research at the Leadership Foundation. Fiona leads research and thought leadership with a particular focus on generating learning for organisations on ‘what works’. Fiona has a background in community health and social policy and has worked as practitioner, teacher, research leader and senior manager over a 35 year career in higher education. She has had academic leadership roles at King’s College London, Kingston University and St George’s, University of London where most recently she was an executive dean. She has published widely on policy and care of older people, public engagement, collaborative practice and leadership of change. In addition to her role with the Leadership Foundation she has a part time professorial appointment at Kingston University and St George’s and does research and writes on collaborative governance and evaluating system wide interventions including Kingston University’s approach to narrowing the attainment gap for students from BME backgrounds. Fiona has recently been appointed Chair of the Board of Trustees of Princess Alice Hospice, which delivers end of life care in Surrey and South West London. She was awarded a CBE in the 2015 New Year’s Honours list for services to health care and higher education.


References

Welsh Government Delivering Science for Wales 2014-15.  Annual Report on the Strategy for Science in Wales p.2, p.6, p.7, p.16

Welsh Government Delivering Science for Wales 2015-16.  Annual Report on the Strategy for Science in Wales p.3, p.12

About the research authors

Professor Peter Halligan is the Chief Executive of the Learned Society of Wales

Dr Louise Bright is Deputy Director of Research and Business Engagement at the University of South Wales and the former Leadership Foundation Associate Director for Wales.

About our research
Our goal is to commission, develop and disseminate path finding research and resources which have originality, utility and impact to the sector. To view our latest research, click here

About Leadership Foundation Membership
We are a membership organisation of and for a sector that has some of the brightest minds in the UK. Our members are key to our strategy and form a community of higher education institutions with a clear commitment to and experience of developing leadership, governance and management capabilities at all levels. Academic and professional services staff from member institutions contribute to our programmes, projects and research and advice on benefits and services. To find out more about membership with us, click here