In the UK, UK Department for Business, Innovation, and Skills provided funding of £150 million for knowledge exchange in 2011–12 to ‘help universities and colleges support the economic recovery and growth, and contribute to wider society’ (Department for Business, Innovation and Skills 2012). This guide explains exactly what you need to write in the two separate impact sections in a Research Council bid (your impact summary and your pathway to impact), and also applies to the impact sections of grant applications for other funders. A university which fails in this respect has no reason for existence. Ideally, systems within universities internationally would be able to share data allowing direct comparisons, accurate storage of information developed in collaborations, and transfer of comparable data as researchers move between institutions. This article aims to explore what is understood by the term ‘research impact’ and to provide a comprehensive assimilation of available literature and information, drawing on global experiences to understand the potential for methods and frameworks of impact assessment being implemented for UK impact assessment. To adequately capture interactions taking place between researchers, institutions, and stakeholders, the introduction of tools to enable this would be very valuable. Wooding et al. What are the challenges associated with understanding and evaluating research impact? One way in which change of opinion and user perceptions can be evidenced is by gathering of stakeholder and user testimonies or undertaking surveys. A comprehensive assessment of impact itself is not undertaken with SIAMPI, which make it a less-suitable method where showcasing the benefits of research is desirable or where this justification of funding based on impact is required. Similar to writing journal articles, impact case studies, both in their underpinning research and details of impact sections, must have a hook. Having undertaken extensive reading and reviewed a number of case studies from REF 2014, I decided the A-Z was best divided into three parts: (1) an impact aim; (2) research and impact evidence; and (3) a narrative to build the story around the blocks of evidence. They are often written with a reader from a particular stakeholder group in mind and will present a view of impact from a particular perspective. In taking a light-hearted approach to REF2021, she has also created a REF2021 Food A-Z and is partway through a REF2021 advent A-Z on Twitter. The university imparts information, but it imparts it imaginatively. The ability to record and log these type of data is important for enabling the path from research to impact to be established and the development of systems that can capture this would be very valuable. 2010; Hanney and González-Block 2011) and can be thought of in two parts: a model that allows the research and subsequent dissemination process to be broken into specific components within which the benefits of research can be studied, and second, a multi-dimensional classification scheme into which the various outputs, outcomes, and impacts can be placed (Hanney and Gonzalez Block 2011). 2005; Wooding et al. The Payback Framework systematically links research with the associated benefits (Scoble et al. Prague, Czech Republic, Health Research—Making an Impact. The Goldsmith report (Cooke and Nadim 2011) recommended making indicators ‘value free’, enabling the value or quality to be established in an impact descriptor that could be assessed by expert panels. Figure 2 demonstrates the information that systems will need to capture and link. What have you achieved that hasn’t been evidenced sufficiently? HEIs overview. In this case, a specific definition may be required, for example, in the Research Excellence Framework (REF), ‘Assessment framework and guidance on submissions’ (REF2014 2011b), which defines ‘impact’ as, an effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia. An outline here is really useful as the reader can understand how difficult it was, what the challenges were, and how the researcher made a difference. These techniques have the potential to provide a transformation in data capture and impact assessment (Jones and Grant 2013). This presents particular difficulties in research disciplines conducting basic research, such as pure mathematics, where the impact of research is unlikely to be foreseen. A Preferred Framework and Indicators to Measure Returns on Investment in Health Research, Measuring Impact Under CERIF at Goldsmiths, Anti-Impact Campaign’s ‘Poster Boy’ Sticks up for the Ivory Tower. There are areas of basic research where the impacts are so far removed from the research or are impractical to demonstrate; in these cases, it might be prudent to accept the limitations of impact assessment, and provide the potential for exclusion in appropriate circumstances. As Donovan (2011) comments, ‘Impact is a strong weapon for making an evidence based case to governments for enhanced research support’. The University and College Union (University and College Union 2011) organized a petition calling on the UK funding councils to withdraw the inclusion of impact assessment from the REF proposals once plans for the new assessment of university research were released. New Directions for Evaluation, ‘Impact is a Strong Weapon for Making an Evidence-Based Case Study for Enhanced Research Support but a State-of-the-Art Approach to Measurement is Needed’, ‘The Limits of Nonprofit Impact: A Contingency Framework for Measuring Social Performance’, Evaluation in National Research Funding Agencies: Approaches, Experiences and Case Studies, ‘Methodologies for Assessing and Evidencing Research Impact. From 2014, research within UK universities and institutions will be assessed through the REF; this will replace the Research Assessment Exercise, which has been used to assess UK research since the 1980s. What are the reasons behind trying to understand and evaluate research impact? The ability to write a persuasive well-evidenced case study may influence the assessment of impact. It also makes it exciting to follow how that impact was achieved. In the UK, evaluation of academic and broader socio-economic impact takes place separately. Differences between these two assessments include the removal of indicators of esteem and the addition of assessment of socio-economic research impact. To evaluate impact, case studies were interrogated and verifiable indicators assessed to determine whether research had led to reciprocal engagement, adoption of research findings, or public value. However, the Achilles heel of any such attempt, as critics suggest, is the creation of a system that rewards what it can measure and codify, with the knock-on effect of directing research projects to deliver within the measures and categories that reward. Providing advice and guidance within specific disciplines is undoubtedly helpful. Tempted as we are to write in our comfort zone (i.e. Be clear about the beneficiaries – did they approach you, you approach them, or a chance meeting? Dr Alex Conner offers some expert advice. Whether you One might consider that by funding excellent research, impacts (including those that are unforeseen) will follow, and traditionally, assessment of university research focused on academic quality and productivity. The traditional form of evaluation of university research in the UK was based on measuring academic impact and quality through a process of peer review (Grant 2006). Any information on the context of the data will be valuable to understanding the degree to which impact has taken place. It is important to emphasize that ‘Not everyone within the higher education sector itself is convinced that evaluation of higher education activity is a worthwhile task’ (Kelly and McNicoll 2011). Attempting to evaluate impact to justify expenditure, showcase our work, and inform future funding decisions will only prove to be a valuable use of time and resources if we can take measures to ensure that assessment attempts will not ultimately have a negative influence on the impact of our research. Frameworks for assessing impact have been designed and are employed at an organizational level addressing the specific requirements of the organization and stakeholders. The main risks associated with the use of standardized metrics are that, The full impact will not be realized, as we focus on easily quantifiable indicators. Research findings including outputs (e.g., presentations and publications), Communications and interactions with stakeholders and the wider public (emails, visits, workshops, media publicity, etc), Feedback from stakeholders and communication summaries (e.g., testimonials and altmetrics), Research developments (based on stakeholder input and discussions), Outcomes (e.g., commercial and cultural, citations), Impacts (changes, e.g., behavioural and economic). It is now possible to use data-mining tools to extract specific data from narratives or unstructured data (Mugabushaka and Papazoglou 2012). This can involve academic impact, economic and societal impact or both: The impact of research, be it academic, economic and social can include: Knowledge exchange (KE) (external website), the enabling of two-way exchange between researchers and research users to share ideas, research evidence, experiences and skills, is fundamental to our understanding of what makes excellent research. From past experience of undertaking research, I have found the key part is breaking the question down into different “blocks” and analysing them from different perspectives. Developing systems and taxonomies for capturing impact, 7. This might include the citation of a piece of research in policy documents or reference to a piece of research being cited within the media. Where quantitative data were available, for example, audience numbers or book sales, these numbers rarely reflected the degree of impact, as no context or baseline was available. We are in the zeitgeist of impact, where the need to justify our research and evidence societal benefits has never been greater. At least, this is the function which it should perform for society. Rather than in alphabetical order, my A to Z is presented here in these three parts. Your email address will not be published. Search for other works by this author on: A White Paper on Charity Impact Measurement, A Framework to Measure the Impact of Investments in Health Research, European Molecular Biology Organization (EMBO) Reports, ‘Estimating the Economic Value to Societies of the Impact of Health Research: A Critical Review’, Bulletin of the World Health Organization, Canadian Academy of Health Sciences Panel on Return on Investment in Health Research, Making an Impact. While aspects of impact can be adequately interpreted using metrics, narratives, and other evidence, the mixed-method case study approach is an excellent means of pulling all available information, data, and evidence together, allowing a comprehensive summary of the impact within context. The exploitation of research to provide impact occurs through a complex variety of processes, individuals, and organizations, and therefore, attributing the contribution made by a specific individual, piece of research, funding, strategy, or organization to an impact is not straight forward. Clearly there is the possibility that the potential new drug will fail at any one of these phases but each phase can be classed as an interim impact of the original discovery work on route to the delivery of health benefits, but the time at which an impact assessment takes place will influence the degree of impact that has taken place. Case studies are ideal for showcasing impact, but should they be used to critically evaluate impact? However, good KE is as much about approach, mindset, personal qualities and researcher mission. Impact assessments raise concerns over the steer of research towards disciplines and topics in which impact is more easily evidenced and that provide economic impacts that could subsequently lead to a devaluation of ‘blue skies’ research. It is time-intensive to both assimilate and review case studies and we therefore need to ensure that the resources required for this type of evaluation are justified by the knowledge gained. For more extensive reviews of the Payback Framework, see Davies et al. We define research impact as 'the demonstrable contribution that excellent research makes to society and the economy'. Perhaps, SROI indicates the desire to be able to demonstrate the monetary value of investment and impact by some organizations. The case study approach, recommended by the RQF, was combined with ‘significance’ and ‘reach’ as criteria for assessment. Systems need to be able to capture links between and evidence of the full pathway from research to impact, including knowledge exchange, outputs, outcomes, and interim impacts, to allow the route to impact to be traced. ‘The justification for a university is that it preserves the connection between knowledge and the zest of life, by uniting the young and the old in the imaginative consideration of learning. Why should this be the case? There is a great deal of interest in collating terms for impact and indicators of impact. In terms of research impact, organizations and stakeholders may be interested in specific aspects of impact, dependent on their focus. RAND Europe’, Capturing Research Impacts. Here we address types of evidence that need to be captured to enable an overview of impact to be developed. Incorporating assessment of the wider socio-economic impact began using metrics-based indicators such as Intellectual Property registered and commercial income generated (Australian Research Council 2008). Through evaluation and commissioned work on knowledge exchange, we have found that more co-productive forms of research (ie research undertaken with rather than on people in a collaborative, iterative process of shared learning) offer particular potential for impact academically and socially. Impact case studies from consultancy work are easier to define here as the beneficiary (or client) had a clear need. Thalidomide has since been found to have beneficial effects in the treatment of certain types of cancer. We suggest that developing systems that focus on recording impact information alone will not provide all that is required to link research to ensuing events and impacts, systems require the capacity to capture any interactions between researchers, the institution, and external stakeholders and link these with research findings and outputs or interim impacts to provide a network of data. Impact case studies are the opposite: you already have the blocks – evidence in different parts and perspectives – and it’s up to the researcher to use them to create a coherent story. Time, attribution, impact. Nearly there with your case study? Anecdotal feedback from researchers who have participated in co-production have reported benefits for their networks, job satisfaction and careers, which has balanced out some of the challenges they faced in undertaking this form of research. Were you a small part of a bigger picture? Proposed methodology for measuring the effectiveness of policies designed to further research, Understanding and evaluating the impact of integrated problem-oriented research programmes: Concepts and considerations, The program and policy change framework: A new tool to measure research use in low- and middle-income countries, The effect of competitive public funding on scientific output: A comparison between China and the EU. These ‘traditional’ bibliometric techniques can be regarded as giving only a partial picture of full impact (Bornmann and Marx 2013) with no link to causality. Throughout history, the activities of a university have been to provide both education and research, but the fundamental purpose of a university was perhaps described in the writings of mathematician and philosopher Alfred North Whitehead (1929). The Payback Framework enables health and medical research and impact to be linked and the process by which impact occurs to be traced. 2005). We will focus attention towards generating results that enable boxes to be ticked rather than delivering real value for money and innovative research. Although some might find the distinction somewhat marginal or even confusing, this differentiation between outputs, outcomes, and impacts is important, and has been highlighted, not only for the impacts derived from university research (Kelly and McNicol 2011) but also for work done in the charitable sector (Ebrahim and Rangan, 2010; Berg and Månsson 2011; Kelly and McNicoll 2011). (2006) on the impact arising from health research. A key concern here is that we could find that universities which can afford to employ either consultants or impact ‘administrators’ will generate the best case studies. Featured image credit: back to school by Martin Abegglen (licensed under a CC BY-SA 2.0). For systems to be able to capture a full range of systems, definitions and categories of impact need to be determined that can be incorporated into system development. Her ORCID is 0000-0003-1185-1962. Such a framework should be not linear but recursive, including elements from contextual environments that influence and/or interact with various aspects of the system. Collating the evidence and indicators of impact is a significant task that is being undertaken within universities and institutions globally. Accountability. Doing this frames the focus of the case study, which you can then link to the purpose of the research – after all impact is all about you, not me.

How To Write Literary Fiction Article, College Application Essay Editing Services, Past Or Present Tense In Academic Writing Research, Reflective Writing Tasks Research, Writer Sign Up Research,