academic discourse tasks

• Our Mission

Teaching Students How to Have an Academic Discussion

This teacher-created scaffolding tool can elevate classroom dialogue, empowering students to express themselves while learning from the ideas of others.

Ever been in the midst of a classroom discussion where students are bubbling over with ideas, eager to share? It can look and sound like lots of voices calling out and talking over each other while quieter students try to find a place to enter the fray. You don’t want to squash anyone’s enthusiasm, but how do you ensure that every student has the opportunity to truly express themselves and hear the ideas of their classmates?

Students’ ability to clearly communicate with others in conversation is a critical literacy skill. While goals for speaking are a part of elementary school standards, beginning in kindergarten, teaching talk can feel tricky; it’s often hard for students to know where to begin or the next best place to go when responding to someone. A “Progression of Talk” chart can be very helpful in this work—defining starting and ending points for students, as well as all the points in between, when engaging in conversation.

The “Progression of Talk,” by Cynthia Satterlee, begins with a student saying anything at all and grows to a student saying something relevant, adding on to what was previously said or disagreeing, clarifying, improving, revising their thinking, provoking more thought, or extending and elaborating—all necessary skills for any form of discourse.

When Cynthia shared this progression during a workshop, I nodded and thought, “Yes, just what I’ve been looking for!” She helped me see how talk could grow across a range of grade levels and subject areas. As I studied the progression, I thought it could become an even more powerful tool for students and teachers by adding a series of sample talk prompts at each level.

Observation

The first step of teaching kids how to lift the level of their talk begins with observation: simply paying attention to what students are saying to each other. Is it connected to the original speaker’s sentiments or wildly off-topic? Are they putting thoughts into words and able to keep a conversation going or inadvertently shutting it down? Match your observations to steps on the progression and then lean into instruction.

What this looks like inside the classroom:

Many elementary-aged children love to share their opinions when they disagree with others. I have heard children do this with short phrases—“That’s not right!”—or by sharing how they see things differently without acknowledging the original speaker’s ideas.

• Providing students with talking prompts like “I agree…” or “I feel differently…” can help them make a connection between what the speaker is saying and what they are adding to grow the conversation.
• Similarly, empowering kids to use the phrase “I’d like to add…” helps build a bridge from someone else’s idea to their own.

Introducing Talk Prompts

Displaying talk prompts once you have introduced them to your class can be a powerful scaffold. It’s best to write them with students while explicitly explaining the thought process behind the prompts:

“One way we are adding on to what other kids in class have to share is to start by saying, ‘Something you just said…’ Let’s write that on a big sticky note and place it here on the board where we can all see it. We can work on growing our talk by using this language whenever we want to share something connected to the conversation.”

This note can then be referred to prior to conversations, with a reminder: “Remember, we are trying to grow our talking skills by saying something that connects to what the person before you said.”

Next, point to and provide the talk prompts to help scaffold this work. Slowly building a progression tool with students is much more powerful than displaying it in its entirety at the onset of digging into these skills.

The Evolution of Talk

This process takes time. Support conversations with the talk prompts until students make them their own. If a child begins speaking without them, you can gently say the prompt and encourage them to try again, scooping up this language as their running start. When I am doing this, it reminds me of helping someone learn how to float. At first, my hands are firmly under the person, but soon my words become gentle fingertips just beneath the surface—providing reassurance, stability, and support.

I take my figurative hands completely away when I hear kids incorporating the language of this progression in natural back-and-forths. This might sound like a child adding on to something they’ve said to clarify a point—“Let me try to say that in a clearer way” or “I’d like to add on to what I said before.” And another child sharing how their thinking has changed as a result of considering the thoughts and perspectives of others—“I used to think… but now I think…”

Not only does this exchange sound good, but it honors the thoughts of all speakers and helps kids see how their words are interconnected.

It’s important to remember that this progression chart isn’t a set of discrete steps or tied to particular grade levels; it can be more fluid depending on what students are saying, how they are interacting, or the focus of your curriculum. For example, with fifth graders in a debate unit, lean into talk prompts that allow students not only to press for more information but to respectfully express differing viewpoints.

As kids grow in this work and try on other successful ways to lift their talk, notice, name, and celebrate what they are saying, then add it to your shared chart. Empower students to use this cocreated visual to gauge their own progress, connect their language to the progression, and set goals to grow. Elevating talk skills in a dynamic way enables students to truly express themselves both in and out of the classroom, shifting responses like “What?!” to “I’d love to hear you say more…”

The Journal

• Book Reviews

Latest Issue

Vol. 23, No. 4

• Teachers and Teaching

How to Empower Every Student to Talk in Class—and Why It Matters

By creating welcoming, well-facilitated opportunities for in-class discourse, educators can help level the playing field.

Emma Lind Martinez

Students and teachers returning to face-to-face (or mask-to-mask) instruction are encountering a heightened set of challenges around classroom discussion. After a year-plus of learning at home or in varying degrees of isolation, academic discourse can feel unnatural.

Students talking in class might seem unremarkable. But it’s actually a critical factor in creating productive, equitable classrooms. High school educators know that most teenagers love to talk and have no shortage of things to say. When done well, structured class discussions can deepen learning for all students; but if not facilitated equitably, they can be exclusionary for students from nondominant backgrounds or without a strong sense of belonging in class.

What makes talk a tool for learning and equity?

In a productive academic discourse, students address each other rather than communicating through the teacher. By setting up goal-directed discussions about meaningful content, educators can help students transform their ideas into public resources for their classmates, advancing the whole group’s state of understanding. A lesson thus becomes a collective endeavor in building knowledge and deepening skills.

Classroom discourse provides students with authentic opportunities to process new content using methods from oral traditions. This sets them up to become independent, self-directed learners. At the same time, discourse builds and sustains a strong community of learners by making room for student voice and agency while centering class culture around communal talk and tasks.

While many of us think of class discussions as being strictly in the humanities realm, student discourse is just as powerful a tool for learning in math and science classrooms. Research backs this up: the authors of  Ambitious Science Teaching  found that in science classrooms, the most “powerful examples of learning” were characterized by verbal sense-making and “productive and equitable talk.”

What does this have to do with equity? A lot, as it turns out. Culturally and linguistically diverse students are often denied the opportunities to delve into more sophisticated thinking skills like synthesis and analysis because of systemic educational inequities. As a result, a disproportionate number of these students struggle to learn independently even though they have the potential to do so.

In  Culturally Responsive Teaching and the Brain , Zaretta Hammond argues that students learn the most, and develop into the strongest independent thinkers, when they engage in academic risks in an environment that feels safe. By creating welcoming, well-facilitated opportunities for in-class discourse, educators can help level the playing field and give our most vulnerable students a chance to independently flourish.

Some may think of academic discourse as happening only in the classrooms of veteran teachers, or with accelerated students who have “earned” the right to be heard. But student talk belongs in every classroom, and all students deserve opportunities to share their voices.

How to make this happen? 5 Key Steps 

Even adults know that sometimes, the more challenging the topic, the more difficult the conversation. We’ve all had moments where saying nothing would be easier. So how can teachers and schools make sure that all students are set up to engage in and benefit from academic discourse? Here are a few key steps for planning and execution:

• Develop clear discourse norms and expectations, and revisit them throughout the discussion.  Should students take turns talking? What’s a supportive way to respond to each other’s thoughts, and what responses might shut down talk?
• Establish goals and a timeframe up front for the discussion; this is especially crucial for engaging diverse learners and English language learners. What are the guardrails of this conversation? How and when can I participate? What’s the goal of this time?
• Choose a task that asks students to actually  do something with ideas (also known as a reasoning or high cognitive-demand task).  Is this a juicy topic that’s worth talking about?
• Plan 2-3 “discourse moves” to engage all students in the discourse; encourage students to talk to each other rather than the teacher. These moves might include things like wait time, a quiet pause while students synthesize their thoughts and prepare to speak; batch call, where a teacher calls on two or three students to talk one after the other, without the teacher speaking in between them; and universal prompts, which are prompts that can be used to push student thinking in a variety of contexts (“What makes you think that?” “What is evidence to prove…?”).   Do all students have an access point into the conversation? How can I draw out students who haven’t participated yet?
• To wrap up the discussion, script a prompt for students to articulate the key point(s) in their own words.  To optimize independent learning, how can students “own” the major takeaway(s) of the discussion?

Encouraging students to talk to each other in class is a key tool, not only for learning, but for equity. As students, teachers, and school leaders emerge from behind our computer screens, we have a great opportunity to create rich, engaging classroom discourse that maximizes learning for all students.

Emma Lind-Martinez is High School Math and Science Achievement Director at the KIPP Foundation .

Last updated December 21, 2021

License this Content

Education Next is a journal of opinion and research about education policy.

NOT CONVINCED?

Get a Sample copy of The Journal

Notify me when education next posts a big story.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 02 March 2021

Research impact evaluation and academic discourse

• Marta Natalia Wróblewska   ORCID: orcid.org/0000-0001-8575-5215 1 , 2  

Humanities and Social Sciences Communications volume  8 , Article number:  58 ( 2021 ) Cite this article

4321 Accesses

8 Citations

33 Altmetric

Metrics details

• Language and linguistics
• Science, technology and society

The introduction of ‘impact’ as an element of assessment constitutes a major change in the construction of research evaluation systems. While various protocols of impact evaluation exist, the most articulated one was implemented as part of the British Research Excellence Framework (REF). This paper investigates the nature and consequences of the rise of ‘research impact’ as an element of academic evaluation from the perspective of discourse. Drawing from linguistic pragmatics and Foucauldian discourse analysis, the study discusses shifts related to the so-called Impact Agenda on four stages, in chronological order: (1) the ‘problematization’ of the notion of ‘impact’, (2) the establishment of an ‘impact infrastructure’, (3) the consolidation of a new genre of writing–impact case study, and (4) academics’ positioning practices towards the notion of ‘impact’, theorized here as the triggering of new practices of ‘subjectivation’ of the academic self. The description of the basic functioning of the ‘discourse of impact’ is based on the analysis of two corpora: case studies submitted by a selected group of academics (linguists) to REF2014 (no = 78) and interviews ( n  = 25) with their authors. Linguistic pragmatics is particularly useful in analyzing linguistic aspects of the data, while Foucault’s theory helps draw together findings from two datasets in a broader analysis based on a governmentality framework. This approach allows for more general conclusions on the practices of governing (academic) subjects within evaluation contexts.

Introduction

The introduction of ‘research impact’ as an element of evaluation constitutes a major change in the construction of research evaluation systems. ‘Impact’, understood broadly as the influence of academic research beyond the academic sphere, including areas such as business, education, public health, policy, public debate, culture etc., has been progressively implemented in various systems of science evaluation—a trend observable worldwide (Donovan, 2011 ; Grant et al., 2009 ; European Science Foundation, 2012 ). Salient examples of attempts to systematically evaluate research impact include the Australian Research Quality Framework–RQF (Donovan, 2008 ) and the Dutch Standard Evaluation Protocol (VSNU–Association of Universities in the Netherlands, 2016 , see ‘societal relevance’).

The most articulated system of impact evaluation to date was implemented in the British cyclical ex post assessment of academic units, Research Excellence Framework (REF), as part of a broader governmental policy—the Impact Agenda. REF is the most-studied and probably the most influential impact evaluation system to date. It has been used as a model for analogous evaluations in other countries. These include the Norwegian Humeval exercise for the humanities (Research Council of Norway, 2017 , pp. 36–37, Wróblewska, 2019 ) and ensuing evaluations of other fields (Research Council of Norway, 2018 , pp. 32–34; Wróblewska, 2019 , pp. 12–16). REF has also directly inspired impact evaluation protocols in Hong-Kong (Hong Kong University Grants Committee, 2018 ) and Poland (Wróblewska, 2017 ). This study is based on data collected in the context of the British REF2014 but it advances a description of the ‘discourse of impact’ that can be generalized and applied to other national and international contexts.

Although impact evaluation is a new practice, a body of literature has been produced on the topic. This includes policy documents on the first edition of REF in 2014 (HEFCE, 2015 ; Stern, 2016 ) and related reports, be it commissioned (King’s College London and Digital Science, 2015 ; Manville et al., 2014 , 2015 ) or conducted independently (National co-ordinating center for public engagement, 2014 ). There also exists a scholarly literature which reflects on the theoretical underpinnings of impact evaluations (Gunn and Mintrom, 2016 , 2018 ; Watermeyer, 2012 , 2016 ) and the observable consequences of the exercise for academic practice (Chubb and Watermeyer, 2017 ; Chubb et al., 2016 ; Watermeyer, 2014 ). While these reports and studies mainly draw on the methods of philosophy, sociology and management, many of them also allude to changes related to language .

Several publications on impact drew attention to the process of meaning-making around the notion of ‘impact’ in the early stages of its existence. Manville et al. flagged up the necessity for the policy-maker to facilitate the development of common vocabulary to enable a broader ‘cultural shift’ (2015, pp. 16, 26. 37–38, 69). Power wrote of an emerging ‘performance discourse of impact’ (2015, p. 44) while Derrick ( 2018 ) looked at the collective process of defining and delimiting “the ambiguous object” of impact at the stage of panel proceedings. The present paper picks up these observations bringing them together in a unique discursive perspective.

Drawing from linguistic pragmatics and Foucauldian discourse analysis, the paper presents shifts related to the introduction of ‘impact’ as element of evaluation in four stages. These are, in chronological order: (1) the ‘problematisation’ of the notion of ‘impact’ in policy and its appropriation on a local level, (2) the creation of an impact infrastructure to orchestrate practices around impact, (3) the consolidation of a new genre of writing—impact case study, (4) academics’ uptake of the notion of impact and its progressive inclusion in their professional positioning.

Each of these stages is described using theoretical concepts grounded in empirical data. The first stage has to do with the process of ‘problematization’ of a previously non-regulated area, i.e., the process of casting research impact as a ‘problem’ to be addressed and regulated by a set of policy measures. The second stage took place when in rapid response to government policy, new procedures and practices were created within universities, giving rise to an impact ‘infrastructure’ (or ‘apparatus’ in the Foucauldian sense). The third stage is the emergence of a crucial element of the infrastructure—a new genre of academic writing—impact case study. I argue that engaging with the new genre and learning to write impact case studies was key in incorporating ‘impact’ into scholars’ narratives of ‘academic identity’. Hence, the paper presents new practices of ‘subjectivation’ as the fourth stage of incorporation of ‘impact’ into academic discourse. The four stages of the introduction of ‘impact’ into academic discourse are mutually interlinked—each step paves the way for the next.

Of the described four stages, only stage three focuses a classical linguistic task: the description of a new genre of text. The remaining three take a broader view informed by sociology and philosophy, focusing on discursive practices i.e., language used in social context. Other descriptions of the emergence of impact are possible—note for instance Power’s four-fold structure (Power, 2015 ), at points analogous to this study.

Theoretical framework and data

This study builds on a constructivist approach to social phenomena in assuming that language plays a crucial role in establishing and maintaining social practice. In this approach ‘discourse’ is understood as the production of social meaning—or the negotiation of social, political or cultural order—through the means of text and talk (Fairclough, 1989 , 1992 ; Fairclough et al., 1997 ; Gee, 2015 ).

Linguistic pragmatics and Foucauldian approaches to discourse are used to account for the changes related to the rise of ‘impact’ as element of evaluation and discourse on the macro and micro scale. In looking at the micro scale of every-day linguistic practices the analysis makes use of linguistic pragmatics, in particular concepts of positioning (Davies and Harré, 1990 ), stage (Goffman, 1969 ; Robinson, 2013 ), metaphor (Cameron, et al., 2009 ; Musolff, 2004 , 2012 ), as well as genre analysis (Swales, 1990 , 2011 ). Analyzing the macro scale, i.e., the establishment of the concept of ‘impact’ in policy and the creation of an impact infrastructure, it draws on selected concepts of Fouculadian governmentality theory (crucially ‘problematisation’, ‘apparatus’, ‘subjectivation’) (Foucault, 1980 , 1988 , 1990 ; Rose, 1999 , pp. ix–xiii).

While the toolbox of linguistic pragmatics is particularly useful in analyzing linguistic aspects of the datasets, Foucault’s governmental framework helps bring together findings from the two datasets in a broader analysis, allowing more general conclusions on the practices of governing (academic) subjects within evaluation frameworks. Both pragmatic and Foucauldian traditions of discourse analysis have been productively applied in the study of higher education contexts (e.g., Fairclough, 1993 , Gilbert and Mulkey, 1984 , Hyland, 2009 , Myers, 1985 , 1989 ; for an overview see Wróblewska and Angermuller, 2017 ).

The analysis builds on an admittedly heterogenous set of concepts, hailing from different traditions and disciplines. This approach allows for a suitably nuanced description of a broad phenomenon—the discourse of impact—studied here on the basis of two different datasets. To facilitate following the argument, individual theoretical and methodological concepts are defined where they are applied in the analysis.

The studied corpus consists of two datasets: a written and oral one. The written corpus includes 78 impact case studies (CSs) submitted to REF2014 in the discipline of linguistics Footnote 1 . Linguistics was selected as a discipline straddling the social sciences and humanities (SSH). SSH are arguably most challenged by the practice of impact evaluation as they have traditionally resisted subjection to economization and social accountability (Benneworth et al., 2016 ; Bulaitis, 2017 ).

The CSs were downloaded in pdf form from REF’s website: https://www.ref.ac.uk/2014/ . The documents have an identical structure, featuring basic information: name of institution, unit of assessment, title of CS and core content divided into five sections: (1) summary of impact, (2) underpinning research, (3) references to the research, (4) details of impact (5) sources to corroborate impact. Each CS is about 4 pages long (~2400 words). The written dataset (with a word-count of 173,474) was analyzed qualitatively using MAX QDA software with a focus on the generic aspect of the documents.

The oral dataset is composed of semi-structured interviews with authors of the studied CSs ( n  = 20) and other actors involved in the evaluation, including two policy-makers and three academic administrators Footnote 2 . In total, the 25 interviews, each around 60 min long, add up to around 25 h of recordings. The interviews were analyzed in two ways. Firstly, they were coded for themes and topics related to the evaluation process—this was useful for the description of impact infrastructure presented in step 2 of analysis. Secondly, they were considered as a linguistic performance and coded for discursive devices (irony, distancing, metaphor etc.)—this was the basis for findings related to the presentation of one’s ‘academic self’ which are the object of fourth step of analysis. The written corpus allows for an analysis of the functioning of the notion of ‘impact’ in the official, administrative discourse of academia, looking at the emergence of an impact infrastructure and the genre created for the description of impact. The oral dataset in turn sheds light on how academics relate to the notion of impact in informal settings, by focusing on metaphors and pragmatic markers of stage.

The discourse of impact

Problematization of impact.

The introduction of ‘impact’, a new element of evaluation accounting for 20% of the final result, was seen as a surprise and as a significant change in respect to the previous model of evaluation—the Research Assessment Exercise (Warner, 2015 ). The outline of an approach to impact evaluation in REF was developed on the government’s recommendation after a review of international practice in impact assessment (Grant et al., 2009 ). The adopted approach was inspired by the previously-created (but never implemented) Australian RQF framework (Donovan, 2008 ). A pilot evaluation exercise run in 2010 confirmed the viability of the case-study approach to impact evaluation. In July 2011 the Higher Education Council for England (HEFCE) published guidelines regulating the new assessment (HEFCE, 2011 ). The deadline for submissions was set for November 2013.

In the period between July 2011 and November 2013 HEFCE engaged in broad communication and training activities across universities, with the aim of explaining the concept of ‘impact’ and the rules which would govern its evaluation (Power, 2015 , pp. 43–48). Knowledge on the new element of evaluation was articulated and passed down to particular departments, academic administrative staff and individual researchers in a trickle-down process, as explained by a HEFCE policymaker in an account of the run-up to REF2014:

There was no master blue print! There were some ideas, which indeed largely came to pass. But in order to understand where we [HEFCE] might be doing things that were unhelpful and might have adverse outcomes, we had to listen. I was in way over one hundred meetings and talked to thousands of people! (…) [The Impact Agenda] is something that we are doing to universities. Actually, what we wanted to say is: ‘we are doing it with you, you’ve Footnote 3 got to own it’.

Int20, policymaker, example 1 Footnote 4

Due to the importance attributed to the exercise by managers of academic units and the relatively short time for preparing submissions, institutions were responsive to the policy developments. In fact, they actively contributed to the establishment and refinement of concepts related to impact. Institutional learning occurred to a large degree contemporarily to the consolidation of the policy and the refinement of the concepts and definitions related to impact. The initially open, undefined nature of ‘impact’ (“there was no master blue-print”) is described also in accounts of academics who participated in the many rounds of meetings and consultations. See example 2 below:

At that time, they [HEFCE] had not yet come up with this definition [of impact], not yet pinned it down, but they were trying to give an idea of what it was, to get feedback, to get a grip on it. (…) And we realised (…) they didn’t have any more of an idea of this than we did! It was almost like a fishing expedition. (…) I got a sense very early on of, you know, groping.

Int1, academic, example 2

The “pinning down” of an initially fuzzy concept and defining the rules which would come to govern its evaluation was just one aim of the process. The other one was to engage academics and affirm their active role in the policy-making. From an idea which came from outside of the British academic community (from the the government, the research councils) and originally from outside the UK (the Australian RQF exercise), a concept which was imposed on academics (“it is something that we are doing to universities”) the Impact Agenda was to become an accepted, embedded element of the academic life (“you’ve got to own it”). In this sense, the laboriousness of the process, both for the policy-makers and the academics involved, was a necessary price to be paid for the feeling of “ownership” among the academic community. Attitudes of academics, initially quite negative (Chubb et al., 2016 , Watermeyer, 2016 ), changed progressively, as the concept of impact became familiarized and adapted to the pre-existing realities of academic life, as recounted by many of the interviewees, e.g.,:

I think the resentment died down relatively quickly. There was still some resistance. And that was partly academics recognising that they had to [take part in the exercise], they couldn’t ignore it. Partly, the government and the research council has been willing to tweak, amend and qualify the initial very hard-edged guidelines and adapt them for the humanities. So, it was two-way process, a dialogue.

Int16, academic, example 3

The announcement of the final REF regulations (HEFCE, 2011 ) was the climax of the long process of making ‘impact’ into a thinkable and manageable entity. The last iteration of the regulations constituted a co-creation of various actors (initial Australian policymakers of the RQF, HEFCE employees, academics, impact professionals, universities, professional organizations) who had contributed to it at different stages (in many rounds of consultations, workshops, talks and sessions across the country). ‘Impact’ as a notion was ‘talked into being’ in a polyphonic process (Angermuller, 2014a , 2014b ) of debate, critique, consultation (“listening”, “getting feedback”) and adaptation (“tweaking”, “changing”, “amending hard-edged guidelines”) also in view of the pre-existing conditions of academia such as the friction between the ‘soft’ and ‘hard’ sciences (as mentioned in example 3). In effect, impact was constituted as an object of thought, and an area of academic activity begun to emerge around it.

The period of defining ‘impact’ as a new, important notion in academic discourse in the UK, roughly between July 2011 and November 2013, can be conceptualized in terms of the Foucauldian notion of ‘problematization’. This concept describes how spaces, areas of activity, persons, behaviors or practices become targeted by government, separated from others, and cast as ‘problems’ to be addressed with a set of techniques and regulations. ‘Problematisation’ is the moment when a notion “enters into the play of true and false, (…) is constituted as an object of thought (whether in the form of moral reflection, scientific knowledge, political analysis, etc.)” (Foucault, 1988 , p. 257), when it “enters into the field of meaning” (Foucault, 1984 , pp. 84–86). The problematization of an area triggers not only the establishment of new notions and objects but also of new practices and institutions. In consequence, the areas in question become subjugated to a new (political, administrative, financial) domination. This eventually shapes the way in which social subjects conceive of their world and of themselves. But a ‘problematisation’, however influential, cannot persist on its own. It requires an overarching structure in the form of an ‘apparatus’ which will consolidate and perpetuate it.

Impact infrastructure

Soon after the publication of the evaluation guidelines for REF2014, and still during the phase of ‘problematisation’ of impact, universities started collecting data on ‘impactful’ research conducted in their departments and recruiting authors of potential CSs which could be submitted for evaluation. The winding and iterative nature of the process of problematization of ‘impact’ made it difficult for research managers and researchers to keep track of the emerging knowledge around impact (official HEFCE documentation, results of the pilot evaluation, FAQs, workshops and sessions organized around the country, writings published in paper and online). At the stage of collecting drafts of CSs it was still unclear what would ‘count’ as impact and what evidence would be required. Hence, there emerged a need for specific procedures and specialized staff who would prepare the REF submissions.

At most institutions, specific posts were created for employees preparing impact submissions for REF2014. These were both secondment positions such as ‘impact lead’, ‘impact champion’ and full-time ones such as impact officer, impact manager. These professionals soon started organizing between themselves at meetings and workshops. Administrative units focused on impact (such as centers for impact and engagement, offices for impact and innovation) were created at many institutions. A body of knowledge on impact evaluation was soon consolidated, along with a specific vocabulary (‘a REF-able piece of research’, ‘pathways to impact’, ‘REF-readiness’ etc.) and sets of resources. Impact evaluation gave raise to the creation of a new type of specialized university employee, who in turn contributed to turning the ‘generation of impact’, as well as the collection and presentation of related data into a veritable field of professional expertize.

In order to ensure timely delivery of CSs to REF2014, institutions established fixed procedures related to the new practice of impact evaluation (periodic monitoring of impact, reporting on impact-related activities), frames (schedules, document templates), forms of knowledge transfer (workshops on impact generation or on writing in the CS genre), data systems and repositories for logging and storing impact-related data, and finally awards and grants for those with achievements (or potential) related to impact. Consultancy companies started offering commercial services focused on research impact, catering to universities and university departments but also to governments and research councils outside the UK looking at solutions for impact evaluation. There is even an online portal with a specific focus on showcasing researchers’ impact (Impact Story).

In consequence, impact became institutionalized as yet another “box to be ticked” on the list of academic achievements, another component of “academic excellence”. Alongside burdens connected to reporting on impact and following regulations in the area, there came also rewards. The rise of impact as a new (or newly-problematised) area of academic life opened up uncharted areas to be explored and opportunities for those who wished to prove themselves. These included jobs for those who had acquired (or could claim) expertize in the area of impact (Donovan, 2017 , p. 3) and research avenues for those studying higher education and evaluation (after all, entirely new evaluation practices rarely emerge, as stressed by Power, 2015 , p. 43). While much writing on the Impact Agenda highlights negative attitudes towards the exercise (Chubb et al., 2016 ; Sayer, 2015 ), equally worth noting are the opportunities that the establishment of a new element of the exercise opened. It is the energy of all those who engage with the concept (even in a critical way) that contributes to making it visible, real and robust.

The establishment of a specialized vocabulary, of formalized requirements and procedures, the creation of dedicated impact-related positions and departments, etc. contribute to the establishment of what can be described as an ‘impact infrastructure’ (comp. Power, 2015 , p. 50) or in terms of Foucauldian governmentality theory as an ‘apparatus’ Footnote 5 . In Foucault’s terminology, ‘apparatus’ refers to a formation which encompasses the entirety of organizing practices (rituals, mechanisms, technologies) but also assumptions, expectations and values. It is the system of relations established between discursive and non-discursive elements as diverse as “institutions, architectural forms, regulatory decisions, laws, administrative measures, scientific statements, philosophical, moral and philanthropic propositions” (Foucault, 1980 , p. 194). An apparatus servers a specific strategic function—responding to an urgent need which arises in a concrete time in history—for instance, regulating the behavior of a population.

There is a crucial discursive element to all the elements of the ‘impact apparatus’. While the creation of organizational units and jobs, the establishment of procedures and regulations, participation in meetings and workshops are no doubt ‘hard facts’ of academic life, they are nevertheless brought about and made real in discursive acts of naming, defining, delimiting and evaluating. The aim of the apparatus was to support the newly-established problematization of impact. It did so by operating on many levels: first of all, and most visibly, newly-established procedures enabled a timely and organized submission to the upcoming REF. Secondly, the apparatus guided the behavior of social actors. It did so not only through directive methods (enforcing impact-related requirements) but also through nurturing attitudes and dispositions which are necessary for the notion of impact to take root in academia (for instance via impact training delivered to early-career scholars).

Interviewed actors involved in implementing the policy in institutions recognized their role in orchestrating collective learning. An interviewed impact officer stated:

My feeling is that ultimately my post should not exist. In ten or fifteen years’ time, impact officers should have embedded the message [about impact] firmly enough that they [researchers] don’t need us anymore.

Int7, impact officer, example 4

A similar vision was evoked by a HEFCE policymaker who was asked if the notion of impact had become embedded in academic institutions:

I hope [after the next edition of REF] we will be able to say that it has become embedded. I think the question then will be “have we done enough in terms of case studies? Do we need something very much lighter-touch?” “Do we need anything at all?”—that’s a question. (…) If [impact] is embedded you don’t need to talk about it.

Int20, policy-maker, example 5

Rather than being an aim in itself, the Impact Agenda is a means of altering academic culture so that institutions and individual researchers become more mindful of the societal impacts of their research. The instillment of a “new impact culture” (see Manville et al., 2014 , pp. 24–29) would ensure that academic subjects consider the question of ‘impact’ even outside of the framework of REF. The “culture shift” is to occur not just within institutions but ultimately within the subjects—it is in them that the notion of ‘impact’ has to become embedded. Hence, the final purpose of the apparatus would be to obscure the origins of the notion of ‘impact’ and the related practices, neutralizing the notion itself, and giving a guise of necessity to an evaluative reality which in fact is new and contingent.

The genre of impact case study as element of infrastructure

In this section two questions are addressed: (1) what are the features of the genre (or what is it like?) and (2) what are the functions of the genre (or what does it do? what vision of research does it instil?). In addressing the first question, I look at narrative patterns, as well as lexical and grammatical features of the genre. This part of the study draws on classical genre analysis (Bhatia, 1993 ; Swales, 1998 ) Footnote 6 . The second question builds on the recognition, present in discourse studies since the 1970s’, that genres are not merely classes of texts with similar properties, but also veritable ‘dispositives of communication’. A genre is a means of articulation of legitimate speech; it does not just represent facts or reflect ideologies, it also acts on and alters the context in which it operates (Maingueneau, 2010 , pp. 6–7). This awareness has engendered broader sociological approaches to genre which include their pragmatic functioning in institutional realities (Swales, 1998 ).

The genre of CS differs from other academic genres in that it did not emerge organically, but was established with a set of guidelines and a document template at a precise moment in time. The genre is partly reproductive, as it recycles existing patterns of academic texts, such as journal article, grant application, annual review, as well as case study templates applied elsewhere. The studied corpus is strikingly uniform, testifying to an established command of the genre amongst submitting authors. Identical expressions are used to describe impact across the corpus. Only very rarely is non-standard vocabulary used (e.g., “horizontal” and “vertical” impact rather then “reach” and “significance” of impact). This coherence can be contrasted with a much more diversified corpus of impact CSs submitted in Norway to an analogous exercise (Wróblewska, 2019 ). The rapid consolidation of the genre in British academia can be attributed to the perceived importance of impact evaluation exercise, which lead to the establishment of an impact infrastructure, with dedicated employees tasked with instilling the ‘culture of impact’.

In its nature, the CS is a performative, persuasive genre—its purpose is to convince the ‘ideal readers’ (the evaluators) of the quality of the underpinning research and the ‘breadth and significance’ of the described impact. The main characteristics of the genre stem directly from its persuasive aim. These are discussed below in terms of narrative patterns, and grammatical and lexical features.

Narrative patterns

On the level of narrative, there is an observable reliance on a generic pattern of story-telling frequent in fiction genres, such as myths or legends, namely the Situation-Problem–Response–Evaluation (SPRE) structure (also known as the Problem-Solution pattern, see Hoey, 1994 , 2001 pp. 123–124). This is a well-known narrative which follows the SPRE pattern: a mountain ruled by a dragon (situation) which threats the neighboring town (problem) is sieged by a group of heroes (response), to lead to a happy ending or a new adventure (evaluation). Compare this to an example of the SPRE pattern in a sample impact narrative from the studied corpus:

Mosetén is an endangered language spoken by approximately 800 indigenous people (…) (SITUATION). Many Mosetén children only learn the majority language, Spanish (PROBLEM). Research at [University] has resulted in the development of language materials for the Mosetenes. (…) (RESPONSE). It has therefore had a direct influence in avoiding linguistic and cultural loss. (EVALUATION).

CS40828 Footnote 7

The SPRE pattern is complemented by patterns of Further Impact and Further Corroboration. The first one allows elaborating the narrative, e.g., by showing additional (positive) outcomes, so that the impact is not presented as an isolated event, but rather as the beginning of a series of collaborations, e.g.,:

The research was published in [outlet] (…). This led to an invitation from the United Nations Environment Programme for [researcher](FURTHER IMPACT).

Patterns of ‘further impact’ are often built around linking words, such as: “X led to” ( n  = 78) Footnote 8 , “as a result” ( n in the corpus =31), “leading to” ( n  = 24), “resulting in” ( n  = 13), “followed” (“X followed Y”– n  = 14). Figure 1 below shows a ‘word tree’ for a frequent linking structure “led to”. The size of the terms in the diagram represents frequencies of terms in the corpus. Reading the word tree from left to right enables following typical sentence structures built around the ‘led to’ phrase: research led to an impact (fundamental change/development/establishment/production of…); impact “led to” further impact.

Word tree with string ‘led to'. This word tree with string ‘led to’ was prepared with MaxQDA software. It visualises a frequent sentence structure where research led to impact (fundamental change/ development/ establishment/ production of…) or otherwise how impact “led to” further impact.

The ‘Further Corroboration’ pattern provides additional information which strengthens the previously provided corroborative material:

(T)he book has been used on the (…) course rated outstanding by Ofsted, at the University [Name](FURTHER CORROBORATION).

Grammatical and lexical features

Both on a grammatical and lexical level, there is a visible focus on numbers and size. In making the point on the breadth and significance of impact, CS authors frequently showcase (high) numbers related to the research audience (numbers of copies sold, audience sizes, downloads but also, increasingly, tweets, likes, Facebook friends and followers). Adjectives used in the CSs appear frequently in the superlative or with modifiers which intensify them: “Professor [name] undertook a major Footnote 9 ESRC funded project”; “[the database] now hosts one of the world’s largest and richest collections (…) of corpora”; “work which meets the highest standards of international lexicographical practice”; “this experience (…) is extremely empowering for local communities”, “Reach: Worldwide and huge ”.

Use of ‘positive words’ constitutes part of the same phenomenon. These appear often in the main narrative on research and impact, and even more frequently in quoted testimonials. Research is described in the CSs as being new, unique and important with the use of words such as “innovative” ( n  = 29), “influential” ( n  = 16), “outstanding” ( n  = 12), “novel” ( n  = 10), “excellent” ( n  = 8), “ground-breaking” ( n  = 7), “tremendous” ( n  = 4), “path-breaking” ( n  = 2), etc. The same qualities are also rendered descriptively, with the use of words that can be qualified as boosters e.g., “[the research] has enabled a complete rethink of the relationship between [areas]”; “ vitally important [research]”.

Novelty of research is also frequently highlighted with the adjective “first” appearing in the corpus 70 times Footnote 10 . While in itself “first” is not positive or negative, it carries a big charge in the academic world where primacy of discovery is key. Authors often boast about having for the first time produced a type of research—“this was the first handbook of discourse studies written”…, studied a particular area—“This is the first text-oriented discourse analytic study”…, compiled a type of data—“[We] provid[ed] for the first time reliable nationwide data”; “[the] project created the first on-line database of…”, or proven a thesis: “this research was the first to show that”…

Another striking lexical characteristic of the CSs is the presence of fixed expressions in the narrative on research impact. I refer to these as ‘impact speak’. There are several collocations with ‘impact’, the most frequent being “impact on” ( n  = 103) followed by the ‘type’ of impact achieved (impact on knowledge), area/topic (impact on curricula) or audience (Impact on Professional Interpreters). This collocation often includes qualifiers of impact such as “significant”, “wide”, “primary”,“secondary”, “broader”, “key”, and boosters: great, positive, wide, notable, substantial, worldwide, major, fundamental, immense etc. Impact featured in the corpus also as a transitive verb ( n  = 22) in the forms “impacted” and “impacting”—e.g., “[research] has (…) impacted on public values and discourse”. This is interesting, as use of ‘impact’ as a verb is still often considered colloquial. Verb collocations with ‘impact’ are connected to achieving influence (“lead to..”, “maximize…”, “deliver impact”) and proving the existence and quality of impact (“to claim”, “to corroborate” impact, “to vouch for” impact, “to confirm” impact, to “give evidence” for impact). Another salient collocation is “pathways to impact” ( n  = 14), an expression describing channels of interacting with the public, in the corpus occasionally shortened to just “pathways” e.g., “The pathways have been primarily via consultancy”. This phrase has most likely made its way to the genre of CS from the Research Councils UK ‘Pathways to Impact’ format introduced as part of grant applications in 2009 (discontinued in early 2020).

On a syntactic level, CSs are rich in parallel constructions of enumeration, for instance: “ (t)ranslators, lawyers, schools, colleges and the wider public of Welsh speakers are among (…) users [of research]”; “the research has benefited a broad, international user base including endangered language speakers and community members, language activists, poets and others ”; [the users of the research come] “from various countries including India, Turkey, China, South Korea, Venezuela, Uzbekistan, and Japan ”. Listing, alongside providing figures, is one of the standard ways of signaling the breadth and significance of impact. Both lists and superlatives support the persuasive function of the genre. In terms of verbal forms, passive verbs are clearly favored and personal pronouns (“I, we”) are avoided: “research was conducted”, “advice was provided”, “contracts were undertaken”.

Vision of research promoted by the genre of CS

Impact CS is a new, influential genre which affects its academic context by celebrating and inviting a particular vision of successful research and impact. It sets a standard for capturing and describing a newly-problematized academic object. This standard will be a point of reference for future authors of CSs. Hence, it is worth taking a look at the vision on research it instills.

The SPRE pattern used in the studied CSs favors a vision of research that is linear: work proceeds from research question to results without interference. The Situation and Problem elements are underplayed in favor of elaborate descriptions of the researchers’ ‘Reactions’ (research and outreach/impact activities) and flattering ‘Evaluations’ (descriptions of effects of the research and data supporting these claims). Most narratives are devoid of challenges (the ‘Problem’ element is underplayed, possible drawbacks and failures in the research process are mentioned sporadically). Furthermore, narratives are clearly goal-oriented: impact is shown as included in the research design from the beginning (e.g., impact is frequently mentioned already in section 2 ‘Underpinning research’, rather than the latter one ‘Details of the impact’). Elements of chance, luck, serendipity in the research process are erased—this is reinforced by the presence of patterns of ‘further proof’ and ‘further corroboration’. As such, the bulk of studied CSs channel a vision of what is referred to in Science Studies as ‘normal’ (deterministic, linear) science (Kuhn, 1970 , pp. 10–42). From a purely literary perspective this makes for rather dull narratives: “fairy-tales of researcher-heroes… but with no dragons to be slain” (Selby, 2016 ).

The few CSs which do discuss obstacles in the research process or in securing impact stand out as strikingly diverse from the rest of the corpus. Paradoxically, while apparently ‘weakening’ the argumentation, they render it more engaging and convincing. This effect has been observed also in in an analogous corpus of Norwegian CSs which tend to problematize the pathway from research to impact to a much higher degree (Wróblewska, 2019 , pp. 34–35).

The lexical and grammatical features of the CSs—the proliferation of ‘positive words’, including superlatives, and the adjective “first”— contribute to an idealization of the research process. The documents channel a vision of academia where there is no place for simply ‘good’ research—all CSs seem based on ‘excellent’ and ‘ground-breaking’ projects. The quality of research underpinning impact is recognized in CSs in a straightforward, simplistic way (quotation numbers, peer reviewed papers, publications in top journals, submission to REF), which contributes to normalizing the view of research quality as easily measurable. Similarly, testimonials related to impact are not all equal. Sources of corroboration cited in CSs were carefully selected to appear prestigious and trustworthy. Testimonials and statements from high-ranking officials (but also ‘celebrities’ such as famous intellectuals or political leaders) were particularly sought-after. The end effect reinforces a solidified vision of a hierarchy of worth and trustworthiness in academia.

The prevalence of impersonal verbal forms suggests an de-personalized vision of the research process (“work was conducted”, “papers were published”, “evidence was given…”), where individual factors such as personal aspirations, constraints or ambitions are effaced. The importance given to numbers contributes to a strengthening of a ‘quantifiable’ idea of impact. This is in line with a trend observed in academic writing in general – the inflation of ‘positive words’ (boosters and superlatives) (Vinkers et al., 2015 ). This tendency is amplified in the genre of CS, particularly in its British iteration. In a Norwegian corpus claims to excellence of research and breadth and significance of impact were significantly more modest (Wróblewska, 2019 , pp. 28–30).

The genre of impact CS is a core binding component of the impact infrastructure: all the remaining elements of this formation are mutually connected by a common aim – the generation of CSs. While the CS genre, together with the encompassing impact infrastructure, is vested with a seductive/coercive force, the subjects whose work it represents and who produce it take different positions in its face.

Academics’ positioning towards the Impact Agenda

Academics position themselves towards the concept of impact in many explicit and implicit ways. ‘Positioning’ is understood here as performance-based claims to identity and subjectivity (Davies and Harré, 1990 , Harré and Van Langenhove, 1998 ). Rejecting the idea of stable “inherent” identities, positioning theorists stress how different roles are invoked and enacted in a continuous game of positioning (oneself) and being positioned (by others). Positioning in academic contexts may take the form of indexing identities such as “professor”, “linguist”, “research manager”, “SSH scholar”, “intellectual”, “maverick” etc. (Angermuller, 2013 ; Baert, 2012 , Hamann, 2016 , Hah, 2019 , 2020 ). Also many daily interactions which do not include explicit identity claims involve subject positioning, as they carry value judgments, thereby also evoking counter-statements and colliding social contexts (Tirado and Galvaz, 2008 , pp. 32–45).

My analysis draws attention to the process of incorporating impact into academic subjectivities. I look firstly at the mechanics of academics’ positioning towards impact: the game of opposite discursive acts of distancing and endorsement. Academics reject the notion of ‘impact’ by ironizing, stage management and use of metaphors. Conversely, they may actively incorporate impact into their presentation of academic ‘self’. This discursive engagement with the notion of impact can be described as ‘subjectivation’, i.e., the process whereby subjects re(establish) themselves in relation to the grid of power/knowledge in which they function (in this case the emergent ‘impact infrastructure’).

The relatively high response rate of this study (~50%) and the visible eagerness of respondents to discuss the question of impact suggest an emotional response of academics to the topic of impact evaluation. Yet, respondents visibly struggled with the notion of ‘impact’, often distancing themselves from it through discursive devices, the most salient being ironizing, use of metaphors and stage management.

Ironizing the notion of impact

In many cases, before proceeding to explain their attitude to impact, interviewed academics elaborated on the notion of impact, explaining how the notion applied to their discipline or field and what it meant for them personally. This often meant rejecting the official definition of impact or redefining the concept. In excerpt 6, the interviewee picks up the notion:

Impact… I don’t even like the word! (…) It sounds [like] a very aggressive word, you know, impact, impact ! I don’t want to imp act ! What you want, and what has happened with [my research] really is… more of a dialogue.

Int21, academic, example 6

Another respondent brought up the notion of impact when discussing ethical challenges arising from public dissemination of research.

When you manage to go through that and navigate successfully, and keep producing research, to be honest, that’s impact for me.

Int9, academic, example 7

An analogous distinction was made by a third respondent who discussed the effect of his work on an area of professional activity. While, as he explained, this application of his research has been a source of personal satisfaction, he refused to describe his work in terms of ‘impact’. He stressed that the type of influence he aims for does not lend itself to producing a CS (is not ‘REF-able’):

That’s not impact in the way this government wants it! Cause I have no evidence. I just changed someone’s view. Is that impact? Yes, for me it is. But it is not impact as understood by the bloody REF.

Int3, academic, example 8

These are but three examples of many in the studied corpus where speakers take up the notion of impact to redefine or nuance it, often juxtaposing it with adjacent notions of public engagement, dissemination, outreach, social responsibility, activism etc. A previous section highlighted how the definition of impact was collectively constructed by a community in a process of problematization. The above-cited examples illustrate the reverse of this phenomenon—namely, how individual social actors actively relate to an existing notion in a process of denying, re-defining, and delimiting.

These opposite tendencies of narrowing down and again widening a definition are in line with the theory of the double role of descriptions in discourse. Definitions are both constructions and constructive —while they are effects of discourse, they can also become ‘building blocks’ for ideas, identities and attitudes (Potter, 1996 , p. 99). By participating in impact-related workshops academics ‘reify’ the existing, official definition by enacting it within the impact infrastructure. Fragments cited above exemplify the opposite strategy of undermining the adequacy of the description or ‘ironizing’ the notion (Ibid, p.107). The tension between reifying and ironizing points to the winding, conflictual nature of the process of accepting and endorsing the new ‘culture of impact’. A recognition of the multiple meanings given to the notion of ‘impact’ by policy-makers, academic managers and scholars may caution us in relation to studies on attitudes towards impact which take the notion at face value.

Respondents nuanced the notion of impact also through the use of metaphors. In discourse analysis metaphors are seen in not just as stylistic devices but as vehicles for attitudes and values (Mussolf, 2004 , 2012 ). Many of the respondents make remarks on the ‘realness’ or ‘seriousness’ of the exercise, emphasizing its conventional, artificial nature. Interviewees admitted that claims made in the CSs tend to be exaggerated. At the same time, they stressed that this was in line with the convention of the genre, the nature of which was clear for authors and panelists alike. The practice of impact evaluation was frequently represented metaphorically as a game. See excerpt 9 below:

To be perfectly honest, I view the REF and all of this sort of regulatory mechanisms as something of a game that everybody has to play. The motivation [to submit to REF] was really: if they are going to make us jump through that hoop, we are clever enough to jump through any hoops that any politician can set.

Int14, academic, example 9

Regarding the relation of the narratives in the CSs to truth see example 10:

[A CS] is creative stuff. Given that this is anonymous, I can say that it’s just creative fiction. I wouldn’t say we [authors of CSs] lie, because we don’t, but we kind of… spin. We try to show a reality which, by some stretch of imagination is there. (It’s) a truth. I’m not lying. Can it be shown in different ways? Yes, it can, and then it would be possibly less. But I choose, for obvious reasons, to say that my external funding is X million, which is a truth.

Int3, academic, example 10

The metaphors of “playing a game”, “jumping through hoops” suggest a competition which one does not enter voluntarily (“everybody has to play it”) while those of “creative fiction”, “spinning”, presenting “ a truth” point to an element of power struggle over defining the rules of the game. Doing well in the exercise can mean outsmarting those who establish the framework (politicians) by “performing” particularly well. This can be achieved by eagerly fulfilling the requirements of the genre of CS, and at the same time maintaining a disengaged position from the “regulatory mechanism” of the impact infrastructure.

Stage management

Academics’ positioning towards impact plays out also through management of ‘stage’ of discursive performance, often taking the form of frontstage and backstage markers (in the sense of Goffman’s dramaturgy–1969, pp. 92–122). For instance, references to the confidential nature of the interview (see example 10 above) or the expression “to be perfectly honest” (example 9), are backstage markers. Most of the study’s participants have authored narratives about their work in the strict, formalized genre of CS, thereby performing on the Goffmanian ‘front stage’ for an audience composed of senior management, REF panelists and, ultimately, perhaps “politicians”, “the government”. However, when speaking on the ‘back stage’ context of an anonymous interview, many researchers actively reject the accuracy of the submitted CSs as representations of their work. Many express a nuanced, often critical, view on impact.

Respondents frequently differentiate between the way they perceive ‘impact’ on different ‘levels’, or from the viewpoint of their different ‘roles’ (scholar, research manager, citizen…). One academic can hold different (even contradictory) views on the assessment of impact. Someone who strongly criticizes the Impact Agenda as an administrative practice might be supportive of ‘impact’ on a personal level or vice versa. See the answer of a linguist asked whether ‘impact’ enters into play when he assesses the work of other academics:

When I look at other people’s work work as a linguist, I don’t worry about that stuff. (…) As an administrator, I think that linguistics, like many sciences, has neglected the public. (…) At some point, when we would be talking about promotion (…) I would want to take a look at the impact of their work. (…) And that would come into my thinking in different times.

Int13, academic, example 11

Interestingly, in the studied corpus there isn’t a simple correlation between conducting research which easily ‘lends itself to impact’ and a positive overall attitude to impact evaluation.

Subjectivation

The most interesting data excerpts in this study are perhaps the ones where respondents wittingly or unwittingly expose their hesitations, uncertainties and struggles in positioning themselves towards the concept of impact. In theoretical terms, these can be interpreted as symptoms of an ongoing process of ‘subjectivation’.

‘Subjectivation’ is another concept rooted in Foucauldian governmentality theory. According to Foucault, individuals come to the ‘truth’ about their subjectivity by actively relating to a pre-existent set of codes, patterns, rules and rituals suggested by their culture or social group (Castellani, 1999 , pp. 257–258; Foucault, 1988 , p. 11). The term ‘subjectivation’ refers to the process in which an individual establishes oneself in relation to the grid of power/knowledge in which they function. This includes actions subjects take on their performance, competences, attitudes, self-esteem, desires etc. in order to improve, regulate or reform themselves (Dean, 1999 , p. 20; Lemke, 2002 ; Rose, 1999 , p. xii).

Academics often distance themselves from the assessment exercise, as shown in previous sections. And yet, the data hints that having taken part in the evaluation and engaged with the impact infrastructure was not without influence on the way they present their research, also in nonofficial, non-evaluative contexts, such as the research interview. This effect is visible in vocabulary choices—interviewees routinely spoke about ‘pathways to impact’, ‘impact generation’, ‘REF-ability’ etc. ‘Impact speak’ has made its way into every-day, casual academic conversations. Beyond changes to vocabulary, there is a more deep-running process—the discursive work of reframing one’s research in view of the evaluation exercise and in its terms. Many respondents seemed to adjust the presentation of their research, its focus and aims, when the topic of REF surfaced in the exchange. Interestingly, such shifts occurred even in the case of respondents who did not submit to the exercise, for instance because they were already retired, or because they refused to take part in it. For those who have submitted CSs to REF, the effect of having re-framed the narrative of their research in this new genre often had a tremendous effect.

Below presented is the example of a scholar who did not initially volunteer to submit a CS, and was reluctant to take part when she was encouraged by a supervisor. During the interview the respondent distanced herself from the exercise and the concept of impact through the discursive devices of ironizing, metaphors, stage management, and humor. The respondent was consistently critical towards impact in course of the interview. Therefore the researcher expected a firm negative answer to the final question: “did the exercise affect your perception of your work?”. See excerpt 13 below for her the respondent’s somewhat surprising answer.

Do you know what? It did, it did, it did. Almost a kind of a massive influence it had. Maybe this is the answer that you didn’t see coming ((laughing)). (…) It did [have an influence] but maybe from a different route as for people who were signed up for [the REF submission] from the outset. (…) When I saw this [CS narrative] being shaped up and people [who gave testimonies] I kind of thought: goodness me! And there were other moving things.

Int21, academic, example 13

Through the preparation of the CS and particularly through familiarizing herself with the underpinning testimonials, the respondent gained greater awareness of an area of practice which was influenced by her research. The interviewee’s attitude changed not only in the course of the evaluation exercise, but also—as if mirroring this process—during the interview. In both cases, elements which were up to that moment implicit (the response of end-users of the work, the researcher’s own emotional response to the exercise and to the written-up narrative of her impact) were made explicit. It is the process of recounting one’s story in a different framework, according to other norms and values (and in a different genre) that triggers the process of subjectivation. This example of a change of attitude in an initially reluctant subject demonstrates the difficulty in opposing the overwhelming force of the impact infrastructure, particularly in view of the (sometimes unexpected) rewards that it offers.

Many respondents found taking part in the REF submission—including the discursive work on the narrative of their research—an exhausting experience. In some cases however, the process of reshaping one’s academic identity triggered by the Agenda was a welcome development. Several interviewees claimed that the exercise valorized their extra-academic involvement which previously went unnoticed at their department. These scholars embraced the genre of CS as an opportunity to present their impact-related activities as an inherent part of their academic work. One academic stated:

At last, I can take my academic identity and my activist identity and roll them up into one.

Int11, academic, example 14

Existing studies have focused on situating academics’ attitudes towards the Impact Agenda on a positive-negative scale (e.g., Chubb et al., 2016 ), and studied divergences depending on career stage or disciplinary affiliation etc. (Chikoore, 2016 ; Chikoore and Probets, 2016 ; Weinstein et al., 2019 ). My data shows that there are many dimensions to each academic’s view of impact. Scholars have complex (sometimes even contradictory) views on ‘impact’ and the discursive work in incorporating impact into a coherent academic ‘self’ is ongoing. While an often overwhelming ‘impact infrastructure’ looms over professional discursive positioning practices, academic subjects are by no means passive recipients of governmental new-managerial policies. On the contrary, they are agents actively involved in accepting, rejecting and negotiating them on a local level—both in front-stage and back-stage contexts.

Looking at the front stage, most CSs seem compliant in their eagerness to demonstrate impact in all its breadth and significance. The documents showcase large numbers and data once considered trivial in the academic context (Facebook likes, Twitter followers, endorsement of celebrities…) and faithfully follow the policy documents in adopting ‘impact speak’. Interviews with academics paint a different picture: the respondents may be playing according to the rules of the evaluation “game”, but they are playing consciously , often in an emotionally detached, distanced manner. Other scholars adjust to the regulations, but not in the name of compliance, but in view of an alignment between the goals of the Agenda and their personal ones. Finally, some academics perceive the evaluation of impact as an opportunity to re-position themselves professionally or re-claim areas of activity which were long considered non-essential for an academic career, like public engagement, outreach and activism.

Concluding remarks

The initial, dynamic phases of the introduction of impact to British academia represent, in terms of Foucauldian theory, the phase of ‘emergence’. This notion draws attention to the moment when discursive concepts (‘impact’, ‘impact case study’…) surface and consolidate. It is in these terms that the previously non-regulated area of academic activity will be thereon described, assessed, evaluated. New notions, definitions, procedures related to impact and the genre of CS will continue to circulate, emerging in other evaluation exercises, at other institutions, in other countries.

The stage of emergence is characterized by a struggle of forces, an often violent conflict between opposing ideas—“it is their eruption, the leap from the wings to centre stage” (Foucault, 1984 , p. 84). The shape that an emergent idea will eventually take is the effect of clashes of these forces and it does not fully depend on any of them. Importantly, emergence is merely “the entry of forces” (p. 84), and “not the final term of historical development” (p. 83). For Foucault, a concept, in its inception, is essentially an empty word, which addresses the needs of a field that is being problematized and satisfies the powers which target it. A problematization (of an object, practice, area of activity) is a response to particular desires or problems—these constitute an instigation, but do not determine the shape of the problematization. As Foucault urges “to one single set of difficulties, several responses can be made” (2003, p. 24).

With the emergence of the Impact Agenda, an area of activity which has always existed (the collaboration of academics with the non-academic world) was targeted, delimited and described with new notions in a process of problematization. The notion of ‘impact’ together with the genre created for capturing it became the core of an administrative machinery—the impact infrastructure. This was a new reality that academics had to quickly come to terms with, positioning themselves towards it in a process of subjectification.

The run-up to REF2014 was a crucial and defining phase, but it was only the first stage of a longer process—the emergence of the concept of ‘impact’, the establishment of basic rules which would govern its generation, documentation, evaluation. Let’s recall Foucault’s argument that “rules are empty in themselves, violent and unfinalized; they are impersonal and can be bent to any purpose. The successes of history belong to those who are capable of seizing these rules”… (pp. 85–86). The rules embodied in the REF guidelines, the new genre of CS, the principals of ‘impact speak’ were in the first instance still “empty and unfinalized”. It was up to those subject to the rules to fill them with meaning.

The data analyzed in this study shows that despite dealing with a new powerful problematization and functioning in the framework of a complex infrastructure, academics continue to be active and highly reflective subjects, who discursively negotiate key concepts of the impact infrastructure and their own position within it. It will be fascinating to study the emergence of analogous evaluation systems in other countries and institutions. ‘Impact infrastructure’ and ‘genre’ are two excellent starting points for an analysis of ensuing changes to academic realities and subjectivities.

Data availability

The interview data analyzed in this paper is not publicly available, due to the confidential nature of the interview data. It can be made available by the corresponding author in anonymised form on reasonable request. The cited case studies were sourced from the REF database ( https://www.ref.ac.uk/2014/ ) and may be consulted online. The coded dataset is considered part of the analysis (and hence protected by copyright), but may be made available on reasonable request.

Most of the studied documents—71 CSs—have been submitted to the Unit of Assessment (UoA) 28—Linguistics and Modern Languages, the remaining seven have been submitted to five different UoAs but fall under the field of linguistics.

Some interviewees were involved in REF in more than just one role. ‘Authors’ of CSs authored the documents to a different degree, some (no = 5) were also engaged in the evaluation process in managerial roles.

Words underlined in interview excerpts were stressed by the speaker.

When citing interview data I give numbers attributed to individual interviews in the corpus, type of interviewee, and number of cited example.

‘Apparatus’ is one of the existing translations of the French ‘dispositif’, another one is ‘historical construct’ (Sembou, 2015 , p. 38) or ‘grid of intelligibility’ (Dreyfus and Rabinow, 1983 , p. 121). The French original is also sometimes used in English texts. In this paper, I use ‘apparatus’ and ‘infrastructure’, as the notion of ‘infrastructure’ has already become current in referring to resources dedicated to impact generation at universities, both in scholarly literature (Power, 2015 ) and in managerial ‘impact speak’.

A full version of the analysis may be found in Wróblewska, 2018 .

CS numbers are those found in the REF impact case study base: https://impact.ref.ac.uk/casestudies/ . I only provide CS numbers for cited fragments of one sentence or longer; exact sources for cited phrases may be given on request or easily identified in the CS database.

The figures given for appearances of certain elements of the genre in the studied corpus are drawn from the computer-assisted qualitative analysis conducted with MaxQDA software. They serve as an illustration of the relative frequency of particular elements for the reader, but since they are not the result of a rigorous corpus analytical study of a larger body of CSs, the researcher does can not claim statistical relevance.

Words underlined in CS excerpts are emphasized by the author of the analysis.

Number of occurrences of string ‘the first’ in the context of quality of research, excluding phrases like “the first workshop took place…” etc.

Angermuller J (2013) How to become an academic philosopher: academic discourse as a multileveled positioning practice. Sociol Hist 2:263–289

Google Scholar  

Angermuller J (2014a) Poststructuralist discourse analysis. subjectivity in enunciative pragmatics. Palgrave Macmillan, Houndmills/Basingstoke

Angermuller J (2014b) Subject positions in polyphonic discourse. In:Angermuller J, Maingueneau D, Wodak R (eds) The Discourse Studies Reader. Main currents in theory and analysis. John Benjamins Publishing Company, Amsterdam/Philadelphia, p 176–186

Baert P (2012) Positioning theory and intellectual interventions. J Theory Soc Behav 42(3):304–324

Article   Google Scholar  

Benneworth P, Gulbrandsen M, Hazelkorn E (2016) The impact and future of arts and humanities research. Palgrave Macmillan, London

Book   Google Scholar  

Bhatia VK (1993) Analysing genre: language use in professional settings. Longman, London

Bulaitis Z (2017) Measuring impact in the humanities: Learning from accountability and economics in a contemporary history of cultural value. Pal Commun 3(7). https://doi.org/10.1057/s41599-017-0002-7

Cameron L, Maslen R, Todd Z, Maule J, Stratton P, Stanley N (2009) The discourse dynamics approach to metaphor and metaphor-led discourse analysis. Metaphor Symbol 24(2):63–89. https://doi.org/10.1080/10926480902830821

Castellani B (1999) Michel Foucault and symbolic interactionism: the making of a new theory of interaction. Stud Symbolic Interact 22:247–272

Chikoore L (2016) Perceptions, motivations and behaviours towards ‘research impact’: a cross-disciplinary perspective. Loughborough University. Loughborough University Institutional Repository. https://dspace.lboro.ac.uk/2134/22942 . Accessed 30 Dec 2020

Chikoore L, Probets S (2016) How are UK academics engaging the public with their research? a cross-disciplinary perspective. High Educ Q 70(2):145–169. https://doi.org/10.1111/hequ.12088

Chubb J, Watermeyer R, Wakeling P (2016) Fear and loathing in the Academy? The role of emotion in response to an impact agenda in the UK and Australia. High Educ Res Dev 36(3):555–568. https://doi.org/10.1080/07294360.2017.1288709

Chubb J, Watermeyer R (2017) Artifice or integrity in the marketization of research impact? Investigating the moral economy of (pathways to) impact statements within research funding proposals in the UK and Australia. Stud High Educ 42(12):2360–2372

Davies B, Harré R (1990) Positioning: the discursive production of selves. J Theory Soc Behav 20(1):43–63

Dean MM (1999) Governmentality: power and rule in modern society. SAGE Publications, Thousand Oaks, California

Derrick G (2018) The evaluators’ eye: Impact assessment and academic peer review. Palgrave Macmillan, London

Donovan C (2008) The Australian Research Quality Framework: A live experiment in capturing the social, economic, environmental, and cultural returns of publicly funded research. New Dir for Eval 118:47–60. https://doi.org/10.1002/ev.260

Donovan C (2011) State of the art in assessing research impact: introduction to a special issue. Res. Eval. 20(3):175–179. https://doi.org/10.3152/095820211X13118583635918

Donovan C (2017) For ethical ‘impactology’. J Responsible Innov 6(1):78–83. https://doi.org/10.1080/23299460.2017.1300756

Dreyfus HL, Rabinow P (1983) Michel Foucault: beyond structuralism and hermeneutics. University of Chicago Press, Chicago

European Science Foundation (2012) The Challenges of Impact Assessment. Working Group 2: Impact Assessment. ESF Archives. http://archives.esf.org/index.php?eID=tx_nawsecuredl&u=0&g=0&t=1609373495&hash=08da8bb115e95209bcea2af78de6e84c0052f3c8&file=/fileadmin/be_user/CEO_Unit/MO_FORA/MOFORUM_Eval_PFR__II_/Publications/WG2_new.pdf . Accessed 30 Dec 2020

Fairclough N (1989) Language and power. Longman, London/New York

Fairclough N (1992) Discourse and social change. Polity Press, Cambridge, UK/Cambridge

Fairclough N (1993) Critical discourse analysis and the marketization of public discourse: The Universities. Discourse Soc 4(2):133–168

Fairclough N, Mulderrig J, Wodak R (1997) Critical discourse analysis. In: Van Dijk TA (ed) Discourse studies: a multidisciplinary introduction. SAGE Publications Ltd, New York, pp. 258–284

Foucault M (1980) The confession of the flesh. In: Gordon C (ed) Power/knowledge: selected interviews and other writings 1972–1977. Vintage Books, New York

Foucault M (1984) Nietzsche, genealogy, history. In: Rabinow P (ed) The Foucault Reader. Pantheon Books, New York

Foucault M (1988) Politics, philosophy, culture: Interviews and other writings, 1977–1984. Routledge, New York

Foucault M (1990) The use of pleasure. The history of sexuality, vol. 2. Vintage Books, New York

Gee J (2015) Social linguistics and literacies ideology in discourses. Taylor and Francis, Florence

Gilbert GN, Mulkay M (1984) Opening Pandora’s Box: a sociological analysis of scientists’ discourse. Cambridge University Press, Cambridge

Goffman E (1969) The presentation of self in everyday life. Allen Lane The Pinguin Press, London

Grant J, Brutscher, PB, Kirk S, Butler L, Wooding S (2009) Capturing Research Impacts. A review of international practice. Rand Corporation. RAND Europe. https://www.rand.org/content/dam/rand/pubs/documented_briefings/2010/RAND_DB578.pdf . Accessed 30 Dec 2020

Gunn A, Mintrom M (2016) Higher education policy change in Europe: academic research funding and the impact agenda. Eur Educ 48(4):241–257. https://doi.org/10.1080/10564934.2016.1237703

Gunn A, Mintrom M (2018) Measuring research impact in Australia. Aust Universit Rev 60(1):9–15

Hah S (2019) Disciplinary positioning struggles: perspectives from early career academics. J Appl Linguist Prof Pract 12(2). https://doi.org/10.1558/jalpp.32820

Hah S (2020) Valuation discourses and disciplinary positioning struggles of academic researchers–a case study of ‘maverick’ academics. Pal Commun 6(1):1–11. https://doi.org/10.1057/s41599-020-0427-2

Hamann J (2016) “Let us salute one of our kind.” How academic obituaries consecrate research biographies. Poetics 56:1–14. https://doi.org/10.1016/j.poetic.2016.02.005

Harré R, Van Langenhove L (1998) Positioning theory: moral contexts of international action. Wiley-Blackwell, Chichester

HEFCE (2015) Research Excellence Framework 2014: Manager’s report. HEFCE. https://www.ref.ac.uk/2014/media/ref/content/pub/REF_managers_report.pdf . Accessed 30 Dec 2020

HEFCE (2011) Assessment framework and guidance on submissions. HEFCE: https://www.ref.ac.uk/2014/media/ref/content/pub/assessmentframeworkandguidanceonsubmissions/GOS%20including%20addendum.pdf . Accessed 30 Dec 2020

Hoey M (1994) Signalling in discourse: A functional analysis of a common discourse pattern in written and spoken English. In: Coulthard M (ed) Advances in written text analysis. Routledge, London

Hoey M (2001) Textual interaction: an introduction to written discourse analysis. Routledge, London

Hong Kong University Grants Committee (2018) Research Assessment Exercise 2020. Draft General Panel Guidelines. UGC. https://www.ugc.edu.hk/doc/eng/ugc/rae/2020/draft_gpg_feb18.pdf Accessed 30 Dec 2020

Hyland K (2009) Academic discourse English in a global context. Continuum, London

King’s College London and Digital Science (2015) The nature, scale and beneficiaries of research impact: an initial analysis of Research Excellence Framework (REF) 2014 impact case studies. Dera: http://dera.ioe.ac.uk/22540/1/Analysis_of_REF_impact.pdf . Accessed 30 Dec 2020

Kuhn TS (1970) The structure of scientific revolutions. University of Chicago Press, Chicago

Lemke T (2002) Foucault, governmentality, and critique. Rethink Marx 14(3):49–64

Maingueneau D (2010) Le discours politique et son « environnement ». Mots. Les langages du politique 94. https://doi.org/10.4000/mots.19868

Manville C, Jones MM, Frearson M, Castle-Clarke S, Henham ML, Gunashekar S, Grant J (2014) Preparing impact submissions for REF 2014: An evaluation. Findings and observations. RAND Corporation: https://www.rand.org/pubs/research_reports/RR726.html . Accessed 30 Dec 2020

Manville C, Guthrie S, Henham ML, Garrod B, Sousa S, Kirtley A, Castle-Clarke S, Ling T (2015) Assessing impact submissions for REF 2014: an evaluation. Rand Corporation. https://www.rand.org/content/dam/rand/pubs/research_reports/RR1000/RR1032/RAND_RR1032.pdf . Accessed 30 Dec 2020

Myers G (1985) Texts as knowledge claims: the social construction of two biology articles. Soc Stud Sci 15(4):593–630

Myers G (1989) The pragmatics of politeness in scientific articles. Appl Linguist 10(1):1–35

Article   ADS   Google Scholar  

Musolff A (2004) Metaphor and political discourse. Analogical reasoning in debates about Europe. Palgrave Macmillan, Basingstoke

Musolff A (2012) The study of metaphor as part of critical discourse analysis. Crit. Discourse Stud. 9(3):301–310. https://doi.org/10.1080/17405904.2012.688300

National Co-ordinating Centre For Public Engagement (2014) After the REF-Taking Stock: summary of feedback. NCCFPE. https://www.publicengagement.ac.uk/sites/default/files/publication/nccpe_after_the_ref_write_up_final.pdf . Accessed 30 Dec 2020

Potter J (1996) Representing reality: discourse, rhetoric and social construction. Sage, London

Power M (2015) How accounting begins: object formation and the accretion of infrastructure. Account Org Soc 47:43–55. https://doi.org/10.1016/j.aos.2015.10.005

Research Council of Norway (2017) Evaluation of the Humanities in Norway. Report from the Principal Evaluation Committee. The Research Council of Norway. Evaluation Division for Science. RCN. https://www.forskningsradet.no/siteassets/publikasjoner/1254027749230.pdf . Accessed 30 Dec 2020

Research Council of Norway (2018) Evaluation of the Social Sciences in Norway. Report from the Principal Evaluation Committee. The Research Council of Norway.Division for Science and the Research System RCN. https://www.forskningsradet.no/siteassets/publikasjoner/1254035773885.pdf Accessed 30 Dec 2020

Robinson D (2013) Introducing performative pragmatics. Routledge, London/New York

Rose N (1999) Governing the soul: the shaping of the private self. Free Association Books, Sidmouth

Sayer D (2015) Rank hypocrisies: the insult of the REF. Sage, Thousand Oaks

Selby J (2016) Critical IR and the Impact Agenda, Paper presented at Pais Impact Conference. Warwick University, Coventry, pp. 22–23 November 2016

Sembou E (2015) Hegel’s Phenomenology and Foucault’s Genealogy. Routledge, New York

Stern N (2016) Building on Success and Learning from Experience. an Independent Review of the Research Excellence Framework. Department for Business, Energy and Industrial Strategy. Assets Publishing Service. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/541338/ind-16-9-ref-stern-review.pdf . Accessed 30 Dec 2020

Swales JM (1998) Other floors, other voices: a textography of a small university building. Routledge, London/New York

Swales JM (1990) Genre analysis: English in academic and research settings. Cambridge University Press, Cambridge

Swales JM (2011) Aspects of Article Introductions. University of Michigan Press, Ann Arbor

Tirado F, Gálvez A (2008) Positioning theory and discourse analysis: some tools for social interaction analysis. Historical Social Res 8(2):224–251

Vinkers CH, Tijdink JK, Otte WM (2015) Use of positive and negative words in scientific PubMed abstracts between 1974 and 2014: retrospective analysis. BMJ 351:h6467. https://doi.org/10.1136/bmj.h6467

Article   PubMed   PubMed Central   Google Scholar  

VSNU–Association of Universities in the Netherlands (2016) Standard Evaluation Protocol (SEP). Protocol for Research Assessments in the Netherlands. VSNU. https://vsnu.nl/files/documenten/Domeinen/Onderzoek/SEP2015-2021.pdf . Accessed 30 Dec 2020

Watermeyer R (2012) From engagement to impact? Articulating the public value of academic research. Tertiary Educ Manag 18(2):115–130. https://doi.org/10.1080/13583883.2011.641578

Watermeyer R (2014) Issues in the articulation of ‘impact’: the responses of UK academics to ‘impact’ as a new measure of research assessment. Stud High Educ 39(2):359–377. https://doi.org/10.1080/03075079.2012.709490

Watermeyer R (2016) Impact in the REF: issues and obstacles. Stud High Educ 41(2):199–214. https://doi.org/10.1080/03075079.2014.915303

Warner M (2015) Learning my lesson. London Rev Books 37(6):8–14

Weinstein N, Wilsdon J, Chubb J, Haddock G (2019) The Real-time REF review: a pilot study to examine the feasibility of a longitudinal evaluation of perceptions and attitudes towards REF 2021. SocArXiv: https://osf.io/preprints/socarxiv/78aqu/ . Accessed 30 Dec 2020

Wróblewska MN, Angermuller J (2017) Dyskurs akademicki jako praktyka społeczna. Zwrot dyskursywny i społeczne badania szkolnictwa wyższego. Kultura–Społeczeństwo–Edukacja 12(2):105–128. https://doi.org/10.14746/kse.2017.12.510.14746/kse.2017.12.5

Wróblewska MN (2017) Ewaluacja wpływu społecznego nauki. Przykład REF 2014 a kontekst polski. NaukaiSzkolnicwo Wyższe 49(1):79–104. https://doi.org/10.14746/nisw.2017.1.5

Wróblewska MN (2018) The making of the Impact Agenda. A study in discourse and governmnetality. Unpublished doctoral dissertation. Warwick University

Wróblewska MN (2019) Impact evaluation in Norway and in the UK: A comparative study, based on REF 2014 and Humeval 2015-2017. ENRESSH working paper series 1. University of Twente Research Information. https://ris.utwente.nl/ws/portalfiles/portal/102033214/ENRESSH_01_2019.pdf . Accessed 30 Dec 2020

Download references

Acknowledgements

I wish to thank Prof. Johannes Angermuller, the supervisor of the doctoral dissertation in which many of the ideas discussed in this paper were first presented. Prof. Angermuller’s guidance and support were essential for the development of my understanding of the importance of discourse in evaluative contexts. I also thank the reviewers of the aforementioned thesis, Prof. Jo Angouri and Prof. Srikant Sarangi for their feedback which helped me develop and clarify the concepts which I use in my analysis, as well as its presentation. Any errors or omissions are of course my own. The research presented in this paper received funding from the European Research Council (DISCONEX project 313,172). The underpinning research was also facilitated by the author’s membership in EU Cost Action “European Network for Research Evaluation in the Social Sciences and the Humanities”(ENRESSH CA15137-E). Particularly advice and encouragement recieved from the late prof. Paul Benneworth was invaluable.

Author information

Authors and affiliations.

University of Warwick, Coventry, UK

Marta Natalia Wróblewska

National Centre for Research and Development–NCBR, Warsaw, Poland

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Marta Natalia Wróblewska .

Ethics declarations

Competing interests.

The author declares no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and Permissions

About this article

Cite this article.

Wróblewska, M.N. Research impact evaluation and academic discourse. Humanit Soc Sci Commun 8 , 58 (2021). https://doi.org/10.1057/s41599-021-00727-8

Download citation

Received : 12 May 2020

Accepted : 11 January 2021

Published : 02 March 2021

DOI : https://doi.org/10.1057/s41599-021-00727-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

• Articles / Rigor

Staying on Task with Meaningful Student Talk

by MiddleWeb · 10/19/2022

By Barbara R. Blackburn

Too often the discussion is controlled by the teacher, and students focus simply on answering the teacher’s question, usually at a surface level. There is often too much teacher talk, with limited student discussion, oftentimes dominated by a few kids.

In recent years the daily conversation has shifted from classroom talk and discussion to student discourse, which is also called accountable talk or academic discourse. What exactly is the difference between our traditional communication with students and this new kind of dialogue? Let’s break it down.

The goal is deepening understanding

Good academic discourse also focuses on a deeper understanding of content. This requires an intentional shift to higher order tasks. Here are some content-area examples of tasks that encourage academic discourse.

Supporting Student Questioning

Academic Vocabulary: Tiers of Vocabulary

Tiers of Vocabulary were introduced by Isabel Beck and Margaret McKeown in 1987, but the tiers have gained renewed interest recently. As the authors describe them, Tier One words are acquired through everyday speech. These words are common and are typically taught at early grades or learned through everyday use around them.

Instruction and opportunities for discourse using Tier 3 words should occur naturally within your curriculum. One particular challenge is that there are so many Tier 3 words in subject-specific units that students can be overwhelmed. You may choose to use vocabulary cards during discourse or prioritize specific words.

A Final Note

Moving students’ conversations to a more rigorous level can result in academic discourse. However, you’ll want to plan for purposeful opportunities that incorporate student questioning and academic vocabulary.

Barbara is the author of  Rigor and Differentiation in the Classroom: Tools and Strategies   and  Rigor is NOT a Four-Letter Word  from Routledge/Eye On Education. In December 2022 her latest book, Rigor in Your Classroom: A Toolkit for Teachers , will be published.

Share this:

Tags: academic discourse academic vocabulary accountable talk Barbara Blackburn Barbara R. Blackburn on-task talk rigorous conversations tiers of vocabulary

MiddleWeb is all about the middle grades, with great 4-8 resources, book reviews, and guest posts by educators who support the success of young adolescents. And be sure to subscribe to MiddleWeb SmartBrief for the latest middle grades news & commentary from around the USA.

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Notify me of follow-up comments by email.

Notify me of new posts by email.

This site uses Akismet to reduce spam. Learn how your comment data is processed .

• Popular Posts
• Recent Posts
• Recent Comments

Articles / Gratitude

Thanksgiving: A Time of Gratitude and Generosity

Book Reviews / Global Learning

Preparing Our Students to Engage the World

Articles / Reading

Teaching YA Novels with Multiple Perspectives

Articles / Laughter & Humor

Humor in Our Schools Heals and Engages Us

Articles / Mathematics

Avoid These 3 Mistakes During Math Debriefs

Book Reviews / Social & Emotional Learning

A Playbook for Student and Teacher Well-Being

Articles / Literacy

Embed These 7 Skills to Assure Comprehension

I Will Screw This Up / Unstructured Learning

Mining for student gold in unstructured class time

The Unstoppable ML Teacher / Videos in Class

Keys to Making Videos Comprehensible to MLs

Book Reviews / Teaching Insights

Culturally & Historically Responsive Classrooms

• Diane Stegg says: Good suggestions for expanding curriculum and learning
• Concept Mastery says: Independent reading time is crucial for students as it fosters a...
• Dina says: Thank you, Robin!!
• Robin Wixson says: Thank you! Well said. We have so much structured time during...
• Judith Wilson says: Hurray! I'm so glad that low tech has not been forgotten.

Sign Up & Receive the Latest News about Our Content…

Email address:

First Name:

Read our Privacy Policy

BOOK REVIEWS

Filling Your Classroom with Deliberate Optimism

Add the Power of Poetry to All Your ELA Lessons

Genius Hour Can Ignite Innovation and Inquiry

Effective Literacy Walks Begin with Collaboration

Restoring the Joy and Possibility of Teaching

The All-New ‘Classroom Instruction That Works’

Practical Ways to Help Post-Pandemic Students

Brain-Friendly Tools to De-Stress the Test

Collective Efficacy for Schoolwide Success

Leading Meaningful Race Conversations with Kids

How to Use Stories as Catalysts for Reflection

How Fiction Writing Can Transform Students

PBL and Inquiry Learning for Gifted Students

How to Boost Students’ Intrinsic Desire to Learn

Reclaim the Joy in Education This Fall

Help Tweens Develop Resilience Superpowers

Collaborative Coaching to Boost Math Learning

Deepening Student Understanding with Collaborative Discourse

Make Thinking Visible

Designing learning for collaboration, engagement, and meaningful discourse, sample collaborative inquiry task using picture search.

Students are shown a picture of a historical painting (of Washington, D.C.).

-What is this a picture of?

-Approximately what time period is this depicting?

-How do you know? Provide three different credible sources of evidence to support your answer.

-Bonus: Can you also find something interesting or unique about the picture or time period?

Hess, K. (2018). A local assessment toolkit to promote deeper learning: Transforming research into practice . Thousand Oaks, CA: Corwin.

Hess, K., & Gong, B. (2014). Ready for college and career? Achieving the Common Core Standards and beyond through deeper, student-centered learning . Quincy, MA: Nellie Mae Education Foundation. Retrieved from http://www.karin-hess.com/free-resources .

Nottingham, J., Nottingham, J., & Renton, M. (2017). Challenging learning through dialogue: Strategies to engage your students and develop their language of learning . Thousand Oaks, CA: Corwin.

Slavin, R. (1991). Synthesis of research on cooperative learning. Educational Leadership, 48 (5), 71-82.

Sousa, D. (2015). Brain-friendly assessments: What they are and how to use them . West Palm Beach, FL: Learning Sciences International.

Wiggins, G., & McTighe, J. (2005). Understanding by design (expanded 2nd edition) . Alexandria, VA: ASCD.

Karin Hess , EdD, is founder and president of Educational Research in Action and author of the Hess Cognitive Rigor Matrices. She is a former classroom teacher and school administrator with more than 40 years of experience in curriculum, instruction, and assessment. She is an internationally recognized leader in applying the concepts of cognitive rigor, depth of knowledge, and learning progressions to the design and development of state and school-based curriculum and assessment systems. An expert in multiple content areas, she specializes in the design and use of performance-based assessments for preK–12 students.

Hess has authored or coauthored more than a dozen books in the field of education, including A Local Assessment Toolkit to Promote Deeper Learning (Corwin, 2018).

ASCD is a community dedicated to educators' professional growth and well-being.

Let us help you put your vision into action., related articles.

What’s Behind the Rise in Problem Behavior? It Could Be Cognitive

Five Peer Tutoring Strategies for the Classroom

Boosting Sight Reading

Grading Writing Just Got Weird

4 C’s for Better Student Engagement

To process a transaction with a purchase order please send to [email protected].

As part of our annual inventory audit process, ASCD will be pausing all order fulfillment operations beginning at 12 noon eastern (U.S.) on Thursday, November 16, through November 21.

Order fulfillment will resume on Tuesday, November 21, when the audit is complete.

Please contact us at 1-800-933-2723 if you have questions about your order status.

• Published: 19 December 2019

Awareness of discourse organizers and comprehension of academic lectures: The effect of using concordancers

• Javad Zare 1  

Current Psychology volume  39 ,  pages 419–427 ( 2020 ) Cite this article

427 Accesses

5 Citations

Metrics details

Previous research shows that discourse organizers facilitate the students’ comprehension of lectures. Importance markers are considered a crucial part of discourse organization. It is not clear, however, whether understanding of importance markers boosts the students’ understanding of lectures. Besides, it is not clear whether using corpora and concordancers in the classroom aids their comprehension. To this end, a total of 206 English as a Foreign Language (EFL) university students (108 males and 98 females, aged from 18 to 23), studying medicine, humanities, engineering, and basic sciences at three major universities in Iran, whose command of English was pretested on a piloted test of Preliminary English Test (PET), were selected for participation in the study. They were then equally divided into a control and experimental group. Learners in the control group attended 15 one-hour general English training sessions (five weeks, three sessions a week) without specific focus on identifying organizational features of lectures, while students in the experimental group attended 15 one-hour sessions (five weeks, three sessions a week) of instruction on importance marking in English academic lectures, using the British Academic Spoken English corpus (BASE) with a concordancer, namely Sketch Engine. Presentation and practice tasks which were developed based on authentic concordance lines which were derived from the BASE corpus were used for the experimental group. Analysis of the scores of participants on a comprehension test of important points of English academic lectures showed that concordancer-based instruction of importance markers enhances EFL learners’ identification and comprehension of important points in English academic lectures.

This is a preview of subscription content, access via your institution .

Access options

Buy single article.

Instant access to the full article PDF.

Price includes VAT (Russian Federation)

Rent this article via DeepDyve.

Ädel, A. (2006). Metadiscourse in L1 and L2 English. Amsterdam: John Benjamins. https://doi.org/10.1075/scl.24 .

Ädel, A. (2010). Just to give you kind of a map of where we are going: A taxonomy of metadiscourse in spoken and written academic English. Nordic Journal of English Studies, 9 (2), 69–97.

Article   Google Scholar  

Ädel, A. (2012). “What I want you to remember is…”: Audience orientation in monologic academic discourse. English Text Construction, 5 (1), 101–127. https://doi.org/10.1075/etc.5.1.06ade .

Alipour, M., & Jalilifar, A. (2007). How explicit instruction makes a difference: Metadiscourse markers & EFL learners' reading comprehension skills. Journal of College Reading and Learning, 3 (1), 35–52.

Google Scholar  

Allami, H., & Serajfard, H. (2012). Engagement markers: A technique for improving writing skills. Journal of Language, Culture, and Translation, 1 (1), 71–83.

Allison, D., & Tauroza, S. (1995). The effect of discourse organization on lecture comprehension. English for Specific Purposes, 14 (2), 157–173. https://doi.org/10.1016/0889-4906(95)00007-E .

Asik, A., & Cephe, P. T. (2013). Discourse markers and spoken English: Nonnative use in the Turkish EFL setting. English Language Teaching, 6 (12), 144–155.

Biber, D., Conrad, S., & Cortes, V. (2004). If you look at ...: Lexical bundles in university teaching and textbooks. Applied Linguistics , 25, 371–405. doi: https://doi.org/10.1093/applin/25.3.371 .

Cambridge English Language Assessment. (2015). Cambridge English preliminary: Handbook for teachers . Cambridge: Cambridge University Press.

Carrier, C. A., Williams, M. D., & Dalagard, B. R. (1988). College students’ perceptions of notetaking and their relationship to selected learner characteristics and course achievement. Research in Higher Education, 28 (3), 223–239. https://doi.org/10.1007/BF00992232 .

Chaudron, C., & Richards, J. C. (1986). The effect of discourse markers on the comprehension of lectures. Applied Linguistics, 7 (2), 113–127. https://doi.org/10.1093/applin/7.2.113 .

Crawford Camiciottoli, B. (2004). Interactive discourse structuring in L2 guest lectures: Some insights from a comparative corpus-based study. Journal of English for Academic Purposes, 3 , 39–54. https://doi.org/10.1016/S1475-1585(03)00044-4 .

Crawford Camiciottoli, B. (2007). The language of business studies lectures . Amsterdam: John Benjamins.

Book   Google Scholar  

Dabaghi, A., Khorvash, Z., & Tavakoli, M. (2010). The effect of meta-discourse awareness on L2 reading comprehension: A case of Iranian EFL learners. English Language Teaching,3(1), 92-102.

Deroey, K. L. B. (2013). Relevance marking in lectures: A corpus-based study . Ghent University, Gent, Belgium: Unpublished PhD Dissertation.

Dunkel, P. A., & Davis, J. N. (1994). The effects of rhetorical signaling cues on the recall of English lecture information by speakers of English as a native and second language. In J. Flowerdew (Ed.), Academic listening: Research perspectives (pp. 55–74). New York: Cambridge University Press.

Eslami, Z., & Eslami-Rasekh, A. (2007). Discourse markers in academic lectures. Asian EFL Journal, 9 (1), 22–38.

Flowerdew, J. (1994). Research of relevance to second language lecture comprehension: An overview. In J. Flowerdew (Ed.), Academic listening: Research perspectives (pp. 7–29). Cambridge: Cambridge University Press.

Flowerdew, J. (1996). Concordancing in language learning. In M. Pennington (Ed.), The power of CALL (pp. 87–113). Houston: Athelstan.

Flowerdew, L. (2010). Using corpora for writing instruction. In A. O’Keeffe & M. McCarthy (Eds.), The Routledge handbook of corpus linguistics (pp. 444–457). London: Routledge.

Flowerdew, J., & Tauroza, S. (1995). The effect of discourse markers on second language lecture comprehension. Studies in Second Language Acquisition, 17 (4), 435–458. https://doi.org/10.1017/S0272263100014406 .

Gilbert, J. (2013). English for academic purposes. In G. Motteram (Ed.), Innovations in learning technologies for English language teaching (pp. 117–144). London: The British Council.

Hamp-Lyons, L. (2011). English for academic purposes. In E. Hinkel (Ed.), Handbook of research in second language teaching and learning, volume 2 (pp. 89–105). New York: Routledge.

Hashemi, S. M., Khodabakhshzade, H., & Elahi Shirvan, M. (2012). The effect of metadiscourse on EFL learners’ listening comprehension. Journal of Language Teaching and Research, 3 (3), 452–457.

Hassanein, O. H. S. (2016). The effect of suggested program based on interactional metadisourse markers on developing EFL majors’ reading comprehension and the reader-friendliness of their writing. Journal of Language Teaching and Research, 7 (2), 273–283.

Hellerman, J., & Vergun, A. (2007). Language which is not taught: The discourse marker use of beginning adult learners of English. Journal of Pragmatics, 39 , 157–179.

Hyland, K. (2004). Disciplinary interactions: Metadiscourse in L2 postgraduate writing. Journal of Second Language Writing, 13 (2), 133–151. https://doi.org/10.1016/j.jslw.2004.02.001 .

Hyland, K. (2010). Metadiscourse: Mapping interactions in academic writing. Nordic Journal of English Studies, 9 (2), 125–143.

Jalilifar, A. (2008). Discourse markers in composition writing: The ease of Iranian learners of English as a foreign language. English Language Teaching, 1 , 114–122.

Jung, S. (2003). The effects of organization markers on ESL learners’ text understanding. TESOL Quarterly, 37 (4), 749–760. https://doi.org/10.2307/3588223 .

Jung, S. (2006). Misunderstanding of academic monologues by nonnative speakers of English. Journal of Pragmatics, 38 (11), 1928–1942. https://doi.org/10.1016/j.pragma.2005.05.001 .

Kettemann, B. (1995). On the use of concordancing in ELT. TELL&CALL, 4 , 4–15.

Kilgarriff, A., Baisa, V., Bušta, J., Jakubíček, M., Kovář, V., Michelfeit, J., Rychlý, P., & Suchomel, V. (2014). The sketch engine: Ten years on. Lexicography, 1 (1), 7–36.

Lin, C. Y. (2010). that's actually sort of you know trying to get consultants in...': Functions and multifunctionality of modifiers in academic lectures. Journal of Pragmatics, 42 (5), 1173–1183. https://doi.org/10.1016/j.pragma.2009.10.001 .

Lynch, T. (2011). Academic listening in the 21st century: Reviewing a decade of research. Journal of English for Academic Purposes, 10 (2), 79–88. https://doi.org/10.1016/j.jeap.2011.03.001 .

Mauranen, A. (1993). Contrastive ESP rhetoric: Metatext in Finnish-English economics texts. English for Specific Purposes, 12 (1), 3–22. https://doi.org/10.1016/0889-4906(93)90024-I .

Mauranen, A. (2001). Reflexive academic talk: Observations from MICASE. In R. C. Simpson & J. M. Swales (Eds.), Corpus linguistics in North America: Selections from the 1999 symposium (pp. 165–178). Ann Arbor: University of Michigan Press.

Mauranen, A. (2012). Exploring ELF: Academic English shaped by non-native speakers . Cambridge: Cambridge University Press.

Mayer, R. E. (1996). Learning strategies for making sense out of expository text: The SOI model for guiding three cognitive processes in knowledge construction. Educational Psychology Review, 8 (4), 357–371. https://doi.org/10.1007/BF01463939 .

Mayer, R. E. (2008). Applying the science of learning: Evidence-based principles for the design of multimedia instruction. American Psychologist, 63 (8), 760–769. https://doi.org/10.1037/0003-066X.63.8.760 .

Article   PubMed   Google Scholar  

Nattinger, J. R., & DeCarrico, J. S. (1992). Lexical phrases and language teaching . Oxford: Oxford University Press.

Nesi, H. (2001). A corpus-based analysis of academic lectures across disciplines. In J. Cotterill & A. Ife (Eds.), Language across boundaries. BAAL, Vol. 16 (pp. 201–218). London: Continuum.

Olsen, L. A., & Huckin, T. H. (1990). Point-driven understanding in engineering lecture comprehension. English for Specific Purposes, 9 (1), 33–47. https://doi.org/10.1016/0889-4906(90)90027-A .

Pérez, M. A., & Macià, E. A. (2002). Metadiscourse in lecture comprehension: Does it really help foreign language learners? Atlantis, 24 (1), 7–21.

Siepmann, D. (2005). Discourse markers across languages: A contrastive study of second-level discourse markers in native and non-native text with implications for general and pedagogic lexicography . New York: Routledge.

Simpson, R. (2004). Stylistic features of academic speech: The role of formulaic expressions. In U. Connor & T. A. Upton (Eds.), Discourse in the professions: Perspectives from corpus linguistics (pp. 37–64). Amsterdam: John Benjamins.

Chapter   Google Scholar  

Swales, J. M. (2001). Metatalk in American academic talk the cases of point and thing. Journal of English Linguistics, 29 (1), 34–54. https://doi.org/10.1177/00754240122005189 .

Thompson, P. (2006). Assessing the contributions of corpora to EAP practice. In Z. Kantaridou, I. Papadopoulou, & I. Mahili (eds.), Motivation in learning language for specific and academic purposes. Macedonia: University of Macedonia (Online).

Thompson, S. E. (2003). Text-structuring metadiscourse, intonation and the signaling of organization in academic lectures. Journal of English for Academic Purposes, 2 (1), 5–20. https://doi.org/10.1016/S1475-1585(02)00036-X .

Titsworth, B. S., & Kiewra, K. A. (2004). Spoken organizational lecture cues and student note taking as facilitators of student learning. Contemporary Educational Psychology, 29 (4), 447–461. https://doi.org/10.1016/j.cedpsych.2003.12.001 .

Yoon, C. (2011). Concordancing in L2 writing class: An overview of research and issues. Journal of English for Academic Purposes, 10 (3), 130–139.

Warsi, J. (2001). The acquisition of English contrastive discourse markers by advanced Russia ESL students. Unpublished PhD dissertation, Boston University. Retrieved from http://jilaniwarsi.tripod.com/professsional/id14.htm1

Zare, J., & Keivanloo-Shahrestanaki, Z. (2017). The language of English academic lectures: The case of field of study in highlighting importance. Lingua, 193 , 36–50.

Zhang, J. (2012). Discourse markers in college English listening instruction: An empirical study of Chinese learners. English Language Teaching, 5 , 46–60.

Zhao, H. (2013). A study on the pragmatic fossilization of discourse markers among Chinese English learners. Journal of Language Teaching and Research, 4 (4), 707–714.

Download references

Acknowledgments

The researcher gratefully acknowledges financial support from Kosar University of Bojnord. The author, Javad Zare, was supported by Kosar University of Bojnord with the grant number 9805191714.

The author, Javad Zare, was supported by Kosar University of Bojnord with the grant number 9805191714.

Author information

Authors and affiliations.

Kosar University of Bojnord, Bojnord, Iran

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Javad Zare .

Ethics declarations

Author identifying information.

Information related to the author is on the title page that is separate from the manuscript.

Conflict of Interest

The author declares that he/she has no conflict of interest.

Ethical Approval

This article does not contain any studies with animals performed by any of the authors.

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed Consent

Informed consent was obtained from all individual participants included in the study.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Cite this article.

Zare, J. Awareness of discourse organizers and comprehension of academic lectures: The effect of using concordancers. Curr Psychol 39 , 419–427 (2020). https://doi.org/10.1007/s12144-019-00579-1

Download citation

Published : 19 December 2019

Issue Date : April 2020

DOI : https://doi.org/10.1007/s12144-019-00579-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Discourse organizer
• Concordancer
• Academic lecture comprehension
• The English academic lecture

Advertisement

• Find a journal
• Publish with us