Editor’s note: Akin to the UK’s Research Excellence Framework (REF), the newly launched Teaching Excellence Framework (TEF) is an effort to assess the effectiveness and impact of higher education teaching primarily by means of metrics. The Higher Education Funding Council for England (HEFCE) which has been deeply involved in both the REF and TEF, also published an important study, The Metric Tide, on the effectiveness of such forms of metrical evaluation of research; concerns about the TEF have also focused on the metrics. This guest post is authored by Linda Bennett and Annika Bennett of Gold Leaf, founded in 2001 to provide market research and business development support for the academic publishing, bookselling and librarian communities. Both can be reached at info@goldleaf.co.uk.

Towards the end of 2016, participating UK universities began to prepare their submissions in order be awarded their first Teaching Excellence Framework ratings. The submissions had to be in by February; the results were due to be announced this week, but due to the General Election results, the announcement was delayed. The TEF is based on the universities’ capture of six sets of metrics, divided into 3 categories: teaching quality (‘teaching on my course’ and assessment and feedback), learning environment (to include academic support and retention rates) and student outcomes and learning gain (to include employment or further study and employability). The metrics have proved controversial and have attracted a great deal of criticism, as they are proxies which measure effects of teaching, but not the teaching itself. The TEF also offers an important moment for publishers in the academic market to think about how their strategies could or should align with resourcing opportunities and requirements.

Inspired in part by the theme of the 2017 Academic Book Trade Conference, Gold Leaf embarked upon a major study to capture all the stakeholder views at this pivotal moment, supported by the UK Booksellers Association. After eight weeks of intense research we presented the results in the form of a report “Resource Provision in Higher Education: Implications of the TEF and related initiatives.”  We also presented our findings at the ABT Conference in May.

Ironically enough for the study of an initiative that places such emphasis on metrics, it was clear that undertaking a quantitative approach towards gathering information about the TEF would not be possible at this stage. Therefore, we based the bulk of the report on semi-structured interviews. We spoke to a total of 40 TEF administrators, senior librarians, booksellers and publishers; the findings gained from the interviews were supplemented by conducting a student focus group and online surveys with academics and students. We gained additional information from booksellers and librarians via more online surveys. (So many publishers volunteered information that we didn’t need to construct an online survey for them.) Altogether we received responses from more than UK 25 universities; collectively, they represented the full spectrum of types of HE institution. Among them were universities that had opted out of the TEF (but most had opted in).

Our report captures the particular concerns of university lecturers relating to metrics. They are mainly based on National Student Survey [NSS] outcomes; the NSS itself has been the subject of criticism. Unsurprisingly, the government has already been acknowledged that the metrics will need further refinement after a major independent review is undertaken (this to be completed by 2019). Following an unfavorable reaction to the TEF in the House of Lords debate, the much-disputed linking of TEF results to tuition fee increases has been postponed for another three years.

Interviews revealed that all stakeholders saw some changes in approach of teaching and learning through the TEF, even though it became obvious that the TEF is both a symptom and a manifestation of changes in approach to teaching and learning that have been going on for some time. Academics themselves were the group who were least convinced that the TEF would have an influence on this (26% answering the question with no). Looking at learning materials, there was a clear need for more interactive online resources, but when it comes to tailored textbooks, students and academics saw less demand for these than librarians or booksellers did. The study revealed that more cooperation between parties was both needed and wanted (to various extents) in order to make the TEF a success and to develop business models that worked for all sides.

Academic publishers and booksellers are already speculating about the opportunities that the TEF offers, but there are challenges to face as well.

The Gold Leaf report suggests that it is essential that academic publishers and booksellers are aware of the TEF and continue to keep abreast of it as it develops because it has such far-reaching implications for the creation and maintenance of teaching and learning resources at the Higher Education (HE) level, how they are paid for, and how their success is measured. Even though many of those from universities who took part in the interviews agreed that a sea-change in approach to teaching and learning was already underway and would have continued to forge ahead with or without the TEF, most also said that the Framework will accelerate certain processes and force a greater focus on them.

Academic publishers and booksellers are already speculating about the opportunities that the TEF offers, but there are challenges to face as well. A higher level of engagement is essential if the industry is to be an effective part in supplying the HE resources necessary for this developing environment. Although prominent members of the industry tried to gain places on TEF boards, they were told that they could not be admitted because of their commercial interests (even though universities are themselves now often run as commercial enterprises).  Publishers and booksellers must lobby to raise awareness of the crucial impact that professionally prepared and distributed learning resources will have on TEF outcomes. It is important to recognize also that resource needs diverge widely between institutions, and indeed between individual academic courses and syllabuses. One of the key recommendations of our report is that publishers and booksellers should take into account the whole spectrum of comments captured and not only heed those views and initiatives most favorable to the approach they are themselves adopting at present.

While the TEF may be a UK-only initiative, it exemplifies trends in teaching, learning and student engagement that are happening worldwide. New business models are already being developed; imaginative, university-friendly payment methods will be essential when delivering to a sector that may no longer expect the student to pay. While clearly there are opportunities and pitfalls in the current situation, academic publishers and booksellers need to have as much information as possible about the rapidly-changing HE landscape to develop their strategies accordingly.