Everyone is concerned with measuring the return on the investments that they make. Librarians have long studied the circulation and usage statistics for materials provided by the library. As content moved from physical to digital forms, the desire to measure activity only increased. But as digital content began to be widely adopted, there were significant challenges comparing the data that different publishers were providing. Counting and reporting usage data is far from a simple process, since the ultimate data can vary significantly depending on what exactly is counted (HTML, PDF or both), how it is counted (loads, versus reloads, internal links, etc.), and what “counts” (bots, automated usage, clicks to abstracts, or landing pages or the full text). In 2000, the community came together to establish a common set of standards and best practices around how data should be collected. The resulting effort led to the creation of Project COUNTER and its various releases of codes of practice over the ensuing years.
Last fall, COUNTER began publicly discussing work on a draft of the newest version of the COUNTER Code of Practice, which will be released this month. To help the community understand more about these changes, I had an exchange with Lorraine Estelle, Executive Director of COUNTER, Oliver Pesch (EBSCO) Chair of the COUNTER Board of Directors, and Tasha Mellins-Cohen (Highwire) a member of COUNTER Technical Advisory Group, to discuss Release 5 and they’ve provided the following information about Release 5 of the Code of Practice.
Thank you for taking the time to talk with the Scholarly Kitchen community about the forthcoming Release 5 of the COUNTER Code of Practice. I’m sure most everyone is aware, but in case anyone is not familiar with COUNTER, can you give me a short introduction to the organization?
COUNTER has been at the forefront of establishing standards for counting and reporting usage statistics for more than 15 years. COUNTER’s library, publisher and vendor members have contributed to the development of COUNTER Release 5 Draft Code of Practice, which is being released this month.
What are the key differences between Release 4 and the draft of Release 5?
The most significant obvious difference is the reduction in the number of reports and metrics! As a working group, we pushed hard for simplicity and clarity, to try and remove the ambiguities which we all admit plague earlier releases of the Code of Practice. Something to be aware of is that as we worked through Release 5, all the old COUNTER 4 reports were either renamed for consistency, or replaced. This new release will make the Code easier to maintain in the long term.
Are there new areas that Release 5 is focused on that haven’t been the focus of the previous release?
Release 5 introduces a lot more granularity in reporting — for example, we have introduced an attribute called ‘access_type’, which allows us to differentiate between controlled, open access, and other free-to-read content, at the level of the item instead of the title. However, we have tried to keep Release 5 as simple as possible to make it easier for publishers to comply, so we’ve not introduced large new focus areas.
In Release 5, why are there fewer reports? How can there be more flexibility with fewer reports?
One of the key mechanisms by which we have ensured flexibility and future-proofing is through the introduction of attributes which can be associated with multiple metrics. Providing information about access_type, is_archive, and yop (among others) as attributes means that we can roll up or drill down through reports with ease.
Introducing attributes means we’ve been able to replace many of the special purpose reports, such as the multimedia and mobile reports, with ‘expanded’ reports. As an example, the Expanded Item Report (IRx) delivers detailed reporting on metrics captured at the item level, whether that is a multimedia piece, a database entry, a book chapter, or a journal article. The report can be filtered by individual users to drill down to specific item types of interest, without publishers having to deliver lots of different reports.
When will Release 5 become “official” and how much time will publishers have to come into compliance?
We’re aiming to finish our consultation and publish Release 5 this summer, with compliance becoming mandatory 18 months later, with the delivery of reports for January 2019 (due by the end of February 2019).
What are ways that Release 5 makes compliance easier or more flexible?
Changes in the way content is structured and published, and the way it is accessed, meant that we evolved the COUNTER Code of Practice over time. Feedback from the library and publisher community last year told us quite clearly that this evolution resulted in ambiguities, confusions, and in some cases conflicts within the Code of Practice, making it difficult for publishers to comply. Rather than making immediate changes and potentially making the situation worse, we decided to proceed with drafting an entirely new release of the Code of Practice, and a key part of the working group’s mandate was to make it easier for publishers to comply.
We tackled the complexity issue from several angles. One, obviously, is the reduction in the number of metrics (12, down from 25) and reports (11, down from 35). The other goes back to my first answer, about simplicity and clarity. The Release 5 working group have discussed every report and every metric to be certain that it is entirely necessary and, importantly, absolutely unambiguous. Associated with that is our new glossary, which includes a very clear definition for every term we use.
As a working group, we pushed hard for simplicity and clarity, to try and remove the ambiguities which we all admit plague earlier releases of the Code of Practice.
Has there been any changes in the certification or audit process?
We’ve not made any significant changes to the audit process. On the subject of audits, though, we are asking publishers who are already COUNTER Release 4 compliant to continue delivering those reports for one year after they implement Release 5.
What is the process for responding if people have any questions or concerns about the draft?
We welcome feedback from all our stakeholders during the consultation period between now and the end of 12 April 2017. This feedback will inform the final draft of the Code of Practice. A good starting point is to register for one of our consultation webinars:
Thursday 23 February at 15:00 GMT, 10:00 EST, 07:00 PST,16:00 CET, 20:30 IST
Thursday 17 March at 14:00 CET (German language webinar)
Thursday 23 March at 15:00 GMT, 11:00 EDT, 08:00 PDT, 16:00 CET, 20:30 IST
We will also be opening a structured online survey and welcome other unstructured feedback by email. Members can also meet us at our break out session at the UKSG Conference and Exhibition in Harrogate, UK 10th to 12th April.
Several publishers and vendors are also lending us their time by taking an in depth look at the draft Code of Practice from the implementation perspective. We are keen to hear from others that may be interesting in providing this type of review and input.
What tools are available to support publisher or library compliance or use of COUNTER-compliant reports?
We offer a series of Friendly Guides for Release 4 of the Code of Practice, in a multiplicity of languages, and will be creating new ones for Release 5. You can find them on our website at http://projectcounter.org/guides.
There’s also our free tool that will check COUNTER usage reports and SUSHI. The COUNTER Report Validation Tool enables publishers, vendors and usage consolidation providers to test their implementation of the SUSHI protocol and COUNTER reports, ensuring errors are caught and corrected before release. The result will be more efficient deployment and improved interoperability. You can find more information about that here: https://www.projectcounter.org/counter-releases-report-validation-tool/.
We also support USUS, the community website which is a great source of information and help on usage metrics, including COUNTER and SUSHI.
Last but definitely not least, publishers and libraries are welcome to contact us through the website to ask for advice and support from Lorraine and the Technical Advisory Group.
I know that COUNTER does a lot of work related to usage data for our community. Beyond the Release 5, what are the new activities that COUNTER is involved in exploring?
COUNTER publishes a list of internet robots, crawlers, and spiders as the minimum requirement for eliminating robots and rogue usage. However, with the recent trend towards Open Access (OA), it has become apparent that more needs to be done to eliminate usage by ‘bad’ or unknown robots, spammers, hackers, dictionary attackers, and other ‘rogue users’. We have formed a new Robots and Crawlers group to discuss and devise ‘adaptive filtering systems’ — sets of algorithms that will allow publishers/repositories/services to follow a common set of rules to dynamically identify and filter out unusual usage and robot activity based on observed behaviors.
We are also collaborating with Making Data Count (MDC) and Jisc in the development of a Code of Practice for Research Data, with the aim of publication in 2017.
Our research with stakeholders shows that there is a great interest in hidden usage, the usage of the same article or item which can happen on a number of platforms, for example a publisher’s platform and on an institutional repository. We have been contributing to the work of Crossref in creating a service for the community that tracks activities around DOIs recording user content.
There is still a huge amount of demand for our library guides in a variety of languages. Currently we are working on Chinese language editions but we welcome feedback from the community on any further guidance that might help implementation and support of the COUNTER Code of Practice.
Thank you all for your contributions to COUNTER on behalf of the community. We look forward to the finalization of Release 5 and to it taking effect in the community. I’m certain that this information will be well received. Of course, if anyone has any further questions or comments about COUNTER’s work, that should be directed to Project COUNTER. Their contact information is available on their website.
