Longitudinal Study of EJournal Use at a US Chemistry Library

A Longitudinal Study of E-Journal Use at a U.S Academic Chemistry Library: Are we measuring what matters and counting what counts? By Tina E Chrzastowski Chemistry Librarian University of Illinois at Urbana-Champaign Urbana, Illinois USA Submitted July 1, 2010 “Measuring usage and understanding users! E-resources statistics and what they teach us.” Abstract The University of Illinois at Urbana-Champaign (UIUC) Chemistry Library, an early adopter of electronic journals, began collecting e-use data biannually in 2000 A decade later, our longitudinal e-journal use data show that our users were also early adopters who migrated completely from print to e-journals within years During this decade, e-journal use exploded as the Chemistry Library purchased additional content and e-access became the user-preferred format After years of growth in collection size and use, e-use nudged up less than 2% between 2006 and 2008; however, use jumped again in 2010, increasing 9.3% between 2008 and 2010 Over 1.8 million chemistry and science article views or downloads are projected to be accessed by UIUC users in 2010 based on data gathered so far this year These data tell us a lot about our users’ needs, but they also inform cost/use ratios, publisher value, and local citation utilization reports Although enlightening and useful, beyond these broad-brush trend data, what lies ahead in e-journal measurement and analysis? Over 1.8 million chemistry and science views or PDF downloads per year indicate that we have moved beyond user acceptance to a new model However, the next decade will require new measurement tools and a different perspective to help libraries determine user needs and expectations and how to meet them with diminishing budgets Introduction Journal use analysis at the University of Illinois at Urbana-Champaign (UIUC) Chemistry Library was published in 1991 (Chrzastowski) and 1993 (Chrzastowski and Olesko) looking at print use only A 2003 study (Chrzastowski) focused on the migration of users from print to electronic journal use, documenting the leap users made from the established print format to the new electronic format This 2010 study examines a decade (2000-2010) of e-journal use at the UIUC Chemistry Library to look for trends, to benchmark the effectiveness of the collection over time, and to compare publisher use data with other use-related data available in order to develop a better understanding of what “use” of an e-journal article means The decade’s statistics also demonstrate the impact on the library as we moved to the digital age In most science disciplines, journals comprise the subject budget’s largest expenditure This is especially true for chemistry, the discipline lucky enough to host the most expensive journals published (Henderson and Bosch) Pricing fluctuations, in part based on inflation, are annual events that must be balanced with any new money awarded to the fund And since very little new money is finding its way to libraries these days, the result is an evaluation of the collection to determine what cuts must be made or if funds from other material formats might be moved to cover the serial budget Serial cancellation decisions, when necessary, are optimally based on data that include cost and local use by title For the purposes of this study, “cost/use ratio” is used over the term “cost-per-use.” The term cost-per-use implies a very exact number that is comparable to other cost-per-use data But costper-use for a weekly journal cannot fairly be compared to a quarterly journal Another example is trying to compare a journal that has been published for over 20 years in many volumes to a fairly new journal with only a few volumes In order to value each use, the total number of volumes must be taken into consideration to establish a true “cost-per-use.” A much broader term is “cost/use ratio.” This phrase implies a more general formula that is closer to the actual valuations being made when dividing the number of annual uses into the annual cost of a journal Ultimately, the cost/use ratio of a journal can help determine if an annual subscription is warranted or if it is more cost effective to purchase that journal’s articles individually from a document supplier For this collection of research-level chemistry journals, the cutoff is based on the cost of a single document delivery article for that journal For example, if the cost of getting an article from a specific journal from the British Library’s Inside Web (including copyright fees and delivery within 48 hours) is $88.00, then the cost/use ratio for that same journal should be below $88.00 in order to remain part of the subscribed collection This type of valuation is not exact, but for the purposes of determining the relative cost/use ratio for over 800 journals, this process is a way to quickly determine those journals highly cost effective to own and those not pulling their weight The bottom of the ranked listing is where the most attention will be paid during a cancellation project, and this methodology can easily determine high and low cost/use ratios The importance of acquiring, analyzing, and relying on local cost/use journal data cannot be overstated Only local data matter because other libraries’ data have no local relevance In the United States, automobile industry manufacturers must report the mileage range a car can be expected to produce, and these data are always given with the caveat “your mileage may differ” due to the vast types of driving conditions and drivers that will effect the result The same is true for determining the use of library materials – those using/driving the product will ultimately determine use and cost/use ratios Each user/driver is operating under a unique set of research demands, searching skills, needs, time limitations, and subject parameters Therefore, the local group of researchers, all with unique needs, together create the local use that guides the overall collection in a specific discipline Setting The UIUC is a premiere research institution located in the Midwest United States The School of Chemical Sciences is a top-ten, nationally ranked institution comprising the Departments of Chemistry and Chemical and Biomolecular Engineering In addition to collecting and supporting chemistry and chemical engineering, the Chemistry Library is also the home for biochemistry materials Table shows the number of faculty, staff, and graduate students in these subject areas biannually for the past 10 years, the study period for this research (American Chemical Society) Few changes have occurred in the population over the last decade, although recent budget crises have led to a declining number of graduate students and staff over the period The relative stability of these numbers shows that very few changes in our user population have taken place over time, and any increase in e-journal use during the past decade is not due to an increase in the number of users E-journal use data collection began in 2000 based on the print journal collection then housed in the UIUC Chemistry Library The expansive nature of this collection meant that broad science journals such as Nature and Science would be included as well as top journals in physics, a smattering in biology, and Annual Reviews in all science disciplines In order to continue to measure the same collection over time, these journals have remained part of the use study to 2010 However, this means that non-Chemistry users campus-wide are measured, too, because we cannot track individual users to isolate use by Chemistry affiliates only Although this greatly expands the user population, it remains essentially the same from 2000 to 2010 No large increases in user populations campus-wide are discernable during the study period Therefore, these are valid comparisons over time for the e-journal use but not valid compared to print measurement As noted later in this paper, print and e-journal use statistics are not comparable due to a number of factors chemistry e-journals users at UIUC (2001-2009) (American Chemical Society) Not included in this table are secondary users such as food scientists, plant scientists, ecologists, etc Table Estimated number of primary University of Illinois at Urbana-Champaign (USA) 2001 Total "Chemistry" Faculty Total "Chemistry" Graduate Enrollment Total "Chemistry" Full Time Staff Total Potential Chemistry Users 2003 2005 2007 2009 98 89 94 98 100 465 454 527 537 503 75 74 68 70 68 638 617 689 705 671 Methodology As with any longitudinal study, research methods have a way of changing over time due to a number of factors, including technological changes (LOTUS 1-2-3 and 5.25” floppy disks no longer being available), new standards implemented (Project COUNTER), and physical changes (print journals no longer on the shelf cannot be counted) Longitudinal studies therefore result in data that may not precisely match the collection methods of previous studies, even those conducted just a few years earlier This is especially true with e-journal use data over the past decade In 2000, when e-journal use statistics were just beginning to be made available to libraries, few options were available Because technologies now allow for more detail and Project COUNTER has established a standard, publishers have made a number of changes in data reporting since 2000 For example, the American Chemical Society has changed their statistical reporting for e-journal use twice in the past ten years, once mid-month, making it challenging to accurately compare data between these different systems with their different definitions Another issue with e-journal data collection is that in early years statistics are broad-brush with vague definitions, such as “number of article requests,” and often publishers simply could not provide data at all In 2000, the first year we attempted to acquire e-journal use data from publishers, only 15% of titles had corresponding use statistics available Further complicating usage comparisons by title, publishers often choose to separate “backfile” (archive) use and “frontfile” (current) use For the purposes of this study, we have, where possible, combined front- and backfile use under a single title Because of these discrepancies, the methodology for collecting e-journal use data has had to adapt to new definitions and categories of use over time In 2000, the standard was to report the number of times a user from a selected institution attempted to access an article Within a few years, this changed so that the type of download (HTML or PDF) was reported as well as views of Table of Contents and Abstracts Over the past decade, definitions of “use” have become more specific and therefore more precise, which has led to better, more precise counting and, therefore, less broad-brush counts even before COUNTER standards were adopted Currently, when available, we use COUNTER Journal reports and 3: “Number of Successful Full-Text Article Requests by Month and Journal.” Report adds data for the viewing of Table of Contents and Abstracts Although these “viewing” numbers are smaller when compared to the PDF and HTML download statistics, this report most closely replicates data provided beginning in 2000 However, as made clear by the many challenges facing longitudinal data collection during times of change, journal use data are not perfectly matched across the decade of data collection and should be seen as best estimates that can help to identify trends E-journal use data are collected from publishers’ Web sites for January, February, and March of each study year These months not represent our busiest periods (which are mid-to-late semester, i.e November and April) However, the methodology for these studies is remnant from the first (1988) study that recorded print use for six months, January to June The difficulty of this length of study led to a shortened version beginning in 1993, and since that time data have been collected for three months and doubled Those data are doubled again when compared to annual cost in order to establish a cost/use ratio The three month period chosen, although not our busiest time, may still result in a somewhat higher estimate of use than the actual use, but again, the numbers are best estimates and useful for comparing within the study and identifying trends Unfortunately, UIUC is not yet SUSHI compliant but is implementing Verde very soon It is hoped that by the 2012 study, collecting and analyzing use data will be much easier, incorporating COUNTER use data, SUSHI delivery, and local cost data from Verde Table Success rate for acquiring e-journal use data for individual titles from publishers in 2008 and 2010 The “No Statistics Available” category includes Open Access journals Acquiring E-Journal Use Statistics Have Statistics for Journals Statistics "forthcoming" Couldn't find No statistics available No Info / Still looking % Success Rate 2008 714 2010 812 10 60 42 35 84.7% 63 21 38 85.4% Table shows the success rate for acquiring use data in 2008 and 2010 Although nearly 100 more journals have use data in 2010 than in 2008, not all titles are able to provide use data The 42 titles with “no statistics available” in 2010 are mostly open access journals; and the publishers of these journals neither keep nor provide use statistics Print use is shown with e-journal use in Table to provide a baseline of journal use prior to electronic access These data show when our users switched to e-journals as their primary method for accessing journal articles The methodology employed for measuring print journals is found in Chrzastowski (1991), Chrzastowski and Olesko (1997), and Chrzastowski (2003) Although they are both reported in Table 3, print journal use and e-journal use are not comparable Print use is measured when a bound volume or an unbound issue is “used,” for example checked out, taken off the shelf, moved to a cart, or photocopied The print use methodology cannot show how many articles within an issue or a volume were used, whereas ejournal use is specific to the exact article and type of view or download The convenience factor alone will cause e-journals to be used many times more than print, and combined with the more precise use measurement, e-use would be expected to exceed print many times over However, data on print use, although not comparable to electronic use, inform those data by providing another, distinct baseline measurement Identifying the cost of a journal subscription is often more difficult than acquiring use data Journal subscriptions are purchased in “Big Deals,” through consortia, in series packages, tacked on free with another subscription, and, sad to say, sometimes we have no idea how we are getting a journal To determine pricing data, journal titles are first checked locally on subject-fund spreadsheets and against lists of titles purchased in package deals by publisher Ebsco is another source used to identify our local, annual pricing, and when every effort to find the price we paid fails, we use the standard price from the Ebsco database, which may or may not include our discounts As with use data, journal prices are best estimates that lead to identifying trends in cost/use ratios for each title we purchase When the database of titles is populated with use and cost information, the formula to calculate the cost/use ratio is simple: annual cost is divided by annual use and ranked by cost/use ratio with the most cost effective titles at the top of the list Purposes for Data Collection and Analysis Although recent technologies, including the Project COUNTER standards and access to more publishers’ data, have made the job of collecting, organizing, and analyzing e-journal use data easier over time, this process is still very time-consuming, difficult, and more than a little messy, and it results in an inexact product Even Project COUNTER data require local adjustments and the additional time of adding cost data Why continue to it? The primary reason is accountability These data provide the information needed to weigh price increases and to estimate the local worth of the access price The resulting cost/use ratios are extremely important for justifying a very high-priced collection of journals These data provide welcome information when collection decisions must be made that reflect local use while maximizing collection dollars The resulting product, in our case a ranked listing of cost/use ratios for the over 800 titles in the Chemistry Library’s e-journal collection, is extremely valuable The top titles rarely fluctuate from year to year, so most of the attention is focused at the bottom of the list These are titles that have little or no use Why? It may not be as simple as saying no one used them because they were not needed Are they indexed in SciFinder Web, PubMed, Web of Science, or Google? Was our access impeded or turned off during the year? Did the journal undergo changes in editorship, content, or focus? Is the country of origin undergoing strife or unrest leading to an interruption in publication? These questions are applied to titles that not meet the Inside Web document delivery cost comparison discussed previously (a single article from that journal to be delivered in 48 hours, a cost/use ratio averaging $80-$100), that is, use of this journal might be better purchased on demand than subscribed to annually Are there other data sets that can explicate journal use for a defined group of researchers? The inherent problems with acquiring cost and use data have led to researching other available data including Local Journal Utilization Reports and SFX linking data These e-measures also have limitations but can verify information in collection evaluation and answer related questions about journal use and use of indexes by specific database These products are discussed further in the Results section of this paper Results: What Do the Data Show? Table shows longitudinal journal use data for the UIUC Chemistry Library from 1988 to 2010 E-journal use data collection began in 2000, although only 15% of titles had publisher-supplied use data at that time In 2010, that number has grown to 84% compliance, but this is still not the 100% one would expect By late 2006, when the library was remodeled and moved to a new, smaller location, print bound journals duplicated in electronic format were moved off-site to a storage facility and are paged if needed However, as Table shows, by 2006 print journal use was a mere fraction of total journal use, and this trend was firmly established as early as 2002 These longitudinal data, although imperfect, clearly show that the migration to e-journals by chemists at UIUC was swift and decisive Table Print and e-journal use 1988-2010 for the UIUC Chemistry Library Print use data collection ceased in 2008 as many journals were moved off-site Print use data include ILL use of the collection (Chrzastowski, 2003) Data are for six months; multiply by for annual use Print Journal Use E-Journal Use Total use 1988 1993 1996 1998 2000 2002 2004 2006 2008 2010 31,703 46,984 42,490 44,650 31,234 20,498 6,471 618 NA NA 0 0 64,590 323,146 674,110 848,670 863,620 943,066 31,703 46,984 42,490 44,650 95,824 343,644 680,581 849,288 863,620 943,066 There are a number of reasons why e-journal use increased dramatically between 2000 and 2006 As previously mentioned, statistical reporting by publishers did not reach a critical mass until after 2002, and more publisher data increased journal use counts Table shows the number of journal titles in each study period and the percentage of publisher-produced use data we could access for each study year These data, and the reasons for no data available in Table 2, suggest that we may never reach 100% compliance, especially with the increase in open access journals Table The number of e-journals in each study, the number of titles for which data were available and the percentage of data available for that study period 2000 2002 2004 2006 2008 2010 247 37 15% 405 260 64% 687 479 69.7% 881 714 81.0% 836 714 85.4% 959 812 84.7% Total Number of Chemistry e-journals in each study year Statistics Success % Statistics Success Rate The period between 2000 and 2006 was also a time of major investment in backfiles and new electronic content at this library, as well as a time of improvement in journal Web sites, bandwidth, and local networking speeds, which all helped to increase use as e-journals began to show promise in saving researchers’ time and energy Also increasing during this time were the number of open access journals and the prominence of Google and Google Scholar Convenient, free, and reliable indexing combined with more advanced UIUC linking capabilities (SFX) increased e-journal traffic significantly until the plateau that began in 2006 and continued in 2008 The usage plateau that took place between 2006 and 2008 was short-lived and looks to be partly attributable to a plateau in the number of titles in each study In contrast, use rose by 9.3% in 2010 (compared to 2008) This rise may seem to be attributable to an increase in the number of journals in the study, increasing from 836 titles in 2008 to 959 titles in 2010 (Table 4) However, looking closer at the data for individual journals and publishers shows that online use of the major journals did, in fact, increase The Table data also reflect that we are not at 100% compliance in obtaining publisher-based use data Although most major publishers (28 of 30 in the 2010 study) are providing data in COUNTER format, there are still small, often single-journal publishers that don’t provide use data at all, in any format Since we are more likely to cancel a journal with no statistics than low 10 statistics, over time these publishers’ titles will be weeded from the collection and therefore this use study It may be that 100% compliance in gathering use statistics will come from having to move solely to major publishers as the smaller publishers are left behind Another part of the difference in 2010 between the number of journals in the study and those with use data can be attributed to the increasing number of open access journals in 2010 and the fact that use data for these titles (over 40 in the 2010 study) are not made available by their distributors These data would be extremely interesting and open access publishers should be encouraged to collect and release use data for individual academic libraries In response to our request for data, the DOAJ (Directory of Open Access Journals) replied, “Many are interested in statistics so we will keep it in mind for future developments.” The number of conclusions that can be drawn from the combined longitudinal data in Table are limited due to the vast changes that have taken place in this collection over time Overall, we can only say that use continues to rise as number of titles increase, that this collection is costeffective overall, and that our ability to measure use has improved but is not perfect However, the fact that we are sustaining over 1.8 million views and downloads per year in chemistry and science titles (out of an estimated million annual downloads for the UIUC campus) alone is good news Chemists are users that have fully embraced this format and are now moving on to ereference and e-books Tables and show the top titles and the “80/20” rule This rule is derived from Pareto’s principle that 80% of causes come from 20% of effects; the 80/20 rule in libraries predicts that 80% of use will be attributable to 20% of the collection Table shows that the Chemistry Library e-journal collection is slightly higher than this prediction, averaging 86% of use attributable to 20% of the collection (2006-2010) Clearly the top titles continue to generate high use over time, creating a very long “comet tail” of lower-use titles: 14% of use comes from the remaining 80% of the collection (650 titles in 2010) 11 Table The “80/20 Rule” applied to e-journal use data 2006-2010 Data are for six months 80/20 Rule Analysis Total Journals with Statistics 20% of titles with Statistics Use of the top 20% Percentage of use for 20% of titles 2006 2008 2010 714 143 746, 534 88% 714 143 735,778 85% 812 162 811,428 86% Table looks even more closely at the “top” titles, focusing on the twenty most heavily used titles These high performance titles have continually, since 2006, represented about half of all use The 2000 data are uncharacteristically high due to the fact that only 37 titles provided use data at all in that first year of data collection In 2002 and 2004, only about 60% -70% of titles supplied use data; by 2006, that number had reached 81%, allowing a bigger picture to emerge and showing the continued staying power of the top 20 titles overall Table Use of the top 20 titles, 2000-2010, and the percentage of use they generated Data are for six months Year 2000 2002 2004 2006 2008 2010 Total Use months 64,590 323,146 674,110 848,670 863,620 943,066 Use of the Top 20 Titles 62,984 239,672 409,636 448,160 438,094 478,282 Top 20 Titles: Percent of Total Use 97.5% 74.1% 60.8% 52.8% 50.7% 50.7% In addition to the few titles behind high-use, a few publishers also represent the highest-use of this collection Table shows the number of titles and use by six major publishers from 2000 to 2010 This analysis is a good way to compare overall publisher activity, use compared to number of titles, and performance over time Again, these data show local performance only, and use (mileage) will vary by owner These six publishers account for 741 titles (77% of titles) and 79% of total use in 2010, and even a slight slip in use or cost/use ratio over time would not be a consideration for cancellation However, the data show that some publishers are adding titles to their collections that are either out of scope for chemistry users or are still finding an audience 12 This is why title-by-title analysis is critical to journal evaluation unless a “Big Deal” package makes a bulk purchase worthwhile, with most titles carrying the load of new or low-use titles Table Number of titles and use data by publisher with percent change, 2000 and 2010 Use data are for six months Not all publishers could supply use data and full data from these six publishers was not available until 2006 PUBLISHER American Chemical Society Number of Journals Online Use Elsevier Number of Journals Online Use Nature Publishing Group Number of Journals Online Use Royal Society of Chemistry Number of Journals Online Use Springer Number of Journals Online Use Wiley Number of Journals Online Use Total Number of Journals Total Use (6 months) 2000 2002 2004 2006 2008 2010 23 59,744 30 55,474 32 102,702 34 179,254 38 220,826 55 311,716 NA 58 15,144 154 90,298 273 149,264 249 170,230 290 205,166 NA NA 15 126,648 20 97,428 20 60,554 20 83,794 NA 11 3,046 30 10,800 28 31,958 27 33,802 51 20,658 NA NA 23 NA 112 7,666 117 19,210 125 17,518 NA 95 60,086 102 58,746 178 110,078 243 108,440 200 106,520 23 194 356 389,19 133,750 645 694 741 575,648 613,062 745,372 59,744 As previously mentioned, e-journal use should be analyzed by individual title Just as local use is critical to understanding local collections, the broad-brush data shown in Table are not conducive to any kind of decision-making process and are only of limited value to identify broad trends Data by individual title are appropriate for collection decisions because the data are wellcontrolled and therefore create apples-to-apples comparisons The Journal of the American Chemical Society (JACS) is one of the premiere journals in chemistry and fairly consistent use documentation is available over time Table shows print and electronic use of JACS from 1988 to 2010 (for six months) for the UIUC Chemistry Library 13 Table Print and e-use for Journal of the American Chemical Society at the UIUC Chemistry Library Data are for months; double for annual use The asterisks show that these years not include use from the ACS “archive.” J American Chemical Society Print Use (6 months) 1988 1993 1996 1998 2000 2002 2004 2006 2008 2010 3,176 5,334 5,094 5,266 2,894 2,860 304 NA NA NA 15,348* 13,066* 16,230 * 28,321* 30,47 36, 258 18,242 15,926 16,534 28,321 30,47 36,258 Electronic Use (6 months) Total Use (6 months) 3,176 5,334 5,094 5,266 Because of reporting changes that did not include the separately purchased archive collection (pre-1996 articles), 2002 to 2006 data may be undercounts UIUC purchased the archive in May 2002, but archive data from ACS were not separated and ACS did not report archive use by title until 2008 However, the 2008 data confirmed that the older articles, those pre-1996 in the archive collection, did not add significantly to the use count when added beginning in 2008 Similar to the combined journal use data in Table 3, use of JACS rises steeply until 2006, and then use levels off in 2008 And like the composite data, JACS use jumps again in 2010 In this case, however, this is a single title for which e-use data have been available since 2000 This increase in use is not due to better counting or additional titles, or even the additional data added for archive use, but may be due to better indexing and linking through SciFinder Scholar (now SciFinder Web) and local SFX implementation These use data can be further illuminated by comparing annual UIUC JACS use to Local Journal Utilization Reports (LJUR) data obtained from Thompson ISI for the University of Illinois at Urbana-Champaign Table shows this comparison and calculates a percentage of total use resulting in citations by UIUC faculty to JACS This table shows that with the increase in use over time comes a decrease in the number of viewed articles that result in a citation in a UIUCauthored article Citations to JACS articles increase beginning in 2002 and continue to rise into 2010, but are dwarfed in comparison to views and downloads of articles If faculty and graduate students are not reading and downloading JACS articles in order to cite them, why are 14 they viewing them at such high rates? The answer is elusive, but some clarification could come from other data, such as trying to obtain a sense of where these users are coming from before entering the JACS site Unfortunately, and as discussed later in this paper, ACS does not include those data in their reports, and repeated efforts to obtain them have so far been unsuccessful Table Annual UIUC JACS use compared to annual number of UIUC faculty citations in JACS LJUR data are currently only available to 2006 for UIUC 1988 1993 1996 1998 2000 2002 2004 2006 Annual JACS Use (Print and Online) 6,352 10,668 10,188 10,532 36,484 31,852 33,068 56,642 LJUR Data: Annual UIUC Faculty Citations to JACS Articles 1,124 1,216 1,083 1,001 985 1,132 1,212 1,656 17.6% 11.3% 10.6% 9.5% 2.6% 3.5% 3.6% 2.9% Percentage of Use Resulting in Citations Another attempt to answer this question was made using SFX statistics and local use statistics that report use through our proxy server, unfortunately with little success Data were collected through SFX statistical query 11, “Most popular journals selected by source” for use of JACS, January-March 2010 Use of JACS for this period was 19,304 uses and SFX shows links to full text through SciFinder Web, Scopus, Web of Science, Google, and PubMed total only 307 links for this same period; whereas links to full text from our local A-Z list total 741 This analysis shows that only 5% of use of JACS articles for three months comes from a local source (via proxy server) or major indexes Where the other 95% of uses originate? This same URL referral question was posed and answered by Davis through the use of ACS transaction logs that detailed URL referrals to ACS articles for Cornell University (Davis, 2004) Davis’s three-month study of ACS article use by Cornell faculty and students analyzed the URL that referred users to the ACS full-text article He found that Cornell researchers accessed nearly 10,000 articles primarily through three means: library catalog (24.9%), a bibliographic database (23.8%), or a list of e-journals (such as UIUC’s ORR) (18.2%) 15 Based on SFX UIUC data, it is clear that UIUC researchers are being referred to articles outside of easily identifiable channels The most obvious referrals that can’t be quantified as originating with SFX or the local proxy server include linking to an article from their own bibliographic database, through email from colleagues, or from a bookmark Further research is needed to confirm these assumptions, and, as noted earlier, data from ACS have been requested The Missing Pieces: What Is Needed? Now that we have a decade of use data to reflect on, what can it tell us? What can we learn from the past ten years? Certainly the first ten years of access to e-journals is similar to the first huge lift hill of a roller coaster ride Like that lift hill, e-journal use is still climbing upward This upward escalation of use is attributable to 1) increased online content over time, 2) improved network speeds and data transmission, 3) better data reporting; and 4) better access and linking tools However, the data that publishers are sharing, while interesting and useful, not tell us much more than quantity This is a good beginning, but it is time to push the envelope again and direct publishers to tell us more There are two data sets that would help to explicate journal use: URL referral data (which show from which site or URL users entered the journal article) and number of unique versus duplicate downloads by IP address The URL referral data is a critical next step These data will tell us whether local catalogs, A-Z ejournal lists, and the indexes and abstracts we purchase are working and how well they are working The URL referral also tells us more about the pathways scientists take to information retrieval In order to simplify this reporting, a COUNTER report for URL referral could offer a limited number of categories suggested by the results from Davis’s research These include not only referrals from local catalogs, bibliographic databases, and A-Z e-journal lists but also Web pages, Web searches, and email (Davis, 2004) These data would help libraries prioritize our resources and determine just what tools are important to users Data showing the number of unique and duplicate IP addresses accessing each article for a given time period will address repeat uses and users Currently, we have no way of knowing how many uses represent multiple repeat visits to that site by a single IP address and how many are unique 16 visits from a single IP address High use for a single article may represent e-reserves (returning users due to classroom assignments) or may uncover researchers who simply use the e-journal as their personal file cabinet, returning regularly to use a core article as needed Both unique and duplicate uses are legitimate, and both represent a need that libraries are filling Although these two new measurements top my desiderata list, a third comes to mind An abbreviated IP range display would show where users are coming from, whether on campus or off Use showing IP ranges using the VPN or proxy server would shed light on user behavior or number of unique IP addresses accessing each article for a given time period But these data would also show the geographic places (labs, research centers, libraries) where users are conducting online journal searching As long as privacy matters are addressed, perhaps by limiting to a general domain name (i.e., Chrzastowski Lab or Chemistry Library) rather than the full IP address, having a sense of users’ locations and affiliation within the campus would again help libraries to determine better ways to deliver and organize information Are We Measuring what Matters and Counting What Counts? Even though it has been often difficult and frustrating to cajole publishers into sharing use data, the efforts have been overwhelmingly worthwhile, mostly successful, and enlightening for both publishers and libraries After a decade of adjustments and alterations in statistical reporting, we are at least measuring and counting, even if we are still collectively deciding what matters Project COUNTER and the advent of SUSHI implementation have, when available, applied a uniform standard for measurement and a delivery option to speed things along However, the last few publishers who still need to adopt COUNTER and SUSHI must be prodded to make the leap Until then, conducting use studies will continue to be, frankly, a lot of work Libraries must their part, too, employing these data to make decisions and establish baseline use levels Next steps include not only adding more depth of analysis to COUNTER statistics but also determining the value these resources provide to users Value is a serious next step because budgets are likely to be in decline into the foreseeable future; value can help determine how to ultimately serve the user in the most cost-effective and efficient way, leading to the greatest 17 value Studies underway include LibValue, a joint study between the Association of Research Libraries, the University of Tennessee, and the University of Illinois (http://libvalue.cci.utk.edu/) This three-year study, begun in December 2009, is focusing on value and return on investment (ROI) in academic libraries Another value-oriented research study currently underway is the UK’s RIN, Research Information Network (http://www.rin.ac.uk/ ; RIN 2008) However, it is important to remember that in any value or ROI study, use is, and will continue to be, a critical measurement Local use is the fuel that will drive value studies, and every effort publishers and libraries can make to quickly and efficiently produce, collect, and analyze use data is an effort well worthwhile Acknowledgement The author gratefully acknowledges the help and expertise of Meg Griffin, a student in the UIUC Graduate School of Library and Information Science and a Graduate Assistant in the UIUC Chemistry Library Her excellent work in collecting, organizing, and analyzing data for this research has been essential to the success of the project Thank you, Meg! 18 References Cited American Chemical Society, Directory of Graduate Research Online (DGRWeb) http://dgr.rints.com/ Accessed June 11, 2010 Chrzastowski, Tina E 1991 Journal collection cost-effectiveness in an academic chemistry library: results of a cost/use survey at the University of Illinois at Urbana-Champaign [presented at the 1989 ACRL Conference] Collection Management v 14(1/2), p 85-98 Chrzastowski, Tina E 2003 Making the Transition From Print to Electronic Serial Collections: A New Model for Academic Chemistry Libraries? Journal of the American Society for Information Science and Technology v 54(12), p 1141-8 Chrzastowski, Tina E and Brian Olesko 1997 Chemistry journal use and cost: results of a longitudinal study [at the University of Illinois Chemistry Library in 1988, 1993 and 1996] Library Resources & Technical Services v 41, p 101-11 Davis, Philip M 2004 Information-Seeking Behavior of Chemists: A Transaction Log Analysis of Referral URLs Journal of the American Society for Information Science and Technology, 55(4), p 326-332 Henderson, Kittie S and Stephen Bosch, 2010 Seeking the New Normal [Periodical price survey 2010] Library Journal v 135(7), p 36-40 Research Information Network, “E-journals: their use, value and impact.” http://www.rin.ac.uk/our-work/communicating-and-disseminating-research/e-journals-their-usevalue-and-impact Accessed June 27, 2010 19 ... data with other use- related data available in order to develop a better understanding of what ? ?use? ?? of an e-journal article means The decade’s statistics also demonstrate the impact on the library. .. collecting and analyzing use data will be much easier, incorporating COUNTER use data, SUSHI delivery, and local cost data from Verde Table Success rate for acquiring e-journal use data for individual... local cost /use journal data cannot be overstated Only local data matter because other libraries’ data have no local relevance In the United States, automobile industry manufacturers must report