Category Archives: Post Publication: Tracking, Metrics, etc

Video for Gaming Metrics Conference

GamingMetrics

Please follow the links to watch videos of the conference proceedings of “Gaming Metrics: Innovation and Surveillance in Academic Misconduct” held at UC Davis February 3-4th, 2016.

DAY 1

Welcoming remarks (Ralph Hexter, Provost, UC Davis)

FROM PUBLISH OR PERISH TO IMPACT OR PERISH (Mario Biagioli, STS & Law, UC Davis)

GAMING THE GAME ACROSS THE BOARD

  • Timothy Lenoir (UC Davis, Cinema and Digital Media & Science and Technology Studies) (Chair)
  • Alex Csiszar (Harvard University, History of Science) “(Gaming) Metrics Before the Game”
  • Paul Wouters (Leiden University, Science and Technology Studies) “The Mismeasurement of Quality and Impact”
  • Karen Levy (NYU, Media, Culture, and Communication) “Networks of Resistance in Trucking”

UNIVERSITY RANKINGS: GAMING OR COOKING?

  • Martin Kenney (UC Davis, Human Ecology) (Chair)
  • Barbara Kehm (University of Glasgow, School of Education Robert Owen Centre for Education Change) “Global University Rankings: Impacts and Applications”
  • Lior Pachter (UC Berkeley, Mathematics) “How King Abdulaziz University Became a ‘Better’ University than MIT in Mathematics”
  • Daniele Fanelli (Stanford University, METRICS) “Institutional Pressures to Publish: What effects do we see?”

PERSONAL V. INDUSTRIAL CHEATING

  • MacKenzie Smith (UC Davis, University Librarian) (Chair)
  • Finn Brunton (NYU, Media, Culture, and Communication) “Making People and Influencing Friends: Citation Networks and the Appearance of Significance”
  • Sarah de Rijcke (Leiden University, Science and Technology Studies) “System Identity: Predatory publishing as socio-technical disruption”
  • Jeffrey Beall (University of Colorado, Denver, Information Science) “Fake Impact Factors and the Abuse of Bibliometrics”
  • Dan Morgan (University of California Press, Collabra Project) “Cui Bono? Judging Intentions (and Outcomes) of Personal and Industrial Cheating”

META GAMING, META CHEATING

  • Anupam Chander (UC Davis, Law) (Chair)
  • Johan Bollen (Indiana University, School of Informatics and Computing) “From Bibliometric Metrics to Crowd-Sourced Science Funding Systems”
  • Carl T. Bergstrom (University of Washington, Biology) “It’s All a Game: The twin fallacies of epistemic purity and a scholarly invisible hand”
  • Jennifer Lin (Crossref) “Trust through Transparency: O brave new world/ That has such data in’t!”
  • Michael Power (London School of Economics, Accounting) “Research Impact and the Logic of Auditability: Solicited testimony as a case of meta-gaming”
  • James Griesemer (UC Davis, Philosophy) “Taking Goodhart’s Law Meta: Gaming, Meta-Gaming, and Hacking Academic Performance Metrics”

Day 2:

Watch HERE.

Welcoming Remarks (Kevin Johnson, Dean, UC Davis School of Law)

MISCONDUCT WATCHDOGS (I)

  • Jonathan Eisen (UC Davis, Genome Center) (Chair)
  • Ivan Oransky (Retraction Watch & NYU) “Retraction Watch: What We’ve Learned Since 2010”
  • John Bohannon (Science Magazine) “Grey Hat Hacking for Science”
  • Elizabeth Wager (Sideview) “Why Do We Need a Committee on Publication Ethics and What Should It Do?”

MISCONDUCT WATCHDOGS (II)

  • Jonathan Eisen (UC Davis, Genome Center) (Chair)
  • Darren Taichman (Executive Deputy Editor, Annals of Internal Medicine Vice President, American College of Physicians) “A False Sense of Security?”
  • Debora Weber-Wulff (University of Applied Sciences Berlin, HTW, Media and Computing & VroniPlag Wiki) “Documenting Plagiarism in Doctoral Theses: The Work of the VroniPlag Wiki Academic Community in Germany”
  • Brandon Stell (The PubPeer Foundation & CNRS) “Introducing PubPeer”
  • Emmanuel Didier and Catherine Guaspare (EPiDaPo, UCLA) “The Voinnet Affair: New Norms in High-Pressured Science”

COUNTERFEITING BRANDS V. FAKING PRODUCTS

  • Madhavi Sunder (UC Davis, Law) (Chair)
  • Marie-Andree Jacob (Keele University, Law) “Template, Creativity and Publication Ethics”
  • Alessandro Delfanti (University of Toronto, Institute of Communication, Culture, Information and Technology) “ArXiv or viXra? Physics and the quest for the true archive”
  • Sergio Sismondo (Queen’s University, Philosophy) “Leveraging Academic Value in the Pharmaceutical Industry”

CARNIVALESQUE RESPONSES

  • Alexandra Lippman (UC Davis, Innovating Communication in Scholarship) (Chair)
  • Cyril Labbé (Joseph Fourier University – Grenoble I) “Ike Antkare, His Publications and Those of His Disciples”
  • Burkhard Morgenstern (Universität Göttingen, Bioinformatics) “Virtual Editors Can Significantly Improve the Visibility of Junk Journals – A case study”
  • Paul Brookes (University of Rochester, Medicine) “Crossing the Line – Pseudonyms & Snark in Post-Pub Peer Review”

Notes from “Advancing Research Communication & Scholarship”

4dfd167454e9f4745360a9550e189323

I’m currently at the Advancing Research Communication & Scholarship conference. Hackathon, presentations on open data,  open access, and alt metrics. You can follow what’s happening via Twitter at #arcs2015. I’ll add some notes from the conference soon and am excited to present on “Beyond Open: Global Perspective on Research Communication and Knowledge Production.” The panel’s chaired by Brian Rosenblum (University of Kansas Libraries) and I am joining Jane Anderson (NYU) Mónica I. Feliú-Mójer, (Ciencia Puerto Rico and iBiology) and John Willinsky (Stanford). Below is the description:

Current scholarly communication initiatives are focused on expanding access, use, and reuse. This session will explore the relationship between these issues and the needs and goals of the developing world and marginalized communities.  We will consider how new models and expectations affect and address knowledge distribution structures in the developing world, and the control local research communities have over their own legacies and outputs.

Continue reading Notes from “Advancing Research Communication & Scholarship”

The future of Google Scholar

There is an interesting interview out in Nature where Richard van Noorden interviewed Anurag Acharya from Google Scholar: Google Scholar pioneer on search engine’s future : Nature News & Comment.  Definitley worth a look.  It has tidbits on the past, present and future of Google Scholar.

There are also some follow ups to this.  For example on Twitter I saw the following exchange:

I am in general agreement here that the cmmnity needs to start thinking about an open alternative.  Yes, I like Google Scholar (e.g., see my post on the Google Scholar blog: Using Google Scholar in Scholarly Workflows that I wrote in honor of the 10th Anniversary og GS.  But the lack of an API interface and the givng in to publishers demands seems lame.  So I do think we need to start to build up new strategies.  

Worth a look: Article by John Bohannon about arXiv paper from Google Scholar

Definitely worth checking this out (by John Bohannon): Uprising: Less prestigious journals publishing greater share of high-impact papers.   The article discusses a paper recently posted to arXiv by Anurag Acharya (from Google Scholar) and others: Rise of the Rest: The Growing Impact of Non-Elite Journals.

Crosspost: Using Google Scholar in Scholarly Workflows

I wrote this for the Google Scholar blog where it was posted yesterday.  I am reposting it here, in part because comments are not allowed on the GS Blog (not sure why) and some people have asked me about that.  So here it is


When Anurag Acharya asked me recently if I would be interested in writing a guest post for the Google Scholar blog in relation to the 10th anniversary of Google Scholar I immediately responded “Yes.” Literally, that was the full content of my email response email response to his request. Why did I answer so enthusiastically? Well, I can put this down to three main reasons:

So – in thinking about what to write for this post, I came up with three main topics I thought would be good to cover – how I got interested in topics of searching for and sharing scholarly papers, how I use Google Scholar, and some ideas about future possible uses of Google Scholar.

Part 1: Some Background 

One day, in ancient history, my wife came home from work (at a biotech startup up focusing on bioinformatics) raving about this new search engine “Google” that people at her company were talking about. As someone who thought of himself as on the cutting edge of web technology, I was a bit dismayed that I had not somehow discovered this myself. But I got over that and tried it out. And, after searching for my name (and being impressed with how well this new search engine worked on such an important topic) I immediately started playing around with searching for scientific papers and data. I did this, I guess, because ever since I was in college, I had been becoming more and more interested in (or some would say obsessed with) issues relating to finding and sharing scientific knowledge.

Without going into two much detail some of the factors that contributed to my obsession included:

  • Working as a shelver and then assistant in the Museum of Comparative Zoology library in college and seeing how people struggled to find papers of relevance to their work;
  • Spending many years in graduate school (in the 1990s) working on projects that had been largely unstudied since the 1960s, including one (so called adaptive mutation) where researchers claimed to have discovered something new in the 1990s but had in fact missed a rich literature on the topic from the 1950s and 1960s (e.g., see this from 1955).
  • Building and sharing databases where I was trying to include a description of every paper that had been published about specific genes. I note – thanks to the Wayback machine my Stanford website from when I was a PhD student is still available – although alas the specific linked databases are not. I have reposted some of them for people to see what they were like (though many of the links in them are busted). See for example my sites on RecASNF2MutS and more
  • Working on projects to catalog everything known about specific organisms in association with work I was doing to characterize the genomes of these organisms

In these and other projects I had seen and experienced just how much time could be spent on searching for papers and data about a particular topics. I am not sure I had a well-defined strategy in every case but I came to rely upon some preferred methods including:

  • “Citation walking” where one takes a paper of interest and then asks “how has this paper been cited?” and traverses across the literature via citations
  • Searching for keywords in abstracts and titles
  • Browsing through specific journals
  • Looking for papers by specific authors
  • For data, I mostly would look in specific centralized data repositories such as Genbank for DNA sequence information and PDB for three-dimensional structural data on proteins.

And of course many other approaches. Nothing really novel or brilliant here though I do think I got pretty good at how to carry out such searches. But one of the challenges was each approach had to be done in a different system and some of the systems were only available for a fee and some were not even online. And even with lots of time and pain, many things could be missed.

Thus when my wife introduced me to this new fangled Google thing my thoughts rapidly turned to – how can I use this new tool to help in finding and then sharing scientific papers or data about these genes and organisms I was studying? Did Google searches solve all my “issues” in this regard? Alas, no. But jump forward ~15 years to today and I am quite amazed in retrospect how much of my scholarly workflow flows through Google Scholar. But rather than try to recall and write about how my workflow changed with the advent of Google Scholar I thought I would just jump to the present time and discuss some ways that I use Google Scholar now.

Part 2: Using Google Scholar today

When working on this post I started to look around at how I use Google Scholar and I confess I was amazed at how many different ways I use it in my work. Here are some examples:

Tracking and using citations. One major general use of Google Scholar lies in tracking of citations to specific scholarly works. Here are some ways that I use such information:

  • Citations to individual works. A key aspect of scholarly work in many fields is examining how specific works are cited. Such information has many uses include discovering new works on a topic by seeing how specific papers from the past are cited, assessing impact of works, ego satisfying, and more. For many years, information on how a specific work was cited was nearly impossible to come by without paying for access to citation tracking databases. Now, with Google Scholar I (and others) can very rapidly gather such information.
  • Citation from diverse sources. One aspect of using Google Scholar to track citations to individual works is the way GS finds citations in diverse sources – not just in the peer reviewed scholarly literature. Now, in some ways this can be viewed as a limitation (some may not want to count or even know about citations from self published white papers, for example). But in others ways this is a wonderful thing as one can find citations to one’s work from very diverse sources outside of the “normal” mold.
  • Citation metrics. It is not a large conceptual leap to go from the ability to track citations to individual works to the ability to create summary statistics about citations across many works. There are many indices for such purposes – some useful and some not. But whatever you think of such indices – Google Scholar has opened up the ability for people to calculate such metrics for oneself or to offer services to calculate metrics for others. Such indices can be used in many ways but perhaps the most common is to summarize the citations for one individual researcher. Which leads into my next topic …

Google Scholar Pages. Perhaps my favorite development from Google Scholar in the last 10 years has been the introduction of Google Scholar Pages for individuals. I make use of my Google Scholar page and pages of others for dozens of things including these:

  • Citation metrics for myself. See above for a discussion of citation metrics in general. I use Google Scholar pages to examine citation metrics for myself and my papers all the time (right now GS shows two summary statistics H-index and I-10 index). And I use this information in many ways including putting it on my CV, including it in grant reports, and examining which of my scholarly works have had more “impact”.
  • As landing page for my publication list. Once one has a GS page, GS automatically adds new publications to one’s list and also updates citation counts and other information regularly. Thus I now include a link to my GS page on my blogs, my work web sites, and in my email signature.
  • To keep track of my coauthors. I have been blessed (and perhaps a bit cursed) to work in a field (genomics) where many projects involve large-scale collaborations across many institutions, involving many researchers. And I have found that a nice way to track these coauthors is via GS (although – note to GS folks – there used to be a way to show, publically, all coauthors in a list but I cannot seem to figure out how to do this anymore).
  • Author disambiguation. For people like myself with a relatively unique name, when others search for my scholarly works, they are pretty easy to find (although I note the fact that there is another Jonathan Eisen out there who publishes some works with a bit of a conspiracy theory angle has been both good and bad for me at times). But for many others, their name is not a perfect way to find their work. This may be because they have a name that is relatively common, or it may be because they have changed their name (e.g., after marriage). For such people creating a GS page can be very useful because once one trains GS with a set of works, it can find new works by that same person quite well (I first found out about this author disambiguation by GS when Anurag gave a talk at a meeting I organized last year). GS is certainly not the only tool in author disambiguation and others – like author UIDs (e.g., ORCIDare almost certainly better long term options. I note – author disambiguation may seem like a esoteric topic to many but it has major implications on important issues such as gender equity in academia, since women are much more likely to change their names during their career than men are.
  • Automated updates of new papers by specific authors. One option associated with GS author pages I use extensively is the ability to “follow” specific authors and get notified of new publications of theirs.
  • To keep track of a collection of people. Most researchers do not regularly update their individual publication pages on their websites. However, if those researchers have GS pages one can keep track of their new papers quite easily (either by the follow option mentioned above or just by browsing occasionally). For example, for my microBEnet project I curate a list of GS pages for researchers in the whole field with connections to studies of “microbiology of the built environment” and thus (hopefully) help others keep up with what is going on in the field.
  • Who is in a specific field? One feature of GS author pages that is not used a lot as far as I can tell, but which has some value is the “areas of interest” tag one can add to one’s profile. Though not everyone uses such tags, I have found they are a useful tool in finding researchers working on specific topics. For example, I list “symbiosis” as one of my areas of interest and if I click on the link for that on my page I get a list (sorted by citation counts – which is both useful and annoying) of others who have listed that same area of interest. And many of the people in this list I am not familiar with yet they do work on topics in which I am very interested.

Automated discovery of new papers by topic. Pretty much all scholars these days are drowning in information and in keeping up with scholarly works. There are many reasons for this of course, and there are also some solutions. I find, for example, that social media is a great way to keep up to date on what new papers are coming out or have come out recently. But social media does not find everything and as someone who is responsible for keeping others up to date on various fields (e.g., this is one of my jobs at microBEnet) I also rely on both manual and automated searchers of the scholarly literature to find new papers or old papers I have missed. GS has two key ways to help in this regard. The first is relatively simple in concept but takes advantage of the power of Google indexing – which is just directly searching GS for papers on particular topics. And the advanced search options allow some customization of such searches. But as someone who is quite busy, I do not actually end up searching GS for new papers all that often. Instead I rely upon automated searches through various services including PubmedPubchase, and GS. I use GS in two ways for such automated searches:

  • Create an alert. When one does a search on GS, in addition to results one is presented with an option to “Create an alert” (which I think may only come up if one is logged in with a Google account). I now have dozens of such alerts in operation. To avoid getting drowned by the results I set them up to send only once a week and I filter them into a separate mail folder that I only look at when I have time. But I frequently find interesting new papers this way.
  • GS Updates. Another option now available, if one has a GS profile, is to use the GS Updates system (which I have written about before here and here for example). This system uses one’s publication list to scan for new papers that are related in some way to one’s prior work.

Many other uses of GS. I have gone on perhaps way too long here so I am only going to briefly mention a few other uses of GS.

  • Finding online versions of papers. Unquestionably one of the most valuable uses of GS is to find online versions of scholarly works. But since others have written extensively about this I will just say the following: if you publish any scholarly work I recommend you make it freely and openly available AND that you make sure that it gets indexed by GS.
  • Full text searches of the literature. Another critically important aspect of GS is that it facilitates full text searching of the scholarly literature which is important for many reasons.
  • Finding works outside of the “normal” places to publish. Another key feature of GS is that it indexes much more than just publisher’s sites. If one posts a preprint on one’s own web server, that paper may show up in GS (which I think is a good thing). GS also indexes many diverse sources of scholarly works and thus helps in finding works that may otherwise not see the light of day.

Part 3: Where do we go from here?

As an active user of Google Scholar I of course have many comments, complaints, ideas and thoughts about what it could do better and where it might go in the future. And there are SO many things that could be added or improved upon – things like better figure and table searching, better exporting of information, better abilities to curate and create collections and to then use such collections as training sets for automated searchers, and more and more and more.  I have written about some such issues and suggestions from time to time in my blog (see for example, this and this and this).  There is certainly lots of work to be done.

But in thinking about this I realized that making a list of issues and suggestions is only of limited value. What I think GS really needs is a better public forum where GS can discuss what their plans are for the future and also where users and developers can discuss what they would find useful. And though I see some places for such discussions on the Google Scholar blog and in related sites, I don’t see a lot. So – I would like to end with a call for GS to create a better site for such discussions of the future of GS …


Update – Adding some comments and responses from Twitter

Some additional details of my discussion w/ reporter John Bohannon for his Science story on Google Scholar

There is a story in today’s Science magazine on Google Scholar by John Bohannon.  Entitled “Google Scholar Wins Raves, But Can It Be Trusted” the article discusses some of the pros and cons of using Google Scholar.  The author of the article interviewed me on and off over a few weeks about Google Scholar because I have written multiple blog posts on how I use it.  For example see:

And also a diverse array of posts on Twitter which I will not rehash here.

Anyway – the new article covers some interesting points but is very very short (ahh — the fun with page length restrictions).  So I thought I would post here some of the comments I made about Google Scholar in emails with the reporter.

Bohannon wrote to me on December 1, 2013

Dear Dr. Eisen-

I’m writing a news story for Science about Google Scholar. Have you continued using their article recommendation engine? (I saw your blog post about it from last year.)

(and then he wrote some details about what he was working on which I am not sure he would want me to post here and I have not asked so I am leaving them out)

I’d very much like to hear your thoughts on how Google Scholar has developed, and how well it works as a replacement for traditional library/proprietary/non-open literature databases.

cheers and thanks in advance,
John Bohannon

I wrote back, that same day (unusual for me)

Well, was just hacking around with Google Scholar this AM.

Have you dug into their new function – Scholar Library? I am playing around with it but have not quite figured it out. What I am hoping to do is to figure out how to get recommendations based on lists in the Library. Currently, the recommendation engine has one very very big limitation. It bases recommendations on one’s own publications. And if you are trying to move into a new area – well that is pretty useless.

So – some comments

1. I find the recommendation engine to be very very useful still. One of the best ways to find out about new papers. With the limitation mentioned above.

2. I use automated Google Scholar searches to find all sorts of papers of interest. This helps cover topics more broadly than the recommendation engine.

3. There are other recommendation systems out there – but I have not used them too much.

4. As for “free and open” – I don’t think I would use such terminology here. Yes, Google Scholar is free. But is it open? I don’t think so. For example, I am not sure if they publish / release all their code that works behind the scenes. And I am not sure how open the results are (not saying it is not open – I just don’t know).

5. The citation information is a wonderful tool – and it is great to have this information be freely available. I use it routinely for all sorts of purposes including getting around the massive limitations of IF. One issue however with GS is that it takes citations from ALL sorts of sources including non peer reviewed material and even material that people may not have been aware was even publicly available. So – citation counts are generally higher – sometimes much higher – in GS than with other metrics. I note – I put info for my Citations based on GS on my CV and various other places.

And this allows GS to be seriously gamed in terms of citation counts.

6. GS is still clunky in a few ways and I hope that Google puts more effort into it. It could become THE tool for academic scholarship searches and tracking but it has some bugs and minor annoyances still.

Just some quick thoughts. Let me know if you have any other questions.

Jonathan Eisen

 

He then wrote back with some questions.  Again, I won’t share the whole email here at this time but I will share the specific questions he asked.  And I then wrote back with some answers

Question:

** Care to share a recent example? Something nicely illustrates its usefulness.

Answer:

 

Question regarding my comment on other systems

Me: There are other recommendation systems out there –  but I have not used them too much.

Bohannon: ** Are there?  For example?  I should take a gander.

Me: Mendeley has one – called recommended or something like that

Question regarding my comment on free vs. open
Bohannon: Right, good point. Free but not really open. Do you feel like this is a worrying limitation? Is it realistic to make it open?  Or at least, more open?
Me: Well, I am certainly not one to tell others how to run a business.  So I am thankful for any free or open material released / produced by for profits.  But there is a major worry here which is – if we do not know how the system works – we do not know if it is biased or how it can be gamed, etc.  And if it is not fully open then if we invest in making use of it, Google could simply kill it at any time and we would have no source material to use for other purposes.
Question regarding my comment on Impact Factor
Bohannon: What is an example of a massive limitation of IF that GS solves? And I wonder why GS can’t tell the difference between peer-reviewed and non-peer-reviewed sources. I’ll add that to questions below.
Me: GS helps with article level metrics as opposed to journal level metrics like IF.  I can look up citations to all my papers quickly.  And I can track metrics like H index and I10 and others.  IF – being a journal level metric – is not really that informative in my opinion (and that of many others)

Question on other things GS could do:

Bohannon: What do you see as the most important things that need to be fixed before it could really take over?

Me:

1. Transparancy in how it works and it what they are planning
2. Make it open source
3. Ability to create reference collations easily (e.g., like Mendeley or Zotero or CiteULike or Endnote).

Questions for Google

Bohannon: * Any other questions you’d like Google to answer, or features to request?

Me:

Well many features to request.

1. Better hot linking of all authors of a paper. Right now they only seem to link the 1st author to their google scholar page.

2. Better handling of long author lists.

3. Better ability to upload collections exported from other tools.

And many more …

After this Bohannon then wrote me another email with additional questions which I answered, briefly

Bohannon: Questions I should have asked but forgot:
Bohannon: When was the last time you exclusively used paper journals to find articles? (Ever?)
Answer from me:
never – or a very very very long time ago
since I started using email / the web 20+ years ago I have tried to use electronic / digital searches to find papers
Bohannon: What electronic services did you use before GS?
Answer from me:
Pubmed searches of course
And pubmed had some automated email alerts that I used to use
I also used lots of searches of journal web sites
And some journals offered automated searches too
The best thing out there was probably “related articles” in Pubmed
Bohannon: Have you switched entirely to GS?
Answer from me:
No – I still use pubmed searches quite a bit – partly because they are less cluttered than GS searches
I still occasionally search journal web sites but that is rare
And I actually use straight google searches a lot

And then another question

Bohannon: Oh, and when did you start using GS?  And when did it become your main search service?

(Sorry for the shotgun interrogation!)

Answer 1 from me:

heh – I have no idea …

Answer 2 from me:

I note – I think right now Twitter is the best source of information about new papers — far better than GS …

And I looked through my email and found that for a few years I used F1000 automated searches

And then in response to a follow up comment about Twitter from Bohannon I said

Yes, did not say Twitter was fun … but if you follow the right people, they can crowdsource for you all of the right literature better than GS

 

And then on Dec 6 Bohannon wrote me about visualizations

Hi Jonathan-

We’re sorting out some art for the article about Google Scholar.  I’m thinking that the best option would be a cropped screen shot of one of your Google Scholar automatic article recommendations.  Would you be OK with that?
Do they come as an email or appear in browser? If email, you can just fwd it.  If browser, could you take a screen shot and send it?
The idea is to show people what GS article recommendations look like, using your own experience for a visual.

 

So I sent a few things including these

 

Screen Shot 2013-12-06 at 12.09.09 PM

 

Screen Shot 2013-12-06 at 12.05.40 PM

We had a few other discussions about related topics but I thought it might be useful to see that general gist of the discussion and some of my thoughts on Google Scholar in more detail.  I note, at our meeting in February on the Future of Scholarly Publishing and Careers here at UC Davis Anurag Acharya from Google Scholar will be talking and will be on a discussion panel that will focus broadly on “metrics” for scholars so should be interesting.