Looking at Open Science through the Prism of a Social Dilemma

The journal First Monday just published a paper by Kaja Scheliga and Sascha Friesike from the Alexander von Humboldt Institute for Internet and Society. The article is titled “Putting open science into practice: A social dilemma?” What follows is an abstract of the paper.

The essence of open science is to make the whole research process transparent and accessible. The idea of open science can be traced back to the days of the emergence of the scientific journal system when scientists started to publish their insights in the form of scientific papers instead of anagrams. In its current form, open science has gained a new dimension thanks to the internet which provides scientists with the technological means to share their insights on a potentially global scale.

Continue reading

Virtual Itineraries Through the Speculum Romanae

Speculum Romanae Magnificentiae, A125, Special Collections Research Center, University of Chicago Library.
Speculum Romanae Magnificentiae, A125, Special Collections Research Center, University of Chicago Library.

The University of Chicago Library takes a cue from nearly 500 year old publishing practices  to curate personalized, “individually bound” introductions to their digital collection of the Speculum Romanae Magnificentiae:

In 1540 Antonio Lafreri, a native of Besançon transplanted to Rome, began publishing maps and other printed images that depicted major monuments and antiquities in Rome. These images were calculated to appeal to the taste for classical antiquity that fueled the cultural event we call the Renaissance. After Lafreri published a title page in the mid-1570s, collections of these prints came to be known as the Speculum Romanae Magnificentiae, the “Mirror of Roman Magnificence.” Tourists and other collectors who bought prints from Lafreri made their own selections and had them individually bound. Over time, Lafreri’s title page served as starting point for large and eclectic compilations, expanded and rearranged by generations of collectors.

The University of Chicago Library has curated their collection of 994 prints–the largest Speculum collection–by creating miniature exhibitions based on theme, place, or an artist. Visitors can choose a “virtual itinerary to explore the collection” with a specialist in the field as their guide. Professor Evelyn Lincoln, who recently visited UC Davis as one of our invited speakers, is one of the expert guides to the collection. She focuses on “Print and Ritual in Renaissance Rome” and discusses her selection through this lens.

Other “tours” through the archive include “Love and the Gods” led by James Grantham Turner (University of California, Berkeley), “Viewing Ruins” led by Christopher P. Heuer (Princeton University), and “The Eternal City: Maps of Rome in the Speculum” led by Jessica Maier (Mount Holyoke College). The virtual itineraries “allow for a more specialized, but still lively and accessible, introduction to selected works from the collection, draw attention to particular intellectual questions associated with these prints, and serve as a new mode of scholarly publishing.”

I highly recommend extended trips into these curated archives. You will be rewarded  richly.

A152
Speculum Romanae Magnificentiae, A152, Special Collections Research Center, University of Chicago Library.

 

Quick post – papers for sale

Well, this is much more elaborate than I could ever have imagined: For Sale: “Your Name Here” in a Prestigious Science Journal – Scientific American.  Seems that there are services out there to help people write, in essence, bogus scientific papers filled with pithy somewhat reasonable sounding phrases about certain topics.  Seems we could all use some more comprehensive full text analyses of papers to try and flag such activities.

Today’s Open Science Reading: the Open Science Reviewer’s Oath

Well this certainly is interesting: The Open Science Peer Review Oath – F1000Research.  This emerged apparently from the AllBio: Open Science & Reproducibility Best Practice Workshop.  The “Oath” is summarized in the following text from a box in their paper:

Box 1. While reviewing this manuscript:

  1. I will sign my review in order to be able to have an open dialogue with you
  2. I will be honest at all times
  3. I will state my limits
  4. I will turn down reviews I am not qualified to provide
  5. I will not unduly delay the review process
  6. I will not scoop research that I had not planned to do before reading the manuscript
  7. I will be constructive in my criticism
  8. I will treat reviews as scientific discourses
  9. I will encourage discussion, and respond to your and/or editors’ questions
  10. I will try to assist in every way I ethically can to provide criticism and praise that is valid, relevant and cognisant of community norms
  11. I will encourage the application of any other open science best practices relevant to my field that would support transparency, reproducibility, re-use and integrity of your research
  12. If your results contradict earlier findings, I will allow them to stand, provided the methodology is sound and you have discussed them in context
  13. I will check that the data, software code and digital object identifiers are correct, and the models presented are archived, referenced, and accessible
  14. I will comment on how well you have achieved transparency, in terms of materials and methodology, data and code access, versioning, algorithms, software parameters and standards, such that your experiments can be repeated independently
  15. I will encourage deposition with long-term unrestricted access to the data that underpin the published concept, towards transparency and re-use
  16. I will encourage central long-term unrestricted access to any software code and support documentation that underpin the published concept, both for reproducibility of results and software availability
  17. I will remind myself to adhere to this oath by providing a clear statement and link to it in each review I write, hence helping to perpetuate good practice to the authors whose work I review.

I note – I reformatted the presentation a tiny bit here.   The Roman numerals in the paper annoyed me.  Regardless of the formatting, this is a pretty long oath.  I think it is probably too long.  Some of this could be reduced.  I am reposting the Oath below with some comments:

  1. I will sign my review in order to be able to have an open dialogue with you.  I think this is OK to have in the oath. 
  2. I will be honest at all times. Seems unnecessary.
  3. I will state my limits. Not sure what this means or how it differs from #4.  I would suggest deleting or merging with #4.
  4. I will turn down reviews I am not qualified to provide.  This is good though not sure how it differs from #3. 
  5. I will not unduly delay the review process. Good. 
  6. I will not scoop research that I had not planned to do before reading the manuscript. Good. 
  7. I will be constructive in my criticism. Good. 
  8. I will treat reviews as scientific discourses.  Not sure what this means or how it is diffeent from #9. 
  9. I will encourage discussion, and respond to your and/or editors’ questions.  Good though not sure how it differs from #8. 
  10. I will try to assist in every way I ethically can to provide criticism and praise that is valid, relevant and cognisant of community norms. OK though this seems to cancel the need for #7. 
  11. I will encourage the application of any other open science best practices relevant to my field that would support transparency, reproducibility, re-use and integrity of your research.  Good.  Seems to cancel the need for #13, #14, #15, #16. 
  12. If your results contradict earlier findings, I will allow them to stand, provided the methodology is sound and you have discussed them in context. OK though I am not sure why this raises to the level of a part of the oath over other things that should be part of a review. 
  13. I will check that the data, software code and digital object identifiers are correct, and the models presented are archived, referenced, and accessible.  Seems to be covered in #11. 
  14. I will comment on how well you have achieved transparency, in terms of materials and methodology, data and code access, versioning, algorithms, software parameters and standards, such that your experiments can be repeated independently. Seems to be covered in #11. 
  15. I will encourage deposition with long-term unrestricted access to the data that underpin the published concept, towards transparency and re-use. Seems to be covered in #11. 
  16. I will encourage central long-term unrestricted access to any software code and support documentation that underpin the published concept, both for reproducibility of results and software availability. Seems to be covered in #11. 
  17. I will remind myself to adhere to this oath by providing a clear statement and link to it in each review I write, hence helping to perpetuate good practice to the authors whose work I review.  Not sure this is needed.

The paper then goes on to provide what they call a manifesto.  I very much prefer the items in the manifesto over those in the oath:

  • Principle 1: I will sign my name to my review – I will write under my own name
  • Principle 2: I will review with integrity
  • Principle 3: I will treat the review as a discourse with you; in particular, I will provide constructive criticism
  • Principle 4: I will be an ambassador for good science practice
  • Principle 5: Support other reviewers

In fact I propose here that the authors considering reversing the Oath and the Manifesto.  What they call the Manifesto shoud be the Oath.  It is short.  And works as an Oath.  The longer, somewhat repetitive list of specific details would work better as the basis for a Manifesto.

Anyway – the paper is worth taking a look at.  I support the push for more consideration of Open Science in review though I am not sure if this Oath is done right at this point.

The future of Google Scholar

There is an interesting interview out in Nature where Richard van Noorden interviewed Anurag Acharya from Google Scholar: Google Scholar pioneer on search engine’s future : Nature News & Comment.  Definitley worth a look.  It has tidbits on the past, present and future of Google Scholar.

There are also some follow ups to this.  For example on Twitter I saw the following exchange:

I am in general agreement here that the cmmnity needs to start thinking about an open alternative.  Yes, I like Google Scholar (e.g., see my post on the Google Scholar blog: Using Google Scholar in Scholarly Workflows that I wrote in honor of the 10th Anniversary og GS.  But the lack of an API interface and the givng in to publishers demands seems lame.  So I do think we need to start to build up new strategies.  

Crosspost: Using Google Scholar in Scholarly Workflows

I wrote this for the Google Scholar blog where it was posted yesterday.  I am reposting it here, in part because comments are not allowed on the GS Blog (not sure why) and some people have asked me about that.  So here it is


When Anurag Acharya asked me recently if I would be interested in writing a guest post for the Google Scholar blog in relation to the 10th anniversary of Google Scholar I immediately responded “Yes.” Literally, that was the full content of my email response email response to his request. Why did I answer so enthusiastically? Well, I can put this down to three main reasons:

So – in thinking about what to write for this post, I came up with three main topics I thought would be good to cover – how I got interested in topics of searching for and sharing scholarly papers, how I use Google Scholar, and some ideas about future possible uses of Google Scholar.

Part 1: Some Background 

One day, in ancient history, my wife came home from work (at a biotech startup up focusing on bioinformatics) raving about this new search engine “Google” that people at her company were talking about. As someone who thought of himself as on the cutting edge of web technology, I was a bit dismayed that I had not somehow discovered this myself. But I got over that and tried it out. And, after searching for my name (and being impressed with how well this new search engine worked on such an important topic) I immediately started playing around with searching for scientific papers and data. I did this, I guess, because ever since I was in college, I had been becoming more and more interested in (or some would say obsessed with) issues relating to finding and sharing scientific knowledge.

Without going into two much detail some of the factors that contributed to my obsession included:

  • Working as a shelver and then assistant in the Museum of Comparative Zoology library in college and seeing how people struggled to find papers of relevance to their work;
  • Spending many years in graduate school (in the 1990s) working on projects that had been largely unstudied since the 1960s, including one (so called adaptive mutation) where researchers claimed to have discovered something new in the 1990s but had in fact missed a rich literature on the topic from the 1950s and 1960s (e.g., see this from 1955).
  • Building and sharing databases where I was trying to include a description of every paper that had been published about specific genes. I note – thanks to the Wayback machine my Stanford website from when I was a PhD student is still available – although alas the specific linked databases are not. I have reposted some of them for people to see what they were like (though many of the links in them are busted). See for example my sites on RecASNF2MutS and more
  • Working on projects to catalog everything known about specific organisms in association with work I was doing to characterize the genomes of these organisms

In these and other projects I had seen and experienced just how much time could be spent on searching for papers and data about a particular topics. I am not sure I had a well-defined strategy in every case but I came to rely upon some preferred methods including:

  • “Citation walking” where one takes a paper of interest and then asks “how has this paper been cited?” and traverses across the literature via citations
  • Searching for keywords in abstracts and titles
  • Browsing through specific journals
  • Looking for papers by specific authors
  • For data, I mostly would look in specific centralized data repositories such as Genbank for DNA sequence information and PDB for three-dimensional structural data on proteins.

And of course many other approaches. Nothing really novel or brilliant here though I do think I got pretty good at how to carry out such searches. But one of the challenges was each approach had to be done in a different system and some of the systems were only available for a fee and some were not even online. And even with lots of time and pain, many things could be missed.

Thus when my wife introduced me to this new fangled Google thing my thoughts rapidly turned to – how can I use this new tool to help in finding and then sharing scientific papers or data about these genes and organisms I was studying? Did Google searches solve all my “issues” in this regard? Alas, no. But jump forward ~15 years to today and I am quite amazed in retrospect how much of my scholarly workflow flows through Google Scholar. But rather than try to recall and write about how my workflow changed with the advent of Google Scholar I thought I would just jump to the present time and discuss some ways that I use Google Scholar now.

Part 2: Using Google Scholar today

When working on this post I started to look around at how I use Google Scholar and I confess I was amazed at how many different ways I use it in my work. Here are some examples:

Tracking and using citations. One major general use of Google Scholar lies in tracking of citations to specific scholarly works. Here are some ways that I use such information:

  • Citations to individual works. A key aspect of scholarly work in many fields is examining how specific works are cited. Such information has many uses include discovering new works on a topic by seeing how specific papers from the past are cited, assessing impact of works, ego satisfying, and more. For many years, information on how a specific work was cited was nearly impossible to come by without paying for access to citation tracking databases. Now, with Google Scholar I (and others) can very rapidly gather such information.
  • Citation from diverse sources. One aspect of using Google Scholar to track citations to individual works is the way GS finds citations in diverse sources – not just in the peer reviewed scholarly literature. Now, in some ways this can be viewed as a limitation (some may not want to count or even know about citations from self published white papers, for example). But in others ways this is a wonderful thing as one can find citations to one’s work from very diverse sources outside of the “normal” mold.
  • Citation metrics. It is not a large conceptual leap to go from the ability to track citations to individual works to the ability to create summary statistics about citations across many works. There are many indices for such purposes – some useful and some not. But whatever you think of such indices – Google Scholar has opened up the ability for people to calculate such metrics for oneself or to offer services to calculate metrics for others. Such indices can be used in many ways but perhaps the most common is to summarize the citations for one individual researcher. Which leads into my next topic …

Google Scholar Pages. Perhaps my favorite development from Google Scholar in the last 10 years has been the introduction of Google Scholar Pages for individuals. I make use of my Google Scholar page and pages of others for dozens of things including these:

  • Citation metrics for myself. See above for a discussion of citation metrics in general. I use Google Scholar pages to examine citation metrics for myself and my papers all the time (right now GS shows two summary statistics H-index and I-10 index). And I use this information in many ways including putting it on my CV, including it in grant reports, and examining which of my scholarly works have had more “impact”.
  • As landing page for my publication list. Once one has a GS page, GS automatically adds new publications to one’s list and also updates citation counts and other information regularly. Thus I now include a link to my GS page on my blogs, my work web sites, and in my email signature.
  • To keep track of my coauthors. I have been blessed (and perhaps a bit cursed) to work in a field (genomics) where many projects involve large-scale collaborations across many institutions, involving many researchers. And I have found that a nice way to track these coauthors is via GS (although – note to GS folks – there used to be a way to show, publically, all coauthors in a list but I cannot seem to figure out how to do this anymore).
  • Author disambiguation. For people like myself with a relatively unique name, when others search for my scholarly works, they are pretty easy to find (although I note the fact that there is another Jonathan Eisen out there who publishes some works with a bit of a conspiracy theory angle has been both good and bad for me at times). But for many others, their name is not a perfect way to find their work. This may be because they have a name that is relatively common, or it may be because they have changed their name (e.g., after marriage). For such people creating a GS page can be very useful because once one trains GS with a set of works, it can find new works by that same person quite well (I first found out about this author disambiguation by GS when Anurag gave a talk at a meeting I organized last year). GS is certainly not the only tool in author disambiguation and others – like author UIDs (e.g., ORCIDare almost certainly better long term options. I note – author disambiguation may seem like a esoteric topic to many but it has major implications on important issues such as gender equity in academia, since women are much more likely to change their names during their career than men are.
  • Automated updates of new papers by specific authors. One option associated with GS author pages I use extensively is the ability to “follow” specific authors and get notified of new publications of theirs.
  • To keep track of a collection of people. Most researchers do not regularly update their individual publication pages on their websites. However, if those researchers have GS pages one can keep track of their new papers quite easily (either by the follow option mentioned above or just by browsing occasionally). For example, for my microBEnet project I curate a list of GS pages for researchers in the whole field with connections to studies of “microbiology of the built environment” and thus (hopefully) help others keep up with what is going on in the field.
  • Who is in a specific field? One feature of GS author pages that is not used a lot as far as I can tell, but which has some value is the “areas of interest” tag one can add to one’s profile. Though not everyone uses such tags, I have found they are a useful tool in finding researchers working on specific topics. For example, I list “symbiosis” as one of my areas of interest and if I click on the link for that on my page I get a list (sorted by citation counts – which is both useful and annoying) of others who have listed that same area of interest. And many of the people in this list I am not familiar with yet they do work on topics in which I am very interested.

Automated discovery of new papers by topic. Pretty much all scholars these days are drowning in information and in keeping up with scholarly works. There are many reasons for this of course, and there are also some solutions. I find, for example, that social media is a great way to keep up to date on what new papers are coming out or have come out recently. But social media does not find everything and as someone who is responsible for keeping others up to date on various fields (e.g., this is one of my jobs at microBEnet) I also rely on both manual and automated searchers of the scholarly literature to find new papers or old papers I have missed. GS has two key ways to help in this regard. The first is relatively simple in concept but takes advantage of the power of Google indexing – which is just directly searching GS for papers on particular topics. And the advanced search options allow some customization of such searches. But as someone who is quite busy, I do not actually end up searching GS for new papers all that often. Instead I rely upon automated searches through various services including PubmedPubchase, and GS. I use GS in two ways for such automated searches:

  • Create an alert. When one does a search on GS, in addition to results one is presented with an option to “Create an alert” (which I think may only come up if one is logged in with a Google account). I now have dozens of such alerts in operation. To avoid getting drowned by the results I set them up to send only once a week and I filter them into a separate mail folder that I only look at when I have time. But I frequently find interesting new papers this way.
  • GS Updates. Another option now available, if one has a GS profile, is to use the GS Updates system (which I have written about before here and here for example). This system uses one’s publication list to scan for new papers that are related in some way to one’s prior work.

Many other uses of GS. I have gone on perhaps way too long here so I am only going to briefly mention a few other uses of GS.

  • Finding online versions of papers. Unquestionably one of the most valuable uses of GS is to find online versions of scholarly works. But since others have written extensively about this I will just say the following: if you publish any scholarly work I recommend you make it freely and openly available AND that you make sure that it gets indexed by GS.
  • Full text searches of the literature. Another critically important aspect of GS is that it facilitates full text searching of the scholarly literature which is important for many reasons.
  • Finding works outside of the “normal” places to publish. Another key feature of GS is that it indexes much more than just publisher’s sites. If one posts a preprint on one’s own web server, that paper may show up in GS (which I think is a good thing). GS also indexes many diverse sources of scholarly works and thus helps in finding works that may otherwise not see the light of day.

Part 3: Where do we go from here?

As an active user of Google Scholar I of course have many comments, complaints, ideas and thoughts about what it could do better and where it might go in the future. And there are SO many things that could be added or improved upon – things like better figure and table searching, better exporting of information, better abilities to curate and create collections and to then use such collections as training sets for automated searchers, and more and more and more.  I have written about some such issues and suggestions from time to time in my blog (see for example, this and this and this).  There is certainly lots of work to be done.

But in thinking about this I realized that making a list of issues and suggestions is only of limited value. What I think GS really needs is a better public forum where GS can discuss what their plans are for the future and also where users and developers can discuss what they would find useful. And though I see some places for such discussions on the Google Scholar blog and in related sites, I don’t see a lot. So – I would like to end with a call for GS to create a better site for such discussions of the future of GS …


Update – Adding some comments and responses from Twitter