Improved PDF retrieval with Unpaywall integration

As an organization dedicated to developing free and open-source research tools, we care deeply about open access to scholarship. With the latest version of Zotero, we’re excited to make it easier than ever to find PDFs for the items in your Zotero library.

While Zotero has always been able to download PDFs automatically as you save items from the web, these PDFs are often behind publisher paywalls, putting them out of reach of many people.

Enter Unpaywall, a database of legal, full-text articles hosted by publishers and repositories around the world. Starting in Zotero 5.0.56, if you save an item from a webpage where Zotero can’t find or access a PDF, Zotero will automatically search for an open-access PDF using data from Unpaywall.

It can do the same when you use “Add Item by Identifier” to create a new item, and a new “Find Available PDF” option in the item context menu lets you retrieve PDFs for existing items in your library.

We operate our own lookup service for these searches with no logging of the contents of requests.

Unpaywall is produced by Impactstory, a nonprofit dedicated to making scholarly research more open, accessible, and reusable, and we’re proud to support their work by subscribing to the Unpaywall Data Feed.

Zotero can also now take better advantage of PDFs available via institutional subscriptions. When you use “Add Item by Identifier” or “Find Available PDF”, Zotero will load the page associated with the item’s DOI or URL and try to find a PDF to download before looking for OA copies. This will work if you have direct or VPN-based access to the PDF. If you use a web-based proxy, only open-access PDFs will be automatically retrieved using this new functionality, but you can continue to save items with gated PDFs from the browser using the Zotero Connector.

If there are other sources of PDFs you’d like Zotero to use, you can also set up custom PDF resolvers.

Upgrade to Zotero 5.0.56 and Zotero Connector 5.0.41 today to start using these new features.

Introducing ZoteroBib: Perfect bibliographies in minutes

We think Zotero is the best tool for almost anyone doing serious research, but we know that a lot of people — including many students — don’t need all of Zotero’s power just to create the occasional bibliography.

Today, we’re introducing ZoteroBib, a free service to help people quickly create perfect bibliographies.

An animation of viewing a New York Times article, copying the URL to ZoteroBib to add a bibliography item, and copying the bibliography to Google Docs

Powered by the same technology behind Zotero, ZoteroBib lets you seamlessly add items from across the web — using Zotero’s unmatched metadata extraction abilities — and generate bibliographies in more than 9,000 citation styles. There’s no software to install or account to create, and it works on any device, including tablets and phones. Your bibliography is stored right on your device — in your browser’s local storage — unless you create a version to share or load elsewhere, so your data remains entirely under your control.

ZoteroBib is completely free: we don’t bombard you with ads or charge you (or your school) money for full functionality or more advanced citation styles. And since it’s built by the team behind Zotero, and backed by the same open-source community, you can count on the same expertise and attention to detail that people rely on when they write dissertations and scholarly papers using Zotero.

To add an item to your bibliography, simply find it online in another tab and paste the URL into the ZoteroBib search box. You can also paste or type in an ISBN, DOI, PubMed ID, or arXiv ID, or you can search by title. ZoteroBib will fetch bibliographic info for the item and add it to your bibliography. ZoteroBib can import high-quality data from journal articles, books, newspaper and magazine articles, blog posts, webpages, and more. If it doesn’t find what you’re looking for or the data is incomplete, the manual editor allows you to enter data by hand.

As you write, you can quickly copy citations with page numbers to the clipboard for pasting into your document:

Copy Citation dialog with a page number entered

When you’re done, a single click copies a formatted bibliography to the clipboard for pasting into your word processor, or you can generate a bibliography in HTML to add to a webpage. And of course, if you find you need a bit more power, you can easily save your data to Zotero or export it for loading into any other reference manager.

Go to ZoteroBib

Not sure whether ZoteroBib or Zotero is right for you? See the ZoteroBib FAQ.

(Finally, if you’re a happy Zotero user, stay tuned: we’ll be bringing some features from ZoteroBib back to the Zotero web interface soon!)

Zotero 5.0.36: New PDF features, faster citing in large documents, and more

The latest version of Zotero introduces some major improvements for PDF-based workflows, a new citing mode that can greatly speed up the use of the word processor plugin in large documents, and various other improvements and bug fixes.

New PDF features

Improved PDF metadata retrieval

While the “Save to Zotero” button in the Zotero Connector is the best way to save high-quality metadata and associated PDFs to Zotero, Zotero’s “Retrieve Metadata for PDF” feature provides an alternative when you already have a PDF and want to quickly add it to your Zotero library as a citable item.

Traditionally, “Retrieve Metadata for PDF” has been a manual operation that relied on Google Scholar for some files, and long-time Zotero users know that trying to retrieve metadata for many PDFs at once could result in Google temporarily blocking access, causing the feature to stop working.

Zotero 5.0.36 introduces a completely new PDF recognizer, using a Zotero-designed web service that doesn’t rely on Google Scholar and that allows large numbers of PDFs to be recognized without rate limits. The Zotero client will send the first few pages of a PDF to the web service, which uses a variety of extraction algorithms and known metadata from CrossRef, paired with DOI and ISBN lookups from the client as before, to build a parent item for the PDF. The Zotero lookup service doesn’t require a Zotero account and doesn’t log any data about the content or results of searches. No data is now sent to Google Scholar.

Using this new service, Zotero will now automatically attempt to retrieve metadata for PDFs you add, whether that’s via drag-and-drop, “Link to File…”/”Store Copy of File…”, or clicking the “Save to Zotero” button on a PDF in the browser. (In a browser, we still recommend saving from the article page when possible.)

Automatic metadata retrieval after a PDF is dragged into Zotero

If you’re not happy with the results or want to add the parent item another way, you can right-click on the new parent item and select “Undo Retrieve Metadata”. There’s also a “Report Inaccurate Metadata” option to send the first few pages of the PDF and the retrieved metadata to Zotero developers for review.

You can disable automatic metadata retrieval in the General pane of the preferences.

Automatic renaming

When PDFs are added to existing Zotero items or metadata is retrieved for a PDF, Zotero will now automatically rename the file based on the parent metadata. This applies to both linked and stored files. A future version of Zotero will provide additional options for customizing the naming format.

You can disable automatic renaming from the General pane of the preferences. (Files saved along with items from web translators have always been automatically renamed and aren’t affected by this setting.)

Custom PDF viewer setting

Zotero opens PDFs using the system PDF viewer by default, but you can now choose a different PDF viewer to use for files from Zotero. This might come in handy if you’re happy with your system default for most PDFs but there’s a particular program you prefer to use for annotations (or if you’re just not sure how or are unable to change your system default).

'Open PDFs using' setting in the Zotero preferences

Faster citing in large documents

When you insert a citation into a document using Zotero’s word processor plugin, Zotero needs to scan the entire document for citations to ensure correct formatting. Citation style requirements such as ibid or name disambiguation mean that the format of a given citation may depend on the citations that precede it, and bibliographies depend on the presence of, and in some cases the order of, all citations in the document, including any that may have been deleted or moved around since a citation was last inserted.

In longer documents, scanning the entire document can take multiple seconds or even minutes, and these updates can become disruptive to the writing process. Common workarounds include splitting up long documents into chapters or using less-demanding citation styles during writing to increase the speed of citation inserts.

Zotero 5.0.36 adds the ability to disable automatic updates and defer citation updating until a manual refresh is triggered. With automatic updates disabled, citation inserts will remain instantaneous regardless of the size of the document.

To disable automatic updates, click the Document Preferences button in the word processor plugin and uncheck “Automatically Update Citations”:

Zotero Document Preferences window with Automatically Update Citations unchecked

To illustrate how citation inserting works with updates disabled, let’s look at an example. Say we’ve added a citation for a paper by Jessica Smith using APA style:

Citation for '(Smith, 2017)'

If we then insert a paper by James Smith, Zotero will create a citation in the default format required by the style without taking into account other citations in the document:

Citations for '(Smith, 2017)' and '(Smith, 2018)' with a dashed underline on the latter

Zotero adds a dashed underline below newly added citations to remind us that they haven’t been updated (though keep in mind that existing citations later in the document might also now be incorrect).

It also replaces the bibliography with a warning:

Automatic citation updates are disabled. To see the bibliography, click Refresh in the Zotero plugin.

We can quickly insert citations this way without waiting for each update. When we’re ready to submit our document, we click the Zotero plugin’s Refresh button:

Citations for '(Jessica Smith, 2017)' and '(James Smith, 2018)'

Zotero scanned the document and updated the citations and bibliography to conform to the style rules, which in this case require disambiguation for lead authors with the same last name.

To avoid accidentally submitting a paper with unformatted citations, we recommend leaving automatic updates enabled unless you find that inserts are taking too long for a given document.

Other changes

Zotero 5.0.36 includes a number of other improvements and bug fixes, including a cleaner user interface on Windows, lower idle CPU usage, and more. See the changelog for the complete list of changes.

Zotero 5.0 and Firefox: Frequently Asked Questions

In A Unified Zotero Experience, we explained the changes introduced in Zotero 5.0 that affect Zotero for Firefox users. See that post for a full explanation of the change, and read on for some additional answers.

What’s changing?

Zotero 5.0 is available only as a standalone program, and Zotero 4.0 for Firefox is being replaced by a Zotero Connector for Firefox that allows you to save to Zotero as you browse the web, similar to the Chrome and Safari extensions that have been available for years for use with Zotero Standalone.

Why is this happening?

Mozilla is discontinuing the powerful extension framework on which Zotero for Firefox is based in favor of a new, more limited extension framework, and it’s no longer technically possible to create a tool like Zotero for Firefox within the browser.

Will you change your minds?

See above.

Won’t this ruin everything that’s great about Zotero?

We don’t think so. Zotero has been available as standalone version since 2011, and many people have preferred it over the Firefox version — and we’re now able to focus on making it better for everyone, without making compromises to fit everything into a tiny pane or spending time keeping up with constant Firefox changes. The Zotero Connector still provides powerful browser integration and an unmatched ability to save as you browse the web.

In recent months, we’ve made numerous improvements to the Zotero Connector to bring it in line with Zotero for Firefox’s browsing-related features, and the Connector already offers some functionality that Zotero for Firefox never did, with more on the way. Since the Connector works in Firefox, Chrome, and Safari, you can use Zotero with whichever browser you prefer, or even multiple browsers at the same time.

OK, if I’m running Zotero for Firefox, what do I have to do?

First, install the Zotero Connector for Firefox from the download page, which will replace Zotero 4.0 for Firefox. Next, from the same page, install Zotero 5.0 for Mac, Windows, or Linux.

You can also install the Zotero Connector in any other browsers that you use.

If I already have the Zotero Connector for Firefox, what do I have to do?

Make sure you’ve also installed Zotero 5.0 from the download page, and leave it open while you browse the web so you can save directly to it.

If I’m running Zotero Standalone 4.0, what do I have to do?

You can upgrade to Zotero 5.0 via Help -> Check for Updates from within Zotero, or you can reinstall Zotero from the download page.

If you’re also running Zotero 4.0 for Firefox, you should first install the Zotero Connector from the download page and restart Firefox.

If you’re using the Zotero Connector for Chrome, you don’t have to do anything else.

If you’re using the Zotero Connector for Safari, check for updates from the Extensions pane — you should have 5.0.23 or later. You can reinstall the Connector from the download page if you have an older version and an update isn’t showing up.

Will I lose my data?

No. Zotero 5.0 will automatically detect and upgrade your existing data. If your Zotero data directory is located within your Firefox or Zotero profile directory, it will be automatically moved to a “Zotero” directory in your home directory, where it won’t be affected by refreshing or uninstalling Firefox.

Where did the Z button in the Firefox toolbar go? How can I open Zotero without it?

Instead of clicking a Z button in the Firefox toolbar, you now switch to Zotero as you would any other program. (See Switching Between Programs for tips on doing this efficiently. Short version: Use the dock/taskbar/launcher or Cmd-Tab/Alt-Tab. Don’t waste time minimizing or moving windows to access what’s behind.)

You can also arrange your windows so that Zotero is visible while you’re browsing.

What about the “Save to Zotero” button?

The button still exists in the browser toolbar, and, as before, it will show you an icon representing the data Zotero detected on the page: webpage, journal article, newspaper article, etc.

If you don’t see the icon, check your browser’s extensions pane to make sure you have the Zotero Connector installed. In some cases, the button may appear in the overflow panel accessible from the right edge of the toolbar.

How can I choose what collection to save to?

Just as in Zotero for Firefox, select the desired target collection in Zotero before clicking the save button. See Switching Between Programs for tips on accessing Zotero quickly.

An upcoming Zotero Connector version will provide the ability to choose the target collection from within the browser itself. [Update: This is now available.]

Are there any features that are no longer available?

While we’ve worked to make all browsing-related functionality available via the Zotero Connector, a few features either haven’t yet been migrated over or aren’t possible in the new Firefox extension framework.

Planned, but not yet available:

  • “Attach Snapshot of Current Page” (snapshots are still available when saving new items)
  • “Create Zotero Item and Note from Selection”/“Add Selection to Zotero Note”
  • “Save Link As Zotero Item”

No longer possible:

  • “Save to Zotero” option in the Firefox open/save dialog — workarounds: find the article page instead and use the “Save to Zotero” button to download the metadata and PDF automatically (recommended whenever possible); preview the PDF in Firefox and click the “Save to Zotero” button; drag the PDF link to Zotero; save the PDF to disk and drag into Zotero; add Zotero as PDF handler in Firefox and choose it from the open/save dialog (not currently possible, but planned)

What if I can’t install separate programs like Zotero at my institution?

Zotero can be installed without administrative privileges on most systems. You can also ask your IT department to install it for you.

It’s worth noting that the Firefox extension framework used by Zotero for Firefox for many years granted equivalent system access, so from a security standpoint there’s no difference between the previous Firefox extension and the standalone program. If anything, the standalone program is more secure, as the Zotero Connector code running within your browser is limited by Mozilla’s new WebExtension framework.

If you really can’t install separate programs, you can still use the Zotero Connector in your browser and save directly to your online library, but you’ll need to rely on the more limited web interface for managing your data. (Improvements to the web library are planned, but the desktop client will remain the recommended way of interacting with your Zotero data.)

What if I used multiple Firefox profiles to keep my Zotero data separate?

Zotero 5.0 supports the same profile system as Firefox. See Multiple Profiles for more info.

What if I’m using Zotero plugins that haven’t yet been updated for Zotero 5.0?

Most plugins have been updated, at least in beta form, so first check with the plugin author.

If a plugin you rely on hasn’t been updated, you can use the Zotero Connector for Firefox with Zotero Standalone 4.0, which is still available from the download page, for a while longer. Install the plugin from the Tools -> Add-ons -> Extensions pane in Zotero. Some Connector features may not work properly with Zotero 4.0, but you should still be able to save items to Zotero from the Connector.

How much longer can I use Zotero 4.0 for Firefox?

Zotero 4.0 for Firefox will cease to work in Firefox 57, which will be released on November 14, 2017, and existing Zotero for Firefox users will be upgraded to the Zotero Connector shortly before then. If you need to use Zotero 4.0 for Firefox for longer, you should switch to the Firefox 52 Extended Support Release, which will be supported by Mozilla until June 2018, and disable updates for the Zotero extension by clicking “More…” next to Zotero in the Firefox Add-ons pane. (Remember to re-enable updates when you later switch to the Zotero Connector.) If you’re already running the Zotero Connector, you can reinstall Zotero 4.0 for Firefox from the download page. Note that Zotero 4.0 syncing will cease to work in early 2018.

To benefit from the many improvements in Zotero 5.0, and to obtain support in the Zotero Forums, we recommend upgrading as soon as possible.