Zotero Translators

Translators are at the core of one of Zotero’s most popular features: its ability to import and export item metadata from and to a variety of formats. Below we describe how translators work, and how you can write your own.

This page describes the function and structure of translators. For in-depth documentation on how to write translator code, see Coding.

Note: Before writing a translator for a site, look at the documentation on exposing metadata; website authors should try embedding the necessary metadata before attempting to write a translator.

If you're looking for a broken translator to fix, see the recent translator errors and check on one of the top reported errors. You can also check the status of many translators by reviewing the translator test overview.

Zotero translators can operate in four different ways (note that translators are not necessarily restricted to a single type):

• Web translators: can be activated when visiting a website. E.g. with Zotero for Firefox, an icon appears in Firefox's address bar when a translator finds item metadata on the loaded webpage. Clicking this icon will activate the translator, saving the item metadata into your Zotero library.
• Import translators: can import item metadata from one of the standard storage formats, such as BibTeX or RIS, into your Zotero library. The data to be imported may be supplied as a file, through the operating system's clipboard, or it may be delivered through a web translator (in this case, the role of the web translator is typically restricted to retrieving the metadata, with the import translator doing the actual parsing).
• Export translators: can export item metadata from items in your Zotero library to a file in one of the standard storage formats (like BibTeX or RIS).
• Search translators: can look up and retrieve item metadata when supplied with a standard identifier, like a PubMed ID (PMID) or DOI.

Zotero translators are stored as individual JavaScript files in the “translators” subdirectory of the Zotero data directory. Each translator contains a JSON metadata header, followed by the translator’s JavaScript code. This code must include certain top-level JavaScript functions, as determined by the translator type(s).

An example of a JSON metadata header is shown below:

{
	"translatorID": "fcf41bed-0cbc-3704-85c7-8062a0068a7a",
	"label": "NCBI PubMed",
	"creator": "Simon Kornblith, Michael Berkowitz, Avram Lyon, and Rintze Zelle",
	"target": "https?://[^/]*(www|preview)[\\.\\-]ncbi[\\.\\-]nlm[\\.\\-]nih[\\.\\-]gov[^/]*/(?:m/)?(books|pubmed|sites/pubmed|sites/entrez|entrez/query\\.fcgi\\?.*db=PubMed|myncbi/browse/collection/|myncbi/collections/)",
	"minVersion": "2.1.9",
	"maxVersion": "",
	"priority": 100,
	"inRepository": true,
	"translatorType": 13,
	"browserSupport": "gcsbv",
	"lastUpdated": "2014-06-20 04:23:04"
}

The roles of the different metadata fields are:

• translatorID
  The unique internal ID by which Zotero identifies the translator. You must use a stable GUID, as the translatorID is used for automatic updating of translators, and for calling translators within other translators.
• translatorType
  An integer specifying to which type(s) the translator belongs. The value is the sum of the values assigned to each type: import (1), export (2), web (4) and search (8). E.g. the value of translatorType is 2 for an export translator, and 13 for a search/web/import translator, because 13=8+4+1.
• label
  The name of the translator.
• creator
  The author(s) of the translator.
• target
  
  • For web translators, the target should specify a JavaScript regular expression (note that escaping requires two backslashes: one for the regular expression itself, and one for the JSON, e.g. “^https?://(www\\.)?example.com/”. If using Scaffold, the add-on takes care of the JSON escaping, so backslashes do not need to be escaped).
    When only matching a domain, the translator should terminate in a forward slash, so it only matches a non-proxied domain. Zotero will take care of de-proxifying the URL and pass the de-proxified URL to the translator.
    Whenever a webpage is loaded, Zotero tests the target regular expressions of all web translators on the webpage URL. If there is a translator with a matching target, this translator’s detectWeb function is run. If this function finds item metadata, the Zotero translator icon appears or becomes active in the browser. When multiple translators have a matching target, the translator with the lowest priority number is selected. Web translators with the target set to null (e.g. the DOI translator) match every webpage, but normally have a high priority number and are only used when no other translator matches.
  • For import translators, the target is set to the expected extension (e.g. the BibTeX import/export translator has its target set to “bib”; selecting BibTex in Zotero’s import window filters for files with a “.bib” extension).
  • For export translators, the target is set to the extension that should be given to generated files (e.g. the BibTeX translator produces “filename.bib” files).
• minVersion & maxVersion
  Respectively the minimum and maximum version of Zotero (as specified in Zotero’s Install Manifest) with which the translator is compatible.
• browserSupport
  A string containing one or more of the letters g, c, s, i, representing the connectors that the translator can be run in – Gecko (Firefox), Chrome, Safari, Internet Explorer, respectively. b indicates support for the Bookmarklet (zotero-dev thread) and v indicates support for the translation-server. For more information, see Connectors. Warning: Compatible with Zotero 2.1.9 and later only.
• priority
  An integer indicating translator priority. When multiple translators can translate a source, the translator with the lowest priority number is selected. Site-specific web translators normally have a priority of 100. For guidelines on picking an appropriate priority for web translators see this page
• inRepository
  Set to true for translators that are added to the Zotero repo and distributed to all Zotero users, and false for those that are not.
• lastUpdated
  The date and time when the translator was last modified (format “YYYY-MM-DD HH:MM:SS”). For the metadata to be read correctly, this line must be the last line in the JSON block.

In addition to the required metadata fields described above, two optional fields exist, configOptions and displayOptions. Both are JavaScript objects, with several properties that control translator behavior:

• configOptions
  • dataMode
    For import translators, this sets the form in which the input data is presented to the translator. If set to “rdf/xml”, Zotero will parse the input as XML and expose the data through the Zotero.RDF object. If “xml/dom”, Zotero will expose the data through the function Zotero.getXML(). Zotero does not natively support importing N3 representations of RDF. The values “block” and “line” are deprecated and no longer necessary in Zotero 2.1 and later.
  • getCollections
    For export translators, set to true or false. If true, an export translator will have access to the collection names and can recreate them in the exported file.
• displayOptions
  • exportCharset
    The default character set to use for export, defaults to “UTF-8”
  • exportFileData, exportNotes and exportTags
    For each property that is set, a checkbox (respectively “Export Files”, “Export Notes” and “Export Tags”) is added to Zotero's export window, allowing files, notes and/or tags to be exported. A checkbox is checked by default if the corresponding property is set to true, and unchecked if the property is set to false.

An example of how these properties are set (taken from the BibTeX.js translator):

"configOptions": {"getCollections": true},
"displayOptions": {"exportCharset": "UTF-8", "exportNotes": true, "exportFileData": false, "useJournalAbbreviation": false}

Depending on the translator type, each Zotero translator must include certain top-level JavaScript functions:

• Web translators
  • detectWeb
    After a web translator has been selected based by its matching target and its priority ranking, detectWeb is run to determine whether item metadata can indeed be retrieved from the webpage. Should return the detected item type (e.g. “journalArticle”, see the overview of Zotero item types), or, if multiple items are found, “multiple”. If detectWeb does not return a value, the translator with the next-highest priority is selected, until all translators with a matching target have been exhausted.
  • doWeb
    Performs the actual item metadata retrieval.
• Import translators
  • detectImport
    Determines whether the translator can import item metadata. Should return true if it can, and false if it cannot.
  • doImport
    Performs the actual import.
• Export translators
  • doExport
    Performs the export.
• Search translators
  • detectSearch
    Determines whether the translator can look up item metadata. Should return true if it can, and false if it cannot.
  • doSearch
    Performs the actual look up.

See Translator Coding for a detailed description on how these functions can be coded.

The following tools can make coding Zotero translators easier:

• Scaffold - Scaffold is a Firefox add-on developed by CHNM to create and modify web translators. Translators can be quickly tested and debugged, and item saving is simulated, so no changes are made to your Zotero library.
• XPath Tools
  Many web translators rely on XPath to extract information from HTML or XML. XPath construction is made easier by using the XCpath bookmarklet or one of the following Firefox add-ons:
  • Firebug - Useful to inspect the HTML structure of webpages, and find XPath expressions for elements of interest.
  • FirePath - Extension to Firebug to edit, inspect and generate XPath expressions.

If you created or modified a translator and wish to have it added to Zotero, or are looking for support on writing translators, please submit a pull request to the Zotero Translators GitHub repo. You can also ask questions about translator development on Zotero development mailing list.

To submit a pull request, fork the Zotero Translator GitHub repository, commit your changes (i.e., adding or modifying translator files), and create a pull request. You can use your Git client of choice, but for new users we recommend SmartGit, which is free for non-commercial purposes.

When you submit pull request on GitHub, your translator code will be reviewed, and you will receive comments from the Zotero developers or experienced volunteers. Once you've made any necessary changes, your translator will be added to the Zotero translator repository.

Please note that contributed translators need to be licensed in a way that allows the Zotero project to distribute them and modify them. We encourage you to license new translators under the GNU Affero General Public License version 3 (or later), which is the license used for Zotero. To do so, just add a license statement to the top of the file. Take a look a recently committed translator, like “Die Zeit.js”, for an example of such a statement.

While there are no strict coding guidelines for translators, there are some general recommendations:

1. Web translator detect targets should be selective, to minimize the number of detectWeb functions that are run for each page.
2. detectWeb, detectImport and detectSearch should be coded to minimize the likelihood of the corresponding doWeb, etc. function failing. Do your minimum required input checking the detect functions – a failing do function will cause user-visible errors.
3. Make detect functions lightweight– they may be run on pages that a user is not even considering saving. Detect functions should not need to make additional HTTP requests. This obviously runs counter to the preceding point– find a happy medium.
4. Minimize HTTP requests. More HTTP requests slow down the user, cause undue load on servers, and in general are bad.
5. Don't leak user data. HTTP requests should in general not be directed to 3rd-party hosts.
6. Document your code. If there are input data deficiencies and the translator is working around them, document the deficiencies. If there are specific types of pages that a web translator is for, provide example URLs and expected output.
7. Produce translator tests when possible, covering the basic page types that the translator is designed to support.