Differences

This shows you the differences between two versions of the page.

--- dev:translators [2017/11/27 05:10] – [Translator Structure] bwiernik
+++ dev:translators [2023/08/18 09:15] (current) – [Recommendations for Translator Authors] Add one advice about not modifying the live document. zoe
@@ Line 7: / Line 7: @@
 **Note:** Before writing a translator for a site, look at the [[dev:exposing_metadata|documentation on exposing metadata]]; website authors should try embedding the necessary metadata before attempting to write a translator.
-If you're looking for a broken translator to fix, see the [[https://repo.zotero.org/errors|recent translator errors]] and check on one of the top reported errors. You can also check the status of many translators by reviewing the [[dev:translators:testing#running_tests|translator test overview]].
+If you're looking for a broken translator to fix, see the [[https://zotero-translator-tests.s3.amazonaws.com/index.html|recent translator errors]] and check on one of the top reported errors. You can also check the status of many translators by reviewing the [[dev:translators:testing#running_tests|translator test overview]].
 ===== Translator Types - Web, Import, Export and Search =====
@@ Line 45: / Line 45: @@
   * **creator** \\ The author(s) of the translator.
   * **target** \\
-    * For [[dev:translators:coding#web_translators|web translators]], the ''target'' should specify a [[:dev:technologies#regular_expressions|JavaScript regular expression]] (note that escaping requires two backslashes: one for the regular expression itself, and one for the JSON, e.g. "<nowiki>^https?://(www\\.)?example.com/</nowiki>". If using Scaffold, the add-on takes care of the JSON escaping, so backslashes do not need to be escaped).\\ When only matching a domain, the translator should terminate in a forward slash, so it only matches a non-proxied domain. Zotero will take care of de-proxifying the URL and pass the de-proxified URL to the translator.\\ Whenever a webpage is loaded, Zotero tests the target regular expressions of all web translators on the webpage URL. If there is a translator with a matching target, this translator’s ''detectWeb'' function is run. If this function finds item metadata, the Zotero translator icon appears or becomes active in the browser. When multiple translators have a matching target, the translator with the lowest priority number is selected. Web translators with the target set to ''null'' (e.g. the DOI translator) match every webpage, but normally have a high priority number and are only used when no other translator matches.
+    * For [[dev:translators:coding#web_translators|web translators]], the ''target'' should specify a [[:dev:technologies#regular_expressions|JavaScript regular expression]] (note that escaping requires two backslashes: one for the regular expression itself, and one for the JSON, e.g. "<nowiki>^https?://(www\\.)?example.com/</nowiki>". If using Scaffold, it takes care of the JSON escaping, so backslashes do not need to be escaped).\\ When only matching a domain, the translator should terminate in a forward slash, so it only matches a non-proxied domain. Zotero will take care of de-proxifying the URL and pass the de-proxified URL to the translator.\\ Whenever a webpage is loaded, Zotero tests the target regular expressions of all web translators on the webpage URL. If there is a translator with a matching target, this translator’s ''detectWeb'' function is run. If this function finds item metadata, the Zotero translator icon appears or becomes active in the browser. When multiple translators have a matching target, the translator with the lowest priority number is selected. Web translators with an empty ''target'' string (e.g. the DOI translator) match every webpage, but normally have a high priority number and are only used when no other translator matches.
     * For import translators, the ''target'' is set to the expected extension (e.g. the BibTeX import/export translator has its target set to "bib"; selecting BibTex in Zotero’s import window filters for files with a ".bib" extension).
     * For export translators, the ''target'' is set to the extension that should be given to generated files (e.g. the BibTeX translator produces "filename.bib" files).
   * **minVersion** & **maxVersion** \\ Respectively the minimum and maximum version of Zotero (as specified in Zotero’s [[https://developer.mozilla.org/en/install_manifests|Install Manifest]]) with which the translator is compatible.
-  * **browserSupport** \\ A string containing one or more of the letters ''g'', ''c'', ''s'', ''i'', representing the connectors that the translator can be run in -- Gecko (Firefox), Chrome, Safari, Internet Explorer, respectively. ''b'' indicates support for the Bookmarklet ([[https://groups.google.com/forum/#!topic/zotero-dev/ZWCe86B3OCw/discussion|zotero-dev thread]]) and ''v'' indicates support for the [[https://github.com/zotero/translation-server|translation-server]].  For more information, see [[dev:translators:connectors|Connectors]]. **Warning: Compatible with Zotero 2.1.9 and later only.**
+  * **browserSupport** \\ A string containing one or more of the letters ''g'', ''c'', ''s'', ''i'', representing the connectors that the translator can be run in -- Gecko (Firefox), Chrome, Safari, Internet Explorer, respectively. ''b'' indicates support for the Bookmarklet ([[https://groups.google.com/forum/#!topic/zotero-dev/ZWCe86B3OCw/discussion|zotero-dev thread]]) and ''v'' indicates support for the [[https://github.com/zotero/translation-server|translation-server]].  For more information, see [[dev:translators:connectors|Connectors]].
   * **priority** \\ An integer indicating translator priority. When multiple translators can translate a source, the translator with the lowest priority number is selected. Site-specific web translators normally have a priority of 100. For guidelines on picking an appropriate priority for web translators see [[:dev:translators:priority|this page]]
   * **inRepository** \\ Set to ''true'' for translators that are added to the Zotero repo and distributed to all Zotero users, and ''false'' for those that are not.
@@ Line 84: / Line 84: @@
   * **[[dev:translators:coding#search_translators|Search translators]]**
     * //detectSearch// \\ Determines whether the translator can look up item metadata. Should return ''true'' if it can, and ''false'' if it cannot.
-    * //doSearch// \\ Performs the actual look up.
+    * //doSearch// \\ Performs the actual lookup.
 See [[dev:translators:coding|Translator Coding]] for a detailed description on how these functions can be coded.
@@ Line 92: / Line 92: @@
 The following tools can make coding Zotero translators easier:
-  * [[:dev:translators:scaffold|Scaffold]] - Scaffold is a Firefox add-on developed by CHNM to create and modify web translators. Translators can be quickly [[:dev:translators:testing|tested]] and debugged, and item saving is simulated, so no changes are made to your Zotero library.
+  * [[:dev:translators:scaffold|Scaffold]] - Scaffold is an IDE for translators built into Zotero (Tools -> Developer -> Translator Editor). Translators can be quickly [[:dev:translators:testing|tested]] and debugged, and item saving is simulated, so no changes are made to your Zotero library.
-  * XPath Tools \\ Many web translators rely on [[dev:technologies#xpath|XPath]] to extract information from HTML or XML. XPath construction is made easier by using the [[http://dl.dropbox.com/u/848981/it/xp/xp.html|XCpath bookmarklet]] or one of the following Firefox add-ons:
+  * Browser inspector - Web translators generally use [[https://developer.mozilla.org/en-US/docs/Web/API/Document/querySelector|querySelector]] and [[https://developer.mozilla.org/en-US/docs/Web/API/Document/querySelectorAll|querySelectorAll]] to extract content from web pages. Your browser likely provides an inspector tool to help you understand pages' structure. You can access it by right-clicking and selecting Inspect (Firefox) or Inspect Element (Chrome).
-    * [[http://getfirebug.com/|Firebug]] - Useful to inspect the HTML structure of webpages, and [[dev:technologies#using_firebug_to_design_xpath_expressions|find XPath expressions for elements of interest]].
+  * XPath Tools - Many older web translators rely on XPath to extract information from HTML or XML. Various tools can assist with generating and checking XPath expressions, including the DevTools built into browsers. For example, in Firefox, you can get the XPath for any element by finding it in the browser's Inspector tool, right-clicking on the element, and choosing Copy -> XPath.
-    * [[https://addons.mozilla.org/fr/firefox/addon/firepath/|FirePath]] - Extension to Firebug to edit, inspect and generate XPath expressions.
 ===== Contributing Translators =====
@@ Line 102: / Line 102: @@
 To submit a pull request, fork the [[https://github.com/zotero/translators|Zotero Translator GitHub repository]], commit your changes (i.e., adding or modifying translator files), and create a [[http://help.github.com/pull-requests/|pull request]]. You can use your Git client of choice, but for new users we recommend [[http://www.syntevo.com/smartgit/index.html|SmartGit]], which is free for non-commercial purposes.
-When you submit pull request on GitHub, your translator code will be reviewed, and you will receive comments from the Zotero developers or experienced volunteers. Once you've made any necessary changes, your translator will be added to the Zotero translator repository.
+When you submit a pull request on GitHub, your translator code will be reviewed, and you will receive comments from the Zotero developers or experienced volunteers. Once you've made any necessary changes, your translator will be added to the Zotero translator repository.
 ==== Licensing ====
 Please note that contributed translators need to be licensed in a way that allows the Zotero project to distribute them and modify them. We encourage you to license new translators under the [[http://www.gnu.org/licenses/agpl.html|GNU Affero General Public License version 3]] (or later), which is the license used for Zotero. To do so, just add a license statement to the top of the file. Take a look a recently committed translator, like "Die Zeit.js", for an example of such a statement.
 ===== Recommendations for Translator Authors =====
 While there are no strict coding guidelines for translators, there are some general recommendations:
@@ Line 112: / Line 112: @@
   - ''detectWeb'', ''detectImport'' and ''detectSearch'' should be coded to minimize the likelihood of the corresponding ''doWeb'', etc. function failing. Do your minimum required input checking the detect functions -- a failing ''do'' function will cause user-visible errors.
   - Make detect functions lightweight-- they may be run on pages that a user is not even considering saving. Detect functions should not need to make additional HTTP requests. This obviously runs counter to the preceding point-- find a happy medium.
-  - Minimize HTTP requests. More HTTP requests slow down the user, cause undue load on servers, and in general are bad.
+  - When translating the web page in the browser, do not modify any part of its DOM.
+  - Minimize HTTP requests. More HTTP requests slow down the user, cause undue load on servers, risk getting the user rate-limited or blocked, and in general are bad.
   - Don't leak user data. HTTP requests should in general not be directed to 3rd-party hosts.
   - Document your code. If there are input data deficiencies and the translator is working around them, document the deficiencies. If there are specific types of pages that a web translator is for, provide example URLs and expected output.
   - Produce [[dev:translators:testing|translator tests]] when possible, covering the basic page types that the translator is designed to support.
+  - Run ESLint on your code before submitting it. Zotero provides an ESLint plugin for translator development. You can run it on your translator within a clone of the ''zotero/translators'' repository: <code>npm ci && npm run lint -- "Your Translator.js"</code>