This is an old revision of the document!


We’re in the process of updating the documentation for Zotero 5.0. Some documentation may be outdated in the meantime. Thanks for your understanding.

Scaffold - an IDE for Zotero translators

Translators are stored as individual JavaScript files with additional metadata at the top and tests at the bottom in the “translators” subdirectory of the Zotero data directory. While translators can be edited with any tool, Scaffold is dedicated to writing Zotero translators, offering advantages such as real-time testing and debugging.

Scaffold supports editing and testing web translators and import translators. Search and export translators can not yet be tested using the IDE.

Installation

Scaffold does run within Zotero 5.0 as an plugin, which is installed by the following steps:

  • Download the XPI file of the latest release of Scaffold
  • Open Zotero
  • Tools → Add-ons → (gear wheel) → Install Add-on From File

Previously to Zotero Version 5.0, Scaffold was a Firefox Plugin.

Interface

After installation, the “Tools” menu in Zotero should contain a “Scaffold” item.

Selecting the Tools menu's “Scaffold” item opens the main Scaffold window:

Top buttons

Load
Opens the “Load Translator” window. Select one of the currently installed translators, and load the translator metadata and code into Scaffold.

Save
Saves the translator you are currently working on. Provide a unique label and translator ID for your translator if you don't want to overwrite an existing translator. New translator IDs can be automatically generated via the “Generate” button.

Run detectWeb
Saves and runs the detectWeb function of the translator code on the site loaded in the browser tab.

Run doWeb
Saves and runs the doWeb function of the translator code on the site loaded in the browser tab.

Run detectImport
Saves and runs the detectImport function of the translator code on the content entered in the Import tab.

Run doImport
Saves and runs the doImport function of the translator code on the content entered in the Import tab.

Tabs

Metadata
Shows the translator metadata. Translator IDs can be generated via the “Generate” button. The target regular expression can be tested with the “Test Regex” button.

Code
The text box in this tab contains the translator's JavaScript code. The editor supports syntax highlighting, folding of code parts, and search and replace.

Import
For import translator you can copy here test data to import.

Tests
This tab contains the data for all test cases saved with the translator, which is currently open.

Testing
This is a list of all test cases for the current translator, which you can run, check or update.

Browser
You can open specified URLs in the browser tab and then test the detection and data extraction for that website.

Debug Output

The main strength of Scaffold is its ability to provide you with immediate feedback, which can dramatically speed up translator development. After a code change, a single click suffices to run the modified translator and generate debug output. The following types of debug output can be generated:

Metadata

When the “Test Regex” button in the “Metadata” tab is clicked, the regular expression in the target field is applied to the site loaded in the most recently selected Firefox tab. The debug window at the right of the Scaffold window will show whether the regular expression matches (true for a match, false for no match), e.g.:

09:54:11 ===>true<===(boolean)

detectWeb and doWeb

When the “Run detectWeb” button is clicked, the detectWeb function of the translator will be executed. Similarly, clicking the “Run doWeb” button executes the translator's doWeb function.

Debug output for the detectWeb function shows what type of item is found on the loaded webpage, e.g.:

19:19:43 detectWeb returned type "book"

Debug output for the doWeb function shows all the item data that would be saved if the translator would be run by Zotero (when testing translators with Scaffold, no items are actually saved to your Zotero library), e.g.:

19:24:21 Returned item:
             'itemType' => "book"
             'creators' ...
                 '0' ...
                     'firstName' => "Herman"
                     'lastName' => "Melville"
                     'creatorType' => "author"
             'notes' ...
             'tags' ...
             'seeAlso' ...
             'attachments' ...
                 '0' ...
                     'title' => "Google Books Link"
                     'snapshot' => "false"
                     'mimeType' => "text/html"
                     'url' => "http://books.google.com/books?id=cYKYYypj8UAC"
                     'document' => "[object]"
             'date' => "1851"
             'pages' => "504"
             'ISBN' => "1603033742, 9781603033749"
             'publisher' => "Plain Label Books"
             'title' => "Moby Dick"
             'repository' => "Google Books"
             'complete' => function(...){...} 
         
19:24:21 Translation successful

If running detectWeb or doWeb results in an error, the debug window will show an error message. For debugging, additional debug output can be added to the code by adding Zotero.debug(string); statements.

Import Translators

Starting with version 2.1.2, Scaffold can be used to test import translators. To do so, enter the test import data in the “Import” tab and use the “Run detectImport” and “Run doImport” buttons to test the translator against the data. The output will be displayed in the right-hand pane as with web translators.

In the translator load window, import translators are arranged in alphabetical order at the bottom of the list.

Scaffold Troubleshooting

  • Why does my translator work in Scaffold but not when I click on the address bar icon in my browser?

Scaffold runs the functions detectWeb and doWeb against the active document in the top Firefox window. Outside of Scaffold, clicking the address bar icon runs the first web translator with a matching target regular expression that returns a non-false value from the function detectWeb, when run on that document and URL.

This can differ from the active document in Firefox when the target expression or detectWeb conditions are too lax, which can allow embedded documents, such as IFRAMES used for advertising or sharing panes, to be detected as content pages. To see if this is what is happening, look at the debug output for a save attempt from Firefox proper and check the URL that is being processed.

  • Why won't my attachments save?
  • Why do some fields show in the output pane but not in the created item?

The output pane in Scaffold shows the item object as Zotero is about to save it– it does not in fact save the object. As an item is saved, the contents of some fields may change.

The attachment information in the output pane of Scaffold merely says what Zotero is going to try to save; as the attachments are saved, they may be discarded if their actual MIME type differs from the one specified, as can happen when PDFs or other attachments are hidden by providers behind an interstitial terms or copyright notice page.

Fields that are not allowed for the specified item type will be discarded, even if they show up in the output pane of Scaffold.

Further Reading

More detailed instructions are available at MediaWiki. See For development through IDE and Run on Scaffold.

dev/translators/scaffold.1518163592.txt.gz · Last modified: 2018/02/09 03:06 by zuphilip