We’re in the process of updating the documentation for Zotero 5.0. Some documentation may be outdated in the meantime. Thanks for your understanding.

Translator Testing

In order to provide consistent and reliable operation on a variety of pages and inputs, it is possible to define tests for import, web, and search translators. These tests are defined as a JSON blocks within the translator text, including the input URL or text, and the expected result from running the translator on that input. A translator that provides good test coverage of its functionality can then be more easily maintained– in the case of input translators, development can proceed with reduced risk of causing regressions, and in the case of web translators, target site changes that lead to partial or complete translator breakage can be detected and documented without relying on user reports.

Structure

Tests are defined as an array of test objects; each test has a type (“web”, “import”, “search”), an input (“url” or “input”), and the expected resulting items. Items can be either an array of one or more item objects, where the item objects are the same as the item skeletons created by new Zotero.Item(type) from inside a translator, or it can be the string “multiple”, indicating that there should be multiple results, but not indicating specifically which items should result.

In most cases, it is not necessary or desirable to write these tests by hand– they can and should be generated by the testing framework using Scaffold; see below.

/** BEGIN TEST CASES **/
var testCases = [
	{
		"type": "web",
		"url": "http://www.example.com",
		"items": [
                       { item data }
		]
	},
	{
		"type": "web",
		"url": "http://www.example.com/search?term",
		"items": "multiple"
	}
]
/** END TEST CASES **/

In some cases, a translator test may fail where the translator itself would not, since it runs immediately upon the target page loading. In such cases, set “defer”: true in the affected test case to make the test harness wait 10 seconds after the page loads before running:

	{
		"type": "web",
		"url": "http://www.example.com/search?term",
		"items": "multiple",
		"defer": true
	}

Note that Scaffold does not currently produce or maintain the defer annotation, so it will need to be added manually, rather than generated using Scaffold's test builder. The annotation is supported in Zotero 3.0.4 and later.

Running tests

An overview of the currently installed translators, giving the option of running their tests, can be accessed by entering the following into the address bar:

chrome://zotero/content/tools/testTranslators/testTranslators.html

The tests defined by each translator can be viewed and edited using Scaffold 3, which shows the test text in the “Tests” tab, and provides tools for creating new tests, and running and modifying existing ones, in the “Testing” tab. See the Scaffold documentation for more information on using Scaffold for translator development and testing.

Automated Testing

The translator tests are run daily at 9 pm EST by the Zotero project to detect new failures and incompatibilities. Test results are available at https://zotero-translator-tests.s3.amazonaws.com/index.html

Further Reading

More detailed instructions are available at MediaWiki. See Write testCases (with translation-server) or Generate testCases (with Scaffold).

dev/translators/testing.txt · Last modified: 2017/11/12 19:53 by 127.0.0.1