Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
dev:how_to_write_a_zotero_translator_plusplus [2010/07/27 17:36] – just cite the chapter#, prevent later problems tomrochewiki | dev:how_to_write_a_zotero_translator_plusplus [2017/11/19 19:24] (current) – Remove HWZT++ adamsmith | ||
---|---|---|---|
Line 1: | Line 1: | ||
- | ===== Chapter 0: Introduction ===== | + | **Note:** This page used to hold an updated version of Adam Crymble' |
- | [[http://niche-canada.org/member-projects/zotero-guide/chapter1.html|How to Write a Zotero Translator]] by Adam Crymble (aka //HWZT//) is still the best guide to writing | + | * The template code included in [[https://www.zotero.org/support/ |
+ | * The code of [[https://github.com/ | ||
+ | * The documentation for writing Zotero translators | ||
+ | * The [[https:// | ||
+ | * Search | ||
- | * much has changed since HWZT was written, limiting its usability. | ||
- | * HWZT is not wikified, limiting its maintainability. | ||
- | This page (aka //HWZT++//) updates and wikifies HWZT. (Note that HWZT++ is [[http:// | ||
- | ===== Chapter 1: Introduction to Zotero Translators ===== | ||
- | |||
- | //Note:// the following is adapted from [[http:// | ||
- | |||
- | ==== Zotero ==== | ||
- | |||
- | The citation management program Zotero is a wonderful tool for researchers everywhere. Citations from the web may be " | ||
- | |||
- | Most users who know about the citation capture feature are enthralled by it and want more. The [[http:// | ||
- | |||
- | Luckily, Zotero translators are fairly easy to create (as far as computer programming goes). This guide seeks to help take some of that load away from the Zotero staff by teaching the community of Zotero users how to create their own translators and to share them with others. | ||
- | |||
- | ==== Who is this guide for? ==== | ||
- | |||
- | Anyone! No previous experience required! | ||
- | |||
- | In fact, the guide will assume that you have no programming experience whatsoever. You just need to have spent some time using Zotero and grabbing citations. (Check out the [[http:// | ||
- | |||
- | This guide uses plain language throughout and is written for people who are not programmers. You just need to be comfortable with computers, able to think logically, and not afraid to do some Google searching when you find a word or come across an error message you don't recognize. | ||
- | |||
- | Everything will be explained with a new user in mind. Therefore, you might find that you do not need to read all sections. If you are unsure whether or not you should read a section or skip it, it is probably a good idea to jump ahead to the end of the chapter and read the "What you should understand before moving on." If in doubt, it is probably best to read the chapter and refresh your knowledge. Skipping too much background will just leave you frustrated when you start coding. | ||
- | |||
- | If you fit into one of the following categories, chances are you are a great candidate for writing a translator: | ||
- | |||
- | * Website administrator of a searchable database | ||
- | * Librarian or archivist | ||
- | * Researcher or journalist | ||
- | * Graduate student | ||
- | * Someone who wants to learn JavaScript | ||
- | |||
- | ==== What you will learn ==== | ||
- | |||
- | When you are finished with this guide, you should not only know enough to create your own working Zotero translator, but you should understand the following concepts and computer languages: | ||
- | |||
- | * Basic HTML | ||
- | * the Document Object Model (DOM) | ||
- | * XPaths | ||
- | * JavaScript Regular Expressions (RegExp) | ||
- | * Basic JavaScript | ||
- | |||
- | Translators are written in a computer language called " | ||
- | |||
- | You will also learn how to use a number of programs (all free and reputable). Among these are Firefox, Zotero, Scaffold, Komodo Edit, DOM Inspector, and XPather. | ||
- | |||
- | You will not learn how to embed JavaScript into HTML documents or to do DOM scripting; however, after learning how to write a Zotero translator you will be well on your way to understanding these concepts. | ||
- | |||
- | ==== Benefits of Writing a Translator ==== | ||
- | |||
- | If you are a web administrator or work for a company that maintains a searchable database, having a Zotero translator will increase your site's usability. Zotero has over a million users, many of whom judge a website' | ||
- | |||
- | [[dev/ | ||
- | |||
- | If you are an end user rather than an administrator, | ||
- | |||
- | If the website in question is a rather obscure database that may be password protected, you will have to submit your own translator. This is because, even if a Zotero programmer has time to work on your request, without access to the database he or she is powerless to write the translator. | ||
- | |||
- | Finally, Zotero is an open source, freeware project, adding to the software is in the spirit of its creation. Once you've finished a new translator you can submit it to the Zotero team so that everyone can benefit from the fruits of your labour. | ||
- | |||
- | ==== The Three Major Types of Translators ==== | ||
- | |||
- | * Scrapers | ||
- | * Metadata Converters | ||
- | * Exporters | ||
- | |||
- | ==== Scrapers ==== | ||
- | |||
- | This guide will teach you how to create a " | ||
- | |||
- | The advantage of a Scraper is that it is the only kind of translator that can be used on any website. What a scraper does is take (scrape) words off the webpage and tells Zotero which words correlate to which part of the citation. It's sort of like cutting and pasting in a text document, but by using code rather than keystrokes. | ||
- | |||
- | Another advantage of a scraper is that you can tailor it to your exact needs by choosing to gather all, some, or very little of the information available. Scrapers are easy to learn and to make. | ||
- | |||
- | The disadvantage of a scraper is that it relies heavily on format and the consistency of the webpage' | ||
- | |||
- | The other two types of translators are powerful and accurate, but are only possible to use under certain conditions that are almost always out of your control unless you are the website' | ||
- | |||
- | ==== Metadata Converters ==== | ||
- | |||
- | These translators take information that a Webmaster has voluntarily embedded in a webpage, known as metadata, and organizes it into the correct Zotero fields. You can think of metadata as invisible ink that only appears if you know how to find it. Obviously the catch here is that the Webmaster must have included this information in the first place. | ||
- | |||
- | The practice of including metadata is becoming more common, especially in databases. In the past couple of years, how people display metadata has become more standardized. Because of this standardization, | ||
- | |||
- | * [[http:// | ||
- | * [[http:// | ||
- | * [[http:// | ||
- | * [[http:// | ||
- | |||
- | If you are a website administrator and want your site to automatically be Zotero compliant, it is best to use one of these systems rather than writing a translator; they are standardized and reliable. Let me repeat that: //if you are a website administrator and can include standardized metadata in your site, stop reading this guide and add the metadata!// If you are not a website administrator, | ||
- | |||
- | ==== Exporters ==== | ||
- | |||
- | Exporters also rely on a website providing certain information. In this case, we need a link that allows us to download a citation. For the most part, there are very few export formats. You may have come across them before, labeled as "MARC display" | ||
- | |||
- | - Use an XPath to grab the link URL of the citation download. | ||
- | - Use HTTP get to download the page found at that URL. | ||
- | - Call the translator for that type of citation (ie, MARC, Bibtex, etc) to interpret the citation. | ||
- | - Save results into Zotero. | ||
- | - If you are lost, check one of the many Library Catalogue translators by launching Scaffold and loading the translator code. | ||
- | |||
- | You do not need to understand the latter two types of translators or the contents of the paragraph above to be successful at writing scrapers. | ||
- | |||
- | Note: from this point forward, we will be using the word " | ||
- | |||
- | ==== Before we begin ==== | ||
- | |||
- | As with anything new and computer-related, | ||
- | |||
- | There are many short cuts available when writing JavaScript and experienced programmers may tell you to use these. You are free to learn the short cuts if you wish; however, this guide will not use them, for simplicity' | ||
- | |||
- | This is NOT a guide detailing how to use Zotero. It is a guide detailing how to write code to extend the usefulness of Zotero. | ||
- | |||
- | ===== Chapter 2: General Troubleshooting Guidelines ===== | ||
- | |||
- | //Note:// the following is adapted from [[http:// | ||
- | |||
- | Before we start, you should be aware: you will get frustrated — at least once. Here are a few tips to help you solve your problems. | ||
- | |||
- | ==== Search Engines ==== | ||
- | |||
- | If you run into difficulties when writing computer code the great news is: the answer to almost any problem can be found online. All computer programmers have needed help at one time or another, and given their love for computers, most sought that help online. Lucky for you, that means that most of the questions they asked — and the subsequent answers — are still floating around the internet. | ||
- | |||
- | This means the internet is often your best resource for finding help. If you run into a problem, the first thing you should do is type your problem into a search engine. More often than not someone has already asked your exact question, and someone else has provided an answer. You might even find entire websites dedicated to solving your particular problem. As far as coding goes, Zotero translators are quite basic; you will not come across a problem when writing a Zotero translator that no one has encountered before. | ||
- | |||
- | Likewise, if you encounter an error message you don't understand, cut and paste that error message into a search engine and surround it with quotation marks. You will likely find dozens of explanations why this error appeared and how to fix it. | ||
- | |||
- | The more specific you can be about your problem, the better the results you will find. Don't be discouraged if you don't find the answer on your first search. Rephrase the search terms and try again. | ||
- | |||
- | ==== Online Tutorials ==== | ||
- | |||
- | Your second best option is [[http:// | ||
- | |||
- | * [[http:// | ||
- | * [[http:// | ||
- | * [[http:// | ||
- | |||
- | At W3Schools, you can find great reference charts that will show you at a glance all the different capabilities of JavaScript and HTML. These will come in handy when you want to do something and can't remember how. | ||
- | |||
- | Apart from W3Schools, you can find many other tutorials online. Try typing in what you want to learn into a search engine and you will likely find a tutorial. Keep in mind that many tutorials teach you how to accomplish a specific task and may not teach you exactly what you're looking for. | ||
- | |||
- | ==== Forums ==== | ||
- | |||
- | If you've Googled it, Yahoo' | ||
- | |||
- | === Webdeveloper.com === | ||
- | |||
- | The [[http:// | ||
- | |||
- | If you can't figure out why you are getting a particular error message, or why you can't get information from point A to point B, this is the forum for you. At any given time there are over one hundred people logged into the forum just waiting to answer your question. If you post your problem here in a courteous manner, with a little bit of luck you will have a solution within a couple of hours. | ||
- | |||
- | It may not be the instant gratification we've come to expect, but don't forget, these people are volunteering to help you, and most probably if you're desperate enough to ask for help, you could use a few hours away from the keyboard anyway. | ||
- | |||
- | === Zotero Forums === | ||
- | |||
- | If your question is something specific to Zotero, such as "why can't I put anything in the field Loc. in archive?" | ||
- | |||
- | ==== Asking Good Questions ==== | ||
- | |||
- | Clarity and specificity are your friends when it comes to asking for help on a forum. The people who read forums and offer their expertise are busy; make it easy for them by carefully thinking out your problem before you ask. Likewise, make sure you are asking a specific question to a narrowly defined problem. | ||
- | |||
- | For example, don't post something like: "Why won't my translator work?" | ||
- | |||
- | Instead, try: "Why am I getting a syntax error when I try to [[http:// | ||
- | |||
- | Always post the relevant section of your code (and only the relevant section of your code) along with your question. This will make it easier for the experts to help you solve your problem. If the answer you get does not do the trick and you are still stuck, be polite and try rephrasing the question. Remember, don't bite the hand that feeds you; these are volunteers and they' | ||
- | |||
- | ==== Debugging ==== | ||
- | |||
- | To help you ask good questions you'll learn how to use the '' | ||
- | |||
- | When fixing problem code, only change one thing at a time. If you are working on a section of code that has several issues, fix one problem and retry the code before moving on. Sometimes if you make three or four changes before retrying the program, you will inadvertently cause another unexpected problem. This can make you think your fix was incorrect, when in fact only the last change you made was wrong. Change one thing and make sure it works before moving on and you will prevent a lot of confusion. | ||
- | |||
- | ===== Chapter 3: Required Software ===== | ||
- | |||
- | [[http:// | ||
- | |||
- | - Scaffold: don't get Scaffold 1.0 from the link in HWZT, get Scaffold 2.0 [[http:// | ||
- | - Solvent: also downlevel, so instead get the following uplevel Firefox add-ons (from either the [[https:// | ||
- | * XPather: download [[http:// | ||
- | * DOM Inspector: download [[https:// | ||
- | Install all 3, then restart Firefox. | ||
- | |||
- | ===== Chapter 4: DOM & HTML ===== | ||
- | |||
- | [[http:// | ||
- | |||
- | ===== Chapter 5: XPath directions ===== | ||
- | |||
- | [[http:/ | ||
- | |||
- | The {DOM Inspector + XPather} workflow differs from that of Solvent. After opening the [[http:// | ||
- | - Open DOM Inspector (aka //DI//) with C-S-i or from the Firefox main menu with Tools> | ||
- | - Hit button=Inspect at the upper right of the DI window. This will open pane=Browser in the DI window displaying the contents of the first sample page. | ||
- | - To test the XPath string denoting the heading (text=" | ||
- | * in textbox=XPath in the DI window, type < | ||
- | * hit button=Eval next to the text box. This will popup dialog=" | ||
- | * Move dialog=" | ||
- | * In table=" | ||
- | |||
- | ===== Chapter 6: JS Variables ===== | ||
- | |||
- | [[http:// | ||
- | |||
- | A few things have changed since HWZT. To use its first Scaffold example as an example: | ||
- | |||
- | - Ensure the [[http:// | ||
- | - Open Scaffold 2.0 from the Firefox main menu with Tools> | ||
- | - If you are not already in tab=Metadata, | ||
- | - The URI of the sample page has changed since HWZT, so you will need to enter Target=< | ||
- | - Hit button=" | ||
- | - Instead of < | ||
- | Zotero.debug(myVariable); | ||
- | </ | ||
- | - Click on icon=" | ||
- | - To also obtain an evaluation like < | ||
- | |||
- | The subsequent examples behave similarly. | ||
- | |||
- | ===== Chapter 7: JS Methods & Math ===== | ||
- | |||
- | [[http:// | ||
- | |||
- | ===== Chapter 8: JS If Statements ===== | ||
- | |||
- | [[http:// | ||
- | |||
- | ===== Chapter 9: JS Loops ===== | ||
- | |||
- | [[http:// | ||
- | |||
- | ===== Chapter 10: JS Functions ===== | ||
- | |||
- | [[http:// | ||
- | |||
- | ===== Chapter 11: XPath containers ===== | ||
- | |||
- | [[http:// | ||
- | |||
- | Again, a few changes since HWZT, and again, using its first Scaffold example as an example: | ||
- | |||
- | - Close any running Scaffold 2.0 instances. | ||
- | - Ensure the [[http:// | ||
- | - Open Scaffold 2.0 from the Firefox main menu with Tools> | ||
- | - If you are not already in tab=Metadata, | ||
- | - The URI of the sample page has changed since HWZT, so you will need to enter Target=< | ||
- | - Hit button=" | ||
- | - Instead of < | ||
- | - In that tab enter < | ||
- | var namespace = doc.documentElement.namespaceURI; | ||
- | var nsResolver = namespace ? function(prefix) { | ||
- | if (prefix == " | ||
- | } : null; | ||
- | var myXPath = '// | ||
- | var myXPathObject = | ||
- | doc.evaluate(myXPath, | ||
- | Zotero.debug(myXPathObject); | ||
- | }</ | ||
- | - Click on icon=" | ||
- | |||
- | The code for the second complete Scaffold example (from " | ||
- | < | ||
- | var namespace = doc.documentElement.namespaceURI; | ||
- | var nsResolver = namespace ? function(prefix) { | ||
- | if (prefix == " | ||
- | } : null; | ||
- | var myXPath = '// | ||
- | var myXPathObject = doc.evaluate(myXPath, | ||
- | var items = new Object(); | ||
- | var headers; | ||
- | while (headers = myXPathObject.iterateNext()) { | ||
- | items[headers.textContent]=''; | ||
- | } | ||
- | Zotero.debug(items); | ||
- | } | ||
- | </ | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | '' | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | </ | ||
- | |||
- | ===== Chapter 12: Regexps ===== | ||
- | |||
- | [[http:// | ||
- | var x = " | ||
- | x = x.replace(/ | ||
- | Zotero.debug(x); | ||
- | } | ||
- | </ | ||
- | |||
- | ===== Chapter 13: Metadata Tab ===== | ||
- | |||
- | [[http:// | ||
- | |||
- | - this chapter (13) does not describe saving the tutorial translator | ||
- | - the icon's title is merely " | ||
- | |||
- | So, to create and save the tutorial translator, do | ||
- | |||
- | - Close any running Scaffold 2.0 instances. | ||
- | - Ensure the [[http:// | ||
- | - Open Scaffold 2.0 from the Firefox main menu with Tools> | ||
- | - If you are not already in tab=Metadata, | ||
- | - In field=Label, | ||
- | - In field=Creator, | ||
- | - In field=Target, | ||
- | - Scaffold 2.0 will not save a translator without code (and it will silently refuse to do so), so switch to tab=Code, and enter < | ||
- | var x = " | ||
- | x = x.replace(/ | ||
- | Zotero.debug(x); | ||
- | } | ||
- | </ | ||
- | - Click icon=Save (second from left, looks like some tabs): your translator should save silently. | ||
- | |||
- | To test your translator saved properly, | ||
- | |||
- | - Close any running Scaffold 2.0 instances. | ||
- | - Ensure the [[http:// | ||
- | - Open Scaffold 2.0 from the Firefox main menu with Tools> | ||
- | - Hit icon=Load (the non-OS UI item closest to the upper left of the dialog) to popup dialog=" | ||
- | - Scroll through the "Load Translator" | ||
- | |||
- | ===== Chapter 14: DetectWeb Tab ===== | ||
- | |||
- | [[http:// | ||
- | |||
- | - HWZT says "Save your entry and click Execute": | ||
- | - HWZT says "If you would like to see the Icon in the address bar [of the first sample page], you will likely have to relaunch Firefox": | ||
- | |||
- | As HWZT has gotten increasingly casual, here are some step-by-step instructions for this chapter' | ||
- | |||
- | - Close any running Scaffold 2.0 instances. | ||
- | - Ensure the [[http:// | ||
- | - Open Scaffold 2.0 from the Firefox main menu with Tools> | ||
- | - Hit icon=Load (the non-OS UI item closest to the upper left of the dialog). This should popup dialog=" | ||
- | - Scroll through the "Load Translator" | ||
- | - You will return to the main dialog=" | ||
- | - Switch to tab=Code and enter the following:< | ||
- | if (doc.title.match(" | ||
- | return " | ||
- | } else if (doc.title.match(" | ||
- | return " | ||
- | } | ||
- | } | ||
- | </ | ||
- | - Click icon=" | ||
- | - Click icon=Save (second from left): your translator should save silently. | ||
- | - Return focus to the page or tab containing the [[http:// | ||
- | |||
- | ===== Chapter 15: Scraping the Search Results Page ===== | ||
- | |||
- | [[http:// | ||
- | chapter 15 (Scraping the Search Results Page)]]: Again, the tutorial example works, but is rather casually presented (notably, it requires a different sample page and really a separate translator), | ||
- | |||
- | - rename and save the original translator | ||
- | - create the stub for a second translator | ||
- | - populate, test, and save the second translator | ||
- | |||
- | To rename the original translator, | ||
- | |||
- | - Close any running Scaffold 2.0 instances. | ||
- | - Ensure the [[http:// | ||
- | - Open Scaffold 2.0 from the Firefox main menu with Tools> | ||
- | - Hit icon=Load (at the upper left of the dialog). This should popup dialog=" | ||
- | - Scroll through the "Load Translator" | ||
- | - You will return to the main dialog=" | ||
- | - Append to field=Label " (single result)" | ||
- | - Click icon=" | ||
- | - Leave the current instance of Scaffold 2.0 open, since we'll use the same translator in the next section. | ||
- | |||
- | To create the stub for a second translator, | ||
- | |||
- | - Switch to tab=Metadata. | ||
- | - Click button=Generate next to field=" | ||
- | - Change the contents of field=Label from < | ||
- | - Click icon=Save (second from left): your translator should save silently. | ||
- | - Close all running Scaffold 2.0 instances. | ||
- | |||
- | To populate, test, and save the second translator, | ||
- | |||
- | - Ensure the [[http:// | ||
- | - Open Scaffold 2.0 from the Firefox main menu with Tools> | ||
- | - Hit icon=Load (upper left of the dialog) to popup dialog=" | ||
- | - Scroll through the "Load Translator" | ||
- | - You will return to the main dialog=" | ||
- | - Click icon=" | ||
- | - Switch to tab=Code and enter the following code (actually, you're just appending < | ||
- | if (doc.title.match(" | ||
- | return " | ||
- | } else if (doc.title.match(" | ||
- | return " | ||
- | } | ||
- | } | ||
- | function doWeb(doc, url) { | ||
- | var namespace = doc.documentElement.namespaceURI; | ||
- | var nsResolver = namespace ? function(prefix) { | ||
- | if (prefix == ' | ||
- | } : null; | ||
- | var articles = new Array(); | ||
- | var items = new Object(); | ||
- | var nextTitle; | ||
- | if (detectWeb(doc, | ||
- | var titles = doc.evaluate('// | ||
- | while (nextTitle = titles.iterateNext()) { | ||
- | items[nextTitle.href] = nextTitle.textContent; | ||
- | } | ||
- | items = Zotero.selectItems(items); | ||
- | for (var i in items) { | ||
- | articles.push(i); | ||
- | } | ||
- | } else { | ||
- | articles = [url]; | ||
- | } | ||
- | Zotero.Utilities.processDocuments(articles, | ||
- | Zotero.wait(); | ||
- | } | ||
- | </ | ||
- | - Click icon=Save (second from left): your translator should save silently. | ||
- | - Click icon=" | ||
- | - Click icon=Save (second from left): your translator should save silently. | ||
- | - Close all running Scaffold 2.0 instances. | ||
- | - Return focus to the page or tab containing the [[http:// | ||
- | |||
- | ===== Chapter 16: Scraping the Individual Entry Page ===== | ||
- | |||
- | [[http:// | ||
- | |||
- | - open the single-result translator | ||
- | - extend the single-result translator | ||
- | - test the single-result translator on single-result pages | ||
- | - test the single-result translator on a search-results page | ||
- | |||
- | To open the single-result translator: | ||
- | |||
- | - Close any running Scaffold 2.0 instances. | ||
- | - Ensure the [[http:// | ||
- | - Open Scaffold 2.0 from the Firefox main menu with Tools> | ||
- | - Hit icon=Load (at the upper left of the dialog). This should popup dialog=" | ||
- | - Scroll through the "Load Translator" | ||
- | - You will return to the main dialog=" | ||
- | - Click icon=" | ||
- | - Leave the current instance of Scaffold 2.0 open, since we'll use the same translator in the next section. | ||
- | |||
- | To extend the single-result translator: add a < | ||
- | |||
- | - replace the current contents of tab=Code with the code below:< | ||
- | if (doc.title.match(" | ||
- | return " | ||
- | } else if (doc.title.match(" | ||
- | return " | ||
- | } | ||
- | } | ||
- | |||
- | // The function used to save well formatted data to Zotero | ||
- | function associateData (newItem, items, field, zoteroField) { | ||
- | if (items[field]) { | ||
- | newItem[zoteroField] = items[field]; | ||
- | } | ||
- | } | ||
- | |||
- | function scrape(doc, url) { | ||
- | // variable declarations | ||
- | var newItem = new Zotero.Item(' | ||
- | newItem.url = doc.location.href; | ||
- | newItem.title = "No Title Found"; | ||
- | var items = new Object(); | ||
- | var tagsContent = new Array(); | ||
- | |||
- | // scrape page data, save to Zotero | ||
- | getItems(doc, | ||
- | getAuthors(newItem, | ||
- | getImprints(newItem, | ||
- | getTags(newItem, | ||
- | saveToZotero(newItem, | ||
- | } | ||
- | |||
- | function getItems(doc, | ||
- | // namespace code | ||
- | var namespace = doc.documentElement.namespaceURI; | ||
- | var nsResolver = namespace ? | ||
- | function(prefix) { | ||
- | if (prefix == ' | ||
- | } : null; | ||
- | |||
- | // populate " | ||
- | var blankCell = " | ||
- | var headersTemp; | ||
- | var headers; | ||
- | var contents; | ||
- | var myXPathObject = doc.evaluate('// | ||
- | var myXPathObject2 = doc.evaluate('// | ||
- | while (headers = myXPathObject.iterateNext()) { | ||
- | headersTemp = headers.textContent; | ||
- | if (!headersTemp.match(/ | ||
- | headersTemp = blankCell; | ||
- | blankCell = blankCell + " | ||
- | } | ||
- | contents = myXPathObject2.iterateNext().textContent; | ||
- | if (headersTemp.match(" | ||
- | tagsContent.push(contents); | ||
- | } | ||
- | items[headersTemp.replace(/ | ||
- | } | ||
- | } | ||
- | |||
- | function getAuthors(newItem, | ||
- | // | ||
- | if (items[" | ||
- | var author = items[" | ||
- | if (author.match("; | ||
- | var authors = author.split("; | ||
- | for (var i in authors) { | ||
- | newItem.creators.push(Zotero.Utilities.cleanAuthor(authors[i], | ||
- | } | ||
- | } else { | ||
- | newItem.creators.push(Zotero.Utilities.cleanAuthor(author, | ||
- | } | ||
- | } | ||
- | } | ||
- | |||
- | function getImprints(newItem, | ||
- | // Format and save " | ||
- | if (items[" | ||
- | items[" | ||
- | if (items[" | ||
- | var colonLoc = items[" | ||
- | newItem.place = items[" | ||
- | var commaLoc = items[" | ||
- | var date1 =items[" | ||
- | newItem.date = date1.substr(0, | ||
- | newItem.publisher = items[" | ||
- | } else { | ||
- | newItem.publisher = items[" | ||
- | } | ||
- | } | ||
- | } | ||
- | |||
- | function getTags(newItem, | ||
- | if (items[" | ||
- | tagsContent.push(items[" | ||
- | } | ||
- | for (var i = 0; i < tagsContent.length; | ||
- | newItem.tags[i] = tagsContent[i]; | ||
- | } | ||
- | } | ||
- | |||
- | function saveToZotero(newItem, | ||
- | // Associate and save well-formed data to Zotero | ||
- | associateData (newItem, items, " | ||
- | associateData (newItem, items, " | ||
- | associateData (newItem, items, " | ||
- | associateData (newItem, items, " | ||
- | newItem.repository = " | ||
- | newItem.complete(); | ||
- | } | ||
- | |||
- | function doWeb(doc, url) { | ||
- | // namespace code | ||
- | var namespace = doc.documentElement.namespaceURI; | ||
- | var nsResolver = namespace ? function(prefix) { | ||
- | if (prefix == ' | ||
- | } : null; | ||
- | |||
- | // variable declarations | ||
- | var articles = new Array(); | ||
- | var items = new Object(); | ||
- | var nextTitle; | ||
- | |||
- | // If Statement checks if page is a Search Result, then saves requested Items | ||
- | if (detectWeb(doc, | ||
- | var titles = doc.evaluate('// | ||
- | while (nextTitle = titles.iterateNext()) { | ||
- | items[nextTitle.href] = nextTitle.textContent; | ||
- | } | ||
- | items = Zotero.selectItems(items); | ||
- | for (var i in items) { | ||
- | articles.push(i); | ||
- | } | ||
- | } else { | ||
- | //saves single page items | ||
- | articles = [url]; | ||
- | } | ||
- | |||
- | // process everything, calling function=scrape to do the heavy lifting | ||
- | Zotero.Utilities.processDocuments(articles, | ||
- | Zotero.wait(); | ||
- | } | ||
- | </ | ||
- | - Click icon=Save (second from left): your translator should save silently. | ||
- | - Close all running Scaffold 2.0 instances. | ||
- | |||
- | To test the translator on a single-result page: | ||
- | |||
- | - Ensure the [[http:// | ||
- | - Open Scaffold 2.0 from the Firefox main menu with Tools> | ||
- | - Hit icon=Load (at the upper left of the dialog). This should popup dialog=" | ||
- | - Scroll through the "Load Translator" | ||
- | - You will return to the main dialog=" | ||
- | - Click icon=" | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | |||
- | 14:36:30 Translation successful | ||
- | </ | ||
- | - Close all running Scaffold 2.0 instances. | ||
- | |||
- | Test the translator on another single-result page: | ||
- | |||
- | - Ensure the [[http:// | ||
- | - Open Scaffold 2.0 from the Firefox main menu with Tools> | ||
- | - Hit icon=Load (at the upper left of the dialog). This should popup dialog=" | ||
- | - Scroll through the "Load Translator" | ||
- | - You will return to the main dialog=" | ||
- | - Click icon=" | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | |||
- | 18:52:01 Translation successful | ||
- | </ | ||
- | - Close all running Scaffold 2.0 instances. | ||
- | |||
- | To test the translator on a search-results page: | ||
- | |||
- | - Ensure the [[http:// | ||
- | - Open Scaffold 2.0 from the Firefox main menu with Tools> | ||
- | - Hit icon=Load (upper left of the dialog) to popup dialog=" | ||
- | - Scroll through the "Load Translator" | ||
- | - You will return to the main dialog=" | ||
- | - Click icon=" | ||
- | - TODO: test adding an item to your library. | ||
- | - Leave the current instance of Scaffold 2.0 open, since we'll use the same translator in the next section. | ||
- | |||
- | Since it is now clear that our " | ||
- | |||
- | - Switch to tab=Metadata. | ||
- | - In field=Label, | ||
- | - Click icon=Save (second from left): your translator should save silently. | ||
- | - TODO: delete now-unnecessary translator=" | ||
- | - Close all running Scaffold 2.0 instances. | ||
- | |||
- | ===== Chapter 17: Common Problems when Scraping an Individual Entry Page ===== | ||
- | |||
- | [[http:// | ||