Line endings missing in imported RIS notes

[dstillman]

From ​http://forums.zotero.org/discussion/143/:

In Windows XP, I imported my Endnote library which has some pretty extensive notes on some books and articles in the "Notes" fields. Unfortunately, they've been rendered pretty unreadable in Zotero because all the line endings within the notes fields are lost on import.

To export from Endnote, I selected RIS Output Style and then exported to a text file. The line endings, including those in the notes field, are clearly present in the text file when I open it in Notepad.

From a quick glance at the code, it looks like we read the RIS file line-by-line and then append the line to the previous with a space. The newline isn't returned by nsILineInputStream (or whatever we're using--I see a few different read() methods), but can we just have it use a "\n" instead of a space if this is an N1/N2/AB field?

We should probably also not trim whitespace (e.g. line = line.replace(/^\s+/, "")) in a note, since there might be tabs, etc.

In a separate but related matter, it looks like newlines are stripped when exporting as well:

addTag("N1", item.notes[j].note.replace(/[\r\n]/g, " "));

(from RIS's doExport(), ~ line 6426)

Do we have a reason for doing this? Is EndNote not in-spec?

[simon]

The RIS spec says:

If the information following any one tag is more than 70 characters long, it is allowable (though not necessary) to insert a carriage return/line feed at the end of 70 characters, and continue on the next line.

This is the guideline that we're currently following. It might make sense to handle the notes differently, though, if this is what EndNote does.

[dstillman]

Yeah, that seems pretty silly to me, at least for our purposes. (I suppose they might have their reasons, but not allowing for content newlines in a transport format with note and abstract fields seems like somewhat of an oversight.)

I'd say we should just follow EndNote and 1) add explicit \n's in notes and abstracts when importing and 2) not strip \r and \n when exporting (though we should still convert \n in notes to \r\n to stay consistent and in-spec (even if EndNote just uses \n)).

[simon]

(In [940]) closes #428, line endings missing in imported RIS notes