[PATCH] Item with non-ASCII characters corrupted on save when Show Editor is open

[fbennett]

When an item containing a non-ASCII character (such as the name Piedelièvre) is cited, it will save and reopen without error unless Show Editor is selected (citation editor open) when the item is saved. In this case only, the following errors occur:

(1) the next attempt to edit the same citation yields an error "Zotero experienced an error updating your document ... A field code in this document is corrupted.", and

(2) an attempt to edit any other citation highlights the corrupted citation, with the message "The Zotero field corresponding to this citation, which tells Zotero which item in your library this citation represents, has been corrupted. Would you like to reselect the item? ...", followed by the error popup at (1).

Apparently conversion to or from RTF is not working quite right.

(Test was done using OpenOffice 3.1 under Linux.)

[fbennett]

Apparently something in the chain is allergic to non-ascii characters. The attached patch escapes the custom string to ascii before sending it to the plugin for storage in the document, and unescapes it when it is used for display in the browser, or for insertion into the document text. Seems to work, see what you think.

Might be a similar issue around bibliography editing.

[fbennett]

As a side note, I hadn't realized that hitting OK with the editor open, and without making any changes, would customize a citation. From the code, I also figured out that accepting a citation with the editor hidden will overwrite all manual changes. Perhaps a text like this might help people like me who are slow on the uptake:

"Press OK with this editor open to customize the citation. To unset customization (and overwrite any manual edits), hide the editor and press OK. Note: Customized citations are not updated to reflect changes in your document, your database, or in the citation style."

[simon]

(In [6323]) closes #1706, [PATCH] Item with non-ASCII characters corrupted on save when Show Editor is open

The problem here was that entities weren't properly being encoded as Unicode RTF when the editor was used, because TinyMCE was replacing high characters with HTML entities that were not properly decoded. This is now fixed.

[fbennett]

Brilliant, many thanks for your time on this, Simon. Happy to confirm that it works as advertised.