| Both sides previous revisionPrevious revisionNext revision | Previous revisionLast revisionBoth sides next revision |
| dev:client_coding:direct_sqlite_database_access [2017/12/18 09:02] – Move data model information into this page bwiernik | dev:client_coding:direct_sqlite_database_access [2020/12/24 12:40] – [Direct Access to the Zotero SQLite Database] dstillman |
|---|
| <html><p id="zotero-5-update-warning" style="color: red; font-weight: bold">We’re | |
| in the process of updating the documentation for | |
| <a href="https://www.zotero.org/blog/zotero-5-0">Zotero 5.0</a>. Some documentation | |
| may be outdated in the meantime. Thanks for your understanding.</p></html> | |
| |
| |
| ====== Direct Access to the Zotero SQLite Database ====== | ====== Direct Access to the Zotero SQLite Database ====== |
| |
| While it is generally preferable to access Zotero library data via either the [[dev/Server API]] or [[dev/client_coding/JavaScript API]], it is also possible to directly access the SQLite database of the Zotero client using an SQLite client, library, or third-party tool. | While it is generally preferable to access Zotero library data via either the [[dev/Web API]] or [[dev/client_coding/JavaScript API]], it is also possible to directly access the SQLite database of the Zotero client using an SQLite client, library, or third-party tool. All library data is stored in an SQLite database, zotero.sqlite, within the [[:zotero_data|Zotero data directory]]. |
| |
| Zotero uses mozStorage, a database API backed by the SQLite database engine, offering the power of a relational database system within the Mozilla development environment. All system data and item metadata is stored in an SQLite database, zotero.sqlite, within the [[:zotero_data|Zotero data directory]].((If the database doesn’t exist at browser startup, a new one is created from .sql files in the extension directory, which are then used only during schema updates.)) | However, **access to the SQLite database should be done only in a read-only manner.** Modifying the database while Zotero is running can easily result in a corrupted database. A caching layer breaks the normal file-locking in SQLite that allows for safe concurrent file access, and even if Zotero is shut down before accessing the file, modifying the database directly bypasses the data validation and referential integrity checks performed by Zotero and the Zotero web API that are required for Zotero to function properly. Generally, the SQLite database should be viewed as an internal database that has the benefit of being externally readable for people who want to get the data out in other ways. |
| | |
| However, **external access to the SQLite database (including direct access via the mozStorage API) should be done only in a read-only manner.** Modifying the database while Zotero is running can easily result in a corrupted database, as mozStorage caching breaks the normal file-locking in SQLite that allows for safe concurrent file access. And even if Firefox is shut down before accessing the file, modifying the database directly bypasses the data validation and referential integrity checks performed by Zotero and the Zotero server that are required for Zotero to function properly. Generally, the SQLite database should be viewed as an internal database that has the benefit of being externally readable for people who want to get the data out in other ways. | |
| |
| Also be aware that the SQLite database structure can change between Zotero releases. | Also be aware that the SQLite database structure can change between Zotero releases. |
| ====== Zotero Client Data Model ====== | ====== Zotero Client Data Model ====== |
| |
| Each bibliographic record in Zotero has an "item type" (book, journal article, thesis...), a set of fields (author, title, date...), and field values. The selection of fields depends on the item; for instance, a thesis has a "university" field which other items do not have. | Each bibliographic record in Zotero has an "item type" (book, journal article, thesis, …), a set of fields (author, title, date, …), and field values. The selection of fields depends on the item type; for instance, a thesis has a "university" field which other items do not have. |
| |
| Item types and fields are defined in [[https://www.zotero.org/trac/browser/extension/branches/2.0/system.sql|system.sql]] and translated in [[https://www.zotero.org/trac/browser/extension/branches/2.0/chrome/locale/en-US/zotero/zotero.properties|zotero.properties]]. In theory you can change and add item types and fields, but this may make your installation of Zotero incompatible with other users and future releases---import routines and citation formats may not work anymore. If you need to add additional information to your records, you should use the "extra" field that is available for all item types. Zotero can be used in many languages, but you should always refer to item types and fields with their internal name as defined in ''system.sql'' and used in ''zotero.properties''. | Item types and fields, as well as translations into various languages, are defined in the[[https://github.com/zotero/zotero-schema/tree/master|Zotero data schema]]. Zotero can be used in many languages, but you should always refer to item types and fields with their internal name as defined in data schema. |
| |
| In addition to the basic item types and fields, each item can have associated user data as defined in [[https://www.zotero.org/trac/browser/extension/branches/2.0/userdata.sql|userdata.sql]]. This includes: | In theory, you can change and add item types and fields, but this will break Zotero sync and may make break many parts of Zotero functionality, such as item import and citation formatting. If you need to add additional information to your records, you should use the "Extra" field that is available for all item types. |
| | |
| | In addition to the basic item types and fields, each item can have associated user data, including: |
| * a set of notes | * a set of notes |
| * a set of attachments | * a set of attachments |
| Attachments (PDFs, snapshots, etc.) are stored inside the [[:zotero_data|Zotero data directory]], in separate subdirectories named after the ID of the item they belong to. | Attachments (PDFs, snapshots, etc.) are stored inside the [[:zotero_data|Zotero data directory]], in separate subdirectories named after the ID of the item they belong to. |
| |
| For a comprehensive list of the internal names of fields, their English equivalents, and their CSL mappings, if applicable, see the [[http://gsl-nagoya-u.net/http/pub/csl-fields/index.html|CSL/Zotero Metadata Field Index]]. Similar mappings, and the creator roles available for each item type, can be [[dev:web_api:write_api#item_typefield_requests|retrieved from the Zotero server API]]. | For a list of the internal names of fields, their English equivalents, and their CSL mappings, see the [[https://aurimasv.github.io/z2csl/typeMap.xml|CSL/Zotero Metadata Field Index]]. |
| | |
| ===== Schema Spy ===== | |
| | |
| The comprehensive way of learning about Zotero's database structures is by digging into the [[https://www.zotero.org/trac/browser/extension/branches/2.0|.sql files]]. Here is, however, some automatically generated database documentation (created by [[http://schemaspy.sourceforge.net/|SchemaSpy]]): | |
| | |
| * for the [[http://zomark.github.com/zotero-marc/schema/trunk/|trunk (2010-10-19)]] | |
| * for the [[http://zomark.github.com/zotero-marc/schema/multilingual/|multilingual branch (2010-10-19)]] | |
Upgrade Storage