Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
zotero_dataserver [2011/04/25 13:43]
lemaxx [Config.inc.php]
zotero_dataserver [2017/11/12 19:53] (current)
Line 1: Line 1:
-====== Disclaimer ====== +<html><p id="​zotero-5-update-warning" ​style="colorredfont-weightbold">Were 
- +in the process of updating ​the documentation for 
-This guide was written based in part on information found on  zitation.org (unreachable when I was writing this tutorial) as well as the indications found on zotero.org. Some steps may be unnecessary or wrong but keep in mind that this is a first version. Any help or comment is welcome. +<a href="https://​www.zotero.org/​blog/zotero-5-0">Zotero 5.0</a>. Some documentation 
- +may be outdated ​in the meantimeThanks for your understanding.</p></html>
-Some items may seem obvious but I detail some steps not to forget my next installation. +
-This installation was done on Ubuntu 10.10 desktop running in a virtual machine. This allows me to use an unmodified version of Zotero 2.1 +
- +
-====== Dependencies:​ ====== +
- +
-===== Required ===== +
- +
-  * PHP +
-  * PHP modules (?) +
-  * Apache +
-  * Apache modules/​features:​ some sort of for PHP (?), virtual hosts support, mod_rewrite,​ auth (?) +
-  * MySQL with root access +
-  * MongoDB +
-  * ability to configure /etc/hosts or DNS +
-  * subversion client +
- +
-===== Optional ===== +
- +
-  * Solr +
-  * Memcached +
-  * Amazon S3 +
-  * Webdav +
- +
-Download Zotero dataserver sources +
-Use your subversion client to pull down the sources from Zotero’s repository. If using the command-line:​
svn co https://​www.zotero.org/​svn/​dataserver
 You’ll see the following layout (from here on references to zotero source paths will assume the root of dataserver/​trunk) +
- +
-<code> +
-dataserver/​ +
-          trunk/ +
-               ​controllers +
-               ​htdocs +
-               ​include +
-               ​misc +
-               ​model +
-               ​processor +
-               ​tests +
-               tmp</code> +
- +
-These files can be anywhere but this guide will assume they are located in /​home/​UTILISATEUR/​www-dev/​dataserv ←- NOTE without the ‘trunk’ – in other words, the ‘include’ directory is at /​home/​UTILISATEUR/​www-dev/​dataserv/​include. +
- +
-====== Configure Apache and Host ====== +
- +
-===== Host ===== +
- +
-We need to define two hostnames, the “sync” and “api” hosts. For testing on one machine this can be done via an /etc/hosts entry +
-In this exemple i use zotero.org domain because i use a virtual machine and i don’t use a custom Zotero plugin. +
-If you want to use your own domain you need to edit the plugin (see below) +
-<​code>​ +
-127.0.0.1 sync.zotero.org api.zotero.org</​code>​ +
- +
-After that when you ping sync.zotero.org you will see it is resolving to 127.0.0.1 +
- +
-===== Apache ===== +
- +
-From web only HTDOCS folder must be accessible. So you can use a virtual host +
-Your sync server need a ssl connection you have to create two config files in /​etc/​apache2/​sites-available +
- +
-<​code>​ +
-<​VirtualHost ​ sync.zotero.org:​80>​ +
-  +
- ​ServerName sync.zotero.org +
- ​ServerAlias api.zotero.org  +
-  +
- ​DocumentRoot /​home/​UTILISATEUR/​www-dev/​dataserv/​htdocs +
-  +
- <​Directory ​"/​home/​UTILISATEUR/​www-dev/​dataserv/​htdocs">​ +
- ​Options FollowSymLinks MultiViews +
- ​AllowOverride All +
- Order allow,​deny +
- Allow from all +
- </​Directory>​ +
-  +
- ​AllowEncodedSlashes On  +
-  +
-</​VirtualHost>​ +
-</​code>​ +
- +
-<​code>​ +
-<​IfModule mod_ssl.c>​ +
-<​VirtualHost sync.zotero.org:​443>​ +
-  +
- ​ServerName sync.zotero.org +
- ​ServerAlias api.zotero.org  +
-  +
- ​DocumentRoot /​home/​UTILISATEUR/​www-dev/​dataserv/​htdocs +
-  +
- <​Directory "/​home/​UTILISATEUR/​www-dev/​dataserv/​htdocs">​ +
- ​Options FollowSymLinks MultiViews +
- ​AllowOverride All +
- Order allow,​deny +
- Allow from all +
- </​Directory>​ +
-  +
- ​AllowEncodedSlashes On +
-  +
- # ​  ​Enable/​Disable SSL for this virtual host. +
- ​SSLEngine on +
-  +
- ​SSLCertificateFile ​   /​etc/​ssl/​certs/​ssl-cert-snakeoil.pem +
- ​SSLCertificateKeyFile /​etc/​ssl/​private/​ssl-cert-snakeoil.key +
-  +
- <​FilesMatch ​"\.(cgi|shtml|phtml|php)$"+
- ​SSLOptions +StdEnvVars +
- </​FilesMatch>​ +
- <​Directory /​usr/​lib/​cgi-bin>​ +
- ​SSLOptions +StdEnvVars +
- </​Directory>​ +
-  +
- ​BrowserMatch "MSIE [2-6]" \ +
- ​nokeepalive ssl-unclean-shutdown \ +
- ​downgrade-1.0 force-response-1.0 +
- # MSIE 7 and newer should be able to use keepalive +
- ​BrowserMatch "MSIE [17-9]"​ ssl-unclean-shutdown +
-  +
-</​VirtualHost>​ +
-</​IfModule>​ +
- +
-</​code>​ +
- +
-====== Create and configure database ====== +
- +
-===== Mongo ===== +
- +
- +
-Install MongoDB and Mongo php driver +
- +
-use /​misc/​mongo_schema.js to create content +
- +
-===== Mysql ===== +
- +
- +
-To understand how database must be set i used the script « test_reset » locate in « Misc » folder (you can use this script to setup your database). I use the same database name. +
- +
-In this folder you will find every *.sql files need for this installation. +
- +
-You have to create 7 Mysql database. Some of them are used only when you run Zotero server in test mode, for other I do not know yet their use. +
- +
-  * zotero_cache > use cache.sql +
-  * zotero_www (this name is hardcoded) > see below +
-  * zotero_www_test (this name is hardcoded) > see below +
-  * zoterotest_ids > use ids.sql +
-  * zoterotest_master > use master.sql and after coredata.sql +
-  * zoterotest1 > use shard.sql and after trigger.sql +
-  * zoterotest2 > use shard.sql and after trigger.sql +
- +
-There is no .sql files to set up zotero_www and zotero_www_test so use as describe on zotero.org/​trac/​wiki/​dataserver_setup +
- +
-<code mysql> +
-CREATE TABLE  `users` ( +
-  `userID` ​            ​mediumint ​     UNSIGNED auto_increment ​                              NOT NULL, +
-  `username` ​          ​varchar(40) ​   CHARACTER SET utf8 COLLATE utf8_general_ci ​           NOT NULL, +
-  `slug` ​              ​varchar(40) ​   CHARACTER SET utf8 COLLATE utf8_general_ci ​               NULL, +
-  `password` ​          ​char(40) ​      ​CHARACTER SET utf8 COLLATE utf8_bin ​                  NOT NULL, +
-  `email` ​             varchar(100) ​  ​CHARACTER SET utf8 COLLATE utf8_general_ci ​           NOT NULL, +
-  `openid` ​            ​varchar(100) ​  ​CHARACTER SET utf8 COLLATE utf8_general_ci ​               NULL, +
-  `role` ​              ​enum('​member',​ +
-                            '​admin'​) ​ DEFAULT '​member' ​                                     NOT NULL, +
-  `resetValidationKey` char(40) ​      ​CHARACTER SET utf8 COLLATE utf8_bin ​                      ​NULL,​ +
-  `resetValidationExp` int                                                                      NULL, +
-  `emailValidationKey` char(10) ​      ​CHARACTER SET utf8 COLLATE utf8_bin ​                      ​NULL,​ +
-  `emailValidated` ​    ​tinyint(1) ​    ​DEFAULT '​0' ​                                          NOT NULL, +
-  `hasImage` ​          ​tinyint(1) ​    ​DEFAULT '​0' ​                                          NOT NULL, +
-  `lastUpdated` ​       timestamp ​     ON UPDATE CURRENT_TIMESTAMP DEFAULT CURRENT_TIMESTAMP NOT NULL, +
-  `dateRegistered` ​    ​timestamp ​                                                           NOT NULL, +
-  PRIMARY KEY (`userID`),​ +
-  UNIQUE KEY  (`username`),​ +
-  UNIQUE KEY  (`slug`), +
-  UNIQUE KEY  (`email`),​ +
-  UNIQUE KEY  (`resetValidationKey`) +
-) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin;​ +
- +
-</​code>​ +
- +
-You need to add specific user in Mysql +
-If you want to personalize those user you have to change value in shard info (see below) +
-<code mysql> +
- +
-CREATE USER zoterotest0@localhost IDENTIFIED BY '​pass0';​ +
-CREATE USER zoterotest1@localhost IDENTIFIED BY '​pass1';​ +
-CREATE USER zoterotest2@localhost IDENTIFIED BY '​pass2';​ +
-CREATE USER zoterotest_ids@localhost IDENTIFIED BY '​pass1';​ +
-CREATE USER zoterotest_ids@localhost IDENTIFIED BY '​pass2';​ +
-CREATE USER zoterotest_www@localhost IDENTIFIED BY '​pass';​ +
- +
-</​code>​ +
- +
-notegive acces on sync.zotero.org too +
- +
-Give Grant access +
- +
-<code mysql> +
- +
-GRANT SELECT ON zoterotest_master.* TO zoterotest1@localhost; +
-GRANT SELECT ON zoterotest_master.* TO zoterotest2@localhost;​ +
-GRANT SELECT, INSERT, UPDATE, DELETE, CREATE TEMPORARY TABLES ON zoterotest1.* TO zoterotest1@localhost;​ +
-GRANT SELECT, INSERT, UPDATE, DELETE, CREATE TEMPORARY TABLES ON zoterotest2.* TO zoterotest2@localhost;​ +
-GRANT SELECT,​INSERT,​DELETE ON zoterotest_ids.* TO zoterotest_ids@localhost;​ +
-GRANT SELECT,​INSERT,​DELETE ON zoterotest_ids.* TO zoterotest_ids@localhost;​ +
-GRANT SELECT ON zotero_www_test.* TO zoterotest_www@localhost;​ +
-</​code>​ +
- +
-Set up shard info +
- +
-<code mysql> +
-INSERT INTO shardHosts VALUES (1, '​127.0.0.1',​ 3306, '​up'​);​ +
-INSERT INTO shardHosts VALUES (2, '​sync.zotero.org',​ 3306, '​up'​);​ +
-INSERT INTO shards VALUES (1, 1, '​zoterotest1',​ '​pass1',​ '​zoterotest1',​ '​up'​);​ +
-INSERT INTO shards VALUES (2, 1, '​zoterotest2',​ '​pass2',​ '​zoterotest2',​ '​up'​);​ +
-</​code>​ +
-Initial users and groups for tests +
- +
-<code mysql> +
-INSERT INTO libraries VALUES (1, '​user',​ '0000-00-00 00:00:​00',​ 0, 1); +
-INSERT INTO libraries VALUES (2, '​user',​ '​0000-00-00 00:​00:​00',​ 0, 1); +
-INSERT INTO libraries VALUES (3, '​group',​ '​0000-00-00 00:​00:​00',​ 0, 2); +
-INSERT INTO users VALUES (1, 1, '​testuser',​ '​0000-00-00 00:​00:​00',​ '​0000-00-00 00:​00:​00'​);​ +
-INSERT INTO users VALUES (2, 2, '​testuser2',​ '​0000-00-00 00:​00:​00',​ '​0000-00-00 00:​00:​00'​);​ +
-INSERT INTO groups VALUES (1, 3, 'Test Group',​ '​test_group',​ '​Private',​ 1, '​admins',​ '​all',​ '​members',​ '',​ '',​ 0, '​0000-00-00 00:​00:​00',​ '​0000-00-00 00:​00:​00'​);​ +
-INSERT INTO groupUsers VALUES (1, 1, '​owner',​ '​0000-00-00 00:​00:​00',​ '​0000-00-00 00:​00:​00'​);​ +
-  +
-INSERT INTO users VALUES (1, '​testuser',​ '​5d9c68c6c50ed3d02a2fcf54f63993b6',​ '​test@zotero.org',​ NULL, '​member',​ NULL, NULL, NULL, 1, '​0000-00-00 00:​00:​00',​ '​0000-00-00 00:​00:​00',​ 0, '​testuser'​);​ +
-INSERT INTO users VALUES (2, '​testuser2',​ '​5d9c68c6c50ed3d02a2fcf54f63993b6',​ '​test2@zotero.org',​ NULL, '​member',​ NULL, NULL, NULL, 1, '​0000-00-00 00:​00:​00',​ '​0000-00-00 00:​00:​00',​ 0, '​testuser2'​);​ +
-INSERT INTO shardLibraries VALUES (1, '​user'​);​ +
-INSERT INTO shardLibraries VALUES (2, '​user'​);​ +
-INSERT INTO shardLibraries VALUES (3, '​group'​);​ +
-</code> +
- +
-when you sync use ‘testuser’ or ‘testuser2′ password ‘testuser’ +
- +
-====== Config Zotero data server ====== +
- +
- +
-Copy the sample config files under include/​config and remove « sample » from the filenames +
-fill out credentials and paths in config files. +
- +
-===== dbconnect.inc.php ===== +
- +
- +
-db master: ​ zoterotest_master. +
- +
-db id1 and id2: zoterotest_ids +
- +
-db www1 and www2: zotero_www ​ (when you are in test mode server auto link to zotero_www_test) this is hardcoded in /​model/​auth/​Password.inc.php +
- +
-db cache: zotero_cache +
- +
-===== Config.inc.php ===== +
- +
- +
-TODO explain values +
- +
-In the original file $AUTH_URI is not present, you can add it to avoid error log. Usage need to be cleared. +
-<code php> public static $AUTH_URI ='';</​code>​ +
-<code php> +
- +
-<? +
-class Z_CONFIG { +
- public static $API_ENABLED = true; +
- public static $SYNC_ENABLED = true; +
- public static $PROCESSORS_ENABLED = true; +
- public static $MAINTENANCE_MESSAGE = '​Server updates in progress. Please try again in few minutes.';​ +
-  +
- public static $TESTING_SITE = false; +
- public static $DEV_SITE = false; +
- public static $AUTH_URI ='';​ +
- public static $BASE_URI = '​http://​zotero.org';​ +
-  +
- public static $AUTH_SALT = '';​ +
- public static $API_SUPER_USERNAME = '';​ +
- public static $API_SUPER_PASSWORD = '';​ +
-  +
- public static $API_BASE_URI = '​api.zotero.org';​ +
- public static $API_BASE_URI_WWW ='';​ +
- +
- public static $S3_BUCKET = '';​ +
- public static $S3_ACCESS_KEY = '';​ +
- public static $S3_SECRET_KEY = '';​ +
-  +
- public static $MEMCACHED_ENABLED = false; +
- public static $MEMCACHED_SERVERS = array( +
- '​192.168.1.1:​11211:​2',​ '​192.168.1.2:​11211:​1'​ +
- ); +
- public static $MEMCACHED_SERVER_NAME_PREFIX_MAP = array( +
- '​api.example.com'​ => '​sync.example.com'​ +
- ); +
-  +
- public static $MONGO_SERVERS = array( +
- '​localhost:​27017',​ '​localhost:​27017'​ +
- ); +
- public static $MONGO_DB ​= "Name_of_your_database";​ +
- public static $MONGO_SAFE_NUM = 2; +
-  +
- public static $SOLR_SERVER = "​192.168.1.1:8983";​ +
-  +
- public static $CITE_SERVERS = array( +
- "​192.168.1.1:​8080",​ "​192.168.1.2:​8080"​ +
- ); +
-  +
- public static $LOG_TO_SCRIBE = false; +
- public static $LOG_ADDRESS = '';​ +
- public static $LOG_PORT = 1463; +
- public static $LOG_TIMEZONE = '​US/​Eastern';​ +
- public static $LOG_TARGET_DEFAULT = '​errors';​ +
-  +
- public static $PROCESSOR_PORT_DOWNLOAD = 3455; +
- public static $PROCESSOR_PORT_UPLOAD = 3456; +
- public static $PROCESSOR_PORT_ERROR = 3457; +
- public static $PROCESSOR_PORT_INDEX = 3458; +
-  +
- public static $PROCESSOR_LOG_TARGET_DOWNLOAD = '​sync-processor-download';​ +
- public static $PROCESSOR_LOG_TARGET_UPLOAD = '​sync-processor-upload';​ +
- public static $PROCESSOR_LOG_TARGET_ERROR = '​sync-processor-error';​ +
- public static $PROCESSOR_LOG_TARGET_INDEX = '​processor-index';​ +
-  +
- public static $SYNC_DOWNLOAD_SMALLEST_FIRST = false; +
- public static $SYNC_UPLOAD_SMALLEST_FIRST = false; +
-  +
- // Set some things manually for running via command line +
- public static $CLI_PHP_PATH = '/​usr/​bin/​php';​ +
- public static $CLI_DOCUMENT_ROOT = "/var/www/​web/​zserv/";​ +
- public static $CLI_SERVER_NAME = "sync.zotero.org"; +
- public static $CLI_SERVER_PORT = "​80";​ +
-  +
- public static $SYNC_ERROR_PATH = '/var/log/​httpd/​sync-errors/';​ +
- public static $API_ERROR_PATH = '/​var/​log/​httpd/​api-errors/';​ +
-+
-?> +
-</​code>​ +
-===== Launch daemon ===== +
- +
- +
-There are for daemons, each in its own directory under the processor directoryFire up for shells and run each independently in the foreground. +
- +
-<code bash> +
- +
-cd processor/​download +
-  php daemon.php +
-cd processor/​upload +
-  php daemon.php +
-cd processor/​error +
-  php daemon.php +
-cd processor/​index +
-  php daemon.php +
-   +
-</​code>​ +
-In all cases you should see the output similar to this (the daemon name will vary) +
- +
-<code bash> +
- +
-  Starting sync download processor daemon +
-  ​processors, 0 queued processes +
-  Waiting for command +
-</code> +
- +
-====== Use zotero in your virtual machine to sync====== +
- +
-If your ssl is auto signed you must accept certificate before the first synchronization  +
-In Firefox open https://​sync.zotero.org and https://​api.zotero.org +
- +
-in Zotero preferences use login= testuser password= testuser and launch sync +
- +
-====== Miscellaneous ====== +
-===== Add new user ===== +
- +
- +
-You must add new entries in zotero_www (or zotero_www_test if in test mode). +
- +
-  * Necessary fields should ​be userID, username, password, slug. +
-  * Password entries are generated by taking a sha1 hash of the user’s password concatenated to the salt. +
-  * Slugs are generated by taking the username and trimming whitespace, lowercasing,​ preg_replace("/​[^a-z0-9 ._-]/",​ "",​ $input), and replacing spaces with underscores. +
- +
-===== Create group ===== +
- +
- +
-As shown in ‘test_reset’ file: +
- +
-TODO explain values +
- +
-<code mysql> +
-INSERT INTO libraries VALUES (3, '​group',​ '​0000-00-00 00:​00:​00',​ 0, 2); +
-INSERT INTO groups VALUES (1, 3, 'Test Group',​ '​test_group',​ '​Private',​ 1, '​admins',​ '​all',​ '​members',​ '',​ '',​ 0, '​0000-00-00 00:​00:​00',​ '​0000-00-00 00:​00:​00'​);​ +
-INSERT INTO groupUsers VALUES (1, 1, '​owner',​ '​0000-00-00 00:​00:​00',​ '​0000-00-00 00:​00:​00'​);​ +
-INSERT INTO shardLibraries VALUES (3, '​group'​);​ +
-</​code>​ +
- +
-===== Custom Zotero ===== +
- +
-TODO explain how edit zotero.jar +
- +
-If you want to use your own domain you need to edit the plugin. +
-Edit  zotero.js and change SYNC_URL and API_URL with your domain. +
- +
-<code javascript> +
-    const ZOTERO_CONFIG = { +
-    GUID: '​zotero@chnm.gmu.edu',​ +
-    DB_REBUILD:​ false, // erase DB and recreate from schema +
-    REPOSITORY_URL:​ '​https://​api.zotero.org/​repo',​ +
-    REPOSITORY_CHECK_INTERVAL:​ 86400, // 24 hours +
-    REPOSITORY_RETRY_INTERVAL:​ 3600, // 1 hour +
-    BASE_URI: '​http://​zotero.org/',​ +
-    WWW_BASE_URL:​ '​http://​www.zotero.org/',​ +
-    SYNC_URL: '​http://​sync.mydomain.org/',​ +
-    API_URL: '​http://​api.mydomain.org/'​ +
-    }; +
- +
- +
-</code> +
- +
-====== Untested / Not working / Todo====== +
- +
-===== Untested ===== +
-  * Webdav to use files attachment. +
-  * API: i don’t know how use it or configure it. +
- +
-===== Not working ===== +
-  * Change user privileges in groups only work if the user performs a restore from the server +
-  * Sync attachment files in My Library using Zotero server (i suppose you need Amazon S3) +
-  * Sync attachment files in group using Zotero server (i suppose you need Amazon S3) +
- +
-===== Todo ===== +
-  * Create SQL script to install database in one step +
-  * Create interface to add/manage users and groups +
-  * Test with webdav to use files attachment +
- +
  
  
 +== Deprecated, go to: == 
 +http://​www.zotero.org/​support/​dev/​dataserver_setup
zotero_dataserver.txt · Last modified: 2017/11/12 19:53 (external edit)