Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
zotero_dataserver_setup [2011/07/14 08:06]
simonth
zotero_dataserver_setup [2017/11/12 19:53] (current)
Line 1: Line 1:
-This procedure (in 15 steps) is inspired by : https://​www.zotero.org/​trac/wiki/dataserver_setup and http://www.zotero.org/support/zotero_dataserver+<​html><​p id="​zotero-5-update-warning"​ style="​colorred; font-weight:​ bold">​We’re 
 +in the process of updating the documentation for 
 +<a href="https://​www.zotero.org/​blog/zotero-5-0">​Zotero 5.0</a>Some documentation 
 +may be outdated in the meantimeThanks for your understanding.<​/p></html>
  
-__Before starting, here are the packages to setup on your machine (we suppose you have an apache server running and know how to make basic configuration changes to it):__ 
  
-  * PHP (php5) +== Deprecatedgo to: ==  
-  * PHP modules (php5-mysql,​ php5-svn, php5-memcache,​ php5-memcached) +http://www.zotero.org/​support/dev/dataserver_setup
-  * Apache module: mod_rewrite +
-  * MySQL with root access (mysql-server,​ phpmyadmin) +
-  * MongoDB (http://​www.php.net/​manual/​en/​mongo.installation.php#​mongo.installation.nix) +
-  * Memcached (same as MongoDB but don't add "​extension=memcache.so"​ in php.ini) +
-  * Ability to configure /​etc/​hosts +
-  * Subversion client +
- +
-Now, we can begin. +
- +
-===== Step 1 ===== +
-Download Zotero dataserver sources (here: https://​www.zotero.org/​svn/​dataserver/​trunk/​).\\  +
-Use your subversion client to pull down the sources from Zotero'​s repository.\\  +
-These files can be anywhere but this guide will assume they are located in /​home/​UTILISATEUR/​dataserver <- NOTE: without the "​trunk"​ - in other wordsthe "​include"​ directory is at /​home/​UTILISATEUR/​dataserver/​include.\\  +
-You can use the following command-line:​ +
-<​code>​ +
-$ sudo svn co https://​www.zotero.org/​svn/​dataserver/​trunk/​ +
-</​code>​ +
- +
- +
-===== Step 2 ===== +
-Set up an apache VirtualHost for the dataserver.\\  +
-Use this code: +
-<​code>​ +
-NameVirtualHost *:443 +
-<​VirtualHost *:443>  +
- DocumentRoot /​home/​UTILISATEUR/​dataserver/​htdocs  +
- SSLEngine on +
- SSLCertificateFile /​etc/​apache2/​server.crt +
- SSLCertificateKeyFile /​etc/​apache2/​server.key  +
- <​Directory "​home/​UTILISATEUR/​dataserver/​htdocs">​ +
- Options FollowSymLinks MultiViews +
- AllowOverride All +
- Order allow,​deny +
- Allow from all +
- </​Directory>​  +
- AllowEncodedSlashes On   +
-</​VirtualHost>​ +
-</​code>​ +
- +
-Save this code as the file "​default"​ in /​etc/​apache2/​sites-available directory.\\ \\  +
- +
-NOTE: don't forget ​to create SSL certificate and SSL key. Like this: +
-<​code>​ +
-$ sudo mkdir /​tmp/​ssl_conf +
-$ cd /​tmp/​ssl_conf +
-$ sudo openssl req -config /​etc/​ssl/​openssl.cnf -new -out server.csr +
-$ sudo openssl rsa -in privkey.pem -out server.key +
-$ sudo openssl x509 -in server.csr -out server.crt -req -signkey server.key -days 365 +
-$ sudo openssl x509 -in server.crt -out server.der.crt -outform DER +
-$ sudo cp server.crt /​etc/​apache2/​ +
-$ sudo cp server.key /​etc/​apache2/​ +
-$ cd /​etc/​apache2/​mods-enabled +
-$ sudo ln -s ../​mods-available/​ssl* . +
-</​code>​ +
- +
- +
-===== Step 3 ===== +
-Add an /etc/hosts entry for sync.zotero.org and api.zotero.org to the IP address of the server.\\  +
-Command to edit /etc/hosts: +
-<​code>​ +
-$ sudo pico /etc/hosts  +
-</​code>​ +
- +
-It should be like this (for example): +
-<​code>​ +
-127.0.0.1 localhost +
- +
-192.168.142.21 sync.zotero.org ​api.zotero.org +
-</code> +
- +
-Make this for all the machines that will access to the server.  +
- +
- +
-===== Step 4 ===== +
-Copy the sample config files under /include/config and remove "​sample"​ from the filenames. Ex: "​config.inc.php-sample"​ becomes "​config.inc.php"​.\\  +
-You'll need to edit these two files later (cf. Step 8).  +
- +
- +
-===== Step 5 ===== +
-Change MySQL time zone. It must be UTC time.\\  +
-So, edit my.cnf file in /etc/mysql/ directory.\\  +
-Add the "​default-time-zone"​ in [mysqld], like this: +
-<​code>​ +
-[mysqld] +
-... +
-... +
-default-time-zone = '​+0:​00'​ +
-</​code>​ +
- +
-Restart MySQL: +
-<​code>​ +
-$ sudo /​etc/​init.d/​mysql restart +
-</​code>​ +
- +
-===== Step 6 ===== +
-Create zotero_cache database (using cache.sql in /misc/ directory).\\  +
-Create zotero_www_test and zotero_www databases.\\  +
-Create a users table for zotero_www_test and zotero_www (not for zotero_cache):​ +
-<​code>​ +
- CREATE TABLE  `users` ( +
-   `userID` ​            ​mediumint ​     UNSIGNED auto_increment ​                              NOT NULL, +
-   `username` ​          ​varchar(40) ​   CHARACTER SET utf8 COLLATE utf8_general_ci ​           NOT NULL, +
-   `slug` ​              ​varchar(40) ​   CHARACTER SET utf8 COLLATE utf8_general_ci ​               NULL, +
-   `password` ​          ​char(40) ​      ​CHARACTER SET utf8 COLLATE utf8_bin ​                  NOT NULL, +
-   `email` ​             varchar(100) ​  ​CHARACTER SET utf8 COLLATE utf8_general_ci ​           NOT NULL, +
-   `openid` ​            ​varchar(100) ​  ​CHARACTER SET utf8 COLLATE utf8_general_ci ​               NULL, +
-   `role` ​              ​enum('​member','​admin'​) ​ DEFAULT '​member' ​                            NOT NULL, +
-   `resetValidationKey` char(40) ​      ​CHARACTER SET utf8 COLLATE utf8_bin ​                      ​NULL,​ +
-   `resetValidationExp` int                                                                      NULL, +
-   `emailValidationKey` char(10) ​      ​CHARACTER SET utf8 COLLATE utf8_bin ​                      ​NULL,​ +
-   `emailValidated` ​    ​tinyint(1) ​    ​DEFAULT '​0' ​                                          NOT NULL, +
-   `hasImage` ​          ​tinyint(1) ​    ​DEFAULT '​0' ​                                          NOT NULL, +
-   `lastUpdated` ​       timestamp ​     ON UPDATE CURRENT_TIMESTAMP DEFAULT CURRENT_TIMESTAMP NOT NULL, +
-   `dateRegistered` ​    ​timestamp ​                                                           NOT NULL, +
-   PRIMARY KEY (`userID`),​ +
-   UNIQUE KEY  (`username`),​ +
-   UNIQUE KEY  (`slug`), +
-   UNIQUE KEY  (`email`),​ +
-   UNIQUE KEY  (`resetValidationKey`) +
- ) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin;​ +
-</​code>​ +
- +
- +
-===== Step 7 ===== +
-Give write access to the /tmp/ directory to apache. +
- +
- +
-===== Step 8 ===== +
-Edit config.inc.php,​ dbconnect.inc.php and Sync.inc.php files.\\  +
-  * **config.inc.php**:​ +
-<​code>​ +
- <? +
- class Z_CONFIG { +
- public static $API_ENABLED = true; +
- public static $SYNC_ENABLED = true; +
- public static $PROCESSORS_ENABLED = true; +
- public static $MAINTENANCE_MESSAGE = '​Server updates in progress. Please try again in a few minutes.';​ +
- +
- public static $TESTING_SITE = true; +
- public static $DEV_SITE = true; +
- +
- public static $BASE_URI = '';​ +
- +
- public static $AUTH_SALT = '';​ +
- public static $API_SUPER_USERNAME = '';​ +
- public static $API_SUPER_PASSWORD = '';​ +
- +
- public static $API_BASE_URI = '';​ +
- public static $SYNC_DOMAIN = '​sync.zotero.org';​ +
- +
- public static $S3_BUCKET = '';​ +
- public static $S3_ACCESS_KEY = '';​ +
- public static $S3_SECRET_KEY = '';​ +
- +
- public static $MEMCACHED_ENABLED = false; +
- public static $MEMCACHED_SERVERS = array( +
- '​memcached1.localdomain:​11211:​2',​ '​memcached2.localdomain:​11211:​1'​ +
- ); +
- +
- public static $MONGO_SERVERS = array( +
- '​localhost:​27017',​ '​localhost:​27017'​ +
- ); +
- public static $MONGO_DB = "​zoterotest";​ +
- public static $MONGO_SAFE_NUM = 2; +
- +
- public static $TRANSLATE_SERVERS = array( +
- "​translator1.localdomain:​1969"​ +
- ); +
- +
- public static $CITE_SERVERS = array( +
- "​citeserver1.localdomain:​8080",​ "​citeserver2.localdomain:​8080"​ +
- ); +
- +
- public static $LOG_TO_SCRIBE = false; +
- public static $LOG_ADDRESS = '';​ +
- public static $LOG_PORT = 1463; +
- public static $LOG_TIMEZONE = '​US/​Eastern';​ +
- public static $LOG_TARGET_DEFAULT = '​errors';​ +
- +
- public static $PROCESSOR_PORT_DOWNLOAD = 3455; +
- public static $PROCESSOR_PORT_UPLOAD = 3456; +
- public static $PROCESSOR_PORT_ERROR = 3457; +
- public static $PROCESSOR_PORT_INDEX = 3458; +
- +
- public static $PROCESSOR_LOG_TARGET_DOWNLOAD = '​sync-processor-download';​ +
- public static $PROCESSOR_LOG_TARGET_UPLOAD = '​sync-processor-upload';​ +
- public static $PROCESSOR_LOG_TARGET_ERROR = '​sync-processor-error';​ +
- public static $PROCESSOR_LOG_TARGET_INDEX = '​processor-index';​ +
- +
- public static $SYNC_DOWNLOAD_SMALLEST_FIRST = false; +
- public static $SYNC_UPLOAD_SMALLEST_FIRST = false; +
- +
- // Set some things manually for running via command line +
- public static $CLI_PHP_PATH = '/​usr/​bin/​php';​ +
- public static $CLI_DOCUMENT_ROOT = "/​home/​UTILISATEUR/​dataserver/";​ +
- +
- public static $SYNC_ERROR_PATH = '/​var/​log/​httpd/​sync-errors/';​ +
- public static $API_ERROR_PATH = '/​var/​log/​httpd/​api-errors/';​ +
-+
- ?> +
-</​code>​ +
- +
-  * **dbconnect.inc.php**:​ +
-<​code>​ +
- <? +
- function Zotero_dbConnectAuth($db) { +
- if ($db == '​master'​) { +
- $host = '​localhost';​ +
- $port = 3306; +
- $db = '​zoterotest_master';​ +
- $user = '​root';​ +
- $pass = '​root_password';​ +
-+
- +
- else if ($db == '​id1'​) { +
- $host = '​localhost';​ +
- $port = 3306; +
- $db = '​zoterotest_ids';​ +
- $user = '​root';​ +
- $pass = '​root_password';​ +
-+
- +
- else if ($db == '​id2'​) { +
- $host = '​localhost';​ +
- $port = 3306; +
- $db = '​zoterotest_ids';​ +
- $user = '​root';​ +
- $pass = '​root_password';​ +
-+
- +
- else if ($db == '​www1'​) { +
- $host = '​localhost';​ +
- $port = 3306; +
- $db = '​zotero_www';​ +
- $user = '​root';​ +
- $pass = '​root_password';​ +
-+
- +
- else if ($db == '​www2'​) { +
- $host = '​localhost';​ +
- $port = 3306; +
- $db = '​zotero_www';​ +
- $user = '​root';​ +
- $pass = '​root_password';​ +
-+
- +
- else if ($db == '​cache'​) { +
- $host = '​localhost';​ +
- $port = 3306; +
- $db = '​zotero_cache';​ +
- $user = '​root';​ +
- $pass = '​root_password';​ +
-+
- +
- else { +
- throw new Exception("​Invalid db '​$db'"​);​ +
-+
- return array('​host'​=>​$host,​ '​port'​=>​$port,​ '​db'​=>​$db,​ '​user'​=>​$user,​ '​pass'​=>​$pass);​ +
-+
- ?> +
-</​code>​  +
- +
-  * **Sync.inc.php** (in /model/ directory):​\\  +
- +
-Put the following lines in comment (or delete them): +
-<​code>​ +
- if (!Zotero_Libraries::​setTimestampLock($affectedLibraries,​ $timestamp)) { +
- throw new Exception("​Library timestamp already used", Z_ERROR_LIBRARY_TIMESTAMP_ALREADY_USED);​ +
-+
-</​code>​ +
- +
-These are lines 1406 to 1408.\\  +
- +
-I'm now searching why I have a problem with these lines and how to fix it...  +
- +
- +
-===== Step 9 ===== +
-Loading https://​sync.zotero.org and https://​api.zotero.org should give a page saying "​Nothing to see here"​.\\  +
-/!\ You must accept SSL certificates before (on https://​sync.zotero.org AND on https://​api.zotero.org).  +
- +
- +
-===== Step 10 ===== +
-Create or edit .my.cnf in your home directory (/​home/​UTILISATEUR) with: +
-<​code>​ +
-[client] +
-user=root +
-password=root_password +
-</​code>​ +
- +
- +
-===== Step 11 ===== +
-Change directory to /misc/ directory. +
- +
- +
-===== Step 12 ===== +
-Edit test_reset and test_setup.\\  +
-  * In test_reset AND test_setup: replace all "​127.0.0.1"​ by "​localhost"​. +
-  * In test_reset: replace +
-<​code>​ +
-# Set up sample users +
-echo "​DELETE FROM users" | $WWW zotero_www_test +
-echo "​INSERT INTO users VALUES (1, '​testuser',​ '​b7a875fc1ea228b9061041b7cec4bd3c52ab3ce3',​ '​test@zotero.org',​ NULL, '​member',​ NULL, NULL, NULL, 1, '​0000-00-00 00:​00:​00',​ '​0000-00-00 00:​00:​00',​ 0, '​testuser'​)"​ | $WWW zotero_www_test +
-echo "​INSERT INTO users VALUES (2, '​testuser2',​ '​fc707fc0b8c62cfeeafffde7273978d29d6d2374',​ '​test2@zotero.org',​ NULL, '​member',​ NULL, NULL, NULL, 1, '​0000-00-00 00:​00:​00',​ '​0000-00-00 00:​00:​00',​ 0, '​testuser2'​)"​ | $WWW zotero_www_test +
-</​code>​ +
- +
-by +
- +
-<​code>​ +
-# Set up sample user in zotero_www_test +
-echo "​DELETE FROM users" | $WWW zotero_www_test +
-echo "​INSERT INTO users VALUES (1, '​testuser',​ '​testuser',​ '​45c571a156ddcef41351a713bcddee5ba7e95460',​ '​test@zotero.org',​ NULL, '​member',​ NULL, NULL, NULL, 1, 0, '​0000-00-00 00:​00:​00',​ '​0000-00-00 00:​00:​00'​)"​ | $WWW zotero_www_test +
- +
-# Set up sample user in zotero_www +
-echo "​DELETE FROM users" | $WWW zotero_www +
-echo "​INSERT INTO users VALUES (1, '​testuser',​ '​testuser',​ '​45c571a156ddcef41351a713bcddee5ba7e95460',​ '​test@zotero.org',​ NULL, '​member',​ NULL, NULL, NULL, 1, 0, '​0000-00-00 00:​00:​00',​ '​0000-00-00 00:​00:​00'​)"​ | $WWW zotero_www +
-</​code>​ +
- +
- +
-===== Step 13 ===== +
-Launch the command: +
-<​code>​ +
-$ sudo ./​test_reset +
-</​code>​ +
- +
-Normally, there are few little errors but nothing fatal. I didn't pay attention to it.  +
- +
- +
-===== Step 14 ===== +
-Launch daemons.\\  +
-There are 4 daemons, each in its own directory under the processor directory. Fire up for shells and run each independently in the foreground! +
-<​code>​ +
-$ cd processor/​download +
-$ php daemon.php +
-$ cd processor/​upload +
-$ php daemon.php +
-$ cd processor/​error +
-$ php daemon.php +
-$ cd processor/​index +
-$ php daemon.php +
-</​code>​  +
- +
-In all cases you should see the output similar to this (the daemon name will vary): +
-<​code>​ +
-Starting sync download processor daemon +
-0 processors, 0 queued processes +
-</​code>​ +
-  +
- +
-===== Step 15 ===== +
-You can try to sync Zotero with login=testuser and password=testuser.\\  +
-Normally, it works! +
-\\ \\ +
zotero_dataserver_setup.txt · Last modified: 2017/11/12 19:53 (external edit)