Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
dev:dataserver_setup [2011/07/15 04:00] – simonthdev:dataserver_setup [2014/09/18 03:45] (current) – Delete uselessly ancient unofficial instructions, which shouldn't be hosted here anyway dstillman
Line 1: Line 1:
-== This installation was tested on Ubuntu 11.04 with SVN revision 9803. == 
- 
-\\  
-__Before starting, here are the packages to setup on your machine (we suppose you have an apache server running and know how to make basic configuration changes to it):__ 
- 
-  * PHP (php5) 
-  * PHP modules (php5-mysql, php5-svn, php5-memcache, php5-memcached) 
-  * Apache module: mod_rewrite 
-  * MySQL with root access (mysql-server, phpmyadmin) 
-  * MongoDB (http://www.php.net/manual/en/mongo.installation.php#mongo.installation.nix) 
-  * Memcached (same as MongoDB but don't add "extension=memcache.so" in php.ini) 
-  * Ability to configure /etc/hosts 
-  * Subversion client 
- 
-Now, we can begin. 
- 
-===== Step 1 ===== 
-Download Zotero dataserver sources (here: https://www.zotero.org/svn/dataserver/trunk/).\\  
-Use your subversion client to pull down the sources from Zotero's repository.\\  
-These files can be anywhere but this guide will assume they are located in /home/UTILISATEUR/dataserver <- NOTE: without the "trunk" - in other words, the "include" directory is at /home/UTILISATEUR/dataserver/include.\\  
-You can use the following command-line: 
-<code> 
-$ sudo svn co https://www.zotero.org/svn/dataserver/trunk/ 
-</code> 
- 
- 
-===== Step 2 ===== 
-Set up an apache VirtualHost for the dataserver.\\  
-Use this code: 
-<code> 
-NameVirtualHost *:443 
-<VirtualHost *:443>  
- DocumentRoot /home/UTILISATEUR/dataserver/htdocs  
- SSLEngine on 
- SSLCertificateFile /etc/apache2/server.crt 
- SSLCertificateKeyFile /etc/apache2/server.key  
- <Directory "home/UTILISATEUR/dataserver/htdocs"> 
- Options FollowSymLinks MultiViews 
- AllowOverride All 
- Order allow,deny 
- Allow from all 
- </Directory>  
- AllowEncodedSlashes On   
-</VirtualHost> 
-</code> 
- 
-Save this code as the file "default" in /etc/apache2/sites-available directory.\\ \\  
- 
-NOTE: don't forget to create SSL certificate and SSL key. Like this: 
-<code> 
-$ sudo mkdir /tmp/ssl_conf 
-$ cd /tmp/ssl_conf 
-$ sudo openssl req -config /etc/ssl/openssl.cnf -new -out server.csr 
-$ sudo openssl rsa -in privkey.pem -out server.key 
-$ sudo openssl x509 -in server.csr -out server.crt -req -signkey server.key -days 365 
-$ sudo openssl x509 -in server.crt -out server.der.crt -outform DER 
-$ sudo cp server.crt /etc/apache2/ 
-$ sudo cp server.key /etc/apache2/ 
-$ cd /etc/apache2/mods-enabled 
-$ sudo ln -s ../mods-available/ssl* . 
-</code> 
- 
- 
-===== Step 3 ===== 
-Add an /etc/hosts entry for sync.zotero.org and api.zotero.org to the IP address of the server.\\  
-Command to edit /etc/hosts: 
-<code> 
-$ sudo pico /etc/hosts  
-</code> 
- 
-It should be like this (for example): 
-<code> 
-127.0.0.1 localhost 
- 
-192.168.142.21 sync.zotero.org api.zotero.org 
-</code> 
- 
-Make this for all the machines that will access to the server.  
- 
- 
-===== Step 4 ===== 
-Copy the sample config files under /include/config and remove "sample" from the filenames. Ex: "config.inc.php-sample" becomes "config.inc.php".\\  
-You'll need to edit these two files later (cf. Step 8).  
- 
- 
-===== Step 5 ===== 
-Change MySQL time zone. It must be UTC time.\\  
-So, edit my.cnf file in /etc/mysql/ directory.\\  
-Add the "default-time-zone" in [mysqld], like this: 
-<code> 
-[mysqld] 
-... 
-... 
-default-time-zone = '+0:00' 
-</code> 
- 
-Restart MySQL: 
-<code> 
-$ sudo /etc/init.d/mysql restart 
-</code> 
- 
-===== Step 6 ===== 
-Create zotero_cache database (using cache.sql in /misc/ directory).\\  
-Create zotero_www_test and zotero_www databases.\\  
-Create a users table for zotero_www_test and zotero_www (not for zotero_cache): 
-<code> 
-CREATE TABLE  `users` ( 
-  `userID`             mediumint      UNSIGNED auto_increment                               NOT NULL, 
-  `username`           varchar(40)    CHARACTER SET utf8 COLLATE utf8_general_ci            NOT NULL, 
-  `slug`               varchar(40)    CHARACTER SET utf8 COLLATE utf8_general_ci                NULL, 
-  `password`           char(40)       CHARACTER SET utf8 COLLATE utf8_bin                   NOT NULL, 
-  `email`              varchar(100)   CHARACTER SET utf8 COLLATE utf8_general_ci            NOT NULL, 
-  `openid`             varchar(100)   CHARACTER SET utf8 COLLATE utf8_general_ci                NULL, 
-  `role`               enum('member','admin')  DEFAULT 'member'                             NOT NULL, 
-  `resetValidationKey` char(40)       CHARACTER SET utf8 COLLATE utf8_bin                       NULL, 
-  `resetValidationExp` int                                                                      NULL, 
-  `emailValidationKey` char(10)       CHARACTER SET utf8 COLLATE utf8_bin                       NULL, 
-  `emailValidated`     tinyint(1)     DEFAULT '0'                                           NOT NULL, 
-  `hasImage`           tinyint(1)     DEFAULT '0'                                           NOT NULL, 
-  `lastUpdated`        timestamp      ON UPDATE CURRENT_TIMESTAMP DEFAULT CURRENT_TIMESTAMP NOT NULL, 
-  `dateRegistered`     timestamp                                                            NOT NULL, 
-  PRIMARY KEY (`userID`), 
-  UNIQUE KEY  (`username`), 
-  UNIQUE KEY  (`slug`), 
-  UNIQUE KEY  (`email`), 
-  UNIQUE KEY  (`resetValidationKey`) 
-) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin; 
-</code> 
- 
- 
-===== Step 7 ===== 
-Give write access to the /tmp/ directory to apache. 
- 
- 
-===== Step 8 ===== 
-Edit config.inc.php, dbconnect.inc.php and Sync.inc.php files.\\  
-  * **config.inc.php**: 
-<code> 
-<? 
-class Z_CONFIG { 
- public static $API_ENABLED = true; 
- public static $SYNC_ENABLED = true; 
- public static $PROCESSORS_ENABLED = true; 
- public static $MAINTENANCE_MESSAGE = 'Server updates in progress. Please try again in a few minutes.'; 
- 
- public static $TESTING_SITE = true; 
- public static $DEV_SITE = true; 
- 
- public static $BASE_URI = ''; 
- 
- public static $AUTH_SALT = ''; 
- public static $API_SUPER_USERNAME = ''; 
- public static $API_SUPER_PASSWORD = ''; 
- 
- public static $API_BASE_URI = ''; 
- public static $SYNC_DOMAIN = 'sync.zotero.org'; 
- 
- public static $S3_BUCKET = ''; 
- public static $S3_ACCESS_KEY = ''; 
- public static $S3_SECRET_KEY = ''; 
- 
- public static $MEMCACHED_ENABLED = false; 
- public static $MEMCACHED_SERVERS = array( 
- 'memcached1.localdomain:11211:2', 'memcached2.localdomain:11211:1' 
- ); 
- 
- public static $MONGO_SERVERS = array( 
- 'localhost:27017', 'localhost:27017' 
- ); 
- public static $MONGO_DB = "zoterotest"; 
- public static $MONGO_SAFE_NUM = 2; 
- 
- public static $TRANSLATE_SERVERS = array( 
- "translator1.localdomain:1969" 
- ); 
- 
- public static $CITE_SERVERS = array( 
- "citeserver1.localdomain:8080", "citeserver2.localdomain:8080" 
- ); 
- 
- public static $LOG_TO_SCRIBE = false; 
- public static $LOG_ADDRESS = ''; 
- public static $LOG_PORT = 1463; 
- public static $LOG_TIMEZONE = 'US/Eastern'; 
- public static $LOG_TARGET_DEFAULT = 'errors'; 
- 
- public static $PROCESSOR_PORT_DOWNLOAD = 3455; 
- public static $PROCESSOR_PORT_UPLOAD = 3456; 
- public static $PROCESSOR_PORT_ERROR = 3457; 
- public static $PROCESSOR_PORT_INDEX = 3458; 
- 
- public static $PROCESSOR_LOG_TARGET_DOWNLOAD = 'sync-processor-download'; 
- public static $PROCESSOR_LOG_TARGET_UPLOAD = 'sync-processor-upload'; 
- public static $PROCESSOR_LOG_TARGET_ERROR = 'sync-processor-error'; 
- public static $PROCESSOR_LOG_TARGET_INDEX = 'processor-index'; 
- 
- public static $SYNC_DOWNLOAD_SMALLEST_FIRST = false; 
- public static $SYNC_UPLOAD_SMALLEST_FIRST = false; 
- 
- // Set some things manually for running via command line 
- public static $CLI_PHP_PATH = '/usr/bin/php'; 
- public static $CLI_DOCUMENT_ROOT = "/home/UTILISATEUR/dataserver/"; 
- 
- public static $SYNC_ERROR_PATH = '/var/log/httpd/sync-errors/'; 
- public static $API_ERROR_PATH = '/var/log/httpd/api-errors/'; 
-} 
-?> 
-</code> 
- 
-  * **dbconnect.inc.php**: 
-<code> 
-<? 
-function Zotero_dbConnectAuth($db) { 
- if ($db == 'master') { 
- $host = 'localhost'; 
- $port = 3306; 
- $db = 'zoterotest_master'; 
- $user = 'root'; 
- $pass = 'root_password'; 
- } 
- 
- else if ($db == 'id1') { 
- $host = 'localhost'; 
- $port = 3306; 
- $db = 'zoterotest_ids'; 
- $user = 'root'; 
- $pass = 'root_password'; 
- } 
- 
- else if ($db == 'id2') { 
- $host = 'localhost'; 
- $port = 3306; 
- $db = 'zoterotest_ids'; 
- $user = 'root'; 
- $pass = 'root_password'; 
- } 
- 
- else if ($db == 'www1') { 
- $host = 'localhost'; 
- $port = 3306; 
- $db = 'zotero_www'; 
- $user = 'root'; 
- $pass = 'root_password'; 
- } 
- 
- else if ($db == 'www2') { 
- $host = 'localhost'; 
- $port = 3306; 
- $db = 'zotero_www'; 
- $user = 'root'; 
- $pass = 'root_password'; 
- } 
- 
- else if ($db == 'cache') { 
- $host = 'localhost'; 
- $port = 3306; 
- $db = 'zotero_cache'; 
- $user = 'root'; 
- $pass = 'root_password'; 
- } 
- 
- else { 
- throw new Exception("Invalid db '$db'"); 
- } 
- return array('host'=>$host, 'port'=>$port, 'db'=>$db, 'user'=>$user, 'pass'=>$pass); 
-} 
-?> 
-</code>  
- 
-  * **Sync.inc.php** (in /model/ directory):\\  
- 
-Put the following lines in comment (or delete them): 
-<code> 
-if (!Zotero_Libraries::setTimestampLock($affectedLibraries, $timestamp)) { 
- throw new Exception("Library timestamp already used", Z_ERROR_LIBRARY_TIMESTAMP_ALREADY_USED); 
-} 
-</code> 
- 
-These are lines 1406 to 1408.\\  
- 
-A solution to this has been suggested in the mailing list (see http://groups.google.com/group/zotero-dev/browse_thread/thread/9ab0112a8e1be19c). 
- 
- 
-===== Step 9 ===== 
-Loading https://sync.zotero.org and https://api.zotero.org should give a page saying "Nothing to see here".\\  
-/!\ You must accept SSL certificates before (on https://sync.zotero.org AND on https://api.zotero.org).  
- 
- 
-===== Step 10 ===== 
-Create or edit .my.cnf in your home directory (/home/UTILISATEUR) with: 
-<code> 
-[client] 
-user=root 
-password=root_password 
-</code> 
- 
- 
-===== Step 11 ===== 
-Change directory to /misc/ directory. 
- 
- 
-===== Step 12 ===== 
-Edit test_reset and test_setup.\\  
-  * In test_reset AND test_setup: replace all "127.0.0.1" by "localhost". 
-  * In test_reset: replace 
-<code> 
-# Set up sample users 
-echo "DELETE FROM users" | $WWW zotero_www_test 
-echo "INSERT INTO users VALUES (1, 'testuser', 'b7a875fc1ea228b9061041b7cec4bd3c52ab3ce3', 'test@zotero.org', NULL, 'member', NULL, NULL, NULL, 1, '0000-00-00 00:00:00', '0000-00-00 00:00:00', 0, 'testuser')" | $WWW zotero_www_test 
-echo "INSERT INTO users VALUES (2, 'testuser2', 'fc707fc0b8c62cfeeafffde7273978d29d6d2374', 'test2@zotero.org', NULL, 'member', NULL, NULL, NULL, 1, '0000-00-00 00:00:00', '0000-00-00 00:00:00', 0, 'testuser2')" | $WWW zotero_www_test 
-</code> 
- 
-by 
- 
-<code> 
-# Set up sample user in zotero_www_test 
-echo "DELETE FROM users" | $WWW zotero_www_test 
-echo "INSERT INTO users VALUES (1, 'testuser', 'testuser', '45c571a156ddcef41351a713bcddee5ba7e95460', 'test@zotero.org', NULL, 'member', NULL, NULL, NULL, 1, 0, '0000-00-00 00:00:00', '0000-00-00 00:00:00')" | $WWW zotero_www_test 
- 
-# Set up sample user in zotero_www 
-echo "DELETE FROM users" | $WWW zotero_www 
-echo "INSERT INTO users VALUES (1, 'testuser', 'testuser', '45c571a156ddcef41351a713bcddee5ba7e95460', 'test@zotero.org', NULL, 'member', NULL, NULL, NULL, 1, 0, '0000-00-00 00:00:00', '0000-00-00 00:00:00')" | $WWW zotero_www 
-</code> 
- 
- 
-===== Step 13 ===== 
-Launch the command: 
-<code> 
-$ sudo ./test_reset 
-</code> 
- 
-Normally, there are few little errors but nothing fatal. I didn't pay attention to it.  
- 
- 
-===== Step 14 ===== 
-Launch daemons.\\  
-There are 4 daemons, each in its own directory under the processor directory. Fire up for shells and run each independently in the foreground! 
-<code> 
-$ cd processor/download 
-$ php daemon.php 
-$ cd processor/upload 
-$ php daemon.php 
-$ cd processor/error 
-$ php daemon.php 
-$ cd processor/index 
-$ php daemon.php 
-</code>  
- 
-In all cases you should see the output similar to this (the daemon name will vary): 
-<code> 
-Starting sync download processor daemon 
-0 processors, 0 queued processes 
-</code> 
-  
- 
-===== Step 15 ===== 
-You can try to sync Zotero with login=testuser and password=testuser.\\  
-Normally, it works! 
-\\ \\  
- 
-===== Miscellaneous ===== 
- 
-==== Add new user ==== 
- 
-You must add new entries in zotero_www (or zotero_www_test if in test mode). 
- 
-  * Necessary fields should be userID, username, slug, password. 
-  * Password entries are generated by taking a sha1 hash of the user’s password. 
-  * Slugs are generated by taking the username and trimming whitespace, lowercasing, preg_replace("/[^a-z0-9 ._-]/", "", $input), and replacing spaces with underscores. 
- 
-==== Create group ==== 
- 
-As shown in ‘test_reset’ file: 
- 
-TODO explain values 
- 
-<code> 
-INSERT INTO libraries VALUES (3, 'group', '0000-00-00 00:00:00', 0, 2); 
-INSERT INTO groups VALUES (1, 3, 'Test Group', 'test_group', 'Private', 1, 'admins', 'all', 'members', '', '', 0, '0000-00-00 00:00:00', '0000-00-00 00:00:00'); 
-INSERT INTO groupUsers VALUES (1, 1, 'owner', '0000-00-00 00:00:00', '0000-00-00 00:00:00'); 
-INSERT INTO shardLibraries VALUES (3, 'group'); 
-</code> 
- 
-==== Custom Zotero ==== 
- 
-TODO explain how edit zotero.jar 
- 
-If you want to use your own domain you need to edit the plugin. 
-Edit  zotero.js and change SYNC_URL and API_URL with your domain. 
- 
-<code> 
-const ZOTERO_CONFIG = { 
-    GUID: 'zotero@chnm.gmu.edu', 
-    DB_REBUILD: false, // erase DB and recreate from schema 
-    REPOSITORY_URL: 'https://api.zotero.org/repo', 
-    REPOSITORY_CHECK_INTERVAL: 86400, // 24 hours 
-    REPOSITORY_RETRY_INTERVAL: 3600, // 1 hour 
-    BASE_URI: 'http://zotero.org/', 
-    WWW_BASE_URL: 'http://www.zotero.org/', 
-    SYNC_URL: 'http://sync.mydomain.org/', 
-    API_URL: 'http://api.mydomain.org/' 
-}; 
-</code> 
- 
-===== Untested / Not working / Todo ===== 
- 
-==== Untested ==== 
-  * Webdav to use files attachment. 
-  * API: explain how to use it or configure it. 
- 
-==== Not working ==== 
-  * Change user privileges in groups only work if the user performs a restore from the server 
-  * Sync attachment files in My Library using Zotero server (i suppose you need Amazon S3) 
-  * Sync attachment files in group using Zotero server (i suppose you need Amazon S3) 
- 
-==== Todo ==== 
-  * Create SQL script to install database in one step 
-  * Create interface to add/manage users and groups 
-  * Test with webdav to use files attachment 
  
dev/dataserver_setup.1310716819.txt.gz · Last modified: 2011/07/15 04:00 by simonth