This is an old revision of the document!

This procedure (in 15 steps) is inspired by : https://www.zotero.org/trac/wiki/dataserver_setup and http://www.zotero.org/support/zotero_dataserver

Before starting, here are the packages to setup on your machine (we suppose you have an apache server running and know how to make basic configuration changes to it):

  • PHP (php5)
  • PHP modules (php5-mysql, php5-svn, php5-memcache, php5-memcached)
  • Apache module: mod_rewrite
  • MySQL with root access (mysql-server, phpmyadmin)
  • MongoDB
    (http://www.php.net/manual/en/mongo.installation.php#mongo.installation.nix)
  • Memcached (same as MongoDB but don't add “extension=memcache.so” in php.ini)
  • Ability to configure /etc/hosts
  • Subversion client

Now, we can begin.

Step 1

Download Zotero dataserver sources (here: https://www.zotero.org/svn/dataserver/trunk/).
Use your subversion client to pull down the sources from Zotero's repository.
These files can be anywhere but this guide will assume they are located in /home/UTILISATEUR/dataserver ← NOTE: without the “trunk” - in other words, the “include” directory is at /home/UTILISATEUR/dataserver/include.
You can use the following command-line: 

$ sudo svn co https://www.zotero.org/svn/dataserver/trunk/

Step 2

Set up an apache VirtualHost for the dataserver.
Use this code: 

NameVirtualHost *:443
<VirtualHost *:443> 
	DocumentRoot /home/UTILISATEUR/dataserver/htdocs 
	SSLEngine on
	SSLCertificateFile /etc/apache2/server.crt
	SSLCertificateKeyFile /etc/apache2/server.key 
	<Directory "home/UTILISATEUR/dataserver/htdocs">
		Options FollowSymLinks MultiViews
		AllowOverride All
		Order allow,deny
		Allow from all
	</Directory> 
	AllowEncodedSlashes On  
</VirtualHost>

Save this code as the file “default” in /etc/apache2/sites-available directory.

NOTE: don't forget to create SSL certificate and SSL key. Like this: 

$ sudo mkdir /tmp/ssl_conf
$ cd /tmp/ssl_conf
$ sudo openssl req -config /etc/ssl/openssl.cnf -new -out server.csr
$ sudo openssl rsa -in privkey.pem -out server.key
$ sudo openssl x509 -in server.csr -out server.crt -req -signkey server.key -days 365
$ sudo openssl x509 -in server.crt -out server.der.crt -outform DER
$ sudo cp server.crt /etc/apache2/
$ sudo cp server.key /etc/apache2/
$ cd /etc/apache2/mods-enabled
$ sudo ln -s ../mods-available/ssl* .

Step 3

Add an /etc/hosts entry for sync.zotero.org and api.zotero.org to the IP address of the server.
Command to edit /etc/hosts: 

$ sudo pico /etc/hosts

It should be like this (for example): 

127.0.0.1		localhost

192.168.142.21		sync.zotero.org api.zotero.org

Make this for all the machines that will access to the server.

Step 4

Copy the sample config files under /include/config and remove “sample” from the filenames. Ex: “config.inc.php-sample” becomes “config.inc.php”.
You'll need to edit these two files later (cf. Step 8).

Step 5

Change MySQL time zone. It must be UTC time.
So, edit my.cnf file in /etc/mysql/ directory.
Add the “default-time-zone” in [mysqld], like this: 

[mysqld]
...
...
default-time-zone = '+0:00'

Restart MySQL: 

$ sudo /etc/init.d/mysql restart

Step 6

Create zotero_cache database (using cache.sql in /misc/ directory).
Create zotero_www_test and zotero_www databases.
Create a users table for zotero_www_test and zotero_www (not for zotero_cache): 

	CREATE TABLE  `users` (
	  `userID`             mediumint      UNSIGNED auto_increment                               NOT NULL,
	  `username`           varchar(40)    CHARACTER SET utf8 COLLATE utf8_general_ci            NOT NULL,
	  `slug`               varchar(40)    CHARACTER SET utf8 COLLATE utf8_general_ci                NULL,
	  `password`           char(40)       CHARACTER SET utf8 COLLATE utf8_bin                   NOT NULL,
	  `email`              varchar(100)   CHARACTER SET utf8 COLLATE utf8_general_ci            NOT NULL,
	  `openid`             varchar(100)   CHARACTER SET utf8 COLLATE utf8_general_ci                NULL,
	  `role`               enum('member','admin')  DEFAULT 'member'                             NOT NULL,
	  `resetValidationKey` char(40)       CHARACTER SET utf8 COLLATE utf8_bin                       NULL,
	  `resetValidationExp` int                                                                      NULL,
	  `emailValidationKey` char(10)       CHARACTER SET utf8 COLLATE utf8_bin                       NULL,
	  `emailValidated`     tinyint(1)     DEFAULT '0'                                           NOT NULL,
	  `hasImage`           tinyint(1)     DEFAULT '0'                                           NOT NULL,
	  `lastUpdated`        timestamp      ON UPDATE CURRENT_TIMESTAMP DEFAULT CURRENT_TIMESTAMP NOT NULL,
	  `dateRegistered`     timestamp                                                            NOT NULL,
	  PRIMARY KEY (`userID`),
	  UNIQUE KEY  (`username`),
	  UNIQUE KEY  (`slug`),
	  UNIQUE KEY  (`email`),
	  UNIQUE KEY  (`resetValidationKey`)
	) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin;

Step 7

Give write access to the /tmp/ directory to apache.

Step 8

Edit config.inc.php, dbconnect.inc.php and Sync.inc.php files.

  • config.inc.php:
	<?
	class Z_CONFIG {
		public static $API_ENABLED = true;
		public static $SYNC_ENABLED = true;
		public static $PROCESSORS_ENABLED = true;
		public static $MAINTENANCE_MESSAGE = 'Server updates in progress. Please try again in a few minutes.';

		public static $TESTING_SITE = true;
		public static $DEV_SITE = true;

		public static $BASE_URI = '';

		public static $AUTH_SALT = '';
		public static $API_SUPER_USERNAME = '';
		public static $API_SUPER_PASSWORD = '';

		public static $API_BASE_URI = '';
		public static $SYNC_DOMAIN = 'sync.zotero.org';

		public static $S3_BUCKET = '';
		public static $S3_ACCESS_KEY = '';
		public static $S3_SECRET_KEY = '';

		public static $MEMCACHED_ENABLED = false;
		public static $MEMCACHED_SERVERS = array(
		'memcached1.localdomain:11211:2', 'memcached2.localdomain:11211:1'
		);

		public static $MONGO_SERVERS = array(
			'localhost:27017', 'localhost:27017'
		);
		public static $MONGO_DB = "zoterotest";
		public static $MONGO_SAFE_NUM = 2;

		public static $TRANSLATE_SERVERS = array(
			"translator1.localdomain:1969"
		);

		public static $CITE_SERVERS = array(
		"citeserver1.localdomain:8080", "citeserver2.localdomain:8080"
		);

		public static $LOG_TO_SCRIBE = false;
		public static $LOG_ADDRESS = '';
		public static $LOG_PORT = 1463;
		public static $LOG_TIMEZONE = 'US/Eastern';
		public static $LOG_TARGET_DEFAULT = 'errors';

		public static $PROCESSOR_PORT_DOWNLOAD = 3455;
		public static $PROCESSOR_PORT_UPLOAD = 3456;
		public static $PROCESSOR_PORT_ERROR = 3457;
		public static $PROCESSOR_PORT_INDEX = 3458;

		public static $PROCESSOR_LOG_TARGET_DOWNLOAD = 'sync-processor-download';
		public static $PROCESSOR_LOG_TARGET_UPLOAD = 'sync-processor-upload';
		public static $PROCESSOR_LOG_TARGET_ERROR = 'sync-processor-error';
		public static $PROCESSOR_LOG_TARGET_INDEX = 'processor-index';

		public static $SYNC_DOWNLOAD_SMALLEST_FIRST = false;
		public static $SYNC_UPLOAD_SMALLEST_FIRST = false;

		// Set some things manually for running via command line
		public static $CLI_PHP_PATH = '/usr/bin/php';
		public static $CLI_DOCUMENT_ROOT = "/home/UTILISATEUR/dataserver/";

		public static $SYNC_ERROR_PATH = '/var/log/httpd/sync-errors/';
		public static $API_ERROR_PATH = '/var/log/httpd/api-errors/';
	}
	?>
  • dbconnect.inc.php:
	<?
	function Zotero_dbConnectAuth($db) {
		if ($db == 'master') {
			$host = 'localhost';
			$port = 3306;
			$db = 'zoterotest_master';
			$user = 'root';
			$pass = 'root_password';
		}

		else if ($db == 'id1') {
			$host = 'localhost';
			$port = 3306;
			$db = 'zoterotest_ids';
			$user = 'root';
			$pass = 'root_password';
		}

		else if ($db == 'id2') {
			$host = 'localhost';
			$port = 3306;
			$db = 'zoterotest_ids';
			$user = 'root';
			$pass = 'root_password';
		}

		else if ($db == 'www1') {
			$host = 'localhost';
			$port = 3306;
			$db = 'zotero_www';
			$user = 'root';
			$pass = 'root_password';
		}

		else if ($db == 'www2') {
			$host = 'localhost';
			$port = 3306;
			$db = 'zotero_www';
			$user = 'root';
			$pass = 'root_password';
		}

		else if ($db == 'cache') {
			$host = 'localhost';
			$port = 3306;
			$db = 'zotero_cache';
			$user = 'root';
			$pass = 'root_password';
		}

		else {
			throw new Exception("Invalid db '$db'");
		}
		return array('host'=>$host, 'port'=>$port, 'db'=>$db, 'user'=>$user, 'pass'=>$pass);
	}
	?>
  • Sync.inc.php (in /model/ directory):

Put the following lines in comment (or delete them): 

	if (!Zotero_Libraries::setTimestampLock($affectedLibraries, $timestamp)) {
		throw new Exception("Library timestamp already used", Z_ERROR_LIBRARY_TIMESTAMP_ALREADY_USED);
	}

These are lines 1406 to 1408.

I'm now searching why I have a problem with these lines and how to fix it…

Step 9

Loading https://sync.zotero.org and https://api.zotero.org should give a page saying “Nothing to see here”.
/!\ You must accept SSL certificates before (on https://sync.zotero.org AND on https://api.zotero.org).

Step 10

Create or edit .my.cnf in your home directory (/home/UTILISATEUR) with: 

[client]
user=root
password=root_password

Step 11

Change directory to /misc/ directory.

Step 12

Edit test_reset and test_setup.

  • In test_reset AND test_setup: replace all “127.0.0.1” by “localhost”.
  • In test_reset: replace
# Set up sample users
echo "DELETE FROM users" | $WWW zotero_www_test
echo "INSERT INTO users VALUES (1, 'testuser', 'b7a875fc1ea228b9061041b7cec4bd3c52ab3ce3', 'test@zotero.org', NULL, 'member', NULL, NULL, NULL, 1, '0000-00-00 00:00:00', '0000-00-00 00:00:00', 0, 'testuser')" | $WWW zotero_www_test
echo "INSERT INTO users VALUES (2, 'testuser2', 'fc707fc0b8c62cfeeafffde7273978d29d6d2374', 'test2@zotero.org', NULL, 'member', NULL, NULL, NULL, 1, '0000-00-00 00:00:00', '0000-00-00 00:00:00', 0, 'testuser2')" | $WWW zotero_www_test

by 

# Set up sample user in zotero_www_test
echo "DELETE FROM users" | $WWW zotero_www_test
echo "INSERT INTO users VALUES (1, 'testuser', 'testuser', '45c571a156ddcef41351a713bcddee5ba7e95460', 'test@zotero.org', NULL, 'member', NULL, NULL, NULL, 1, 0, '0000-00-00 00:00:00', '0000-00-00 00:00:00')" | $WWW zotero_www_test

# Set up sample user in zotero_www
echo "DELETE FROM users" | $WWW zotero_www
echo "INSERT INTO users VALUES (1, 'testuser', 'testuser', '45c571a156ddcef41351a713bcddee5ba7e95460', 'test@zotero.org', NULL, 'member', NULL, NULL, NULL, 1, 0, '0000-00-00 00:00:00', '0000-00-00 00:00:00')" | $WWW zotero_www

Step 13

Launch the command: 

$ sudo ./test_reset

Normally, there are few little errors but nothing fatal. I didn't pay attention to it.

Step 14

Launch daemons.
There are 4 daemons, each in its own directory under the processor directory. Fire up for shells and run each independently in the foreground! 

$ cd processor/download
$ php daemon.php
$ cd processor/upload
$ php daemon.php
$ cd processor/error
$ php daemon.php
$ cd processor/index
$ php daemon.php

In all cases you should see the output similar to this (the daemon name will vary): 

Starting sync download processor daemon
0 processors, 0 queued processes

Step 15

You can try to sync Zotero with login=testuser and password=testuser.
Normally, it works!

Table of Contents

zotero_dataserver_setup.1310645161.txt.gz · Last modified: 2011/07/14 08:06 by simonth