Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
zotero_dataserver [2011/04/25 07:05] – [Config.inc.php] lemaxxzotero_dataserver [2019/04/19 14:35] (current) – bwiernik
Line 1: Line 1:
-====== Disclaimer ====== +See http://www.zotero.org/support/dev/dataserver_setup
- +
-This guide was written based in part on information found on  zitation.org (unreachable when I was writing this tutorial) as well as the indications found on zotero.org. Some steps may be unnecessary or wrong but keep in mind that this is a first version. Any help or comment is welcome. +
- +
-Some items may seem obvious but I detail some steps not to forget my next installation. +
-This installation was done on Ubuntu 10.10 desktop running in a virtual machine. This allows me to use an unmodified version of Zotero 2.1 +
- +
-====== Dependencies: ====== +
- +
-===== Required ===== +
- +
-  * PHP +
-  * PHP modules (?) +
-  * Apache +
-  * Apache modules/features: some sort of for PHP (?), virtual hosts support, mod_rewrite, auth (?) +
-  * MySQL with root access +
-  * MongoDB +
-  * ability to configure /etc/hosts or DNS +
-  * subversion client +
- +
-===== Optional ===== +
- +
-  * Solr +
-  * Memcached +
-  * Amazon S3 +
-  * Webdav +
- +
-Download Zotero dataserver sources +
-Use your subversion client to pull down the sources from Zotero’s repository. If using the command-line:
svn co https://www.zotero.org/svn/dataserver
 You’ll see the following layout (from here on references to zotero source paths will assume the root of dataserver/trunk) +
- +
-<code> +
-dataserver/ +
-          trunk/ +
-               controllers +
-               htdocs +
-               include +
-               misc +
-               model +
-               processor +
-               tests +
-               tmp</code> +
- +
-These files can be anywhere but this guide will assume they are located in /home/UTILISATEUR/www-dev/dataserv ←- NOTE without the ‘trunk’ – in other words, the ‘include’ directory is at /home/UTILISATEUR/www-dev/dataserv/include. +
- +
-====== Configure Apache and Host ====== +
- +
-===== Host ===== +
- +
-We need to define two hostnames, the “sync” and “api” hosts. For testing on one machine this can be done via an /etc/hosts entry +
-In this exemple i use zotero.org domain because i use a virtual machine and i don’t use a custom Zotero plugin. +
-If you want to use your own domain you need to edit the plugin (see below) +
-<code> +
-127.0.0.1 sync.zotero.org api.zotero.org</code> +
- +
-After that when you ping sync.zotero.org you will see it is resolving to 127.0.0.1 +
- +
-===== Apache ===== +
- +
-From web only HTDOCS folder must be accessible. So you can use a virtual host +
-Your sync server need a ssl connection you have to create two config files in /etc/apache2/sites-available +
- +
-<code> +
-<VirtualHost  sync.zotero.org:80> +
-  +
- ServerName sync.zotero.org +
- ServerAlias api.zotero.org  +
-  +
- DocumentRoot /home/UTILISATEUR/www-dev/dataserv/htdocs +
-  +
- <Directory "/home/UTILISATEUR/www-dev/dataserv/htdocs"> +
- Options FollowSymLinks MultiViews +
- AllowOverride All +
- Order allow,deny +
- Allow from all +
- </Directory> +
-  +
- AllowEncodedSlashes On  +
-  +
-</VirtualHost> +
-</code> +
- +
-<code> +
-<IfModule mod_ssl.c> +
-<VirtualHost sync.zotero.org:443> +
-  +
- ServerName sync.zotero.org +
- ServerAlias api.zotero.org  +
-  +
- DocumentRoot /home/UTILISATEUR/www-dev/dataserv/htdocs +
-  +
- <Directory "/home/UTILISATEUR/www-dev/dataserv/htdocs"> +
- Options FollowSymLinks MultiViews +
- AllowOverride All +
- Order allow,deny +
- Allow from all +
- </Directory> +
-  +
- AllowEncodedSlashes On +
-  +
- #   Enable/Disable SSL for this virtual host. +
- SSLEngine on +
-  +
- SSLCertificateFile    /etc/ssl/certs/ssl-cert-snakeoil.pem +
- SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key +
-  +
- <FilesMatch "\.(cgi|shtml|phtml|php)$"> +
- SSLOptions +StdEnvVars +
- </FilesMatch> +
- <Directory /usr/lib/cgi-bin> +
- SSLOptions +StdEnvVars +
- </Directory> +
-  +
- BrowserMatch "MSIE [2-6]" \ +
- nokeepalive ssl-unclean-shutdown \ +
- downgrade-1.0 force-response-1.0 +
- # MSIE 7 and newer should be able to use keepalive +
- BrowserMatch "MSIE [17-9]" ssl-unclean-shutdown +
-  +
-</VirtualHost> +
-</IfModule> +
- +
-</code> +
- +
-====== Create and configure database ====== +
- +
-===== Mongo ===== +
- +
- +
-Install MongoDB and Mongo php driver +
- +
-use /misc/mongo_schema.js to create content +
- +
-===== Mysql ===== +
- +
- +
-To understand how database must be set i used the script « test_reset » locate in « Misc » folder (you can use this script to setup your database). I use the same database name. +
- +
-In this folder you will find every *.sql files need for this installation. +
- +
-You have to create 7 Mysql database. Some of them are used only when you run Zotero server in test mode, for other I do not know yet their use. +
- +
-  * zotero_cache > use cache.sql +
-  * zotero_www (this name is hardcoded) > see below +
-  * zotero_www_test (this name is hardcoded) > see below +
-  * zoterotest_ids > use ids.sql +
-  * zoterotest_master > use master.sql and after coredata.sql +
-  * zoterotest1 > use shard.sql and after trigger.sql +
-  * zoterotest2 > use shard.sql and after trigger.sql +
- +
-There is no .sql files to set up zotero_www and zotero_www_test so use as describe on zotero.org/trac/wiki/dataserver_setup +
- +
-<code mysql> +
-CREATE TABLE  `users` ( +
-  `userID`             mediumint      UNSIGNED auto_increment                               NOT NULL, +
-  `username`           varchar(40)    CHARACTER SET utf8 COLLATE utf8_general_ci            NOT NULL, +
-  `slug`               varchar(40)    CHARACTER SET utf8 COLLATE utf8_general_ci                NULL, +
-  `password`           char(40)       CHARACTER SET utf8 COLLATE utf8_bin                   NOT NULL, +
-  `email`              varchar(100)   CHARACTER SET utf8 COLLATE utf8_general_ci            NOT NULL, +
-  `openid`             varchar(100)   CHARACTER SET utf8 COLLATE utf8_general_ci                NULL, +
-  `role`               enum('member', +
-                            'admin')  DEFAULT 'member'                                      NOT NULL, +
-  `resetValidationKey` char(40)       CHARACTER SET utf8 COLLATE utf8_bin                       NULL, +
-  `resetValidationExp` int                                                                      NULL, +
-  `emailValidationKey` char(10)       CHARACTER SET utf8 COLLATE utf8_bin                       NULL, +
-  `emailValidated`     tinyint(1)     DEFAULT '0'                                           NOT NULL, +
-  `hasImage`           tinyint(1)     DEFAULT '0'                                           NOT NULL, +
-  `lastUpdated`        timestamp      ON UPDATE CURRENT_TIMESTAMP DEFAULT CURRENT_TIMESTAMP NOT NULL, +
-  `dateRegistered`     timestamp                                                            NOT NULL, +
-  PRIMARY KEY (`userID`), +
-  UNIQUE KEY  (`username`), +
-  UNIQUE KEY  (`slug`), +
-  UNIQUE KEY  (`email`), +
-  UNIQUE KEY  (`resetValidationKey`) +
-) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin; +
- +
-</code> +
- +
-You need to add specific user in Mysql +
-If you want to personalize those user you have to change value in shard info (see below) +
-<code mysql> +
- +
-CREATE USER zoterotest0@localhost IDENTIFIED BY 'pass0'; +
-CREATE USER zoterotest1@localhost IDENTIFIED BY 'pass1'; +
-CREATE USER zoterotest2@localhost IDENTIFIED BY 'pass2'; +
-CREATE USER zoterotest_ids@localhost IDENTIFIED BY 'pass1'; +
-CREATE USER zoterotest_ids@localhost IDENTIFIED BY 'pass2'; +
-CREATE USER zoterotest_www@localhost IDENTIFIED BY 'pass'; +
- +
-</code> +
- +
-note: give acces on sync.zotero.org too +
- +
-Give Grant access +
- +
-<code mysql> +
- +
-GRANT SELECT ON zoterotest_master.* TO zoterotest1@localhost; +
-GRANT SELECT ON zoterotest_master.* TO zoterotest2@localhost; +
-GRANT SELECT, INSERT, UPDATE, DELETE, CREATE TEMPORARY TABLES ON zoterotest1.* TO zoterotest1@localhost; +
-GRANT SELECT, INSERT, UPDATE, DELETE, CREATE TEMPORARY TABLES ON zoterotest2.* TO zoterotest2@localhost; +
-GRANT SELECT,INSERT,DELETE ON zoterotest_ids.* TO zoterotest_ids@localhost; +
-GRANT SELECT,INSERT,DELETE ON zoterotest_ids.* TO zoterotest_ids@localhost; +
-GRANT SELECT ON zotero_www_test.* TO zoterotest_www@localhost; +
-</code> +
- +
-Set up shard info +
- +
-<code mysql> +
-INSERT INTO shardHosts VALUES (1, '127.0.0.1', 3306, 'up'); +
-INSERT INTO shardHosts VALUES (2, 'sync.zotero.org', 3306, 'up'); +
-INSERT INTO shards VALUES (1, 1, 'zoterotest1', 'pass1', 'zoterotest1', 'up'); +
-INSERT INTO shards VALUES (2, 1, 'zoterotest2', 'pass2', 'zoterotest2', 'up'); +
-</code> +
-Initial users and groups for tests +
- +
-<code mysql> +
-INSERT INTO libraries VALUES (1, 'user', '0000-00-00 00:00:00', 0, 1); +
-INSERT INTO libraries VALUES (2, 'user', '0000-00-00 00:00:00', 0, 1); +
-INSERT INTO libraries VALUES (3, 'group', '0000-00-00 00:00:00', 0, 2); +
-INSERT INTO users VALUES (1, 1, 'testuser', '0000-00-00 00:00:00', '0000-00-00 00:00:00'); +
-INSERT INTO users VALUES (2, 2, 'testuser2', '0000-00-00 00:00:00', '0000-00-00 00:00:00'); +
-INSERT INTO groups VALUES (1, 3, 'Test Group', 'test_group', 'Private', 1, 'admins', 'all', 'members', '', '', 0, '0000-00-00 00:00:00', '0000-00-00 00:00:00'); +
-INSERT INTO groupUsers VALUES (1, 1, 'owner', '0000-00-00 00:00:00', '0000-00-00 00:00:00'); +
-  +
-INSERT INTO users VALUES (1, 'testuser', '5d9c68c6c50ed3d02a2fcf54f63993b6', 'test@zotero.org', NULL, 'member', NULL, NULL, NULL, 1, '0000-00-00 00:00:00', '0000-00-00 00:00:00', 0, 'testuser'); +
-INSERT INTO users VALUES (2, 'testuser2', '5d9c68c6c50ed3d02a2fcf54f63993b6', 'test2@zotero.org', NULL, 'member', NULL, NULL, NULL, 1, '0000-00-00 00:00:00', '0000-00-00 00:00:00', 0, 'testuser2'); +
-INSERT INTO shardLibraries VALUES (1, 'user'); +
-INSERT INTO shardLibraries VALUES (2, 'user'); +
-INSERT INTO shardLibraries VALUES (3, 'group'); +
-</code> +
- +
-when you sync use ‘testuser’ or ‘testuser2′ password ‘testuser’ +
- +
-====== Config Zotero data server ====== +
- +
- +
-Copy the sample config files under include/config and remove « sample » from the filenames +
-fill out credentials and paths in config files. +
- +
-===== dbconnect.inc.php ===== +
- +
- +
-db master:  zoterotest_master. +
- +
-db id1 and id2: zoterotest_ids +
- +
-db www1 and www2: zotero_www  (when you are in test mode server auto link to zotero_www_test) this is hardcoded in /model/auth/Password.inc.php +
- +
-db cache: zotero_cache +
- +
-===== Config.inc.php ===== +
- +
- +
-TODO explain values +
- +
-In the original file $AUTH_URI is not present, you can add it to avoid error log. Usage need to be cleared. +
-<code php> public static $AUTH_URI ='';</code> +
-<code php> +
- +
-<? +
-class Z_CONFIG { +
- public static $API_ENABLED = true; +
- public static $SYNC_ENABLED = true; +
- public static $PROCESSORS_ENABLED = true; +
- public static $MAINTENANCE_MESSAGE = 'Server updates in progress. Please try again in a few minutes.'; +
-  +
- public static $TESTING_SITE = false; +
- public static $DEV_SITE = false; +
- public static $AUTH_URI =''; +
- public static $BASE_URI = 'http://zotero.org'; +
-  +
- public static $AUTH_SALT = ''; +
- public static $API_SUPER_USERNAME = ''; +
- public static $API_SUPER_PASSWORD = ''; +
-  +
- public static $API_BASE_URI = 'api.zotero.org'; +
- public static $API_BASE_URI_WWW =''; +
- +
- public static $S3_BUCKET = ''; +
- public static $S3_ACCESS_KEY = ''; +
- public static $S3_SECRET_KEY = ''; +
-  +
- public static $MEMCACHED_ENABLED = false; +
- public static $MEMCACHED_SERVERS = array( +
- '192.168.1.1:11211:2', '192.168.1.2:11211:1' +
- ); +
- public static $MEMCACHED_SERVER_NAME_PREFIX_MAP = array( +
- 'api.example.com' => 'sync.example.com' +
- ); +
-  +
- public static $MONGO_SERVERS = array( +
- 'localhost:27017', 'localhost:27017' +
- ); +
- public static $MONGO_DB = "test"; +
- public static $MONGO_SAFE_NUM = 2; +
-  +
- public static $SOLR_SERVER = "192.168.1.1:8983"; +
-  +
- public static $CITE_SERVERS = array( +
- "192.168.1.1:8080", "192.168.1.2:8080" +
- ); +
-  +
- public static $LOG_TO_SCRIBE = false; +
- public static $LOG_ADDRESS = ''; +
- public static $LOG_PORT = 1463; +
- public static $LOG_TIMEZONE = 'US/Eastern'; +
- public static $LOG_TARGET_DEFAULT = 'errors'; +
-  +
- public static $PROCESSOR_PORT_DOWNLOAD = 3455; +
- public static $PROCESSOR_PORT_UPLOAD = 3456; +
- public static $PROCESSOR_PORT_ERROR = 3457; +
- public static $PROCESSOR_PORT_INDEX = 3458; +
-  +
- public static $PROCESSOR_LOG_TARGET_DOWNLOAD = 'sync-processor-download'; +
- public static $PROCESSOR_LOG_TARGET_UPLOAD = 'sync-processor-upload'; +
- public static $PROCESSOR_LOG_TARGET_ERROR = 'sync-processor-error'; +
- public static $PROCESSOR_LOG_TARGET_INDEX = 'processor-index'; +
-  +
- public static $SYNC_DOWNLOAD_SMALLEST_FIRST = false; +
- public static $SYNC_UPLOAD_SMALLEST_FIRST = false; +
-  +
- // Set some things manually for running via command line +
- public static $CLI_PHP_PATH = '/usr/bin/php'; +
- public static $CLI_DOCUMENT_ROOT = "/var/www/web/zserv/"; +
- public static $CLI_SERVER_NAME = "sync.zotero.org"; +
- public static $CLI_SERVER_PORT = "80"; +
-  +
- public static $SYNC_ERROR_PATH = '/var/log/httpd/sync-errors/'; +
- public static $API_ERROR_PATH = '/var/log/httpd/api-errors/'; +
-} +
-?> +
-</code> +
-===== Launch daemon ===== +
- +
- +
-There are for daemons, each in its own directory under the processor directory. Fire up for shells and run each independently in the foreground. +
- +
-<code bash> +
- +
-cd processor/download +
-  php daemon.php +
-cd processor/upload +
-  php daemon.php +
-cd processor/error +
-  php daemon.php +
-cd processor/index +
-  php daemon.php +
-   +
-</code> +
-In all cases you should see the output similar to this (the daemon name will vary) +
- +
-<code bash> +
- +
-  Starting sync download processor daemon +
-  0 processors, 0 queued processes +
-  Waiting for command +
-</code> +
- +
-====== Use zotero in your virtual machine to sync. ====== +
- +
-If your ssl is auto signed you must accept certificate before the first synchronization  +
-In Firefox open https://sync.zotero.org and https://api.zotero.org +
- +
-in Zotero preferences use login= testuser password= testuser and launch sync +
- +
-====== Miscellaneous ====== +
-===== Add new user ===== +
- +
- +
-You must add new entries in zotero_www (or zotero_www_test if in test mode). +
- +
-  * Necessary fields should be userID, username, password, slug. +
-  * Password entries are generated by taking a sha1 hash of the user’s password concatenated to the salt. +
-  * Slugs are generated by taking the username and trimming whitespace, lowercasing, preg_replace("/[^a-z0-9 ._-]/", "", $input), and replacing spaces with underscores. +
- +
-===== Create group ===== +
- +
- +
-As shown in ‘test_reset’ file: +
- +
-TODO explain values +
- +
-<code mysql> +
-INSERT INTO libraries VALUES (3, 'group', '0000-00-00 00:00:00', 0, 2); +
-INSERT INTO groups VALUES (1, 3, 'Test Group', 'test_group', 'Private', 1, 'admins', 'all', 'members', '', '', 0, '0000-00-00 00:00:00', '0000-00-00 00:00:00'); +
-INSERT INTO groupUsers VALUES (1, 1, 'owner', '0000-00-00 00:00:00', '0000-00-00 00:00:00'); +
-INSERT INTO shardLibraries VALUES (3, 'group'); +
-</code> +
- +
-===== Custom Zotero ===== +
- +
-TODO explain how edit zotero.jar +
- +
-If you want to use your own domain you need to edit the plugin. +
-Edit  zotero.js and change SYNC_URL and API_URL with your domain. +
- +
-<code javascript> +
-    const ZOTERO_CONFIG = { +
-    GUID: 'zotero@chnm.gmu.edu', +
-    DB_REBUILD: false, // erase DB and recreate from schema +
-    REPOSITORY_URL: 'https://api.zotero.org/repo', +
-    REPOSITORY_CHECK_INTERVAL: 86400, // 24 hours +
-    REPOSITORY_RETRY_INTERVAL: 3600, // 1 hour +
-    BASE_URI: 'http://zotero.org/', +
-    WWW_BASE_URL: 'http://www.zotero.org/', +
-    SYNC_URL: 'http://sync.mydomain.org/', +
-    API_URL: 'http://api.mydomain.org/' +
-    }; +
- +
- +
-</code> +
- +
-====== Untested / Not working / Todo====== +
- +
-===== Untested ===== +
-  * Webdav to use files attachment. +
-  * API: i don’t know how use it or configure it. +
- +
-===== Not working ===== +
-  * Change user privileges in groups only work if the user performs a restore from the server +
-  * Sync attachment files in My Library using Zotero server (i suppose you need Amazon S3) +
-  * Sync attachment files in group using Zotero server (i suppose you need Amazon S3) +
- +
-===== Todo ===== +
-  * Create SQL script to install database in one step +
-  * Create interface to add/manage users and groups +
-  * Test with webdav to use files attachment +
- +
- +
- +
zotero_dataserver.1303729501.txt.gz · Last modified: 2011/04/25 07:05 by lemaxx