Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
Last revisionBoth sides next revision
zotero_dataserver [2011/04/25 13:43] – [Config.inc.php] lemaxxzotero_dataserver [2017/11/12 19:53] – external edit 127.0.0.1
Line 1: Line 1:
-====== Disclaimer ====== +<html><p id="zotero-5-update-warning" style="color: red; font-weight: bold">We’re 
- +in the process of updating the documentation for 
-This guide was written based in part on information found on  zitation.org (unreachable when I was writing this tutorial) as well as the indications found on zotero.org. Some steps may be unnecessary or wrong but keep in mind that this is a first version. Any help or comment is welcome. +<a href="https://www.zotero.org/blog/zotero-5-0">Zotero 5.0</a>. Some documentation 
- +may be outdated in the meantime. Thanks for your understanding.</p></html>
-Some items may seem obvious but I detail some steps not to forget my next installation. +
-This installation was done on Ubuntu 10.10 desktop running in a virtual machine. This allows me to use an unmodified version of Zotero 2.1 +
- +
-====== Dependencies: ====== +
- +
-===== Required ===== +
- +
-  * PHP +
-  * PHP modules (?) +
-  * Apache +
-  * Apache modules/features: some sort of for PHP (?), virtual hosts support, mod_rewrite, auth (?) +
-  * MySQL with root access +
-  * MongoDB +
-  * ability to configure /etc/hosts or DNS +
-  * subversion client +
- +
-===== Optional ===== +
- +
-  * Solr +
-  * Memcached +
-  * Amazon S3 +
-  * Webdav +
- +
-Download Zotero dataserver sources +
-Use your subversion client to pull down the sources from Zotero’s repository. If using the command-line:
svn co https://www.zotero.org/svn/dataserver
 You’ll see the following layout (from here on references to zotero source paths will assume the root of dataserver/trunk) +
- +
-<code> +
-dataserver/ +
-          trunk/ +
-               controllers +
-               htdocs +
-               include +
-               misc +
-               model +
-               processor +
-               tests +
-               tmp</code> +
- +
-These files can be anywhere but this guide will assume they are located in /home/UTILISATEUR/www-dev/dataserv ←- NOTE without the ‘trunk’ – in other words, the ‘include’ directory is at /home/UTILISATEUR/www-dev/dataserv/include. +
- +
-====== Configure Apache and Host ====== +
- +
-===== Host ===== +
- +
-We need to define two hostnames, the “sync” and “api” hosts. For testing on one machine this can be done via an /etc/hosts entry +
-In this exemple i use zotero.org domain because i use a virtual machine and i don’t use a custom Zotero plugin. +
-If you want to use your own domain you need to edit the plugin (see below) +
-<code> +
-127.0.0.1 sync.zotero.org api.zotero.org</code> +
- +
-After that when you ping sync.zotero.org you will see it is resolving to 127.0.0.1 +
- +
-===== Apache ===== +
- +
-From web only HTDOCS folder must be accessible. So you can use a virtual host +
-Your sync server need a ssl connection you have to create two config files in /etc/apache2/sites-available +
- +
-<code> +
-<VirtualHost  sync.zotero.org:80> +
-  +
- ServerName sync.zotero.org +
- ServerAlias api.zotero.org  +
-  +
- DocumentRoot /home/UTILISATEUR/www-dev/dataserv/htdocs +
-  +
- <Directory "/home/UTILISATEUR/www-dev/dataserv/htdocs"> +
- Options FollowSymLinks MultiViews +
- AllowOverride All +
- Order allow,deny +
- Allow from all +
- </Directory> +
-  +
- AllowEncodedSlashes On  +
-  +
-</VirtualHost> +
-</code> +
- +
-<code> +
-<IfModule mod_ssl.c> +
-<VirtualHost sync.zotero.org:443> +
-  +
- ServerName sync.zotero.org +
- ServerAlias api.zotero.org  +
-  +
- DocumentRoot /home/UTILISATEUR/www-dev/dataserv/htdocs +
-  +
- <Directory "/home/UTILISATEUR/www-dev/dataserv/htdocs"> +
- Options FollowSymLinks MultiViews +
- AllowOverride All +
- Order allow,deny +
- Allow from all +
- </Directory> +
-  +
- AllowEncodedSlashes On +
-  +
- #   Enable/Disable SSL for this virtual host. +
- SSLEngine on +
-  +
- SSLCertificateFile    /etc/ssl/certs/ssl-cert-snakeoil.pem +
- SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key +
-  +
- <FilesMatch "\.(cgi|shtml|phtml|php)$"> +
- SSLOptions +StdEnvVars +
- </FilesMatch> +
- <Directory /usr/lib/cgi-bin> +
- SSLOptions +StdEnvVars +
- </Directory> +
-  +
- BrowserMatch "MSIE [2-6]" \ +
- nokeepalive ssl-unclean-shutdown \ +
- downgrade-1.0 force-response-1.0 +
- # MSIE 7 and newer should be able to use keepalive +
- BrowserMatch "MSIE [17-9]" ssl-unclean-shutdown +
-  +
-</VirtualHost> +
-</IfModule> +
- +
-</code> +
- +
-====== Create and configure database ====== +
- +
-===== Mongo ===== +
- +
- +
-Install MongoDB and Mongo php driver +
- +
-use /misc/mongo_schema.js to create content +
- +
-===== Mysql ===== +
- +
- +
-To understand how database must be set i used the script « test_reset » locate in « Misc » folder (you can use this script to setup your database). I use the same database name. +
- +
-In this folder you will find every *.sql files need for this installation. +
- +
-You have to create 7 Mysql database. Some of them are used only when you run Zotero server in test mode, for other I do not know yet their use. +
- +
-  * zotero_cache > use cache.sql +
-  * zotero_www (this name is hardcoded) > see below +
-  * zotero_www_test (this name is hardcoded) > see below +
-  * zoterotest_ids > use ids.sql +
-  * zoterotest_master > use master.sql and after coredata.sql +
-  * zoterotest1 > use shard.sql and after trigger.sql +
-  * zoterotest2 > use shard.sql and after trigger.sql +
- +
-There is no .sql files to set up zotero_www and zotero_www_test so use as describe on zotero.org/trac/wiki/dataserver_setup +
- +
-<code mysql> +
-CREATE TABLE  `users` ( +
-  `userID`             mediumint      UNSIGNED auto_increment                               NOT NULL, +
-  `username`           varchar(40)    CHARACTER SET utf8 COLLATE utf8_general_ci            NOT NULL, +
-  `slug`               varchar(40)    CHARACTER SET utf8 COLLATE utf8_general_ci                NULL, +
-  `password`           char(40)       CHARACTER SET utf8 COLLATE utf8_bin                   NOT NULL, +
-  `email`              varchar(100)   CHARACTER SET utf8 COLLATE utf8_general_ci            NOT NULL, +
-  `openid`             varchar(100)   CHARACTER SET utf8 COLLATE utf8_general_ci                NULL, +
-  `role`               enum('member', +
-                            'admin')  DEFAULT 'member'                                      NOT NULL, +
-  `resetValidationKey` char(40)       CHARACTER SET utf8 COLLATE utf8_bin                       NULL, +
-  `resetValidationExp` int                                                                      NULL, +
-  `emailValidationKey` char(10)       CHARACTER SET utf8 COLLATE utf8_bin                       NULL, +
-  `emailValidated`     tinyint(1)     DEFAULT '0'                                           NOT NULL, +
-  `hasImage`           tinyint(1)     DEFAULT '0'                                           NOT NULL, +
-  `lastUpdated`        timestamp      ON UPDATE CURRENT_TIMESTAMP DEFAULT CURRENT_TIMESTAMP NOT NULL, +
-  `dateRegistered`     timestamp                                                            NOT NULL, +
-  PRIMARY KEY (`userID`), +
-  UNIQUE KEY  (`username`), +
-  UNIQUE KEY  (`slug`), +
-  UNIQUE KEY  (`email`), +
-  UNIQUE KEY  (`resetValidationKey`) +
-) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin; +
- +
-</code> +
- +
-You need to add specific user in Mysql +
-If you want to personalize those user you have to change value in shard info (see below) +
-<code mysql> +
- +
-CREATE USER zoterotest0@localhost IDENTIFIED BY 'pass0'; +
-CREATE USER zoterotest1@localhost IDENTIFIED BY 'pass1'; +
-CREATE USER zoterotest2@localhost IDENTIFIED BY 'pass2'; +
-CREATE USER zoterotest_ids@localhost IDENTIFIED BY 'pass1'; +
-CREATE USER zoterotest_ids@localhost IDENTIFIED BY 'pass2'; +
-CREATE USER zoterotest_www@localhost IDENTIFIED BY 'pass'; +
- +
-</code> +
- +
-note: give acces on sync.zotero.org too +
- +
-Give Grant access +
- +
-<code mysql> +
- +
-GRANT SELECT ON zoterotest_master.* TO zoterotest1@localhost; +
-GRANT SELECT ON zoterotest_master.* TO zoterotest2@localhost; +
-GRANT SELECT, INSERT, UPDATE, DELETE, CREATE TEMPORARY TABLES ON zoterotest1.* TO zoterotest1@localhost; +
-GRANT SELECT, INSERT, UPDATE, DELETE, CREATE TEMPORARY TABLES ON zoterotest2.* TO zoterotest2@localhost; +
-GRANT SELECT,INSERT,DELETE ON zoterotest_ids.* TO zoterotest_ids@localhost; +
-GRANT SELECT,INSERT,DELETE ON zoterotest_ids.* TO zoterotest_ids@localhost; +
-GRANT SELECT ON zotero_www_test.* TO zoterotest_www@localhost; +
-</code> +
- +
-Set up shard info +
- +
-<code mysql> +
-INSERT INTO shardHosts VALUES (1, '127.0.0.1', 3306, 'up'); +
-INSERT INTO shardHosts VALUES (2, 'sync.zotero.org', 3306, 'up'); +
-INSERT INTO shards VALUES (1, 1, 'zoterotest1', 'pass1', 'zoterotest1', 'up'); +
-INSERT INTO shards VALUES (2, 1, 'zoterotest2', 'pass2', 'zoterotest2', 'up'); +
-</code> +
-Initial users and groups for tests +
- +
-<code mysql> +
-INSERT INTO libraries VALUES (1, 'user', '0000-00-00 00:00:00', 0, 1); +
-INSERT INTO libraries VALUES (2, 'user', '0000-00-00 00:00:00', 0, 1); +
-INSERT INTO libraries VALUES (3, 'group', '0000-00-00 00:00:00', 0, 2); +
-INSERT INTO users VALUES (1, 1, 'testuser', '0000-00-00 00:00:00', '0000-00-00 00:00:00'); +
-INSERT INTO users VALUES (2, 2, 'testuser2', '0000-00-00 00:00:00', '0000-00-00 00:00:00'); +
-INSERT INTO groups VALUES (1, 3, 'Test Group', 'test_group', 'Private', 1, 'admins', 'all', 'members', '', '', 0, '0000-00-00 00:00:00', '0000-00-00 00:00:00'); +
-INSERT INTO groupUsers VALUES (1, 1, 'owner', '0000-00-00 00:00:00', '0000-00-00 00:00:00'); +
-  +
-INSERT INTO users VALUES (1, 'testuser', '5d9c68c6c50ed3d02a2fcf54f63993b6', 'test@zotero.org', NULL, 'member', NULL, NULL, NULL, 1, '0000-00-00 00:00:00', '0000-00-00 00:00:00', 0, 'testuser'); +
-INSERT INTO users VALUES (2, 'testuser2', '5d9c68c6c50ed3d02a2fcf54f63993b6', 'test2@zotero.org', NULL, 'member', NULL, NULL, NULL, 1, '0000-00-00 00:00:00', '0000-00-00 00:00:00', 0, 'testuser2'); +
-INSERT INTO shardLibraries VALUES (1, 'user'); +
-INSERT INTO shardLibraries VALUES (2, 'user'); +
-INSERT INTO shardLibraries VALUES (3, 'group'); +
-</code> +
- +
-when you sync use ‘testuser’ or ‘testuser2′ password ‘testuser’ +
- +
-====== Config Zotero data server ====== +
- +
- +
-Copy the sample config files under include/config and remove « sample » from the filenames +
-fill out credentials and paths in config files. +
- +
-===== dbconnect.inc.php ===== +
- +
- +
-db master:  zoterotest_master. +
- +
-db id1 and id2: zoterotest_ids +
- +
-db www1 and www2: zotero_www  (when you are in test mode server auto link to zotero_www_test) this is hardcoded in /model/auth/Password.inc.php +
- +
-db cache: zotero_cache +
- +
-===== Config.inc.php ===== +
- +
- +
-TODO explain values +
- +
-In the original file $AUTH_URI is not present, you can add it to avoid error log. Usage need to be cleared. +
-<code php> public static $AUTH_URI ='';</code> +
-<code php> +
- +
-<? +
-class Z_CONFIG { +
- public static $API_ENABLED = true; +
- public static $SYNC_ENABLED = true; +
- public static $PROCESSORS_ENABLED = true; +
- public static $MAINTENANCE_MESSAGE = 'Server updates in progress. Please try again in a few minutes.'; +
-  +
- public static $TESTING_SITE = false; +
- public static $DEV_SITE = false; +
- public static $AUTH_URI =''; +
- public static $BASE_URI = 'http://zotero.org'; +
-  +
- public static $AUTH_SALT = ''; +
- public static $API_SUPER_USERNAME = ''; +
- public static $API_SUPER_PASSWORD = ''; +
-  +
- public static $API_BASE_URI = 'api.zotero.org'; +
- public static $API_BASE_URI_WWW =''; +
- +
- public static $S3_BUCKET = ''; +
- public static $S3_ACCESS_KEY = ''; +
- public static $S3_SECRET_KEY = ''; +
-  +
- public static $MEMCACHED_ENABLED = false; +
- public static $MEMCACHED_SERVERS = array( +
- '192.168.1.1:11211:2', '192.168.1.2:11211:1' +
- ); +
- public static $MEMCACHED_SERVER_NAME_PREFIX_MAP = array( +
- 'api.example.com' => 'sync.example.com' +
- ); +
-  +
- public static $MONGO_SERVERS = array( +
- 'localhost:27017', 'localhost:27017' +
- ); +
- public static $MONGO_DB = "Name_of_your_database"; +
- public static $MONGO_SAFE_NUM = 2; +
-  +
- public static $SOLR_SERVER = "192.168.1.1:8983"; +
-  +
- public static $CITE_SERVERS = array( +
- "192.168.1.1:8080", "192.168.1.2:8080" +
- ); +
-  +
- public static $LOG_TO_SCRIBE = false; +
- public static $LOG_ADDRESS = ''; +
- public static $LOG_PORT = 1463; +
- public static $LOG_TIMEZONE = 'US/Eastern'; +
- public static $LOG_TARGET_DEFAULT = 'errors'; +
-  +
- public static $PROCESSOR_PORT_DOWNLOAD = 3455; +
- public static $PROCESSOR_PORT_UPLOAD = 3456; +
- public static $PROCESSOR_PORT_ERROR = 3457; +
- public static $PROCESSOR_PORT_INDEX = 3458; +
-  +
- public static $PROCESSOR_LOG_TARGET_DOWNLOAD = 'sync-processor-download'; +
- public static $PROCESSOR_LOG_TARGET_UPLOAD = 'sync-processor-upload'; +
- public static $PROCESSOR_LOG_TARGET_ERROR = 'sync-processor-error'; +
- public static $PROCESSOR_LOG_TARGET_INDEX = 'processor-index'; +
-  +
- public static $SYNC_DOWNLOAD_SMALLEST_FIRST = false; +
- public static $SYNC_UPLOAD_SMALLEST_FIRST = false; +
-  +
- // Set some things manually for running via command line +
- public static $CLI_PHP_PATH = '/usr/bin/php'; +
- public static $CLI_DOCUMENT_ROOT = "/var/www/web/zserv/"; +
- public static $CLI_SERVER_NAME = "sync.zotero.org"; +
- public static $CLI_SERVER_PORT = "80"; +
-  +
- public static $SYNC_ERROR_PATH = '/var/log/httpd/sync-errors/'; +
- public static $API_ERROR_PATH = '/var/log/httpd/api-errors/'; +
-} +
-?> +
-</code> +
-===== Launch daemon ===== +
- +
- +
-There are for daemons, each in its own directory under the processor directory. Fire up for shells and run each independently in the foreground. +
- +
-<code bash> +
- +
-cd processor/download +
-  php daemon.php +
-cd processor/upload +
-  php daemon.php +
-cd processor/error +
-  php daemon.php +
-cd processor/index +
-  php daemon.php +
-   +
-</code> +
-In all cases you should see the output similar to this (the daemon name will vary) +
- +
-<code bash> +
- +
-  Starting sync download processor daemon +
-  0 processors, 0 queued processes +
-  Waiting for command +
-</code> +
- +
-====== Use zotero in your virtual machine to sync. ====== +
- +
-If your ssl is auto signed you must accept certificate before the first synchronization  +
-In Firefox open https://sync.zotero.org and https://api.zotero.org +
- +
-in Zotero preferences use login= testuser password= testuser and launch sync +
- +
-====== Miscellaneous ====== +
-===== Add new user ===== +
- +
- +
-You must add new entries in zotero_www (or zotero_www_test if in test mode). +
- +
-  * Necessary fields should be userID, username, password, slug. +
-  * Password entries are generated by taking a sha1 hash of the user’s password concatenated to the salt. +
-  * Slugs are generated by taking the username and trimming whitespace, lowercasing, preg_replace("/[^a-z0-9 ._-]/", "", $input), and replacing spaces with underscores. +
- +
-===== Create group ===== +
- +
- +
-As shown in ‘test_reset’ file: +
- +
-TODO explain values +
- +
-<code mysql> +
-INSERT INTO libraries VALUES (3, 'group', '0000-00-00 00:00:00', 0, 2); +
-INSERT INTO groups VALUES (1, 3, 'Test Group', 'test_group', 'Private', 1, 'admins', 'all', 'members', '', '', 0, '0000-00-00 00:00:00', '0000-00-00 00:00:00'); +
-INSERT INTO groupUsers VALUES (1, 1, 'owner', '0000-00-00 00:00:00', '0000-00-00 00:00:00'); +
-INSERT INTO shardLibraries VALUES (3, 'group'); +
-</code> +
- +
-===== Custom Zotero ===== +
- +
-TODO explain how edit zotero.jar +
- +
-If you want to use your own domain you need to edit the plugin. +
-Edit  zotero.js and change SYNC_URL and API_URL with your domain. +
- +
-<code javascript> +
-    const ZOTERO_CONFIG = { +
-    GUID: 'zotero@chnm.gmu.edu', +
-    DB_REBUILD: false, // erase DB and recreate from schema +
-    REPOSITORY_URL: 'https://api.zotero.org/repo', +
-    REPOSITORY_CHECK_INTERVAL: 86400, // 24 hours +
-    REPOSITORY_RETRY_INTERVAL: 3600, // 1 hour +
-    BASE_URI: 'http://zotero.org/', +
-    WWW_BASE_URL: 'http://www.zotero.org/', +
-    SYNC_URL: 'http://sync.mydomain.org/', +
-    API_URL: 'http://api.mydomain.org/' +
-    }; +
- +
- +
-</code> +
- +
-====== Untested / Not working / Todo====== +
- +
-===== Untested ===== +
-  * Webdav to use files attachment. +
-  * API: i don’t know how use it or configure it. +
- +
-===== Not working ===== +
-  * Change user privileges in groups only work if the user performs a restore from the server +
-  * Sync attachment files in My Library using Zotero server (i suppose you need Amazon S3) +
-  * Sync attachment files in group using Zotero server (i suppose you need Amazon S3) +
- +
-===== Todo ===== +
-  * Create SQL script to install database in one step +
-  * Create interface to add/manage users and groups +
-  * Test with webdav to use files attachment +
- +
  
  
 +== Deprecated, go to: == 
 +http://www.zotero.org/support/dev/dataserver_setup
zotero_dataserver.txt · Last modified: 2019/04/19 14:35 by bwiernik