3. Set-up of a CubicWeb environment

You can configure the database system of your choice:

For advanced features, have a look to:

3.1. Cubicweb resources configuration

3.1.1. Resource mode

3.1.1.1. Standard resource mode

A resource mode is a predefined set of settings for various resources directories, such as cubes, instances, etc. to ease development with the framework. There are two running modes with CubicWeb:

  • system: resources are searched / created in the system directories (eg usually requiring root access):

    • instances are stored in <INSTALL_PREFIX>/etc/cubicweb.d
    • temporary files (such as pid file) in <INSTALL_PREFIX>/var/run/cubicweb

    where <INSTALL_PREFIX> is the detected installation prefix (‘/usr/local’ for instance).

  • user: resources are searched / created in the user home directory:

    • instances are stored in ~/etc/cubicweb.d
    • temporary files (such as pid file) in /tmp

3.1.1.2. Within virtual environment

When installed within a virtualenv, CubicWeb will look for instances data as in user mode by default, that is in $HOME/etc/cubicweb.d. However the CW_INSTANCES_DIR environment variable should be preferably used.

3.1.1.3. Custom resource location

Notice that each resource path may be explicitly set using an environment variable if the default doesn’t suit your needs. Here are the default resource directories that are affected according to mode:

  • system:

    CW_INSTANCES_DIR = <INSTALL_PREFIX>/etc/cubicweb.d/
    CW_INSTANCES_DATA_DIR = <INSTALL_PREFIX>/var/lib/cubicweb/instances/
    CW_RUNTIME_DIR = <INSTALL_PREFIX>/var/run/cubicweb/
    
  • user:

    CW_INSTANCES_DIR = ~/etc/cubicweb.d/
    CW_INSTANCES_DATA_DIR = ~/etc/cubicweb.d/
    CW_RUNTIME_DIR = /tmp
    

Cubes search path is also affected, see the Cubes section.

3.1.1.4. Setting Cubicweb Mode

By default, the mode is set to ‘system’ for standard installation. The mode is set to ‘user’ if cubicweb is used from a mercurial repository. You can force this by setting the CW_MODE environment variable to either ‘user’ or ‘system’ so you can easily:

  • use system wide installation but user specific instances and all, without root privileges on the system (export CW_MODE=user)
  • use local checkout of cubicweb on system wide instances (requires root privileges on the system (export CW_MODE=system)

If you’ve a doubt about the mode you’re currently running, check the first line outputed by the cubicweb-ctl list command.

3.1.1.5. Development Mode (source)

If .hg directory is found into the cubicweb package, there are specific resource rules.

<CW_SOFTWARE_ROOT> is the source checkout’s cubicweb directory:

  • cubicweb migration files are searched in <CW_SOFTWARE_ROOT>/misc/migration instead of <INSTALL_PREFIX>/share/cubicweb/migration/.

3.1.1.6. Development Mode (virtualenv)

If a virtualenv is found to be activated (i.e. a VIRTUAL_ENV variable is found in environment), the virtualenv root is used as <INSTALL_PREFIX>. This, in particular, makes it possible to work in setuptools development mode (python setup.py develop) without any further configuration.

3.1.2. Environment configuration

3.1.2.1. Python

If you installed CubicWeb by cloning the Mercurial shell repository or from source distribution, then you will need to update the environment variable PYTHONPATH by adding the path to cubicweb:

Add the following lines to either .bashrc or .bash_profile to configure your development environment

export PYTHONPATH=/full/path/to/grshell-cubicweb

If you installed CubicWeb with packages, no configuration is required and your new cubes will be placed in /usr/share/cubicweb/cubes and your instances will be placed in /etc/cubicweb.d.

3.1.2.2. CubicWeb

Here are all environment variables that may be used to configure CubicWeb:

CW_MODE

Resource mode: user or system, as explained in Resource mode.

CW_INSTANCES_DIR

Directory where cubicweb instances will be found.

CW_INSTANCES_DATA_DIR

Directory where cubicweb instances data will be written (backup file…)

CW_RUNTIME_DIR

Directory where pid files will be written

3.2. Databases configuration

Each instance can be configured with its own database connection information, that will be stored in the instance’s sources file. The database to use will be chosen when creating the instance. CubicWeb is known to run with Postgresql (recommended), SQLServer and SQLite.

Other possible sources of data include CubicWeb, Subversion, LDAP and Mercurial, but at least one relational database is required for CubicWeb to work. You do not need to install a backend that you do not intend to use for one of your instances. SQLite is not fit for production use, but it works well for testing and ships with Python, which saves installation time when you want to get started quickly.

3.2.1. PostgreSQL

Many Linux distributions ship with the appropriate PostgreSQL packages. Basically, you need to install the following packages:

  • postgresql and postgresql-client, which will pull the respective versioned packages (e.g. postgresql-9.1 and postgresql-client-9.1) and, optionally,
  • a postgresql-plpython-X.Y package with a version corresponding to that of the aforementioned packages (e.g. postgresql-plpython-9.1).

If you run postgres version prior to 8.3, you’ll also need the postgresql-contrib-8.X package for full-text search extension.

If you run postgres on another host than the CubicWeb repository, you should install the postgresql-client package on the CubicWeb host, and others on the database host.

For extra details concerning installation, please refer to the PostgreSQL project online documentation.

3.2.1.1. Database cluster

If you already have an existing cluster and PostgreSQL server running, you do not need to execute the initilization step of your PostgreSQL database unless you want a specific cluster for CubicWeb databases or if your existing cluster doesn’t use the UTF8 encoding (see note below).

To initialize a PostgreSQL cluster, use the command initdb:

$ initdb -E UTF8 -D /path/to/pgsql

Note: initdb might not be in the PATH, so you may have to use its absolute path instead (usually something like /usr/lib/postgresql/9.4/bin/initdb).

Notice the encoding specification. This is necessary since CubicWeb usually want UTF8 encoded database. If you use a cluster with the wrong encoding, you’ll get error like:

new encoding (UTF8) is incompatible with the encoding of the template database (SQL_ASCII)
HINT:  Use the same encoding as in the template database, or use template0 as template.

Once initialized, start the database server PostgreSQL with the command:

$ postgres -D /path/to/psql

If you cannot execute this command due to permission issues, please make sure that your username has write access on the database.

$ chown username /path/to/pgsql

3.2.1.2. Database authentication

The database authentication is configured in pg_hba.conf. It can be either set to ident sameuser or md5. If set to md5, make sure to use an existing user of your database. If set to ident sameuser, make sure that your client’s operating system user name has a matching user in the database. If not, please do as follow to create a user:

$ su
$ su - postgres
$ createuser -s -P <dbuser>

The option -P (for password prompt), will encrypt the password with the method set in the configuration file pg_hba.conf. If you do not use this option -P, then the default value will be null and you will need to set it with:

$ su postgres -c "echo ALTER USER <dbuser> WITH PASSWORD '<dbpassword>' | psql"

The above login/password will be requested when you will create an instance with cubicweb-ctl create to initialize the database of your instance.

3.2.1.3. Database creation

If you create the database by hand (instead of using the cubicweb-ctl db-create tool), you may want to make sure that the local settings are properly set. For example, if you need to handle french accents properly for indexing and sorting, you may need to create the database with something like:

$ createdb --encoding=UTF-8 --locale=fr_FR.UTF-8 -t template0 -O <owner> <dbname>

Notice that the cubicweb-ctl db-create does database initialization that may requires a postgres superuser. That’s why a login/password is explicitly asked at this step, so you can use there a superuser without using this user when running the instance. Things that require special privileges at this step:

  • database creation, require the ‘create database’ permission
  • install the plpython extension language (require superuser)

To avoid using a super user each time you create an install, a nice trick is to install plpython (and tsearch when needed) on the special template1 database, so they will be installed automatically when cubicweb databases are created without needs for special access rights. To do so, run

# Installation of plpythonu language by default ::
$ createlang -U pgadmin plpythonu template1
$ psql -U pgadmin template1
template1=# update pg_language set lanpltrusted=TRUE where lanname='plpythonu';

Where pgadmin is a postgres superuser. The last command is necessary since by default plpython is an ‘untrusted’ language and as such can’t be used by non superuser. This update fix that problem by making it trusted.

3.2.2. SQLServer

As of this writing, support for SQLServer 2005 is functional but incomplete. You should be able to connect, create a database and go quite far, but some of the SQL generated from RQL queries is still currently not accepted by the backend. Porting to SQLServer 2008 is also an item on the backlog.

The source configuration file may look like this (specific parts only are shown):

[system]
db-driver=sqlserver2005
db-user=someuser
# database password not needed
#db-password=toto123
#db-create/init may ask for a pwd: just say anything
db-extra-arguments=Trusted_Connection
db-encoding=utf8

You need to change the default settings on the database by running:

ALTER DATABASE <databasename> SET READ_COMMITTED_SNAPSHOT ON;

The ALTER DATABASE command above requires some permissions that your user may not have. In that case you will have to ask your local DBA to run the query for you.

You can check that the setting is correct by running the following query which must return ‘1’:

SELECT is_read_committed_snapshot_on
  FROM sys.databases WHERE name='<databasename>';

3.2.3. SQLite

SQLite has the great advantage of requiring almost no configuration. Simply use ‘sqlite’ as db-driver, and set path to the dabase as db-name. Don’t specify anything for db-user and db-password, they will be ignore anyway.

Note

SQLite is great for testing and to play with cubicweb but is not suited for production environments.