The PkgForge Registry requires a PostgreSQL database. It is not currently possible to use any other database as a number of triggers and rules rely on the plpgsql language. This document assumes that you already have a database running and have administrator rights to create new users and databases (normally this is done as the postgres user).
If LCFG is being used to manage the system then the best method for managing the database is to use the LCFG postgresql component. This can be used to configure the ACLs and run backups.
The PkgForge and PkgForge-Registry packages must be
installed. These are provided as RPMs but it is also possible to build
and install them from the .tar.gz
format in the same way
as any other Perl module.
Perl version 5.8.8 or newer is required. The PkgForge-Server package requires the following modules.
DBD::Pg
DBIx::Class
DateTime
DateTime::Format::Pg
Moose
MooseX::Types
Text::Abbrev
If you are using LCFG this can all be installed by including
the lcfg/options/pkgforge-registry.h
header.
Note that steps 1 and 2 can be done using
the registry-init.sh
script provided in the
PkgForge-Registry package. They are detailed here so that a full
explanation can be provided.
As the database administrator issue the following commands to create the database roles:
createuser --no-createrole --no-createdb --no-superuser pkgforge_admin createuser --no-createrole --no-createdb --no-superuser pkgforge_incoming createuser --no-createrole --no-createdb --no-superuser pkgforge_builder createuser --no-createrole --no-createdb --no-superuser pkgforge_web
Each distinct part of the service runs as a different database user to make it possible to restrict write access to only the essential tables. This is to prevent deliberate or accidental corruption of the data. The admin user owns the database and has complete access. In normal operation it is rare that admin access is required for modifying the data.
As the database administrator issue the following commands to create the database and add the necessary language support:
createdb --owner pkgforge_admin pkgforge createlang plpgsql pkgforge
The final step is to create the tables, functions and rules for the
database. This is done using the registry-setup.sql
script which is provided as part of the PkgForge-Registry package. The
SQL can be run using the psql
command-line tool, like
this:
psql --dbname pkgforge --host pkgforge --username pkgforge_admin --file registry-setup.sql
This assumes that the database server is running on a machine named pkgforge and that you have already configured it appropriately to allow access as the pkgforge_admin database user. If you have not done this then details are provided later in this document. The setup script can instead be run as the postgres database administrator user.
It is essential to take regular backups of the PkgForge Registry to
provide security against loss of data because of a crash. The best way
to do this is with pg_dump
as part of a nightly cron
job.
If the LCFG postgresql component is being used this cron job can be added using the LCFG cron component. For example:
!cron.objects mADD(pkgforgedb) cron.object_pkgforgedb postgresql cron.method_pkgforgedb run cron.args_pkgforgedb backupdb pkgforge cron.run_pkgforgedb 00 22 * * *
That creates a cron job which will run at 22:00 each night and do a dump of the pkgforge database using the backupdb method of the LCFG postgresql component.
Alternatively this can be done by creating a cron job which calls pg_dump directly, for example:
pg_dump --create pkgforge > pkgforge.$(date '+%Y%m%d').dump.sql
This will dump the pkgforge database as plain-text into a file which the date as part its name. The command to create the database will be prepended to the SQL file.
In the event of a problem occurring, the storing of the data from each dump separately is incredibly useful. It makes it possible to walk back in time through the dumps to search for where corruption was first introduced.
With the dumps done like this it is very easy to restore the database. Firstly, do the initialisation step 1, "Create the Roles", which is explained above. After the roles are created the following command (as the database administrator) is all that is necessary.
psql -f pkgforge.20110501.dump.sql
This is also the best approach for moving the pkgforge database to a new host. Just take an additional dump then stop the database on the old server. To bring up the database on the new server do the restore process described above.
Access control to PostgreSQL databases is managed through
the pg_hba.conf
and pg_ident.conf
configuration files. On Redhat/Fedora systems these files are usually
found in the /var/lib/pgsql/data/
directory along with
all the other configuration files for the server. You can usually
access this most easily as ~postgres/data/
PostgreSQL supports numerous different authentication mechanisms, for details see the PostgreSQL documentation. PkgForge should support any authentication mechanism which is supported by the Perl DBI module. Only password and kerberos based authentication has been tested though.
The design of the PkgForge system is that each sub-system (i.e. incoming queue processor, build daemon and web interface) have different database roles. This makes it possible to limit the write-access to just the essential set of tables for each role. This improves data security and reduces the risk of accidental or deliberate corruption. Going beyond that it is sensible to also restrict access to the database by IP address and to use SSL for the connections.
If you want to support SSL connections to the database server you
need to add an X509 certificate and activate the
postgresql ssl
option in the postgresql.conf
file. Using the LCFG components it can be done like this:
#include <lcfg/options/postgresql-server.h> !postgresql.options mADD(ssl) !postgresql.option_ssl mSET(on) #include <lcfg/options/x509-client.h> !x509.keys mADD(pgssl) x509.service_pgssl <%profile.node%>.<%profile.domain%> x509.keyfile_pgssl <%postgresql.pgauthdir%>/server.key x509.certfile_pgssl <%postgresql.pgauthdir%>/server.crt x509.chainfile_pgssl <%postgresql.pgauthdir%>/root.crt x509.uid_pgssl <%postgresql.pgowner%> x509.gid_pgssl <%postgresql.pggroup%> x509.component_pgssl postgresql
If you are not using the LCFG x509 component you will need to create the SSL certificate manually.
If you want support for kerberos authentication in your database
you will need a keytab and you will need to set
the krb_server_keyfile
configuration option in
the postgresql.conf
file. With LCFG you can do this using
the kerberos component:
#include <lcfg/options/kerberos-client.h> !kerberos.keys mADD(postgres) kerberos.keytab_postgres <%postgresql.pgauthdir%>/postgres.keytab kerberos.keytabuid_postgres <%postgresql.pgowner%> kerberos.keytabgid_postgres <%postgresql.pggroup%> !postgresql.options mADD(krb_server_keyfile) postgresql.option_krb_server_keyfile '<%kerberos.keytab_postgres%>'
If you are not using the LCFG x509 component you will need to create the Kerberos keytab file manually. Note also that you would need to ensure that the UID and GID for the file are set correctly.
On an LCFG-managed host the ACLs are handled by the LCFG postgresql component. Here is an example which uses kerberos authentication:
#include <lcfg/options/postgresql-server.h> !postgresql.ident mADD(pkgf1) postgresql.idmap_pkgf1 pkgforge postgresql.sysname_pkgf1 example1 postgresql.pgname_pkgf1 pkgforge_admin !postgresql.hostssl mADD(pkgf1) postgresql.ssldatabase_pkgf1 pkgforge postgresql.ssluser_pkgf1 pkgforge_admin !postgresql.ssladdress_pkgf1 mHOSTIP(host1.example.org) postgresql.sslnetmask_pkgf1 255.255.255.255 postgresql.sslauthtype_pkgf1 krb5 !postgresql.sslauthopt_pkgf1 mSET(map=<%postgresql.idmap_pkgf1%>)
First, this creates a user mapping from the example1
kerberos principal to the pkgforge_admin
database
role. That mapping appears in the pg_ident.conf
file. The
same mapping can be applied from any username to database role.
Secondly, this adds an entry into the pg_hba.conf
configuration file to allow kerberised aaccess for that database role
from the host1.example.org
host address.
With this ACL in place the example1
user can access
the pkgforge database as the pkgforge_admin
database role
from the host1.example.org
host using kerberos
authentication like this:
% psql --dbname pkgforge --host pkgforge --username pkgforge_admin
This example LCFG configuration creates the following entry in
the pg_ident.conf
configuration file:
pkgforge example1 pkgforge_admin
The first entry is the map name, the second is the system username and the third part is the database role.
This example LCFG configuration creates an entry like this in
the pg_hba.conf
configuration file:
hostssl pkgforge pkgforge_admin 192.168.1.175 255.255.255.255 krb5 map=pkgforge
If you are not using the LCFG postgresql component you will need to
edit these files manually. Once the configuration is done you will
then also need to reload the database using the init script
(usually /etc/init.d/postgresql
).