A+ Deployment Instructions

These instructions are checked against Ubuntu, but they should work for Debian and probably for many derivatives of it.

$ lsb_release -a
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.1 LTS
Release:        18.04
Codename:       bionic

A+ can be deployed with Apache 2 or NGINX. Bot web servers work well enough and can support Shibboleth authentication too. Sadly, combination of NGINX, uWSGI protocol, uWSGI and Shibboleth is not yet tested, but the HTTP proxy version works.

Aalto University is currently running following combinations:

Apache, uWSGI protocol, uWSGI, Shibboleth, uWSGI logs written to a file
NGINX, HTTP proxy, gunicorn, Shibboleth, Gunicorn logs written to journald
NGINX, uWSGI protocol, uWSGI, uWSGI logs written to journald

Common system configuration

Install required packages

sudo apt-get install \
  git gettext \
  postgresql memcached \
  uwsgi-core uwsgi-plugin-python3 \
  python3-virtualenv \
  python3-certifi python3-lz4 python3-psycopg2 python3-reportlab python3-reportlab-accel

Create a new user for a-plus

sudo adduser --system --group \
  --shell /bin/bash --home /srv/aplus \
  --gecos "A-plus LMS webapp server" \
  aplus

Create a database and add permissions

sudo -Hu postgres createuser aplus
sudo -Hu postgres createdb -O aplus aplus

Create a run directory

echo "d /run/aplus 0750 aplus www-data - -" \
  | sudo tee /etc/tmpfiles.d/aplus.conf > /dev/null
sudo systemd-tmpfiles --create

The Application

Change to our service user
```
sudo -u aplus -Hi
```

Clone the Django application

# as user aplus in /srv/aplus
git clone --branch production https://github.com/Aalto-LeTech/a-plus.git
mkdir a-plus/static \
      a-plus/media

Install virtualenv

# as user aplus in /srv/aplus
python3 -m virtualenv -p python3 --system-site-packages venv
. venv/bin/activate
pip install -r a-plus/requirements.txt
pip install -r a-plus/requirements_prod.txt

Configure Django

# as user aplus in /srv/aplus
cat > a-plus/aplus/local_settings.py <<EOF
BASE_URL = "https://$(hostname)/"
SERVER_EMAIL = "aplus@$(hostname)"
EOF

awk '/(BASE_URL|DEBUG|SECRET_KEY|SERVER_EMAIL)/ {next};
     /^## (Database|Cache|Logging)/ {while (/^#/ && getline>0); next} 1' \
  a-plus/aplus/local_settings.example.py >> a-plus/aplus/local_settings.py

cat >> a-plus/aplus/local_settings.py <<EOF
## Database
DATABASES = {
  'default': {
    'ENGINE': 'django.db.backends.postgresql',
    'NAME': 'aplus',
  }
}
## Cache
CACHES = {
  'default': {
    'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
    'LOCATION': '127.0.0.1:11211',
  }
}
EOF

Run Django deployment tasks

# as user aplus in /srv/aplus
. venv/bin/activate
pushd a-plus

# migrate db
./manage.py migrate

# compile localisation files
./manage.py compilemessages

# collect static files to be served via dedicated web server
./manage.py collectstatic --no-input

popd

Create uWSGI configuration
```
# as user aplus in /srv/aplus
cat > uwsgi-aplus-web.ini <<EOF
[uwsgi]
home=/srv/aplus/venv
module=aplus.wsgi:application
enable-threads=True
processes=4
threads=2
max-requests=10000
harakiri=40
env=LANG=en_US.UTF-8
EOF

cat > uwsgi-aplus-api.ini <<EOF
[uwsgi]
home=/srv/aplus/venv
module=aplus.wsgi:application
enable-threads=False
processes=2
threads=1
max-requests=10000
harakiri=40
env=LANG=en_US.UTF-8
EOF
```
NOTE: Select number of processes and threads based on number of of CPUs. Probably a good number is around two times cpus for the web (as there is a lot of io wait) and number of cpus for the API.

You can gracefully chain reload uWSGI services by touching (editing) above files. In addition, sudo systemctl restart aplus-*-uwsgi.service will also do a chain reload. To fully reload python engine, do stop and then start for these services.
End

Exit the aplus user shell and return to the admin
```
exit
```

Common Shibboleth configuration

To configure Shibboleth, follow a guide for your federation. Because A+ is used a lot in Finnish Universities, we have some examples using HAKA, the identity federation for Finnish universities.

For HAKA, you need at least schemas from here and metadata certificate from here. In addition, you can start from apache2/shibboleth2.xml or nginx/shibboleth2.xml. More details below for both web servers.

Key generation

Here is a command to create sp-key.pem and sp-cert.pem files. As of writing, shib-keygen does create too small keys and uses sha1 for hashing. Due to security considerations, you should use following instead:

# set your domain here and then copy-paste command below
host=$(hostname)
entityid=https://$host

cd /etc/shibboleth
printf '[req]\ndistinguished_name=req\n[san]\nsubjectAltName=DNS:%s, URI:%s\n' "$host" "$entityid" | \
openssl req -x509 -sha256 -nodes \
  -newkey rsa:4096 -keyout sp-key.pem \
  -days 3650 -out sp-cert.pem \
  -subj "/CN=$host" -extensions san -config /dev/stdin
chown _shibd:_shibd sp-cert.pem sp-key.pem
chmod 0400 sp-key.pem

You can print the certificate information with this command:

openssl x509 -in /etc/shibboleth/sp-cert.pem -noout -text

Apache 2 configuration

Resources for Apache 2 configuration can be found under apache2 directory. Following instructions expect that the applocation is installed under /srv/aplus/a-plus/.

Install packages

sudo apt-get install apache2 libapache2-mod-uwsgi
sudo a2enmod uwsgi
sudo a2enmod ssl

Configure Apache 2

sudo ln -s ../sites-available/$(hostname).conf /etc/apache2/sites-enabled/000-$(hostname).conf
sed -e "s/__HOSTNAME__/$(hostname)/g" /srv/aplus/a-plus/doc/apache2/aplus-apache2.conf \
 | sudo tee /etc/apache2/sites-available/$(hostname).conf > /dev/null

Create systemd service files for uWSGI processes

sudo cp /srv/aplus/a-plus/doc/apache2/aplus-*-uwsgi.service \
        /etc/systemd/system
sudo systemctl daemon-reload
sudo systemctl enable aplus-web-uwsgi.service aplus-api-uwsgi.service
sudo systemctl start aplus-web-uwsgi.service aplus-api-uwsgi.service

Reload Apache 2
```
sudo systemctl restart apahce2.service
```

Shibboleth with Apache 2

If you are using shibboleth

Install Shibboleth 2

sudo apt-get install libapache2-mod-shib2
sudo a2enmod shib2

Configure Shibboleth for your federation or organization

Configuration is under directory /etc/shibboleth. For HAKA, there is apache2/shibboleth2.xml.
Shibboleth configuration in The local_settings.py

Map your federations variables to ones used in A+. Most of the values are common, so only defining PREFIX should be enough. Currently, STUDENT_DOMAIN is a required variable, as A+ presumes student numbers to be from a single domain. Rest of the options are documented in settings.py.
```
sudo tee -a /srv/aplus/a-plus/aplus/local_settings.py << EOF
# Shibboleth
SHIBBOLETH_ENVIRONMENT_VARS = {
    'PREFIX': 'SHIB_',
    'STUDENT_DOMAIN': 'example.com', # XXX: change this!
}
EOF
```
Reload shibboleth
```
sudo systemctl restart shibd.service
```

NGINX configuration

Resources for NGINX configuration can be found under nginx directory. Following instructions expect that the applocation is installed under /srv/aplus/a-plus/.

Install packges
```
sudo apt-get install nginx
```

Configure NGINX

if [ -d /etc/nginx/sites-available ] && grep -qs sites-enabled /etc/nginx/nginx.conf; then
    dest=/etc/nginx/sites-available/$(hostname).conf
    sudo ln -s ../sites-available/$(hostname).conf /etc/nginx/sites-enabled/$(hostname).conf
else
    dest=/etc/nginx/conf.d/$(hostname).conf
fi
sed -e "s/__HOSTNAME__/$(hostname)/g" /srv/aplus/a-plus/doc/nginx/aplus-nginx.conf \
 | sudo tee "$dest" > /dev/null

NOTE: If you are going to use shibboleth, then use file nginx/aplus-nginx-shib.conf.

Create systemd service files for uWSGI processes

sudo cp /srv/aplus/a-plus/doc/nginx/aplus-*-uwsgi.service \
        /etc/systemd/system
sudo systemctl daemon-reload
sudo systemctl enable aplus-web-uwsgi.service aplus-api-uwsgi.service
sudo systemctl start aplus-web-uwsgi.service aplus-api-uwsgi.service

If you prefer to use Gunicorn, then you can use nginx/aplus-gunicorn.service.

Reload NGINX
```
sudo systemctl restart nginx.service
```

Shibboleth with NGINX

This guide bases on NGINX module nginx-http-shibboleth. This module uses fastcgi and shibboleth scripts to provide similar integration as Apache 2 plugin.

With Ubuntu xenial or before NGINX 1.11

With Ubuntu xenial (16.04) and before NGINX 1.11, dynamic modules are not supported, so you need to rebuild the whole NGINX package.

sudo -i
cd /usr/src

apt-get install build-essential devscripts
apt-get source nginx
apt-get build-dep nginx

git clone https://github.com/nginx-shib/nginx-http-shibboleth.git

pushd nginx-1.*/

# add shib module to be build: debian/rules
patch -l -p1 <<PATCH
--- a/debian/rules
+++ b/debian/rules
@@ -98,6 +98,8 @@
   --with-mail \\
   --with-mail_ssl_module \\
   --with-threads \\
+  --add-module=\$(MODULESDIR)/headers-more-nginx-module \\
+  --add-module=/usr/src/nginx-http-shibboleth/ \\
   --add-module=\$(MODULESDIR)/nginx-auth-pam \\
   --add-module=\$(MODULESDIR)/nginx-dav-ext-module \\
   --add-module=\$(MODULESDIR)/nginx-echo \\
@@ -126,6 +128,7 @@
   --with-stream_ssl_module \\
   --with-threads \\
   --add-module=\$(MODULESDIR)/headers-more-nginx-module \\
+  --add-module=/usr/src/nginx-http-shibboleth/ \\
   --add-module=\$(MODULESDIR)/nginx-auth-pam \\
   --add-module=\$(MODULESDIR)/nginx-cache-purge \\
   --add-module=\$(MODULESDIR)/nginx-dav-ext-module \\
PATCH

# Create debian release
dch -lshib "Add Shibboleth"
dch -r ""

# build debian packages
dpkg-buildpackage -uc -us
popd

# install the packages
dpkg -i $(ls nginx-common_*shib*_all.deb | sort | tail -n1) \
        $(ls nginx-full_*shib*_amd64.deb | sort | tail -n1)

exit # exit sudo session

Starting from Ubuntu Bionic or NGINX 1.11 we can dynamic modules

Write down instructions for this.

Get NGINX supporting files

cd /etc/nginx
wget https://raw.githubusercontent.com/nginx-shib/nginx-http-shibboleth/master/includes/shib_clear_headers

Install required packages

sudo apt-get install \
  shibboleth-sp2-common \
  shibboleth-sp2-utils \
  shibboleth-sp2-schemas \
  xmltooling-schemas

Create systemd services and sockets for shibboleth scripts

sudp cp /srv/aplus/a-plus/doc/nginx/shib*.socket \
        /srv/aplus/a-plus/doc/nginx/shib*.service \
        /etc/systemd/system
sudo systemctl daemon-reload
sudo systemctl enable shibauthorizer.socket shibresponder.socket
sudo systemctl start shibauthorizer.socket shibresponder.socket

Configure Shibboleth for your federation or organization

Configuration is under directory /etc/shibboleth. For HAKA, there is nginx/shibboleth2.xml.
Shibboleth configuration in The local_settings.py

Map your federations variables to ones used in A+. Most of the values are common, so only defining PREFIX should be enough. Currently, STUDENT_DOMAIN is a required variable, as A+ presumes student numbers to be from a single domain. Rest of the options are documented in settings.py.
```
sudo tee -a /srv/aplus/a-plus/aplus/local_settings.py << EOF
# Shibboleth
SHIBBOLETH_ENVIRONMENT_VARS = {
    'PREFIX': 'HTTP_SHIB_',
    'STUDENT_DOMAIN': 'example.com', # XXX: change this!
}
EOF
```
Reload shibboleth
```
sudo systemctl restart shibd.service
```

Final steps

Create or set a superuser.

It is not typically needed to create local superuser, thus you can skip the first part.
```
sudo -Hu aplus sh -c "cd /srv/aplus/a-plus;
  ../venv/bin/python3 ./manage.py createsuperuser"
```
Instead, it is recommended to setup Shibboleth or Social Auth and login your first admin. After that, you can make an existing user a superuser:
```
sudo -Hu aplus sh -c "cd /srv/aplus/a-plus;
  ../venv/bin/python3 ./manage.py set_superuser"
```
NOTE: You might need to use --first-name, --last-name or --email to limit the number of users.

Create privacy notices

Copy privacy notice templates

sudo -Hu aplus sh -c "
  mkdir -p /srv/aplus/a-plus/local_templates/
  cp /srv/aplus/a-plus/templates/privacy_notice_* \
     /srv/aplus/a-plus/local_templates/
"

Edit your local templates in /srv/aplus/a-plus/local_templates/ to match your local information.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DEPLOYMENT.md

DEPLOYMENT.md

A+ Deployment Instructions

Table of contents

Common system configuration

The Application

Common Shibboleth configuration

Key generation

Apache 2 configuration

Shibboleth with Apache 2

NGINX configuration

Shibboleth with NGINX

Final steps

Files

DEPLOYMENT.md

Latest commit

History

DEPLOYMENT.md

File metadata and controls

A+ Deployment Instructions

Table of contents

Common system configuration

The Application

Common Shibboleth configuration

Key generation

Apache 2 configuration

Shibboleth with Apache 2

NGINX configuration

Shibboleth with NGINX

Final steps