Galaxy deployment options

[sebastian-luna-valero]

Hi @micafer

@bgruening is currently updating the docker-galaxy repository.

I see there are different templates and artifacts in IM (including an Ansible role) to deploy Galaxy and I am opening this thread to discuss how to converge/update the available options to deploy Galaxy with IM.

Initially I am testing the docker-compose TOSCA template in IM to deploy galaxy using docker-compose.

However, we may want to add new TOSCA templates to give Galaxy users the option to configure certain parameters.

What do you think?

[micafer]

It is a good idea.

[bgruening]

The ansible deployment is most likely more scalable I would assume. The single-docker container is meant for smaller groups or single-user deployments.

The cool thing with this container is, we can configure it completely with ENV vars. So if we could allow to set ENV vars in the form this would give this container a LOT of flexibility already.

[sebastian-luna-valero]

Thanks both.

@bgruening the main idea with this repo is to serve smaller groups and/or single-user deployments, right? In other words, should we then focus on the docker-compose deployment or on the ansible deployment?

Indeed we can configure the TOSCA template to capture user input and put them in ENV vars for the deployment of the container.

[bgruening]

@bgruening the main idea with this repo is to serve smaller groups and/or single-user deployments, right? In other words, should we then focus on the docker-compose deployment or on the ansible deployment?

I don't know. It was probably a lot of work to set this up and I don't want to abandon this. The Docker one is in some way not very best-practise. But it is so easy to modify and maintain...

[sebastian-luna-valero]

I would like to focus first on the docker-galaxy deployment with docker compose. We can look at the ansible deployment later on.

I had issues deploying docker-galaxy with the existing docker compose template in IM and I proposed a fix here: grycap/tosca#215

The cool thing with this container is, we can configure it completely with ENV vars. So if we could allow to set ENV vars in the form this would give this container a LOT of flexibility already.

@bgruening could you please provide the list of ENV vars that we need to show to the user in the IM Dashboard, and what should be their values? I would like to create an initial template to implement this.

[bgruening]

Excellent question ... I don't know :) Everything can be configured with it. Can we make it a free text field?

Maybe we can start with:

https://github.com/bgruening/docker-galaxy?tab=readme-ov-file#magic-environment-variables--toc
https://github.com/bgruening/docker-galaxy?tab=readme-ov-file#https-support--toc

GALAXY_CONFIG_BRAND
PROXY_PREFIX
NONUSE
GALAXY_CONFIG_ENABLE_MULLED_CONTAINERS

[micafer]

Hi @sebastian-luna-valero

In current template:
https://github.com/grycap/tosca/blob/main/templates/galaxy.yaml

I can add this env variables as input values to improve it.

[sebastian-luna-valero]

Ah! I wasn't following your updates to the ansible role for Galaxy!

That's great, then please update the current template and I will stop my work on it.

Many thanks Miguel!

[micafer]

Current template already uses last version of the @bgruening galaxy docker image.

[sebastian-luna-valero]

Yes, I checked.

I think what remains now is to capture the input from the user in the IM dashboard for the envvars stated above by @bgruening

[sebastian-luna-valero]

In addition to the envvars stated above, I think it would be also nice to allow the user to select:

• DOCKER_REGISTRY
• DOCKER_REGISTRY_USERNAME
• IMAGE_TAG

With defaults

• DOCKER_REGISTRY=quay.io
• DOCKER_REGISTRY_USERNAME=bgruening
• IMAGE_TAG=latest

xref: https://github.com/bgruening/docker-galaxy/blob/1b33426c90f27cad7e9a3ffecdb5c7f822e1ea41/galaxy/docker-compose.yaml#L5

I will then close my ongoing PR grycap/tosca#215 so instead of the docker-compose template we use the updated galaxy one.

[micafer]

I really prefer not to set the input values, because I think is better to restrict the image to the one build by @bgruening that we know that it works.
We could add the "IMAGE_TAG", enabling the user to select the version of Galaxy to use, but the rest of values I think can confuse the user.

[micafer]

Added Brand and version input values.

[sebastian-luna-valero]

Thanks!

A first test shows this

galaxy_portal  | Traceback (most recent call last):
galaxy_portal  |   File "/usr/local/bin/check_database.py", line 19, in <module>
galaxy_portal  |     mapping = init('/tmp/', db_url)
galaxy_portal  |   File "/galaxy/lib/galaxy/model/mapping.py", line 45, in init
galaxy_portal  |     engine = build_engine(
galaxy_portal  |   File "/galaxy/lib/galaxy/model/orm/engine_factory.py", line 111, in build_engine
galaxy_portal  |     engine = create_engine(url, **engine_options)
galaxy_portal  |   File "<string>", line 2, in create_engine
galaxy_portal  |   File "/galaxy_venv/lib/python3.10/site-packages/sqlalchemy/util/deprecations.py", line 281, in warned
galaxy_portal  |     return fn(*args, **kwargs)  # type: ignore[no-any-return]
galaxy_portal  |   File "/galaxy_venv/lib/python3.10/site-packages/sqlalchemy/engine/create.py", line 599, in create_engine
galaxy_portal  |     dbapi = dbapi_meth(**dbapi_args)
galaxy_portal  |   File "/galaxy_venv/lib/python3.10/site-packages/sqlalchemy/dialects/postgresql/psycopg2.py", line 690, in import_dbapi
galaxy_portal  |     import psycopg2
galaxy_portal  | ModuleNotFoundError: No module named 'psycopg2'
galaxy_portal  | Waiting for database

and Galaxy does not complete the start up.

Could you please have a look?

[micafer]

I tested twice and it worked for me.
What O.S. are you using?

[sebastian-luna-valero]

I have reproduced the error in my second attempt.

I am currently using https://appdb.egi.eu/store/vappliance/egi.ubuntu.22.04

Sorry, is that the reason?

[micafer]

I tried in IISAS site with this image and it worked.
Try again, and if it fails, send me the details of the VM by mail.

[sebastian-luna-valero]

This worked for me: grycap/ansible-role-galaxy#59

Could you please have a look and confirm?

[micafer]

I do not have access to CESNET site, could you try now?

[sebastian-luna-valero]

Great, it works!

Many thanks for your help @micafer

@bgruening would you like to run a test and check what else would you like to see? Happy to follow up via PM in Matrix.