Use omero-user-token in Bio-Formats cache regeneration

[will-moore]

Workflow described at
https://github.com/IDR/deployment/blob/master/docs/operating-procedures.md#bio-formats-cache-regeneration
can result in connection issues as all the processes try to connect to OMERO at the same time. E.g. see IDR/idr-metadata#686 (comment)

One possible solution is to use https://github.com/glencoesoftware/omero-user-token so we are not creating many new sessions at the start.

[will-moore]

Ah - my bad. I checked the help options and this worked...

$ omero_user_token set -u will

token=$(omero_user_token get)
if [ $? -ne 0]; then
    echo "No valid token found"
    exit 1
fi
key=$(echo "${token}" | sed -e 's/^\(.*\)@.*:.*$/\1/')
host=$(echo "${token}" | sed -e 's/^.*@\(.*\):.*$/\1/')
port=$(echo "${token}" | sed -e 's/^.*@.*:\(.*\)$/\1/')
echo "Connecting to ${host}:${port} with key ${key}"

omero login -k $key -s $host -p $port

[will-moore]

Thinking how to apply this to the parallel-based omero render command....

Since the existing workflow connects to all 5 servers in the same way -s localhost -u public -w public this works OK:

$ screen -dmS cache parallel --eta --sshloginfile nodes -a ids.txt --results /tmp/cache/ -j10 '/opt/omero/server/OMERO.server/bin/omero render -s localhost -u public -w public test --force'

But if we create a different session on each server then we can't use a single token to connect to all the servers.

Maybe we need an updated omero render command that can use $ omero_user_token get to connect itself.

Any other ideas? cc @sbesson

[sbesson]

Double checked and you're correct, a session key is only valid within the context of a server so you'll need to create one token per node.

The workflow highlighted in #426 (comment) is certainly the way these session tokens are being used with the CLI i.e. retrieve it, split it into key vs server and consume it with omero login -k <key> -s <server> or subcommands supporting the same options. The alternative would be to wrap this logic in a Python script.

[sbesson]

Proposing to close this as per limitation mentioned above. We need to review the cache regeneration process, possibly leveraging the memo regeneration tool included in the image-region micro-service if this is in scope for an upcoming IDR release /cc @jburel