[Fuseki] Reloadable configuration files

[afs]

Version

5.0.0 dev

Feature

Add a server action that reloads the services defined in a configuration file and replace the server data services configuration with zero-downtime, no HTTP request lost or aborted while the server is running.

This allows:

1. Reloading an in-memory graph with data from a file.
2. Change the location of a TDB2 database
3. Change of database configuration (e.g. (re)setting per dataset or per-endpoint query timeouts)

This is a subset of the configuration file capabilities. Some changes still require a server restart.

Limitation: transient data (in-memory, data loaded via HTTP) may be lost.

Are you interested in contributing a solution yourself?

Yes

[flange-ipb]

Hello Andy,

this feature is very welcome, thanks for implementing!

Some ideas:

• As an alternative to calling the reload endpoint, should it also be possible to send a signal to the Java process to achieve the configuration reload? That's typically done with SIGHUP.
• Let's assume the configuration for text indexing has changed. It should be possible to rerun text indexing. That's probably out of scope of this issue, but maybe an idea for another endpoint?

[namedgraph]

Can't Fuseki simply detect when the config file changes? I think ontop is able to do that.
Calling an endpoint seems a bit like overkill here.

Looks like something called WatchService can detect file changes: https://docs.oracle.com/javase/tutorial/essential/io/notification.html

[flange-ipb]

WatchService could be unreliable, especially for non-local file systems. It seems there are even problems inside Docker containers. 😢

[rvesse]

I've used Apache Commons IO Monitor APIs for watching the configuration file in the past. Though as @flange-ipb says it can be quite unreliable especially when running in a containerised environment

For example in a K8S environment where the config file is coming from a ConfigMap the way K8S actually mounts the file into the container means that the actual "file" ends up being a symbolic link and there's a hidden sub-directory lurking with the actual current version of the mapped file(s) from the ConfigMap bind mounted into your container

[rvesse]

• Let's assume the configuration for text indexing has changed. It should be possible to rerun text indexing. That's probably out of scope of this issue, but maybe an idea for another endpoint?

That functionality could be achieved by creating a custom FusekiModule though not sure if those are going to get called again when reloading a server or not, probably a question for @afs