Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add some simple end-to-end workflow with NEMO / FESOM and Pangaea #5

Open
willirath opened this issue Jun 2, 2020 · 1 comment
Open

Comments

@willirath
Copy link
Member

We should add a more realistic example here.

@benbovy
Copy link
Member

benbovy commented Jun 16, 2020

I think that the current, multi-apps architecture (xpublish's FastAPI apps + Flask app) will quickly become problematic if we want to add more realistic examples. Currently:

  1. For each dataset, we need to run a xpublish FastAPI application
  2. Additionally, we need to run a top-level FastAPI application, acting like a reverse proxy, in which we mount all single dataset applications
  3. If we want to add or remove one or more datasets, we need to manage 1 and 2 manually or automatically and restart the top-level application.
  4. The Flask application mirrors some API provided by xpublish (duplicated maintenance)
  5. Accessing the datasets in the Flask application using fsspec's HTTPFileSystem introduces overhead.

I'm not sure if there is any benefit of having all those layers here. It's probably not worth the extra complexity.

Running a single FastAPI application would be much simpler, IMO. That's not currently possible with xpublish, but it could be supported with little effort (see https://github.com/jhamman/xpublish/issues/25).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants