Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

External requests could be async/await #44

Open
bnewbold opened this issue Jan 30, 2021 · 1 comment
Open

External requests could be async/await #44

bnewbold opened this issue Jan 30, 2021 · 1 comment
Assignees
Labels
enhancement New feature or request

Comments

@bnewbold
Copy link
Contributor

Calls to Elasticsearch, GROBID, and Fatcat API are done as regular synchronous HTTP fetches. FastAPI is an async framework, and we should be making these calls using async code paths (to be clear, meaning async/await syntax, not the /_async_search/ API).

For the citation query code path (requests to GROBID, fatcat elasticsearch, and fatcat API), plan is to move all this code into fuzzycat, and use async versions of code (eg, aiohttp for GROBID and fatcat API, async/await version of elasticsearch python library). For the primary queries, against the fulltext index, waiting on support in elasticsearch-dsl (elastic/elasticsearch-dsl-py#1480). Alternatively, we might be able to use existing elasticsearch-dsl to generate query object (as a dict), and pass that to an async elasticsearch-py function call. Not sure.

@bnewbold bnewbold added the enhancement New feature or request label Jan 30, 2021
@bnewbold bnewbold self-assigned this Jan 30, 2021
@bnewbold
Copy link
Contributor Author

Somewhat embarrassingly, the urgent aspect of this issue has been resolved by simply removing the async and await keywords from methods that make blocking calls. FastAPI will detect this and schedule the calls appropriately.

FastAPI docs: https://fastapi.tiangolo.com/async/

@bnewbold bnewbold changed the title External requests are not async External requests could be async/await Mar 30, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant