speed up requests

[maxheld83]

the requests are too damn slow, at least an order of magnitude slower than necessary as per the API I think.

Gotta do some profiling to figure out what's going on here.

[maxheld83]

@njahn82 with caching now implemented, we're back to manageable runtimes:

bench::mark(metacheck = metacheck::render_email(sample(metacheck:::dois_many(), 10)))
#> Warning: Some expressions had a GC in every iteration; so filtering is disabled.
#> # A tibble: 1 x 6
#>   expression      min   median `itr/sec` mem_alloc `gc/sec`
#>   <bch:expr> <bch:tm> <bch:tm>     <dbl> <bch:byt>    <dbl>
#> 1 metacheck     23.8s    23.8s    0.0419    68.3MB    0.545

^{Created on 2021-03-08 by the reprex package (v1.0.0)}

This scales up to 40mins for 1000 DOIs.

This is still not great.

Next steps would be:

• also cache doi.org (some, but not great impact probably, https://github.com/subugoe/biblids/issues/20)
• profile rcrossref calls (what makes them slow? Is it really just the API?)
• stick very close to the advertised crossref rate limit respect crossref rate limit #173
• run parallel and/or async httr requests (if allowed)

[njahn82]

Thanks. Do you have an example, how biblids::is_doi_found and biblids::is_doi_resolvable differ. They seem somehow redundant to me as DOIs must resolve to an URL. Maybe one function / api call testing if a DOI resolves is enough.

[maxheld83]

Thanks. Do you have an example, how biblids::is_doi_found and biblids::is_doi_resolvable differ. They seem somehow redundant to me as DOIs must resolve to an URL. Maybe one function / api call testing if a DOI resolves is enough.

Moving discussion to #190.

[maxheld83]

doi.org calls are now also cached on disc, should speed up by ~20%.