Jobqueue-specific Dask examples?

[willirath]

Are there any existing Dask examples that specifically focus on dask-jobqueue?

What I have in mind is something for users with some familiarity with Dask, who want to learn how to best decide about cluster specs on HPC. (I know this is a wide field!)

[guillaumeeb]

Hi @willirath, just mentioning what I use currently as we discussed:

• Something very similar to https://github.com/dask/dask-examples/blob/master/applications/embarrassingly-parallel.ipynb, but with dask-jobqueue and some adaptive show.
• A Dataframe example based on the time series generation tooling included in dask or distributed (don't remember where I took it).
• I've copied some Pangeo cloud examples and data, notably https://github.com/pangeo-data/pangeo-example-notebooks/blob/master/sea-surface-height.ipynb.
• I really like your Pi example : Add adaptive example of a Monte Carlo estimate pangeo-data/pangeo-example-notebooks#19

I'm curious if there is some material from NCAR (cc @kmpaul @jhamman @andersy005 @mnlevy1981) or others?

[guillaumeeb]

Just came into @pbranson work here: https://github.com/pbranson/c3dis-2019-pangeo, maybe there some interesting things adapted to HPC setup.