Skip to content

Commit

Permalink
Vignette: Use I(.) for manually tested clusters [#725]
Browse files Browse the repository at this point in the history
  • Loading branch information
HenrikBengtsson committed Jun 30, 2024
1 parent 8935f1b commit 97eaf42
Show file tree
Hide file tree
Showing 3 changed files with 9 additions and 9 deletions.
4 changes: 2 additions & 2 deletions DESCRIPTION
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
Package: future
Version: 1.33.2-9007
Version: 1.33.2-9008
Title: Unified Parallel and Distributed Processing in R for Everyone
Imports:
digest,
Expand Down Expand Up @@ -40,5 +40,5 @@ ByteCompile: TRUE
URL: https://future.futureverse.org, https://github.com/HenrikBengtsson/future
BugReports: https://github.com/HenrikBengtsson/future/issues
Encoding: UTF-8
RoxygenNote: 7.3.1
RoxygenNote: 7.3.2
Roxygen: list(markdown = TRUE)
2 changes: 1 addition & 1 deletion pkgdown/_pkgdown.yml.rsp
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ url: https://<%= pkg %>.futureverse.org
home:
links:
- text: Roadmap/Milestones
href: https://github.com/HenrikBengtsson/<%= pkg %>/milestones
href: https://github.com/<%= gsub("(^.*:|[.]git$)", "", subset(gert::git_remote_list(), name == "origin")$url) %>/milestones
- text: The Futureverse Project
href: https://www.futureverse.org/
- text: Futureverse User Forum
Expand Down
12 changes: 6 additions & 6 deletions vignettes/future-3-topologies.md.rsp
Original file line number Diff line number Diff line change
Expand Up @@ -102,15 +102,15 @@ Although this does not give an error, we will find that the inner layer of futur
Now, we could imagine that we process the outer layer with, say, two parallel futures, and then the inner layer with four parallel futures. In that case, we would end up running on at most eight cores (= 2 * 4). This can be achieved by forcing a fixed number of workers at each layer:

```r
plan(list(tweak(multisession, workers = 2), tweak(multisession, workers = 4)))
plan(list(tweak(multisession, workers = 2), tweak(multisession, workers = I(4))))
```

When using this approach, there is a risk of setting up too many concurrent workers. To make sure the setup respects `availableCores()`, use something like:
When using this approach, there is a risk of setting up too many concurrent workers. Because Futureverse has a built-in protection, we need to declare nested workers using the As-Is `I(.)` function, which basically tells the parallel framework "trust us, we know what we are doing". To minimize the risk of mistakes and to make sure our setup respects `availableCores()`, use something like:

```r
plan(list(
tweak(multisession, workers = availableCores() %/% 4),
tweak(multisession, workers = 4)
tweak(multisession, workers = I(4))
))
```

Expand All @@ -134,7 +134,7 @@ One possible downside to the above setup is that we might not utilize all availa
nodes <- rep(c("n1", "n2", "n3"), each = 8)
plan(list(
tweak(cluster, workers = nodes),
tweak(multisession, workers = 2)
tweak(multisession, workers = I(2))
))
```

Expand Down Expand Up @@ -229,7 +229,7 @@ tweaking the `multisession` plan by passing a function to `workers`;
halfCores <- function() { max(1, round(0.5 * availableCores()))
plan(list(
tweak(cluster, workers = nodes),
tweak(multisession, workers = halfCores)
tweak(multisession, workers = I(halfCores))
))
```

Expand All @@ -250,7 +250,7 @@ customWorkers <- function() {
}
plan(list(
tweak(cluster, workers = nodes),
tweak(multisession, workers = customWorkers)
tweak(multisession, workers = I(customWorkers))
))
```

Expand Down

0 comments on commit 97eaf42

Please sign in to comment.