Strategy for testing examples with interactive app

[pawelru]

Most of our app examples looks like this:

#' app <- init(...)
#' if (interactive()) {
#'   shinyApp(app$ui, app$server)
#' }

When checking the code, only app variable will be initiated and we do not (!) run the app which will cover server code. It's like defining the function vs executing it. As a result our examples might fail and this already happened (e.g. #1046). We need to came up with the strategy how to prevent it.

I have an idea to use \dontshow{} roxygen2 syntax to put some basic tests to cover server codes against some obvious mistakes. It won't be included in the final docs but it will be run during check (see the docs). In there, we might use shinytest2 to and put some basic tests. If there will be a lot of overhead code - we can create an utility function. Just an idea - haven't really tried this (will do it tomorrow).

Please let me know your thoughts on this.

This might be relevant for modules packages as well.

[chlebowa]

I was wondering if it's possible to have an app start and kill itself after, say a second or two. That way an automated pipeline could actually run all all the apps but not capture the session indefinitely.

Anyway, I think the answer may lie in what @vedhav has been cooking up.

[vedhav]

@pawelru do check this draft PR which does a very similar thing but with cypress to do the integration tests. In fact, this helped in capturing an actual bug which is somewhere in teal.transform or packages that used its features so easy to miss without actually running the module in a teal app.

It follows these steps to do the tests:

1. Get the examples for the functions that start with tm_*. Note that this is not 100% true because I remember seeing one or two modules that were non-teal-module but had this nomenclature, we can just ignore them once we find them.
2. Run them mocking interactivity so that the apps will run. Make sure that the shiny's default port is set to 3333
3. Now that the app is running, we can trigger the cypress test and listen to this.
4. We can record the test status for every example and show them at the end.

P.S this is also part of a larger testing conversation

[pawelru]

When working on strict testing (incl. examples) I came up with mocking shiny::runApp(). Please take a look at this draft PR. My aim is to trigger discussion and compare available solutions.

[pawelru]

I'm going to close it not to duplicate the space for discussion. Let's move it to the linked issue.

[averissimo]

How about creating a new issue to leverage \dontshow to hide the if (interactive()) { on examples?

• while allowing them to still pass R CMD check?

{shiny} package still uses it, which would be a reason to leave it as is.

However, we do have the options as the diff below passes R CMD check and shows like this

diff --git a/R/init.R b/R/init.R
index 01af3c0ee..52391d08a 100644
--- a/R/init.R
+++ b/R/init.R
@@ -88,9 +88,9 @@
 #'   header = tags$h1("Sample App"),
 #'   footer = tags$p("Copyright 2017 - 2023")
 #' )
-#' if (interactive()) {
-#'   shinyApp(app$ui, app$server)
-#' }
+#' \dontshow{if (interactive()) \{ # is interactive }
+#' shinyApp(app$ui, app$server)
+#' \dontshow{ \} # is interactive }
 #'
 init <- function(data,
                  modules,

[chlebowa]

The parts in \dontshow are still run, so checking examples during R CMD CHECK will start an app that will keep running and the process will hang.