nbstata compiles document but does not work in interactive mode in quarto documents (in RStudio)

[avila]

Description

I was trying to use nbstata interactively in RStudio but I cant get it to work as expected. Based on the example document from friosavila. First the chunks never finish running, and if I click the Stop button, then it returns an error (see screenshots below for more context).

The error is because the do command before the filename is missing. But one could expect the error to be returned more quickly.

. /tmp/Rtmpc5adhy/chunk-code-9d3d86c3c832d.txt 
/ is not a valid command name
r(199);

Click to expand (Screenshots)

The whole document is rendered fine though.

Click to expand (Screenshots)

The document was created with jupyter engine and nbstata kernel.

Click to expand (Screenshots)

I am also able to use nbstata kernel in jupyter notebooks, both interactively and also to compile the whole document. I might revert to this solution for the time being, but it would be great to have it working in Rstudio because I find it's notebook implementation very superior to the jupyter's interface.

Please let me know if there is anything I could try or am doing wrong. I am happy to debug and then document it for future users.

System

• Linux fedora 6.6.3-200.fc39.x86_64 SMP PREEMPT_DYNAMIC Tue Nov 28 19:11:52 UTC 2023 x86_64 GNU/Linux
• $mode [1] "desktop"
• $version [1] ‘2023.9.1.494’
• $long_version [1] "2023.09.1+494"
• $release_name [1] "Desert Sunflower"

quarto check output

Click to expand (Long output)

[✓] Checking versions of quarto binary dependencies...
Pandoc version 3.1.1: OK
Dart Sass version 1.55.0: OK
[✓] Checking versions of quarto dependencies......OK
[✓] Checking Quarto installation......OK
Version: 1.3.450
Path: /home/avila/opt/quarto-1.3.450/bin

[✓] Checking basic markdown render....OK

[✓] Checking Python 3 installation....OK
Version: 3.12.0
Path: /usr/bin/python3
Jupyter: 5.5.0
Kernels: python3, nbstata

() Checking Jupyter engine render....0.00s - Debugger warning: It seems that frozen modules are being used, which may
0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off
0.00s - to python to disable frozen modules.
0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation.
0.00s - Debugger warning: It seems that frozen modules are being used, which may
0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off
0.00s - to python to disable frozen modules.
0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation.
[✓] Checking Jupyter engine render....OK

[✓] Checking R installation...........OK
Version: 4.3.2
Path: /usr/lib64/R
LibPaths:
- /home/avila/R/x86_64-redhat-linux-gnu-library/4.3
- /usr/local/lib/R/library
- /usr/lib64/R/library
- /usr/share/R/library
knitr: 1.45
rmarkdown: 2.25

[✓] Checking Knitr engine render......OK

[hugetim]

Thank you for reporting. I think that when you run stata interactively in RStudio, it's not actually using nbstata.[1] I bet you're using the knitr version of Stata instead (about which I only have a vague awareness). Maybe Stata 18 broke something about how knitr stata worked? I think you should instead report this to the knitr project (but do report back if you would).[2]

By contrast, according to my hypothesis, when you render using Quarto, it is using nbstata and working.

[1] One clue is that the Stata "splash," that several-line display of the Stata version and so on, is disabled by default in nbstata. It's also hard for me to imagine how nbstata could trigger an error like that.
[2] An additional piece of info that may be helpful to them is the Stata revision date.

[avila]

Thank you for the quick reply. Although that is not the exact issue, it did point me in the right direction. I am not very familiar with quarto but it does seem that in interactive mode, knitr is doing the job of executing the cells. I'll try to illustrate the issue here, as it might be helpful to others (or at least me in the future!)

It seems that knitr creates a temporary sh file with the following content and tries to call that from the default shell.

{engine} /tmp/tempfilename.txt

So it enters Stata in normal (not batch mode) and it never returns any output or error until I cancel the call via the stop button, when I can then see Stata's splash and the error (/ is not a valid command). See, for example, if I rename the engine to something random:

Having said that, there is indeed a knitr engine called stata. So I don't know why that is not being called. I did set jupyter (and nbstata as the kernel) when creating the document (see next screenshot).

Having said all that, it is very likely that the issue is either with quarto or with knitr. I'll try to figure it out, and I'll link back to this issue any relevant info I find.

---

By the way (not relevant to this issue) I also opened an issue in quarto's repository asking about support for nbstata. Is there any documentation on using nbstata with quarto (from VSCode or any other editor?).

[avila]

One possible workaround would be setting a custom knitr engine and trying to make sure it points to python (jupyter) and uses nbstata. (here for context: https://bookdown.org/yihui/rmarkdown-cookbook/custom-engine.html)

Disclaimer: that I have a very shallow understanding of how knitr and jupyter kernels work.

And here is a minumum reproducible example of a document.

---
title: "nbstata"
format: html
jupyter: nbstata
---

## nbstata 

```{stata}
di 1 + 1 
```

Note that the document does not include the usual knitr setup chunk that a Rmarkdown usually includes.

[hugetim]

Thank you, @avila, for that quarto issue which led to nbstata support in the Quarto VSCode extension.

I'd like to add further documentation directing people toward that extension (and also how to use nbstata with RStudio if there is a way), but I won't have time to work on it myself until July at the earliest.

[avila]

That's great that quarto is now supporting nbstata! I just did a quick test with simple documents, and it ran fine on VSCode (interactive execution and compilation).

On RStudio the conflict between the default stata engine (from knitr) and nbstata via jupyter is still present. However! I found out a few weeks ago that due to Fedora's packaging guidelines, quarto is not bundled-up with RStudio as distributed by Fedora (see: rstudio/rstudio#12609). So I am not sure if the conflict is only in Fedora distributions or a more generic issue.

In any case, a quarto document with nbstata kernel can be compiled with quarto::quarto_render("file.qmd") from R or with quarto render file.qmd from the terminal. The source file can be edited from RStudio but (at least in my system) interactive code execution does not work. If I get the chance to play with another distribution, I'll make sure to test it and report back.