bmdf

Bash markdown fences: embed commands, their output, and tables of contents in markdown

• Install
• Usage
  • bmd: format bash command and output as Markdown
    • bmdf (bmd -f): command+output mode
    • bmdff (bmd -ff): two-fence mode
    • bmdfff (bmd -fff): <details> mode
    • Piping
    • Env vars
    • -w/--workdir / $BMDF_WORKDIR
  • mdcmd: update commands embedded in Markdown files
  • mktoc: update table of contents
  • Reference
• Examples

This package provides 3 CLIs:

• bmd: run Bash commands, wrap output for Markdown embedding
  • bmdf, bmdff, bmdfff: different levels of "fencing" for command output
• mdcmd: find bmd blocks in Markdown files, execute commands, update Markdown files with output
• mktoc: update Markdown table of contents (with custom "id" anchors)

Install

pip install bmdf

Usage

bmd: format bash command and output as Markdown

bmd (and aliases bmdf, bmdff, bmdfff) takes a bash command as input, and renders the command and/or its output in various Markdown-friendly formats:

bmdf (bmd -f): command+output mode

Suppose you want to embed a command and its output in a README.md, like this:

seq 3
# 1
# 2
# 3

(Note how the command is bash-highlighted, and output lines are rendered as comments)

Put a placeholder like this in your README.md:

<!-- `bmdf seq 3` -->

then run mdcmd to update your README containing this embedded command block.

bmdff (bmd -ff): two-fence mode

bmdff (alias for bmd -ff) renders two code fences, one with the Bash command (syntax-highlighted appropriately), and a second (non-highlighted) block with the output, e.g.:

<!-- `bmdff seq 5` -->

becomes:

seq 5

1
2
3
4
5

bmdfff (bmd -fff): <details> mode

When a command's output is large, rendering it as a <details><summary> (with the output collapsed, by default) may be preferable.

bmdfff (3 fs, alias for bmd -fff) transforms placeholders like this:

<!-- `bmdfff seq 10` -->

to:

seq 10

1
2
3
4
5
6
7
8
9
10

Piping

Piping works too, e.g.:

<!-- `bmdf -- seq 10 | wc -l` -->

will become:

seq 10 | wc -l
# 10

(the -- is needed so that that -l isn't parsed as an opt to bmdf)

Env vars

By default, shell=True is passed to subprocess calls (but can be disabled via -S).

This means env vars are expanded; they can also be set via -E, e.g.:

<!-- `bmdf -E FOO=bar echo $FOO` -->

yields:

FOO=bar echo '$FOO'
# bar

More examples of quoting/splitting behavior

Quoting "$FOO":

<!-- `bmdf -E FOO=bar echo "$FOO"` -->

yields:

FOO=bar echo '$FOO'
# bar

Arg with spaces:

<!-- `bmdf -E FOO=bar echo "FOO: $FOO"` -->

yields:

FOO=bar echo 'FOO: $FOO'
# FOO: bar

Escaping $:

<!-- `bmdf -E FOO=bar echo "\$FOO=$FOO"` -->

yields:

FOO=bar echo '\$FOO=$FOO'
# $FOO=bar

-w/--workdir / $BMDF_WORKDIR

By default, bmdf runs in the current working directory. This can be overridden with -w:

<!-- `bmdf -w .github ls` -->

ls
# workflows

mdcmd: update commands embedded in Markdown files

# Modify README.md in-place
mdcmd -i README.md
# Same as above; no args defaults to `-i README.md`
mdcmd

The placeholder block above will now contain seq 3 and its output; that's how first block above is rendered!

The full README.md block will now look like:

<!-- `bmdf seq 3` -->
```bash
seq 3
# 1
# 2
# 3
```

and running mdcmd again will rewrite the same content.

Note: bmdf (alias for bmd -f) is used because it wraps the output of whatever it's passed in a "Bash fence" block. You don't have to use it, but most commands will fail to output a Markdown "fence" block, and subsequent mdcmd invocations will fail to parse them.

mktoc: update table of contents

Put a block like this in your README.md:

<!-- toc -->
<!-- /toc -->

Then put empty <a> tags next to the headings you want to include, e.g.:

## My section heading <a id="my-section"></a>

(This allows for customizing and shortening the ids, as well as skipping sections)

Then run:

# Modify README.md in-place
mktoc -i README.md
# Same as above; no args defaults to `-i README.md`
mktoc

And the <!-- toc --> section will have a table of contents injected (like the one at the top of this file).

Reference

bmd
# Usage: bmd [OPTIONS] COMMAND...
#
#   Format a command and its output to markdown, either in a `bash`-fence or
#   <details> block, and copy it to the clipboard.
#
# Options:
#   -A, --strip-ansi                Strip ANSI escape sequences from output
#   -C, --no-copy                   Disable copying output to clipboard
#                                   (normally uses first available executable
#                                   from ['pbcopy', 'xclip', 'clip']
#   -e, --error-fmt TEXT            If the wrapped command exits non-zero,
#                                   append a line of output formatted with this
#                                   string. One "%d" placeholder may be used,
#                                   for the returncode. Defaults to
#                                   $BMDF_ERR_FMT
#   -E, --env TEXT                  k=v env vars to set, for the wrapped command
#   -f, --fence                     Pass 0-3x to configure output style: 0x:
#                                   print output lines, prepended by "# "; 1x:
#                                   print a "```bash" fence block including the
#                                   <command> and commented output lines; 2x:
#                                   print a bash-fenced command followed by
#                                   plain-fenced output lines; 3x: print a
#                                   <details/> block, with command <summary/>
#                                   and collapsed output lines in a plain fence.
#   -i, --include-stderr / -I, --no-include-stderr
#                                   Capture and interleave both stdout and
#                                   stderr streams; falls back to
#                                   $BMDF_INCLUDE_STDERR
#   -s, --shell / -S, --no-shell    Disable "shell" mode for the command; falls
#                                   back to $BMDF_SHELL, but defaults to True if
#                                   neither is set
#   -t, --fence-type TEXT           When -f/--fence is 2 or 3, this customizes
#                                   the fence syntax type that the output is
#                                   wrapped in
#   -u, --expanduser / -U, --no-expanduser
#                                   Pass commands through `os.path.expanduser`
#                                   before `subprocess`; falls back to
#                                   $BMDF_EXPANDUSER
#   -v, --expandvars / -V, --no-expandvars
#                                   Pass commands through `os.path.expandvars`
#                                   before `subprocess`; falls back to
#                                   $BMDF_EXPANDVARS
#   -w, --workdir TEXT              `cd` to this directory before executing
#                                   (falls back to $BMDF_WORKDIR
#   -x, --executable TEXT           `shell_executable` to pass to Popen
#                                   pipelines (default: $SHELL)
#   --help                          Show this message and exit.

mdcmd --help
# Usage: mdcmd [OPTIONS] [PATH] [OUT_PATH]
#
#   Parse a Markdown file, updating blocks preceded by <!-- `[cmd...]` -->
#   delimiters.
#
#   If no paths are provided, will look for a README.md, and operate "in-place"
#   (same as ``mdcmd -i README.md``).
#
# Options:
#   -a, --amend                     Squash changes onto the previous Git commit;
#                                   suitable for use with `git rebase -x`
#   -i, --inplace / -I, --no-inplace
#                                   Edit the file in-place
#   -n, --dry-run                   Print the commands that would be run, but
#                                   don't execute them
#   -T, --no-cwd-tmpdir             In in-place mode, use a system temporary-
#                                   directory (instead of the current workdir,
#                                   which is the default)
#   -x, --execute TEXT              Only execute commands that match these
#                                   regular expressions
#   -X, --exclude TEXT              Only execute commands that don't match these
#                                   regular expressions
#   --help                          Show this message and exit.

mktoc --help
# Usage: mktoc [OPTIONS] [PATH] [OUT_PATH]
#
#   Insert a table of contents (TOC) in a markdown file.
#
#   Looks for a pair of sentinel lines to insert or update the TOC between:
#   ``<!-- toc -->``, ``<!-- /toc -->``.
#
#   If an empty line follows the opening ``<!-- toc -->`` line, the TOC will be
#   inserted there (along with the closing sentinel); this is useful when
#   initially generating a TOC.
#
#   If no ``out_path`` is provided, will operate "in-place" on ``README.md`` (as
#   if ``mktoc -i README.md`` was passed).
#
# Options:
#   -a, --amend                     Squash changes onto the previous Git commit;
#                                   suitable for use with `git rebase -x`
#   -i, --inplace / -I, --no-inplace
#                                   Edit the file in-place
#   -n, --indent-size INTEGER       Indent size (spaces)
#   -T, --no-cwd-tmpdir             In in-place mode, use a system temporary-
#                                   directory (instead of the current workdir,
#                                   which is the default)
#   --help                          Show this message and exit.

(these blocks are self-hosted, using bmdf and mdcmd; likewise the table of contents up top, via mktoc 😎)

Examples

These repos' READMEs use bmdf / mdcmd / mktoc to execute README examples (and in some cases are verified by GitHub Actions):

• runsascoded/juq
• runsascoded/utz
• runsascoded/dvc-utils
• ryan-williams/dvc-helpers (GHA)
• ryan-williams/git-helpers
• ryan-williams/parquet-helpers (GHA)
• ryan-williams/tdbs-dask
• TileDB-Inc/scverse-ml-workshop-2024