Skip to content

Commit

Permalink
add docgen tools binary
Browse files Browse the repository at this point in the history
This commit adds a `docgen` binary to the tools crate. This tool can
generate a .json file describing the API based on parsing the `cbindgen`
generated `rustls.h` header file using a tree-sitter C grammar.

The produced JSON can in turn be used to generate markdown web
documentation, or any other format required.

Along with generating docs this tool applies some basic policy. In
particular it requires that **all** public API items be documented or it
will produce an error instead of the JSON data. We can extend this in
the future to require (for example) describing arguments and return
values in doc comments.

The generated JSON has the following structure:

```json
{
  "structs": [ .. ],
  "functions": : [ .. ],
  "callbacks": [ .. ],
  "enums": [ .. ],
  "externs": [ .. ],
  "aliases": [ .. ]
}
```

Where each key is a general category of item:

* Structure definitions
* API function prototypes
* Callback typedefs
* Enum typedefs
* Extern typedefs
* Simple type alias typdefs

Within each category items are described like:

```json
{
  "anchor": "rustls-accepted",
  "comment": "A parsed ClientHello produced by a ...",
  "feature": null,
  "name": "rustls_accepted",
  "text": "```c\nstruct rustls_accepted\n```"
},
```

The anchor field is useful for creating anchor links that identify the
item. The name is the actual name of the item. The comment is the
C block comment content (with the block comment syntax removed, but
other whitespace left as-is). If the item is surrounded by an ifdef
guard for a crate feature, then the required feature name is used
as the feature field. Lastly the text field holds the actual
text that defines the item in rustls.h (without comment) as
a C formatted markdown code block.

Backtick references in comments that match anchor names (after replacing
`_` with `-` and stripping an optional `()` suffix) are automatically
hyperlinked.
  • Loading branch information
cpu committed Dec 31, 2024
1 parent c8dbf73 commit 2c7cb79
Show file tree
Hide file tree
Showing 5 changed files with 891 additions and 4 deletions.
62 changes: 62 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 5 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -22,3 +22,8 @@ regex = "1.9.6"
toml = { version = "0.6.0", default-features = false, features = ["parse"] }
hickory-resolver = { version = "=0.25.0-alpha.4", features = ["dns-over-https-rustls", "webpki-roots"] }
tokio = { version = "1.42.0", features = ["io-util", "macros", "net", "rt"] }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
tree-sitter = "0.23" # TODO(@cpu): handle breaking API changes for 0.24
tree-sitter-c = "0.23"
tree-sitter-md = "0.3"
9 changes: 5 additions & 4 deletions tools/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,8 @@ publish = false
rustls = { workspace = true }
hickory-resolver = { workspace = true }
tokio = { workspace = true }

[[bin]]
name = "ech_fetch"
path = "src/ech_fetch.rs"
serde = { workspace = true }
serde_json = { workspace = true }
tree-sitter = { workspace = true }
tree-sitter-c = { workspace = true }
tree-sitter-md = { workspace = true, features = ["parser"] }
Loading

0 comments on commit 2c7cb79

Please sign in to comment.