Merge pull request #30 from lquerel/26-weaver-registry-check-subcommand

Registry sub-commands

Cargo.toml

@@ -3,9 +3,9 @@ name = "weaver"
 version = "0.1.0"
 authors = ["Laurent Querel <[email protected]>"]
 edition = "2021"
-repository = "https://github.com/f5/otel-weaver"
-description = "OTel Weaver - A Schema-Driven Client SDK Generator for OpenTelemetry"
-keywords = ["opentelemetry", "client", "schema", "arrow", "generator"]
+repository = "https://github.com/open-telemetry/weaver"
+description = "Manage semantic convention registry and telemetry schema workflows (OpenTelemetry Project)"
+keywords = ["opentelemetry", "semconv", "schema", "registry", "generator"]
 categories = ["command-line-utilities"]
 license = "Apache-2.0"
 readme = "README.md"
@@ -29,11 +29,15 @@ serde = { version = "1.0.196", features = ["derive"] }
 serde_yaml = "0.9.31"
 serde_json = "1.0.113"
 thiserror = "1.0.56"
-ureq = "2.9.5"
+ureq = "2.9.6"
 regex = "1.10.3"
 rayon = "1.8.1"
 ordered-float = { version = "4.2.0", features = ["serde"] }
 
+# Features definition =========================================================
+[features]
+experimental = []
+
 # Crate definitions ===========================================================
 [[bin]]
 bench = false
@@ -49,7 +53,7 @@ weaver_semconv = { path = "crates/weaver_semconv" }
 weaver_schema = { path = "crates/weaver_schema" }
 weaver_cache = { path = "crates/weaver_cache" }
 
-clap = { version = "4.5.0", features = ["derive"] }
+clap = { version = "4.5.1", features = ["derive"] }
 crossterm = "0.27.0"
 ratatui = "0.26.1"
 tui-textarea = "0.4.0"

README.md

@@ -10,9 +10,10 @@
 
 OpenTelemetry Weaver is a CLI tool that enables users to:
 
-- Search for and retrieve information from a semantic convention registry or a telemetry schema.
-- Resolve a semantic convention registry or a telemetry schema.
-- Generate a client SDK/API from a telemetry schema.
+- Manage Semantic Convention Registries: check, generate, resolve, search, stats commands.
+- Manage Telemetry Schemas: check, generate, resolve, search, stats commands.
+
+Note: Telemetry Schema commands are only available with the --features experimental flag.
 
 ## Install
 
@@ -36,22 +37,42 @@ for debug mode or the `target/release` directory for release mode.
 ## Usage
 
 ```
+Manage semantic convention registry and telemetry schema workflows (OpenTelemetry Project)
+
 Usage: weaver [OPTIONS] [COMMAND]
 
 Commands:
-  resolve     Resolve a semantic convention registry or a telemetry schema
-  gen-client  Generate a client SDK or client API
-  languages   List all supported languages
-  search      Search in a semantic convention registry or a telemetry schema
-  help        Print this message or the help of the given subcommand(s)
+  registry  Manage Semantic Convention Registry
+  help      Print this message or the help of the given subcommand(s)
 
 Options:
   -d, --debug...  Turn debugging information on
   -h, --help      Print help
   -V, --version   Print version
 ```
 
-### Command `search`
+### Command `registry`
+
+This command provides subcommands to manage semantic convention registries.
+
+```
+Manage Semantic Convention Registry
+
+Usage: weaver registry <COMMAND>
+
+Commands:
+  check     Validates a registry (i.e., parsing, resolution of references, extends clauses, and constraints)
+  generate  Generates documentation or code for a registry (not yet implemented)
+  resolve   Resolves a registry (not yet implemented)
+  search    Searches a registry (not yet implemented)
+  stats     Calculate and display a set of general statistics on a registry (not yet implemented)
+  help      Print this message or the help of the given subcommand(s)
+
+Options:
+  -h, --help  Print help
+```
+
+### Command `search` (Experimental)
 
 This command provides an interactive terminal UI, allowing users to search for
 attributes and metrics specified within a given semantic convention registry or
@@ -74,7 +95,7 @@ This search engine leverages [Tantivy](https://github.com/quickwit-oss/tantivy)
 and supports a simple [search syntax](https://docs.rs/tantivy/latest/tantivy/query/struct.QueryParser.html)
 in the search bar.
 
-### Command `resolve`
+### Command `resolve` (Experimental)
 
 This command resolves a schema or a semantic convention registry (not yet
 implemented) and displays the result on the standard output.
@@ -91,7 +112,7 @@ A "resolved schema" is one where:
 - All overrides have been applied.
 - This resolved schema is what the code generator and upcoming plugins utilize.
 
-### Command `gen-client`
+### Command `gen-client` (Experimental)
 
 This command generates a client SDK from a telemetry schema for a given language
 specified with the `--language` option.
@@ -103,7 +124,7 @@ weaver gen-client --schema telemetry-schema.yaml --language go
 In the future, users will be able to specify the protocol to use for the generated
 client SDK (i.e. OTLP or OTel Arrow Protocol) and few others options.
 
-### Command `languages`
+### Command `languages` (Experimental)
 
 This command displays all the languages for which a client SDK/API can
 be generated.

crates/weaver_resolver/README.md

@@ -11,4 +11,34 @@ validate telemetry data, generate code, and perform other tasks.
 Important Note: Currently, 2 versions of the resolution process are present in
 the `weaver_resolver` crate. The first version of the resolution process is
 incomplete and still used by the `weaver` CLI. A second version is under active
-development and is expected to replace the first version in the near future.
+development and is expected to replace the first version in the near future.
+
+## Semantic Conventions - Parsing, Resolution and Validation
+
+The parsing is implemented using the serde library. A collection of
+serde-annotated Rust structures is used to represent semantic convention
+entities. Optional fields are represented as `Option` types.
+
+The attribute macro `#[serde(deny_unknown_fields)]` is used to ensure that
+unknown fields are not allowed in the semantic convention entities. This,
+combined with the distinction between optional and required fields in the
+entities, ensures that the semantic conventions are validated in terms of
+structure during the parsing process.
+
+The resolution process for semantic conventions is a multistep process that
+involves the following steps:
+- Load all semantic conventions from the registry
+- Resolve iteratively all semantic conventions. This involves the maintenance
+  of an unresolved semantic convention list and a resolved semantic convention
+  list. The resolution process involves the following steps:
+  - Resolve iteratively all attributes `ref` until no more resolvable `ref` are
+    found.
+  - Resolve iteratively all `extends` parent/child clauses until no more
+    resolvable `extends` are found. The extended entity inherits prefix,
+    attributes, and constraints from the parent entity.
+- Apply constraints `any_of` and `include` (not yet supported).
+- Validate the resolved semantic conventions
+  - No more unresolved `ref` or `extends` clauses. The unresolved list should
+    be empty.
+  - All constraints satisfied.
+

crates/weaver_resolver/data/registry-test-1-single-attr-ref/expected-registry.json

@@ -1,5 +1,5 @@
 {
-  "registry_url": "https://semconv-registry.com",
+  "registry_url": "https://127.0.0.1",
   "groups": [
     {
       "id": "metric.messaging.attributes",