Initial skeleton of some component model processing (bytecodealliance…

…#4005)

* Initial skeleton of some component model processing

This commit is the first of what will likely be many to implement the
component model proposal in Wasmtime. This will be structured as a
series of incremental commits, most of which haven't been written yet.
My hope is to make this incremental and over time to make this easier to
review and easier to test each step in isolation.

Here much of the skeleton of how components are going to work in
Wasmtime is sketched out. This is not a complete implementation of the
component model so it's not all that useful yet, but some things you can
do are:

* Process the type section into a representation amenable for working
  with in Wasmtime.
* Process the module section and register core wasm modules.
* Process the instance section for core wasm modules.
* Process core wasm module imports.
* Process core wasm instance aliasing.
* Ability to compile a component with core wasm embedded.
* Ability to instantiate a component with no imports.
* Ability to get functions from this component.

This is already starting to diverge from the previous module linking
representation where a `Component` will try to avoid unnecessary
metadata about the component and instead internally only have the bare
minimum necessary to instantiate the module. My hope is we can avoid
constructing most of the index spaces during instantiation only for it
to all ge thrown away. Additionally I'm predicting that we'll need to
see through processing where possible to know how to generate adapters
and where they are fused.

At this time you can't actually call a component's functions, and that's
the next PR that I would like to make.

* Add tests for the component model support

This commit uses the recently updated wasm-tools crates to add tests for
the component model added in the previous commit. This involved updating
the `wasmtime-wast` crate for component-model changes. Currently the
component support there is quite primitive, but enough to at least
instantiate components and verify the internals of Wasmtime are all
working correctly. Additionally some simple tests for the embedding API
have also been added.

.github/workflows/main.yml

@@ -137,7 +137,9 @@ jobs:
     - run: cargo check -p wasmtime --no-default-features --features pooling-allocator
     - run: cargo check -p wasmtime --no-default-features --features cranelift
     - run: cargo check -p wasmtime --no-default-features --features wasm-backtrace
+    - run: cargo check -p wasmtime --no-default-features --features component-model
     - run: cargo check -p wasmtime --no-default-features --features cranelift,wat,async,cache,wasm-backtrace
+    - run: cargo check --features component-model
 
     # Check that benchmarks of the cranelift project build
     - run: cargo check --benches -p cranelift-codegen
@@ -297,6 +299,10 @@ jobs:
       env:
         RUST_BACKTRACE: 1
 
+    # Test the component-model related functionality which is gated behind a
+    # compile-time feature
+    - run: cargo test --features component-model
+
   # Build and test the wasi-nn module.
   test_wasi_nn:
     name: Test wasi-nn module

Cargo.toml

@@ -109,6 +109,7 @@ pooling-allocator = ["wasmtime/pooling-allocator", "wasmtime-cli-flags/pooling-a
 all-arch = ["wasmtime/all-arch"]
 posix-signals-on-macos = ["wasmtime/posix-signals-on-macos"]
 wasm-backtrace = ["wasmtime/wasm-backtrace", "wasmtime-cli-flags/wasm-backtrace"]
+component-model = ["wasmtime/component-model", "wasmtime-wast/component-model", "wasmtime-cli-flags/component-model"]
 
 # Stub feature that does nothing, for Cargo-features compatibility: the new
 # backend is the default now.

build.rs

@@ -28,6 +28,9 @@ fn main() -> anyhow::Result<()> {
             test_directory_module(out, "tests/misc_testsuite/simd", strategy)?;
             test_directory_module(out, "tests/misc_testsuite/threads", strategy)?;
             test_directory_module(out, "tests/misc_testsuite/memory64", strategy)?;
+            if cfg!(feature = "component-model") {
+                test_directory_module(out, "tests/misc_testsuite/component-model", strategy)?;
+            }
             Ok(())
         })?;
 

crates/cli-flags/Cargo.toml

@@ -26,3 +26,4 @@ default = [
 pooling-allocator = []
 memory-init-cow = []
 wasm-backtrace = []
+component-model = []

crates/cli-flags/src/lib.rs

@@ -39,6 +39,8 @@ pub const SUPPORTED_WASM_FEATURES: &[(&str, &str)] = &[
     ("simd", "enables support for proposed SIMD instructions"),
     ("threads", "enables support for WebAssembly threads"),
     ("memory64", "enables support for 64-bit memories"),
+    #[cfg(feature = "component-model")]
+    ("component-model", "enables support for the component model"),
 ];
 
 pub const SUPPORTED_WASI_MODULES: &[(&str, &str)] = &[
@@ -333,6 +335,8 @@ impl CommonOptions {
             threads,
             multi_memory,
             memory64,
+            #[cfg(feature = "component-model")]
+            component_model,
         } = self.wasm_features.unwrap_or_default();
 
         if let Some(enable) = simd {
@@ -358,6 +362,10 @@ impl CommonOptions {
         if let Some(enable) = memory64 {
             config.wasm_memory64(enable);
         }
+        #[cfg(feature = "component-model")]
+        if let Some(enable) = component_model {
+            config.wasm_component_model(enable);
+        }
     }
 
     pub fn opt_level(&self) -> wasmtime::OptLevel {
@@ -391,6 +399,8 @@ pub struct WasmFeatures {
     pub threads: Option<bool>,
     pub multi_memory: Option<bool>,
     pub memory64: Option<bool>,
+    #[cfg(feature = "component-model")]
+    pub component_model: Option<bool>,
 }
 
 fn parse_wasm_features(features: &str) -> Result<WasmFeatures> {
@@ -439,6 +449,8 @@ fn parse_wasm_features(features: &str) -> Result<WasmFeatures> {
         threads: all.or(values["threads"]),
         multi_memory: all.or(values["multi-memory"]),
         memory64: all.or(values["memory64"]),
+        #[cfg(feature = "component-model")]
+        component_model: all.or(values["component-model"]),
     })
 }
 

crates/cranelift/src/compiler.rs

@@ -28,8 +28,8 @@ use std::mem;
 use std::sync::Mutex;
 use wasmtime_environ::{
     AddressMapSection, CompileError, FilePos, FlagValue, FunctionBodyData, FunctionInfo,
-    InstructionAddressMap, Module, ModuleTranslation, StackMapInformation, Trampoline, TrapCode,
-    TrapEncodingBuilder, TrapInformation, Tunables, TypeTables, VMOffsets,
+    InstructionAddressMap, Module, ModuleTranslation, ModuleTypes, StackMapInformation, Trampoline,
+    TrapCode, TrapEncodingBuilder, TrapInformation, Tunables, VMOffsets,
 };
 
 /// A compiler that compiles a WebAssembly module with Compiler, translating
@@ -109,7 +109,7 @@ impl wasmtime_environ::Compiler for Compiler {
         func_index: DefinedFuncIndex,
         mut input: FunctionBodyData<'_>,
         tunables: &Tunables,
-        types: &TypeTables,
+        types: &ModuleTypes,
     ) -> Result<Box<dyn Any + Send>, CompileError> {
         let isa = &*self.isa;
         let module = &translation.module;
@@ -237,7 +237,7 @@ impl wasmtime_environ::Compiler for Compiler {
     fn emit_obj(
         &self,
         translation: &ModuleTranslation,
-        types: &TypeTables,
+        types: &ModuleTypes,
         funcs: PrimaryMap<DefinedFuncIndex, Box<dyn Any + Send>>,
         tunables: &Tunables,
         obj: &mut Object<'static>,

crates/cranelift/src/func_environ.rs

@@ -16,8 +16,8 @@ use std::convert::TryFrom;
 use std::mem;
 use wasmparser::Operator;
 use wasmtime_environ::{
-    BuiltinFunctionIndex, MemoryPlan, MemoryStyle, Module, ModuleTranslation, TableStyle, Tunables,
-    TypeTables, VMOffsets, WASM_PAGE_SIZE,
+    BuiltinFunctionIndex, MemoryPlan, MemoryStyle, Module, ModuleTranslation, ModuleTypes,
+    TableStyle, Tunables, VMOffsets, WASM_PAGE_SIZE,
 };
 use wasmtime_environ::{FUNCREF_INIT_BIT, FUNCREF_MASK};
 
@@ -113,7 +113,7 @@ pub struct FuncEnvironment<'module_environment> {
     isa: &'module_environment (dyn TargetIsa + 'module_environment),
     module: &'module_environment Module,
     translation: &'module_environment ModuleTranslation<'module_environment>,
-    types: &'module_environment TypeTables,
+    types: &'module_environment ModuleTypes,
 
     /// The Cranelift global holding the vmctx address.
     vmctx: Option<ir::GlobalValue>,
@@ -158,7 +158,7 @@ impl<'module_environment> FuncEnvironment<'module_environment> {
     pub fn new(
         isa: &'module_environment (dyn TargetIsa + 'module_environment),
         translation: &'module_environment ModuleTranslation<'module_environment>,
-        types: &'module_environment TypeTables,
+        types: &'module_environment ModuleTypes,
         tunables: &'module_environment Tunables,
     ) -> Self {
         let builtin_function_signatures = BuiltinFunctionSignatures::new(

crates/cranelift/src/lib.rs

@@ -51,7 +51,7 @@ use cranelift_entity::PrimaryMap;
 use cranelift_wasm::{DefinedFuncIndex, FuncIndex, WasmFuncType, WasmType};
 use target_lexicon::CallingConvention;
 use wasmtime_environ::{
-    FilePos, FunctionInfo, InstructionAddressMap, ModuleTranslation, TrapInformation, TypeTables,
+    FilePos, FunctionInfo, InstructionAddressMap, ModuleTranslation, ModuleTypes, TrapInformation,
 };
 
 pub use builder::builder;
@@ -208,7 +208,7 @@ fn indirect_signature(isa: &dyn TargetIsa, wasm: &WasmFuncType) -> ir::Signature
 fn func_signature(
     isa: &dyn TargetIsa,
     translation: &ModuleTranslation,
-    types: &TypeTables,
+    types: &ModuleTypes,
     index: FuncIndex,
 ) -> ir::Signature {
     let func = &translation.module.functions[index];

crates/environ/Cargo.toml

@@ -26,3 +26,6 @@ target-lexicon = "0.12"
 
 [badges]
 maintenance = { status = "actively-developed" }
+
+[features]
+component-model = []

crates/environ/src/compilation.rs

@@ -2,8 +2,8 @@
 //! module.
 
 use crate::{
-    DefinedFuncIndex, FilePos, FunctionBodyData, ModuleTranslation, PrimaryMap, SignatureIndex,
-    StackMap, Tunables, TypeTables, WasmError, WasmFuncType,
+    DefinedFuncIndex, FilePos, FunctionBodyData, ModuleTranslation, ModuleTypes, PrimaryMap,
+    SignatureIndex, StackMap, Tunables, WasmError, WasmFuncType,
 };
 use anyhow::Result;
 use object::write::Object;
@@ -148,7 +148,7 @@ pub trait Compiler: Send + Sync {
         index: DefinedFuncIndex,
         data: FunctionBodyData<'_>,
         tunables: &Tunables,
-        types: &TypeTables,
+        types: &ModuleTypes,
     ) -> Result<Box<dyn Any + Send>, CompileError>;
 
     /// Collects the results of compilation into an in-memory object.
@@ -169,7 +169,7 @@ pub trait Compiler: Send + Sync {
     fn emit_obj(
         &self,
         module: &ModuleTranslation,
-        types: &TypeTables,
+        types: &ModuleTypes,
         funcs: PrimaryMap<DefinedFuncIndex, Box<dyn Any + Send>>,
         tunables: &Tunables,
         obj: &mut Object<'static>,

crates/environ/src/component.rs

@@ -0,0 +1,34 @@
+//! Support for the component model in Wasmtime.
+//!
+//! This module contains all of the internal type definitions used by Wasmtime
+//! to process the component model. Despite everything being `pub` here this is
+//! not the public interface of Wasmtime to the component model. Instead this is
+//! the internal support to mirror the core wasm support that Wasmtime already
+//! implements.
+//!
+//! Some main items contained within here are:
+//!
+//! * Type hierarchy information for the component model
+//! * Translation of a component into Wasmtime's representation
+//! * Type information about a component used at runtime
+//!
+//! This module also contains a lot of Serialize/Deserialize types which are
+//! encoded in the final compiled image for a component.
+//!
+//! Note that this entire module is gated behind the `component-model` Cargo
+//! feature.
+//!
+//! ## Warning: In-progress
+//!
+//! As-of the time of this writing this module is incomplete and under
+//! development. It will be added to incrementally over time as more features
+//! are implemented. Current design decisions are also susceptible to change at
+//! any time. Some comments may reflect historical rather than current state as
+//! well (sorry).
+
+mod info;
+mod translate;
+mod types;
+pub use self::info::*;
+pub use self::translate::*;
+pub use self::types::*;