microsoft/openvmm
Publicmirrored fromhttps://github.com/microsoft/openvmmAvailable
Guide/src/dev_guide/dev_tools/flowey/variables.md
101lines · modecode
| 1 | # Variables |
| 2 | |
| 3 | Variables are flowey's mechanism for creating typed data dependencies between steps. When a node emits steps, it uses `ReadVar<T>` and `WriteVar<T>` to declare what data each step consumes and produces. This creates explicit edges in the dependency graph: if step B reads from a variable that step A writes to, flowey ensures step A executes before step B. |
| 4 | |
| 5 | ## Claiming Variables |
| 6 | |
| 7 | Before a step can use a [`ReadVar`](https://openvmm.dev/rustdoc/linux/flowey/node/prelude/struct.ReadVar.html) or [`WriteVar`](https://openvmm.dev/rustdoc/linux/flowey/node/prelude/struct.WriteVar.html), it must **claim** it. Claiming serves several purposes: |
| 8 | |
| 9 | 1. Registers that this step depends on (or produces) this variable |
| 10 | 2. Converts `ReadVar<T, VarNotClaimed>` to `ReadVar<T, VarClaimed>` |
| 11 | 3. Allows flowey to track variable usage for graph construction |
| 12 | |
| 13 | Variables can only be claimed inside step closures using the `claim()` method. |
| 14 | |
| 15 | **Nested closure pattern and related contexts:** |
| 16 | |
| 17 | ```rust,ignore |
| 18 | // Inside a SimpleFlowNode's process_request() method |
| 19 | fn process_request(&self, request: Self::Request, ctx: &mut NodeCtx<'_>) { |
| 20 | // Assume a single Request provided an input ReadVar and output WriteVar |
| 21 | let input_var: ReadVar<String> = /* from one of the requests */; |
| 22 | let output_var: WriteVar<i32> = /* from one of the requests */; |
| 23 | |
| 24 | // Declare a step (still build-time). This adds a node to the DAG. |
| 25 | ctx.emit_rust_step("compute length", |step| { |
| 26 | // step : StepCtx (outer closure, build-time) |
| 27 | // Claim dependencies so the graph knows: this step READS input_var, WRITES output_var. |
| 28 | let input_var = input_var.claim(step); |
| 29 | let output_var = output_var.claim(step); |
| 30 | |
| 31 | // Return the runtime closure. |
| 32 | move |rt| { |
| 33 | // rt : RustRuntimeServices (runtime phase) |
| 34 | let input = rt.read(input_var); // consume value |
| 35 | let len = input.len() as i32; |
| 36 | rt.write(output_var, &len); // fulfill promise |
| 37 | Ok(()) |
| 38 | } |
| 39 | }); |
| 40 | } |
| 41 | ``` |
| 42 | |
| 43 | **Why the nested closure dance?** |
| 44 | |
| 45 | The nested closure pattern is fundamental to flowey's two-phase execution model: |
| 46 | |
| 47 | 1. **Build-Time (Outer Closure)**: When flowey constructs the DAG, the outer closure runs to: |
| 48 | - Claim variables, which registers dependencies in the graph |
| 49 | - Determine what this step depends on (reads) and produces (writes) |
| 50 | - Allow flowey determine execution order |
| 51 | - Returns an inner closure that gets invoked during the job's runtime |
| 52 | 2. **Runtime (Inner Closure)**: When the pipeline actually executes, the inner closure runs to: |
| 53 | - Read actual values from claimed `ReadVar`s |
| 54 | - Perform the real work (computations, running commands, etc.) |
| 55 | - Write actual values to claimed `WriteVar`s |
| 56 | |
| 57 | - [**`NodeCtx`**](https://openvmm.dev/rustdoc/linux/flowey/node/prelude/struct.NodeCtx.html): Used when emitting steps (during the build-time phase). Provides `emit_*` methods, `new_var()`, `req()`, etc. |
| 58 | |
| 59 | - [**`StepCtx`**](https://openvmm.dev/rustdoc/linux/flowey/node/prelude/struct.StepCtx.html): Used inside step closures (during runtime execution). Provides access to `claim()` for variables, and basic environment info (`backend()`, `platform()`). |
| 60 | |
| 61 | The type system enforces this separation: `claim()` requires `StepCtx` (only available in the outer closure), while `read()`/`write()` require `RustRuntimeServices` (only available in the inner closure). |
| 62 | |
| 63 | ## ClaimedReadVar and ClaimedWriteVar |
| 64 | |
| 65 | These are type aliases for claimed variables: |
| 66 | |
| 67 | - [`ClaimedReadVar<T>`](https://openvmm.dev/rustdoc/linux/flowey/node/prelude/type.ClaimedReadVar.html) = `ReadVar<T, VarClaimed>` |
| 68 | - [`ClaimedWriteVar<T>`](https://openvmm.dev/rustdoc/linux/flowey/node/prelude/type.ClaimedWriteVar.html) = `WriteVar<T, VarClaimed>` |
| 69 | |
| 70 | Only claimed variables can be read/written at runtime. |
| 71 | |
| 72 | ### Implementation Detail: Zero-Sized Types (ZSTs) |
| 73 | |
| 74 | The claim state markers [`VarClaimed`](https://openvmm.dev/rustdoc/linux/flowey/node/prelude/enum.VarClaimed.html) and [`VarNotClaimed`](https://openvmm.dev/rustdoc/linux/flowey/node/prelude/enum.VarNotClaimed.html) are zero-sized types (ZSTs) - they exist purely at the type level. It allows Rust to statically verify that all variables used in a runtime block have been claimed by that block. |
| 75 | |
| 76 | The type system ensures that `claim()` is the only way to convert from `VarNotClaimed` to `VarClaimed`, and this conversion can only happen within the outer closure where `StepCtx` is available. |
| 77 | |
| 78 | ## Static Values vs Runtime Values |
| 79 | |
| 80 | Sometimes you know a value at build-time: |
| 81 | |
| 82 | ```rust,ignore |
| 83 | // Create a ReadVar with a static value |
| 84 | let version = ReadVar::from_static("1.2.3".to_string()); |
| 85 | |
| 86 | // This is encoded directly in the pipeline, not computed at runtime |
| 87 | // WARNING: Never use this for secrets! |
| 88 | ``` |
| 89 | |
| 90 | This can be used as an escape hatch when you have a Request (that expects a value to be determined at runtime), but in a given instance you know the value at build-time. |
| 91 | |
| 92 | ## Variable Operations |
| 93 | |
| 94 | `ReadVar` provides operations for transforming and combining variables: |
| 95 | |
| 96 | - **`map()`**: Transform a `ReadVar<T>` into a `ReadVar<U>` |
| 97 | - **`zip()`**: Combine two ReadVars into `ReadVar<(T, U)>` |
| 98 | - **`into_side_effect()`**: Convert `ReadVar<T>` to `ReadVar<SideEffect>` when you only care about ordering, not the value |
| 99 | - **`depending_on()`**: Create a new ReadVar with an explicit dependency |
| 100 | |
| 101 | For detailed examples, see the [`ReadVar` documentation](https://openvmm.dev/rustdoc/linux/flowey/node/prelude/struct.ReadVar.html). |
| 102 | |