GitVita
New repositoryImport repositoryNew issue
Your repositories Your dashboard Sign in

microsoft/openvmm

Public

mirrored fromhttps://github.com/microsoft/openvmmAvailable

Watch0Fork0Star0
CodeCommitsIssuesPull requestsActionsInsightsSecurity
2d321ee34e4dd620a030b11b532fa4d8a6f3ed73

Branches

Tags

  • No tags available.
0Branches0Tags
Go to file
Add file
Code

Clone

HTTPS

Download ZIP

openvmm/Guide/src/dev_guide/dev_tools/flowey

Guide/src/dev_guide/dev_tools/flowey/nodes.md

224lines · modecode

RawDownload
Latest commit unavailable.
1# Nodes
2
3At a conceptual level, a Flowey node is analogous to a strongly typed function: you "invoke" it by submitting one or more Request values (its parameters), and it responds by emitting steps that perform work and produce outputs (values written to `WriteVar`s, published artifacts, or side-effect dependencies).
4
5## The Node/Request Pattern
6
7Every node has an associated **Request** type that defines what operations the node can perform. Requests are defined using the [`flowey_request!`](https://openvmm.dev/rustdoc/linux/flowey_core/macro.flowey_request.html) macro and registered with [`new_flow_node!`](https://openvmm.dev/rustdoc/linux/flowey_core/macro.new_flow_node.html) or [`new_simple_flow_node!`](https://openvmm.dev/rustdoc/linux/flowey_core/macro.new_simple_flow_node.html) macros.
8
9For complete examples, see the [`FlowNode` trait documentation](https://openvmm.dev/rustdoc/linux/flowey_core/node/trait.FlowNode.html).
10
11## FlowNode vs SimpleFlowNode
12
13Flowey provides two node implementation patterns with a fundamental difference in their Request structure and complexity:
14
15[**`SimpleFlowNode`**](https://openvmm.dev/rustdoc/linux/flowey_core/node/trait.SimpleFlowNode.html) - for straightforward, function-like operations:
16
17- Uses a **single struct Request** type
18- Processes one request at a time independently
19- Behaves like a "plain old function" that resolves its single request type
20- Each invocation is isolated - no shared state or coordination between requests
21- Simpler implementation with less boilerplate
22- Ideal for straightforward operations like running a command or transforming data
23
24**Example use case**: A node that runs `cargo build` - each request is independent and just needs to know what to build.
25
26[**`FlowNode`**](https://openvmm.dev/rustdoc/linux/flowey_core/node/trait.FlowNode.html) - for complex nodes requiring coordination and non-local configuration:
27
28- Often uses an **enum Request** with multiple variants
29- Receives all requests as a `Vec<Request>` and processes them together
30- Can aggregate, optimize, and consolidate multiple requests into fewer steps
31- Enables **non-local configuration** - critical for simplifying complex pipelines
32
33### The Non-Local Configuration Pattern
34
35The key advantage of `FlowNode` is its ability to accept configuration from different parts of the node graph without forcing intermediate nodes to be aware of that configuration. This is the "non-local" aspect.
36
37For nodes that accept **config values** — versions, feature flags, local paths — use `FlowNodeWithConfig` with a typed config struct. Config is declared with the `flowey_config!` macro, set via `ctx.config(...)` by any caller, and automatically merged across all callers before being delivered to `emit()` separately from action requests.
38
39Consider an "install Rust toolchain" node:
40
41```rust,ignore
42flowey_config! {
43 pub struct Config {
44 pub version: Option<String>,
45 pub auto_install: Option<bool>,
46 }
47}
48
49flowey_request! {
50 pub enum Request {
51 GetToolchain(WriteVar<PathBuf>),
52 }
53}
54
55new_flow_node_with_config!(struct Node);
56
57impl FlowNodeWithConfig for Node {
58 type Request = Request;
59 type Config = Config;
60
61 fn imports(ctx: &mut ImportCtx<'_>) { /* ... */ }
62
63 fn emit(
64 config: Config,
65 requests: Vec<Self::Request>,
66 ctx: &mut NodeCtx<'_>,
67 ) -> anyhow::Result<()> {
68 let version = config.version
69 .expect("version must be set by cfg_versions");
70 // ... use version to install, then fulfill GetToolchain requests
71 Ok(())
72 }
73}
74```
75
76Callers set config and submit requests independently:
77
78```rust,ignore
79// A top-level job configuration node sets the version once:
80ctx.config(install_rust::Config {
81 version: Some("1.75".into()),
82 ..Default::default()
83});
84
85// Any node that needs the Rust toolchain just requests it:
86let toolchain = ctx.reqv(|v| install_rust::Request::GetToolchain(v));
87```
88
89```txt
90Root Node → config(version: "1.75")
91 → Node A
92 → Node B
93 → Node C → req(GetToolchain)
94```
95
96Flowey merges all config partials automatically. If two callers set the same field, the values must agree or the build fails. The intermediate nodes (A, B, C) never need to know about or pass through the version.
97
98This pattern:
99
100- **Eliminates plumbing complexity** in large pipelines
101- **Allows global configuration** to be set once at the top level
102- **Keeps unrelated nodes decoupled** from configuration they don't need
103- **Validates consistency** — conflicting config values are caught at build time
104- **Separates concerns** — config (what version?) is distinct from requests (give me the toolchain path)
105
106**Additional Benefits of FlowNode:**
107
108- Optimize and consolidate multiple similar requests into fewer steps (e.g., installing a tool once for many consumers)
109- Resolve conflicts or enforce consistency across requests
110
111For detailed comparisons and examples, see the [`FlowNode`](https://openvmm.dev/rustdoc/linux/flowey_core/node/trait.FlowNode.html) and [`SimpleFlowNode`](https://openvmm.dev/rustdoc/linux/flowey_core/node/trait.SimpleFlowNode.html) documentation.
112
113## Node Registration
114
115Nodes are automatically registered using macros that handle most of the boilerplate:
116
117- [`new_flow_node!(struct Node)`](https://openvmm.dev/rustdoc/linux/flowey_core/macro.new_flow_node.html) - registers a FlowNode
118- [`new_flow_node_with_config!(struct Node)`](https://openvmm.dev/rustdoc/linux/flowey_core/macro.new_flow_node_with_config.html) - registers a FlowNodeWithConfig (a FlowNode that also accepts typed config)
119- [`new_simple_flow_node!(struct Node)`](https://openvmm.dev/rustdoc/linux/flowey_core/macro.new_simple_flow_node.html) - registers a SimpleFlowNode
120- [`flowey_request!`](https://openvmm.dev/rustdoc/linux/flowey_core/macro.flowey_request.html) - defines the Request type and implements [`IntoRequest`](https://openvmm.dev/rustdoc/linux/flowey_core/node/trait.IntoRequest.html)
121- [`flowey_config!`](https://openvmm.dev/rustdoc/linux/flowey_core/macro.flowey_config.html) - defines a Config struct with automatic merge support and implements [`IntoConfig`](https://openvmm.dev/rustdoc/linux/flowey_core/node/trait.IntoConfig.html)
122
123## The imports() Method
124
125The `imports()` method declares which other nodes this node might depend on. This enables flowey to:
126
127- Validate that all dependencies are available
128- Build the complete dependency graph
129- Catch missing dependencies at build-time
130
131```admonish warning
132Flowey does not catch unused imports today as part of its build-time validation step.
133```
134
135**Why declare imports?** Flowey needs to know the full set of potentially-used nodes at compilation time to properly resolve the dependency graph.
136
137For more on node imports, see the [`FlowNode::imports` documentation](https://openvmm.dev/rustdoc/linux/flowey_core/node/trait.FlowNode.html#tymethod.imports).
138
139## The emit() Method
140
141The [`emit()`](https://openvmm.dev/rustdoc/linux/flowey_core/node/trait.FlowNode.html#tymethod.emit) method is where a node's actual logic lives. For [`FlowNode`](https://openvmm.dev/rustdoc/linux/flowey_core/node/trait.FlowNode.html), it receives all requests together and must:
142
1431. Aggregate and validate requests (ensuring consistency where needed)
1442. Emit steps to perform the work
1453. Wire up dependencies between steps via variables
146
147For [`SimpleFlowNode`](https://openvmm.dev/rustdoc/linux/flowey_core/node/trait.SimpleFlowNode.html), the equivalent [`process_request()`](https://openvmm.dev/rustdoc/linux/flowey_core/node/trait.SimpleFlowNode.html#tymethod.process_request) method processes one request at a time.
148
149For complete implementation examples, see the [`FlowNode::emit` documentation](https://openvmm.dev/rustdoc/linux/flowey_core/node/trait.FlowNode.html#tymethod.emit).
150
151## Node Design Philosophy
152
153Flowey nodes are designed around several key principles:
154
155### 1. Composability
156
157Nodes should be reusable building blocks that can be combined to build complex
158workflows. Each node should have a single, well-defined responsibility.
159
160❌ **Bad**: A node that "builds and tests the project"
161✅ **Good**: Separate nodes for "build project" and "run tests"
162
163### 2. Explicit Dependencies
164
165Dependencies between steps should be explicit through variables, not implicit
166through side effects.
167
168❌ **Bad**: Assuming a tool is already installed
169✅ **Good**: Taking a `ReadVar<SideEffect>` that proves installation happened
170
171### 3. Backend Abstraction
172
173Nodes should work across all backends when possible. Backend-specific behavior
174should be isolated and documented.
175
176### 4. Separation of Concerns
177
178Keep node definition (request types, dependencies) separate from step
179implementation (runtime logic):
180
181- **Node definition**: What the node does, what it depends on
182- **Step implementation**: How it does it
183
184## Common Patterns
185
186### Node Config vs Request
187
188When designing a `FlowNode`, separate **config** (values that must be consistent across all callers) from **requests** (actions the node performs):
189
190- **Config** (`flowey_config!`): version strings, feature flags, local override paths, auto-install toggles. Use `FlowNodeWithConfig` and `ctx.config(...)`. Config fields are `Option<T>` or `BTreeMap<K, V>` — flowey merges them automatically and errors on conflicts.
191- **Requests** (`flowey_request!`): actions that produce outputs, e.g. `GetBinary(WriteVar<PathBuf>)`. Multiple callers can each submit requests, and the node fulfills all of them.
192
193```admonish tip
194If a value needs `same_across_all_reqs` validation, it should probably be a config field instead. The `flowey_config!` macro provides this validation automatically during merge.
195```
196
197### Request Aggregation and Validation
198
199When a FlowNode receives multiple requests, it may need to aggregate or validate them. Common techniques:
200
201- Iterate through all requests and separate them by variant
202- Collect output variables that can have multiple instances
203- Validate that required values were provided
204
205For values that must be consistent across all callers (versions, flags), prefer using `flowey_config!` with `FlowNodeWithConfig` instead of manual validation — see [The Non-Local Configuration Pattern](#the-non-local-configuration-pattern) above.
206
207### Conditional Execution Based on Backend/Platform
208
209Nodes can query the current backend and platform to emit platform-specific or backend-specific steps. This allows nodes to adapt their behavior based on the execution environment.
210
211**Key concepts:**
212
213- Use `ctx.backend()` to check if running locally, on ADO, or on GitHub Actions
214- Use `ctx.platform()` to check the operating system (Windows, Linux, macOS)
215- Use `ctx.arch()` to check the architecture (x86_64, Aarch64)
216- Emit different steps or use different tool configurations based on these values
217
218**When to use:**
219
220- Installing platform-specific tools or dependencies
221- Using different commands on Windows vs Unix systems
222- Optimizing for local development vs CI environments
223
224For more on backend and platform APIs, see the [`NodeCtx` documentation](https://openvmm.dev/rustdoc/linux/flowey_core/node/struct.NodeCtx.html).
225