microsoft/openvmm
Publicmirrored from https://github.com/microsoft/openvmmAvailable
Guide/src/dev_guide/getting_started/build_openhcl.md
171lines · modecode
| 1 | # Building OpenHCL |
| 2 | |
| 3 | **Prerequisites:** |
| 4 | |
| 5 | - [Getting started on Linux / WSL2](./linux.md). |
| 6 | |
| 7 | Reminder: OpenHCL cannot currently be built on Windows hosts! |
| 8 | |
| 9 | * * * |
| 10 | |
| 11 | An OpenHCL IGVM firmware image is composed of several distinct binaries and |
| 12 | artifacts. For example: the `openvmm_hcl` usermode binary, the OpenHCL boot |
| 13 | shim, the OpenHCL Linux kernel and initrd, etc.... |
| 14 | |
| 15 | Some of these components are built directly out of the OpenVMM repo, whereas |
| 16 | others must be downloaded as pre-built artifacts from other associated repos. |
| 17 | Various tools and scripts will then transform, package, and re-package these |
| 18 | artifacts into a final OpenHCL IGVM firmware binary. |
| 19 | |
| 20 | Fortunately, we don't expect you do to all those steps manually! |
| 21 | |
| 22 | All the complexity of installing the correct system dependencies, building the |
| 23 | right binaries, downloading the right artifacts, etc... is neatly encapsulated |
| 24 | behind a single `cargo xflowey build-igvm` command, which orchestrates the |
| 25 | entire end-to-end OpenHCL build process. |
| 26 | |
| 27 | Using the `build-igvm` flow is as simple as running: |
| 28 | |
| 29 | ```bash |
| 30 | cargo xflowey build-igvm [RECIPE] |
| 31 | ``` |
| 32 | |
| 33 | The first build will take some time as all the dependencies are |
| 34 | installed/downloaded/built. |
| 35 | |
| 36 | **Note: At this time, OpenHCL can only be built on Linux (or WSL2)!** |
| 37 | |
| 38 | A "recipe" corresponds to one of the pre-defined IGVM SKUs that are actively |
| 39 | supported and tested in OpenVMM's build infrastructure. |
| 40 | |
| 41 | A single recipe encodes _all_ the details of what goes into an individual IGVM |
| 42 | file, such as what build flags `openvmm_hcl` should be built with, what goes |
| 43 | into a VTL2 initrd, what `igvmfilegen` manifest is being used, etc... |
| 44 | |
| 45 | - e.g: `x64`, for a "standard" x64 IGVM |
| 46 | - e.g: `aarch64`, for a "standard" aarch64 IGVM |
| 47 | - e.g: `x64-cvm`, for a x64 CVM IGVM |
| 48 | - e.g: `x64-test-linux-direct`, for x64 IGVM booting a test linux direct image |
| 49 | - _for a full list of available recipes, please run `cargo xflowey build-igvm --help`_ |
| 50 | |
| 51 | New recipes can be added by modifying the `build-igvm` source code. |
| 52 | |
| 53 | Build output is then binplaced to: `flowey-out/artifacts/build-igvm/{release-mode}/{recipe}/openhcl-{recipe}.bin` |
| 54 | |
| 55 | So, for example: |
| 56 | |
| 57 | ```bash |
| 58 | cargo xflowey build-igvm x64-cvm |
| 59 | # output: flowey-out/artifacts/build-igvm/debug/x64-cvm/openhcl-x64-cvm.bin |
| 60 | |
| 61 | cargo xflowey build-igvm x64 --release |
| 62 | # output: flowey-out/artifacts/build-igvm/release/x64/openhcl-x64.bin |
| 63 | ``` |
| 64 | |
| 65 | ```admonish warning |
| 66 | `cargo xflowey build-igvm` is designed to be used as part of the |
| 67 | developer inner-loop, and does _NOT_ have a stable CLI suitable for CI or any |
| 68 | other form of production automation! |
| 69 | |
| 70 | In-tree pipelines and automation should interface with the underlying `flowey` |
| 71 | infrastructure that powers `cargo xflowey build-igvm`, _without_ relying on |
| 72 | the details of its CLI. |
| 73 | ``` |
| 74 | |
| 75 | ## Building ohcldiag-dev |
| 76 | |
| 77 | `ohcldiag-dev` is typically built as a Windows binary. |
| 78 | |
| 79 | This can be done directly from Windows, or using cross-compilation from WSL2, as |
| 80 | described in the [Suggested Dev Environment](../getting_started/suggested_dev_env.md#wsl2-cross-compiling-from-wsl2-to-windows) |
| 81 | section of the Guide. |
| 82 | |
| 83 | The command to build `ohcldiag-dev` is simply: |
| 84 | |
| 85 | ```sh |
| 86 | # you may need to run `rustup target add x86_64-pc-windows-msvc` first |
| 87 | cargo build -p ohcldiag-dev --target x86_64-pc-windows-msvc |
| 88 | ``` |
| 89 | |
| 90 | **Note:** Thanks to x86 emulation built into Windows, `ohcldiag-dev.exe` that is |
| 91 | built for x64 Windows will work on Aarch64 Windows as well. |
| 92 | |
| 93 | ## Troubleshooting |
| 94 | |
| 95 | This section documents some common errors you may encounter while building |
| 96 | OpenHCL. |
| 97 | |
| 98 | If you are still running into issues, consider filing an issue on the OpenVMM |
| 99 | GitHub Issue tracker. |
| 100 | |
| 101 | ### Help! The build failed due to a missing dependency! |
| 102 | |
| 103 | If you don't mind having `xflowey` install some dependencies globally on your |
| 104 | machine (i.e: via `apt install`, or `rustup toolchain add`), you can pass |
| 105 | `--auto-install-deps` to your invocation of `build-igvm`. |
| 106 | |
| 107 | Alternatively - `build-igvm` _should_ emit useful human-readable error messages |
| 108 | when it encounters a dependency that isn't installed, with a suggestion on how |
| 109 | to install it. |
| 110 | |
| 111 | If it doesn't - please file an Issue! |
| 112 | |
| 113 | ### Help! Everything is rebuilding even though I only made a small change! |
| 114 | |
| 115 | Cargo's target triple handling can be a bit buggy. Try running with: |
| 116 | |
| 117 | ```bash |
| 118 | CARGO_BUILD_TARGET=x86_64-unknown-linux-gnu cargo build-igvm [RECIPE] |
| 119 | ``` |
| 120 | |
| 121 | or adding the below to your .bashrc: |
| 122 | |
| 123 | ```bash |
| 124 | export CARGO_BUILD_TARGET=x86_64-unknown-linux-gnu |
| 125 | ``` |
| 126 | |
| 127 | ## Build Customization |
| 128 | |
| 129 | Aside from building IGVM files corresponding the the built-in IGVM recipes, |
| 130 | `build-igvm` also offers a plethora of customization options for developers who |
| 131 | wish to build specialized custom IGVM files for local testing. |
| 132 | |
| 133 | Some examples of potentially useful customization include: |
| 134 | |
| 135 | * `--override-manifest`: Override the recipe's `igvmfilegen` manifest file |
| 136 | via, in order to tweak different kernel command line options, different VTL0 |
| 137 | boot configuration, or different VTL2 memory sizes. |
| 138 | |
| 139 | * `--custom-openvmm-hcl`: Specify a pre-built `openvmm_hcl` binary. This is |
| 140 | useful in case you have already built it with some custom settings, e.g.: |
| 141 | |
| 142 | ```bash |
| 143 | cargo build --target x86_64-unknown-linux-musl -p openvmm_hcl --features myfeature |
| 144 | cargo xflowey build-igvm x64 --custom-openvmm-hcl target/x86_64-unknown-linux-musl/debug/openvmm_hcl |
| 145 | ``` |
| 146 | |
| 147 | * Specify a custom VTL2 kernel `vmlinux` / `Image`, instead of using a |
| 148 | pre-packed stable/dev kernel. |
| 149 | |
| 150 | ```bash |
| 151 | cargo xflowey build-igvm x64 --custom-kernel path/to/my/prebuilt/vmlinux |
| 152 | ``` |
| 153 | |
| 154 | For a full list of available customizations, refer to `build-igvm --help`. |
| 155 | |
| 156 | ### Advanced |
| 157 | |
| 158 | Depending on what you're doing, you may need to build the individual components |
| 159 | that go into an OpenHCL IGVM build. |
| 160 | |
| 161 | Our `flowey`-based pipelines handle the complexities of properly invoking and |
| 162 | orchestrating the various individual build tools / scripts used to construct |
| 163 | IGVM files, but a sufficiently motivated user can go through these steps |
| 164 | manually. |
| 165 | |
| 166 | Please consult the source code for `cargo xflowey build-igvm` for a breakdown of |
| 167 | all build steps and available customization options. |
| 168 | |
| 169 | Note that the canonical "source of truth" for how to build end-to-end OpenHCL |
| 170 | IGVM files are these build scripts themselves, and the specific flow is subject |
| 171 | to change over time! |