# Hardware Debugging (gdbstub)
Think EXDI from Hyper-V, except instead of using the EXDI interface, we use the
[GDB Remote Serial Protocol](https://sourceware.org/gdb/onlinedocs/gdb/Remote-Protocol.html)
(via the [`gdbstub`](https://github.com/daniel5151/gdbstub/) Rust library).
Hardware debugging has several benefits over using an in-guest / in-kernel debugger:
- Debugging early-boot scenarios (before UEFI / Windows / Linux debuggers are set up)
- Debugging low-level ISRs
- Non-intrusive debugging = easier to repro certain bugs
- Debugging SNP/TDX/VBS Confidential VMs
## Enabling the Debugger
### OpenVMM
1. Pass the `--gdb <port>` flag at startup to enable the
debug worker. e.g., `--gdb 9001`
To pause the VM until the debugger has been attached, pass `--paused` at startup.
### OpenHCL
1. Pass the `OPENHCL_GDBSTUB=1` `OPENHCL_GDBSTUB_PORT=<gdbstub port>` parameters to enable gdbstub. e.g., `Set-VmFirmwareParameters -Name UhVM -CommandLine OPENHCL_GDBSTUB=1 OPENHCL_GDBSTUB_PORT=5900`.
2. To expose a TCP port, run `ohcldiag-dev.exe <name> vsock-tcp-relay --allow-remote --reconnect <gdbstub port> <tcp port>`.
To pause VTL0 boot until desired, pass `OPENHCL_VTL0_STARTS_PAUSED=1` as a parameter. Then once the debugger is attached, you can start VTL0 with `ohcldiag-dev.exe <name> resume`.
## Connecting via GDB
The quickest way to get connected to a OpenVMM VM is via `gdb` directly.
Note that GDB does _not_ support debugging PDBs, so if you're trying to debug
Windows, you'll be limited to plain disassembly. See the [`Connecting via
WinDbg`](#connecting-via-windbg) section below if this is your use-case.
On the flipside, if you're trying to debug ELF images with DWARF debug info
(e.g., a vmlinux binary), then you'll likely want to use `gdb` directly, as it
will support source-level debugging with symbols, whereas WinDbg will not.
You can install `gdb` via your distro's package manager. e.g., on Ubuntu:
```bash
sudo apt install gdb
```
Once `gdb` is installed, run it, and enter the following `gdb` command (swapping
`9001` for whatever port you specified at the CLI)
```
target remote :9001
```
If all goes well, you should get output similar to this:
```
(gdb) target remote :9001
Remote debugging using :9001
warning: No executable has been specified and target does not support
determining executable automatically. Try using the "file" command.
0xfffff8015c054c1f in ?? ()
(gdb)
```
At this point, you can try some basic GDB commands to make sure things are working.
e.g., start / interrupt the VM's execution using `cont` and `ctrl-c`
```
(gdb) cont
Continuing.
^C # <-- hit ctrl-c in the terminal
Thread 1 received signal SIGINT, Interrupt.
0xfffff8015c054c1f in ?? ()
(gdb)
```
e.g., inspecting register state
```
(gdb) info registers
rax 0x0 0
rbx 0x0 0
rcx 0x40086 262278
rdx 0x0 0
rsi 0xffff960d4eea5010 -116491073990640
rdi 0x0 0
rbp 0x0 0x0
rsp 0xfffff8015b3f5ec8 0xfffff8015b3f5ec8
r8 0x0 0
r9 0xffffffff 4294967295
r10 0xfffff8015bfff1f0 -8790254554640
r11 0x0 0
r12 0xffffffff 4294967295
...etc...
```
e.g., setting data breakpoints
```
(gdb) awatch *0xfffff804683190e0
Hardware access (read/write) watchpoint 1: *0xfffff804683190e0
```
e.g., single stepping
```
0xfffff8047a309686 in ?? ()
(gdb) si
0xfffff8047a309689 in ?? ()
```
You may find [this blog post](https://blog.mattjustice.com/2018/08/24/gdb-for-windbg-users/)
useful, as it includes a table of common `gdb` commands along with their WinDbg
counterparts.
## Connecting via WinDbg
WinDbg doesn't understand the GDB Remote Serial Protocol directly, but
thankfully, some smart folks over on the WinDbg team have developed a GDB Remote
Serial Protocol <-> WinDbg translation layer!
For more information, see
[Setting Up QEMU Kernel-Mode Debugging using EXDI](https://learn.microsoft.com/en-us/windows-hardware/drivers/debugger/setting-up-qemu-kernel-mode-debugging-using-exdi)
Getting this working with OpenVMM or OpenHCL is as easy as following the guide,
except you'll need to [enable our debugger](#enabling-the-debugger) instead of
running QEMU.
It's easiest to connect through the GUI. The steps are relatively simple: Open Windbgx -> File -> Attach to kernel -> EXDI. On the form, fill out:
- Target Type: `QEMU`
- Target Architecture: `X64`
- Target OS: `Windows`
- Image Screening heuristic size: `0xFFFE - NT`
- Gdb server and port: `<server>:<port>` e.g., `127.0.0.1:1337` (use whatever port you set above)
### Known WinDbg Bugs
- Hardware breakpoints are issued with [`ba`](https://learn.microsoft.com/en-us/windows-hardware/drivers/debuggercmds/ba--break-on-access-). The `Access Size` parameter is incorrectly multiplied by 8 when sent to the stub. Consequently, it _must_ be set to 1.
- Unlike GDB, WinDbg doesn't implicitly set software breakpoints via our offered write_addrs implementation.
---
## Supported Features
At the time of writing (8/16/24) the debugger supports the following operations:
- read/write guest memory
- read guest registers \*
- start/interrupt execution
- watchpoints
- hardware breakpoints
- single stepping
## TODO Features
If you're looking for work, and want to improve the debugging experience for
everyone, consider implementing one or more of the following features:
- \* reading _all_ guest registers, including fpu, xmm, and various key msrs
- software breakpoints:
- Intercept guest breakpoint exceptions into VTL2
- writing guest registers
- exposing the OpenVMM interactive console via the
[`MonitorCmd`](https://docs.rs/gdbstub/latest/gdbstub/target/ext/monitor_cmd/trait.MonitorCmd.html)
interface
- Custom commands sent using `monitor` (gdb) / `.exdicmd` (WinDbg)
- e.g., being able to invoke `x device/to/inspect` directly from the debugger
- [any other features supported by the `gdbstub` library](https://github.com/daniel5151/gdbstub#debugging-features)microsoft/openvmm
Publicmirrored fromhttps://github.com/microsoft/openvmmAvailable
Guide/src/reference/dev_feats/gdbstub.md
167lines · modepreview