microsoft/hve-core
Publicmirrored fromhttps://github.com/microsoft/hve-coreAvailable
docs/architecture/workflows.md
303lines · modecode
| 1 | --- |
| 2 | title: Build Workflows |
| 3 | description: GitHub Actions CI/CD pipeline architecture for validation, security, and release automation |
| 4 | author: WilliamBerryiii |
| 5 | ms.date: 2026-02-10 |
| 6 | ms.topic: overview |
| 7 | --- |
| 8 | |
| 9 | HVE Core uses GitHub Actions for continuous integration, quality validation, security scanning, and release automation. The workflow architecture emphasizes reusable components and parallel execution for fast feedback. |
| 10 | |
| 11 | ## Pipeline Overview |
| 12 | |
| 13 | ```mermaid |
| 14 | flowchart TD |
| 15 | subgraph PR["Pull Request"] |
| 16 | direction TB |
| 17 | PR1[PR Opened/Updated] --> PV[pr-validation.yml] |
| 18 | PV --> LINT[Linting Jobs] |
| 19 | PV --> SEC[Security Jobs] |
| 20 | PV --> TEST[Test Jobs] |
| 21 | end |
| 22 | |
| 23 | subgraph MAIN["Main Branch"] |
| 24 | direction TB |
| 25 | MERGE[Merge to Main] --> MN[main.yml] |
| 26 | MN --> VAL[Validation] |
| 27 | VAL --> PKG[Extension Package] |
| 28 | PKG --> REL[Release Please] |
| 29 | end |
| 30 | |
| 31 | subgraph SCHED["Scheduled"] |
| 32 | direction TB |
| 33 | CRON[Weekly Sunday 2AM] --> WEEKLY[weekly-security-maintenance.yml] |
| 34 | WEEKLY --> SECCHECK[Security Checks] |
| 35 | end |
| 36 | |
| 37 | subgraph MANUAL["Manual"] |
| 38 | direction TB |
| 39 | DISPATCH[Manual Trigger] --> PUB[extension-publish.yml] |
| 40 | PUB --> VSCE[Publish to Marketplace] |
| 41 | end |
| 42 | ``` |
| 43 | |
| 44 | ## Workflow Inventory |
| 45 | |
| 46 | | Workflow | Trigger | Purpose | |
| 47 | |------------------------------------|-------------------------|-------------------------------------------------| |
| 48 | | `pr-validation.yml` | Pull request, manual | Pre-merge quality gate with parallel validation | |
| 49 | | `main.yml` | Push to main, manual | Post-merge validation and release automation | |
| 50 | | `weekly-security-maintenance.yml` | Sunday 2 AM UTC, manual | Scheduled security posture review | |
| 51 | | `security-scan.yml` | Push to main/develop | CodeQL security validation | |
| 52 | | `extension-publish.yml` | Manual | VS Code extension marketplace publishing | |
| 53 | | `extension-publish-prerelease.yml` | Manual | VS Code extension pre-release publishing | |
| 54 | | `copilot-setup-steps.yml` | Manual | Coding agent environment setup | |
| 55 | | `prerelease-release.yml` | PR closed | Pre-release tag and publish on merge to main | |
| 56 | | `prerelease.yml` | Push to main | Pre-release companion PR management | |
| 57 | | `scorecard.yml` | Schedule, push | OpenSSF Scorecard security analysis | |
| 58 | | `codeql-analysis.yml` | Schedule | Weekly CodeQL security scan (also reusable) | |
| 59 | | `dependency-review.yml` | Pull request | Dependency vulnerability review (also reusable) | |
| 60 | | `sha-staleness-check.yml` | Manual | SHA reference freshness check (also reusable) | |
| 61 | |
| 62 | ### Reusable Workflows |
| 63 | |
| 64 | Individual validation workflows called by orchestration workflows: |
| 65 | |
| 66 | | Workflow | Purpose | npm Script | |
| 67 | |-------------------------------------|----------------------------------|-------------------------------------| |
| 68 | | `markdown-lint.yml` | Markdownlint validation | `npm run lint:md` | |
| 69 | | `spell-check.yml` | cspell dictionary check | `npm run spell-check` | |
| 70 | | `frontmatter-validation.yml` | AI artifact frontmatter schemas | `npm run lint:frontmatter` | |
| 71 | | `markdown-link-check.yml` | Broken link detection | `npm run lint:md-links` | |
| 72 | | `link-lang-check.yml` | Link language validation | `npm run lint:links` | |
| 73 | | `yaml-lint.yml` | YAML syntax validation | `npm run lint:yaml` | |
| 74 | | `ps-script-analyzer.yml` | PowerShell static analysis | `npm run lint:ps` | |
| 75 | | `table-format.yml` | Markdown table formatting | `npm run format:tables` | |
| 76 | | `pester-tests.yml` | PowerShell unit tests | `npm run test:ps` | |
| 77 | | `skill-validation.yml` | Skill structure validation | `npm run validate:skills` | |
| 78 | | `dependency-pinning-scan.yml` | GitHub Actions pinning | N/A (PowerShell direct) | |
| 79 | | `sha-staleness-check.yml` | SHA reference freshness* | N/A (PowerShell direct) | |
| 80 | | `codeql-analysis.yml` | CodeQL security scanning* | N/A (GitHub native) | |
| 81 | | `dependency-review.yml` | Dependency vulnerability review* | N/A (GitHub native) | |
| 82 | | `extension-package.yml` | VS Code extension packaging | `npm run extension:package` | |
| 83 | | `copyright-headers.yml` | Copyright header validation | `npm run validate:copyright` | |
| 84 | | `gitleaks-scan.yml` | Secret detection scanning | N/A (gitleaks direct) | |
| 85 | | `plugin-package.yml` | Plugin collection packaging | N/A | |
| 86 | | `plugin-validation.yml` | Plugin and collection metadata | `npm run lint:collections-metadata` | |
| 87 | | `extension-publish-marketplace.yml` | Extension marketplace publishing | N/A | |
| 88 | |
| 89 | Workflows marked with `*` are dual-purpose: they accept `workflow_call` for reuse by orchestration workflows and also run independently via their own triggers. |
| 90 | |
| 91 | ## PR Validation Pipeline |
| 92 | |
| 93 | The `pr-validation.yml` workflow serves as the primary quality gate for all pull requests. It runs 16 parallel jobs covering linting, security, and testing. |
| 94 | |
| 95 | ```mermaid |
| 96 | flowchart LR |
| 97 | subgraph "Linting" |
| 98 | ML[markdown-lint] |
| 99 | SC[spell-check] |
| 100 | TF[table-format] |
| 101 | YL[yaml-lint] |
| 102 | FV[frontmatter-validation] |
| 103 | LLC[link-lang-check] |
| 104 | MLC[markdown-link-check] |
| 105 | CH[copyright-headers] |
| 106 | end |
| 107 | |
| 108 | subgraph "Analysis" |
| 109 | PSA[psscriptanalyzer] |
| 110 | PT[pester-tests] |
| 111 | SV[skill-validation] |
| 112 | PV[plugin-validation] |
| 113 | end |
| 114 | |
| 115 | subgraph "Security" |
| 116 | DPC[dependency-pinning-check] |
| 117 | NA[npm-audit] |
| 118 | CQL[codeql] |
| 119 | GLS[gitleaks-scan] |
| 120 | end |
| 121 | ``` |
| 122 | |
| 123 | ### Jobs |
| 124 | |
| 125 | | Job | Reusable Workflow | Validates | |
| 126 | |--------------------------|-------------------------------|--------------------------------| |
| 127 | | spell-check | `spell-check.yml` | Spelling across all files | |
| 128 | | markdown-lint | `markdown-lint.yml` | Markdown formatting rules | |
| 129 | | table-format | `table-format.yml` | Markdown table structure | |
| 130 | | psscriptanalyzer | `ps-script-analyzer.yml` | PowerShell code quality | |
| 131 | | yaml-lint | `yaml-lint.yml` | YAML syntax | |
| 132 | | pester-tests | `pester-tests.yml` | PowerShell unit tests | |
| 133 | | frontmatter-validation | `frontmatter-validation.yml` | AI artifact metadata | |
| 134 | | skill-validation | `skill-validation.yml` | Skill directory structure | |
| 135 | | link-lang-check | `link-lang-check.yml` | Link accessibility | |
| 136 | | markdown-link-check | `markdown-link-check.yml` | Broken links | |
| 137 | | dependency-pinning-check | `dependency-pinning-scan.yml` | Action SHA pinning | |
| 138 | | npm-audit | Inline | npm dependency vulnerabilities | |
| 139 | | codeql | `codeql-analysis.yml` | Code security patterns | |
| 140 | | copyright-headers | `copyright-headers.yml` | Copyright header compliance | |
| 141 | | plugin-validation | `plugin-validation.yml` | Plugin and collection metadata | |
| 142 | | gitleaks-scan | `gitleaks-scan.yml` | Secret detection | |
| 143 | |
| 144 | All jobs run in parallel with no dependencies, enabling fast feedback (typically under 3 minutes). |
| 145 | |
| 146 | ## Main Branch Pipeline |
| 147 | |
| 148 | The `main.yml` workflow runs after merges to main, performing validation and release automation. |
| 149 | |
| 150 | ```mermaid |
| 151 | flowchart LR |
| 152 | V1[spell-check] --> RP[release-please] |
| 153 | V2[markdown-lint] --> RP |
| 154 | V3[table-format] --> RP |
| 155 | V4[dependency-pinning-scan] --> RP |
| 156 | V5[gitleaks-scan] --> RP |
| 157 | V6[pester-tests] --> RP |
| 158 | RP --> RST[reset-prerelease] |
| 159 | RP -->|release_created| EPR[extension-package-release] |
| 160 | RP -->|release_created| PPR[plugin-package-release] |
| 161 | RP -->|release_created| SBOM[generate-dependency-sbom] |
| 162 | EPR --> ATT[attest-and-upload] |
| 163 | SBOM --> ATT |
| 164 | PPR --> UPP[upload-plugin-packages] |
| 165 | SBOM --> SD[sbom-diff] |
| 166 | ATT --> PUB[publish-release] |
| 167 | UPP --> PUB |
| 168 | SD --> PUB |
| 169 | style RP fill:#f9f,stroke:#333 |
| 170 | ``` |
| 171 | |
| 172 | Release-please v4 handles `chore`-type commits natively. They are not releasable and do not produce spurious release PRs, so no commit-message guard is needed. |
| 173 | |
| 174 | ### Main Branch Jobs |
| 175 | |
| 176 | | Job | Purpose | Dependencies | |
| 177 | |---------------------------|--------------------------------|----------------------------------------------------------------------| |
| 178 | | spell-check | Post-merge spelling validation | None | |
| 179 | | markdown-lint | Post-merge markdown validation | None | |
| 180 | | table-format | Post-merge table validation | None | |
| 181 | | dependency-pinning-scan | Security pinning check | None | |
| 182 | | gitleaks-scan | Secret detection scanning | None | |
| 183 | | pester-tests | PowerShell unit tests | None | |
| 184 | | release-please | Automated release management | All validation jobs | |
| 185 | | reset-prerelease | Reset pre-release tracking | release-please | |
| 186 | | extension-package-release | Build release VSIX | release-please (conditional) | |
| 187 | | plugin-package-release | Build release plugin packages | release-please (conditional) | |
| 188 | | generate-dependency-sbom | Generate dependency SBOM | release-please (conditional) | |
| 189 | | attest-and-upload | Sign and upload VSIX | release-please, extension-package-release, generate-dependency-sbom | |
| 190 | | upload-plugin-packages | Upload plugin packages | release-please, plugin-package-release | |
| 191 | | sbom-diff | Compare SBOM changes | release-please, generate-dependency-sbom | |
| 192 | | publish-release | Finalize GitHub Release | release-please, attest-and-upload, upload-plugin-packages, sbom-diff | |
| 193 | |
| 194 | When release-please creates a release, parallel jobs build the extension VSIX (`extension-package-release`), package plugin collections (`plugin-package-release`), and generate an SBOM (`generate-dependency-sbom`). The `attest-and-upload` job signs the VSIX with Sigstore attestation, `upload-plugin-packages` uploads collection artifacts, and `sbom-diff` compares dependency changes. The `publish-release` job finalizes the GitHub Release after all artifacts are ready. |
| 195 | |
| 196 | ## Security Workflows |
| 197 | |
| 198 | ### Weekly Security Maintenance |
| 199 | |
| 200 | The `weekly-security-maintenance.yml` workflow runs every Sunday at 2AM UTC, providing scheduled security posture review. |
| 201 | |
| 202 | | Job | Purpose | |
| 203 | |------------------|---------------------------------------| |
| 204 | | validate-pinning | Verify GitHub Actions use SHA pinning | |
| 205 | | check-staleness | Detect outdated SHA references | |
| 206 | | codeql-analysis | Full CodeQL security scan | |
| 207 | | summary | Aggregate security status report | |
| 208 | |
| 209 | ### Security Validation Tools |
| 210 | |
| 211 | | Tool | Script | Checks | |
| 212 | |--------------------|------------------------------|------------------------------------------| |
| 213 | | Dependency Pinning | `Test-DependencyPinning.ps1` | Actions use SHA refs, not tags | |
| 214 | | SHA Staleness | `Test-SHAStaleness.ps1` | SHAs reference recent commits | |
| 215 | | npm Audit | `npm audit` | Known vulnerabilities in dependencies | |
| 216 | | CodeQL | GitHub native | Code patterns indicating security issues | |
| 217 | | Gitleaks | `gitleaks` | Secret detection in repository history | |
| 218 | | Dependency Review | GitHub native | Dependency vulnerability analysis | |
| 219 | |
| 220 | ## Extension Publishing |
| 221 | |
| 222 | The `extension-publish.yml` and `extension-publish-prerelease.yml` workflows handle VS Code extension marketplace publishing through manual dispatch. Both workflows use collection-based packaging to produce and publish a separate VSIX per collection. |
| 223 | |
| 224 | ```mermaid |
| 225 | flowchart TD |
| 226 | subgraph Stable["extension-publish.yml"] |
| 227 | NV[normalize-version] --> PKG1["package (matrix)"] |
| 228 | PKG1 --> PUB1["publish (matrix)"] |
| 229 | end |
| 230 | subgraph PreRelease["extension-publish-prerelease.yml"] |
| 231 | VV[validate-version] --> PKG2["package (matrix)"] |
| 232 | PKG2 --> PUB2["publish (matrix)"] |
| 233 | end |
| 234 | ``` |
| 235 | |
| 236 | ### Publishing Jobs |
| 237 | |
| 238 | | Job | Purpose | Workflow | |
| 239 | |-------------------|-------------------------------------------------------------|------------------------------------| |
| 240 | | normalize-version | Ensure version consistency | `extension-publish.yml` | |
| 241 | | validate-version | Enforce odd minor version for pre-release channel | `extension-publish-prerelease.yml` | |
| 242 | | package (matrix) | Build one VSIX per collection using `extension-package.yml` | Both | |
| 243 | | publish (matrix) | Upload each VSIX to VS Code Marketplace via OIDC + vsce | Both | |
| 244 | |
| 245 | ### Collection-Based Packaging |
| 246 | |
| 247 | Collection manifests in `collections/*.collection.yml` define collection-scoped subsets of the full artifact set. The `extension-package.yml` reusable workflow discovers these manifests, filters by maturity and channel, and packages each as an independent VSIX. |
| 248 | |
| 249 | | Collection | Maturity | Included In | |
| 250 | |----------------|--------------|--------------------| |
| 251 | | `hve-core-all` | Stable | Stable, PreRelease | |
| 252 | | `developer` | Experimental | PreRelease only | |
| 253 | |
| 254 | Maturity filtering rules: |
| 255 | |
| 256 | * **Deprecated** collections are always excluded. |
| 257 | * **Experimental** collections are excluded from Stable channel builds. |
| 258 | * **Stable** collections are included in all channel builds. |
| 259 | |
| 260 | ### Version Channels |
| 261 | |
| 262 | | Channel | Version Pattern | Marketplace | |
| 263 | |-------------|--------------------|------------------| |
| 264 | | Stable | Even minor (1.2.0) | Main listing | |
| 265 | | Pre-release | Odd minor (1.3.0) | Pre-release flag | |
| 266 | |
| 267 | ## npm Script Mapping |
| 268 | |
| 269 | Workflows invoke validation through npm scripts defined in `package.json`: |
| 270 | |
| 271 | | npm Script | Command | Used By | |
| 272 | |--------------------------------|---------------------------------------------|----------------------------| |
| 273 | | `lint:md` | `markdownlint-cli2` | markdown-lint.yml | |
| 274 | | `lint:md:fix` | `markdownlint-cli2 --fix` | Local | |
| 275 | | `spell-check` | `cspell` | spell-check.yml | |
| 276 | | `spell-check:fix` | `cspell --show-suggestions` | Local | |
| 277 | | `lint:frontmatter` | `Validate-MarkdownFrontmatter.ps1` | frontmatter-validation.yml | |
| 278 | | `lint:md-links` | `Markdown-Link-Check.ps1` | markdown-link-check.yml | |
| 279 | | `lint:links` | `Invoke-LinkLanguageCheck.ps1` | link-lang-check.yml | |
| 280 | | `lint:yaml` | `Invoke-YamlLint.ps1` | yaml-lint.yml | |
| 281 | | `lint:ps` | `Invoke-PSScriptAnalyzer.ps1` | ps-script-analyzer.yml | |
| 282 | | `lint:collections-metadata` | `Validate-Collections.ps1` | plugin-validation.yml | |
| 283 | | `lint:marketplace` | `Validate-Marketplace.ps1` | plugin-validation.yml | |
| 284 | | `lint:version-consistency` | `Test-ActionVersionConsistency.ps1` | Local | |
| 285 | | `lint:all` | Chains all linters | Local | |
| 286 | | `format:tables` | `markdown-table-formatter` | table-format.yml | |
| 287 | | `test:ps` | `Invoke-PesterTests.ps1` | pester-tests.yml | |
| 288 | | `validate:skills` | `Validate-SkillStructure.ps1` | skill-validation.yml | |
| 289 | | `validate:copyright` | `Test-CopyrightHeaders.ps1` | copyright-headers.yml | |
| 290 | | `extension:prepare` | `Prepare-Extension.ps1` | extension-package.yml | |
| 291 | | `extension:prepare:prerelease` | `Prepare-Extension.ps1 -Channel PreRelease` | extension-package.yml | |
| 292 | | `extension:package` | `Package-Extension.ps1` | extension-package.yml | |
| 293 | | `package:extension` | Alias for `extension:package` | extension-package.yml | |
| 294 | | `extension:package:prerelease` | `Package-Extension.ps1 -PreRelease` | extension-package.yml | |
| 295 | | `plugin:generate` | `Generate-Plugins.ps1` + post-process | plugin-package.yml | |
| 296 | | `plugin:validate` | Alias for `lint:collections-metadata` | plugin-validation.yml | |
| 297 | |
| 298 | ## Related Documentation |
| 299 | |
| 300 | * [Testing Architecture](testing.md) - PowerShell Pester test infrastructure |
| 301 | * [Scripts README](../../scripts/README.md) - Script organization and usage |
| 302 | |
| 303 | 🤖 *Crafted with precision by ✨Copilot following brilliant human instruction, then carefully refined by our team of discerning human reviewers.* |
| 304 | |