GitVita
New repositoryImport repositoryNew issue
Your repositories Your dashboard Sign in

microsoft/hve-core

Public

mirrored fromhttps://github.com/microsoft/hve-coreAvailable

Watch0Fork0Star0
CodeCommitsIssuesPull requestsActionsInsightsSecurity
c5fcf0b3766ea51ef3e9e9317f1d596622255f70

Branches

Tags

  • No tags available.
0Branches0Tags
Go to file
Add file
Code

Clone

HTTPS

Download ZIP

hve-core/docs/architecture

docs/architecture/workflows.md

337lines · modecode

RawDownload
Latest commit unavailable.
1---
2title: Build Workflows
3description: GitHub Actions CI/CD pipeline architecture for validation, security, and release automation
4sidebar_position: 3
5author: WilliamBerryiii
6ms.date: 2026-05-20
7ms.topic: overview
8---
9
10HVE Core uses GitHub Actions for continuous integration, quality validation, security scanning, and release automation. The workflow architecture emphasizes reusable components and parallel execution for fast feedback.
11
12## Pipeline Overview
13
14```mermaid
15flowchart TD
16 subgraph PR["Pull Request"]
17 direction TB
18 PR1[PR Opened/Updated] --> PV[pr-validation.yml]
19 PV --> LINT[Linting Jobs]
20 PV --> SEC[Security Jobs]
21 PV --> TEST[Test Jobs]
22 end
23
24 subgraph MAIN["Main Branch"]
25 direction TB
26 MERGE[Merge to Main] --> MN[release-stable.yml]
27 MN --> VAL[Validation]
28 VAL --> PKG[Extension Package]
29 PKG --> REL[Release Please]
30 end
31
32 subgraph SCHED["Scheduled"]
33 direction TB
34 CRON[Weekly Sunday 2AM] --> WEEKLY[weekly-security-maintenance.yml]
35 WEEKLY --> SECCHECK[Security Checks]
36 end
37
38 subgraph MANUAL["Manual"]
39 direction TB
40 DISPATCH[Manual Trigger] --> PUB[release-marketplace-stable.yml]
41 PUB --> VSCE[Publish to Marketplace]
42 end
43```
44
45## Workflow Inventory
46
47| Workflow | Trigger | Purpose |
48|--------------------------------------|-------------------------|-------------------------------------------------------------------|
49| `pr-validation.yml` | Pull request, manual | Pre-merge quality gate with parallel validation |
50| `release-stable.yml` | Push to main, manual | Post-merge validation and release automation |
51| `weekly-security-maintenance.yml` | Sunday 2 AM UTC, manual | Scheduled security posture review |
52| `weekly-validation.yml` | Schedule, manual | Weekly full validation sweep |
53| `security-scan.yml` | Push to main/develop | CodeQL security validation |
54| `release-marketplace-stable.yml` | Manual | VS Code extension marketplace publishing |
55| `release-marketplace-prerelease.yml` | Manual | VS Code extension pre-release publishing |
56| `copilot-setup-steps.yml` | Manual | Coding agent environment setup |
57| `devcontainer-change-log.yml` | Push to main/develop | Logs devcontainer infrastructure file changes to the step summary |
58| `release-prerelease.yml` | PR closed | Pre-release tag and publish on merge to main |
59| `release-prerelease-pr.yml` | Push to main | Pre-release companion PR management |
60| `scorecard.yml` | Schedule, push | OpenSSF Scorecard security analysis |
61| `codeql-analysis.yml` | Schedule | Weekly CodeQL security scan (also reusable) |
62| `dependency-review.yml` | Pull request | Dependency vulnerability review (also reusable) |
63| `sha-staleness-check.yml` | Manual | SHA reference freshness check (also reusable) |
64| `deploy-docs.yml` | Push to main, manual | Docusaurus documentation site deployment |
65| `create-stale-docs-issues.yml` | Schedule | Automated stale docs issue creation from ms.date freshness |
66| `msdate-freshness-check.yml` | Schedule, manual | ms.date freshness validation across documentation |
67| `label-sync.yml` | Push to main, manual | Repository label synchronization |
68| `workflow-permissions-scan.yml` | Schedule, manual | GitHub Actions permissions audit |
69
70### Reusable Workflows
71
72Individual validation workflows called by orchestration workflows:
73
74| Workflow | Purpose | npm Script |
75|---------------------------------------|----------------------------------|-------------------------------------|
76| `markdown-lint.yml` | Markdownlint validation | `npm run lint:md` |
77| `spell-check.yml` | cspell dictionary check | `npm run spell-check` |
78| `frontmatter-validation.yml` | AI artifact frontmatter schemas | `npm run lint:frontmatter` |
79| `markdown-link-check.yml` | Broken link detection | `npm run lint:md-links` |
80| `link-lang-check.yml` | Link language validation | `npm run lint:links` |
81| `yaml-lint.yml` | YAML syntax validation | `npm run lint:yaml` |
82| `ps-script-analyzer.yml` | PowerShell static analysis | `npm run lint:ps` |
83| `table-format.yml` | Markdown table formatting | `npm run format:tables` |
84| `pester-tests.yml` | PowerShell unit tests | `npm run test:ps` |
85| `skill-validation.yml` | Skill structure validation | `npm run validate:skills` |
86| `dependency-pinning-scan.yml` | Dependency pinning validation | N/A (PowerShell direct) |
87| `sha-staleness-check.yml` | SHA reference freshness* | N/A (PowerShell direct) |
88| `codeql-analysis.yml` | CodeQL security scanning* | N/A (GitHub native) |
89| `dependency-review.yml` | Dependency vulnerability review* | N/A (GitHub native) |
90| `extension-package.yml` | VS Code extension packaging | `npm run extension:package` |
91| `copyright-headers.yml` | Copyright header validation | `npm run validate:copyright` |
92| `gitleaks-scan.yml` | Secret detection scanning | N/A (gitleaks direct) |
93| `plugin-package.yml` | Plugin collection packaging | N/A |
94| `plugin-validation.yml` | Plugin and collection metadata | `npm run lint:collections-metadata` |
95| `extension-marketplace-publish.yml` | Extension marketplace publishing | N/A |
96| `python-lint.yml` | Python linting (ruff) | `npm run lint:py` |
97| `pytest-tests.yml` | Python unit tests | `npm run test:py` |
98| `pip-audit.yml` | Python dependency auditing | N/A (pip-audit direct) |
99| `fuzz-tests.yml` | Python fuzz testing | N/A (pytest direct) |
100| `docusaurus-tests.yml` | Docusaurus test suite | N/A (npm test) |
101| `model-validation.yml` | Model reference validation | `npm run lint:models` |
102| `ai-artifact-validation.yml` | AI artifact structure validation | `npm run lint:ai-artifacts` |
103| `action-version-consistency-scan.yml` | Action version consistency | `npm run lint:version-consistency` |
104
105Workflows marked with `*` are dual-purpose: they accept `workflow_call` for reuse by orchestration workflows and also run independently via their own triggers.
106
107## PR Validation Pipeline
108
109The `pr-validation.yml` workflow serves as the primary quality gate for all pull requests. It runs 16 parallel jobs covering linting, security, and testing.
110
111```mermaid
112flowchart LR
113 subgraph "Linting"
114 ML[markdown-lint]
115 SC[spell-check]
116 TF[table-format]
117 YL[yaml-lint]
118 FV[frontmatter-validation]
119 LLC[link-lang-check]
120 MLC[markdown-link-check]
121 CH[copyright-headers]
122 end
123
124 subgraph "Analysis"
125 PSA[psscriptanalyzer]
126 PT[pester-tests]
127 SV[skill-validation]
128 PV[plugin-validation]
129 end
130
131 subgraph "Security"
132 DPC[dependency-pinning-check]
133 NA[npm-audit]
134 CQL[codeql]
135 GLS[gitleaks-scan]
136 end
137```
138
139### Jobs
140
141| Job | Reusable Workflow | Validates |
142|--------------------------|-------------------------------|--------------------------------|
143| spell-check | `spell-check.yml` | Spelling across all files |
144| markdown-lint | `markdown-lint.yml` | Markdown formatting rules |
145| table-format | `table-format.yml` | Markdown table structure |
146| psscriptanalyzer | `ps-script-analyzer.yml` | PowerShell code quality |
147| yaml-lint | `yaml-lint.yml` | YAML syntax |
148| pester-tests | `pester-tests.yml` | PowerShell unit tests |
149| frontmatter-validation | `frontmatter-validation.yml` | AI artifact metadata |
150| skill-validation | `skill-validation.yml` | Skill directory structure |
151| link-lang-check | `link-lang-check.yml` | Link accessibility |
152| markdown-link-check | `markdown-link-check.yml` | Broken links |
153| dependency-pinning-check | `dependency-pinning-scan.yml` | Dependency pinning |
154| npm-audit | Inline | npm dependency vulnerabilities |
155| codeql | `codeql-analysis.yml` | Code security patterns |
156| copyright-headers | `copyright-headers.yml` | Copyright header compliance |
157| plugin-validation | `plugin-validation.yml` | Plugin and collection metadata |
158| gitleaks-scan | `gitleaks-scan.yml` | Secret detection |
159
160All jobs run in parallel with no dependencies, enabling fast feedback (typically under 3 minutes).
161
162## Main Branch Pipeline
163
164The `release-stable.yml` workflow runs after merges to main, performing validation and release automation.
165
166```mermaid
167flowchart LR
168 V1[spell-check] --> RP[release-please]
169 V2[markdown-lint] --> RP
170 V3[table-format] --> RP
171 V4[dependency-pinning-scan] --> RP
172 V5[gitleaks-scan] --> RP
173 V6[pester-tests] --> RP
174 RP --> RST[reset-prerelease]
175 RP -->|release_created| EPR[extension-package-release]
176 RP -->|release_created| PPR[plugin-package-release]
177 RP -->|release_created| SBOM[generate-dependency-sbom]
178 EPR --> ATT[attest-and-upload]
179 SBOM --> ATT
180 PPR --> UPP[upload-plugin-packages]
181 SBOM --> SD[sbom-diff]
182 ATT --> PUB[publish-release]
183 UPP --> PUB
184 SD --> PUB
185 style RP fill:#f9f,stroke:#333
186```
187
188Release-please v4 handles `chore`-type commits natively. They are not releasable and do not produce spurious release PRs, so no commit-message guard is needed.
189
190### Main Branch Jobs
191
192| Job | Purpose | Dependencies |
193|---------------------------|--------------------------------|----------------------------------------------------------------------|
194| spell-check | Post-merge spelling validation | None |
195| markdown-lint | Post-merge markdown validation | None |
196| table-format | Post-merge table validation | None |
197| dependency-pinning-scan | Security pinning check | None |
198| gitleaks-scan | Secret detection scanning | None |
199| pester-tests | PowerShell unit tests | None |
200| release-please | Automated release management | All validation jobs |
201| reset-prerelease | Reset pre-release tracking | release-please |
202| extension-package-release | Build release VSIX | release-please (conditional) |
203| plugin-package-release | Build release plugin packages | release-please (conditional) |
204| generate-dependency-sbom | Generate dependency SBOM | release-please (conditional) |
205| attest-and-upload | Sign and upload VSIX | release-please, extension-package-release, generate-dependency-sbom |
206| upload-plugin-packages | Upload plugin packages | release-please, plugin-package-release |
207| sbom-diff | Compare SBOM changes | release-please, generate-dependency-sbom |
208| publish-release | Finalize GitHub Release | release-please, attest-and-upload, upload-plugin-packages, sbom-diff |
209
210When release-please creates a release, parallel jobs build the extension VSIX (`extension-package-release`), package plugin collections (`plugin-package-release`), and generate an SBOM (`generate-dependency-sbom`). The `attest-and-upload` job signs the VSIX with Sigstore attestation, `upload-plugin-packages` uploads collection artifacts, and `sbom-diff` compares dependency changes. The `publish-release` job finalizes the GitHub Release after all artifacts are ready.
211
212## Security Workflows
213
214### Weekly Security Maintenance
215
216The `weekly-security-maintenance.yml` workflow runs every Sunday at 2AM UTC, providing scheduled security posture review.
217
218| Job | Purpose |
219|------------------|--------------------------------------|
220| validate-pinning | Verify dependency pinning compliance |
221| check-staleness | Detect outdated SHA references |
222| codeql-analysis | Full CodeQL security scan |
223| summary | Aggregate security status report |
224
225### Security Validation Tools
226
227| Tool | Script | Checks |
228|--------------------|------------------------------|-----------------------------------------------|
229| Dependency Pinning | `Test-DependencyPinning.ps1` | Actions use SHA refs; npm uses exact versions |
230| SHA Staleness | `Test-SHAStaleness.ps1` | SHAs reference recent commits |
231| npm Audit | `npm audit` | Known vulnerabilities in dependencies |
232| CodeQL | GitHub native | Code patterns indicating security issues |
233| Gitleaks | `gitleaks` | Secret detection in repository history |
234| Dependency Review | GitHub native | Dependency vulnerability analysis |
235
236## Extension Publishing
237
238The `release-marketplace-stable.yml` and `release-marketplace-prerelease.yml` workflows handle VS Code extension marketplace publishing through manual dispatch. Both workflows use collection-based packaging to produce and publish a separate VSIX per collection.
239
240```mermaid
241flowchart TD
242 subgraph Stable["release-marketplace-stable.yml"]
243 NV[normalize-version] --> PKG1["package (matrix)"]
244 PKG1 --> PUB1["publish (matrix)"]
245 end
246 subgraph PreRelease["release-marketplace-prerelease.yml"]
247 VV[validate-version] --> PKG2["package (matrix)"]
248 PKG2 --> PUB2["publish (matrix)"]
249 end
250```
251
252### Publishing Jobs
253
254| Job | Purpose | Workflow |
255|-------------------|-------------------------------------------------------------|--------------------------------------|
256| normalize-version | Ensure version consistency | `release-marketplace-stable.yml` |
257| validate-version | Enforce odd minor version for pre-release channel | `release-marketplace-prerelease.yml` |
258| package (matrix) | Build one VSIX per collection using `extension-package.yml` | Both |
259| publish (matrix) | Upload each VSIX to VS Code Marketplace via OIDC + vsce | Both |
260
261### Collection-Based Packaging
262
263Collection manifests in `collections/*.collection.yml` define collection-scoped subsets of the full artifact set. The `extension-package.yml` reusable workflow discovers these manifests, filters by maturity and channel, and packages each as an independent VSIX.
264
265| Collection | Maturity | Included In |
266|--------------------|--------------|--------------------|
267| `hve-core-all` | Stable | Stable, PreRelease |
268| `hve-core` | Stable | Stable, PreRelease |
269| `ado` | Stable | Stable, PreRelease |
270| `github` | Stable | Stable, PreRelease |
271| `project-planning` | Stable | Stable, PreRelease |
272| `coding-standards` | Stable | Stable, PreRelease |
273| `data-science` | Stable | Stable, PreRelease |
274| `security` | Experimental | Stable, PreRelease |
275| `design-thinking` | Preview | Stable, PreRelease |
276| `installer` | Stable | Stable, PreRelease |
277| `experimental` | Experimental | PreRelease only |
278
279Maturity filtering rules:
280
281| Maturity Level | Build Inclusion |
282|----------------|-------------------------------------------------|
283| Deprecated | Always excluded |
284| Experimental | Excluded from Stable channel builds |
285| Preview | Included in both Stable and PreRelease channels |
286| Stable | Included in all channel builds |
287
288### Version Channels
289
290| Channel | Version Pattern | Marketplace |
291|-------------|--------------------|------------------|
292| Stable | Even minor (1.2.0) | Main listing |
293| Pre-release | Odd minor (1.3.0) | Pre-release flag |
294
295## npm Script Mapping
296
297Workflows invoke validation through npm scripts defined in `package.json`:
298
299| npm Script | Command | Used By |
300|--------------------------------|---------------------------------------------|-------------------------------|
301| `lint:md` | `markdownlint-cli2` | markdown-lint.yml |
302| `lint:md:fix` | `markdownlint-cli2 --fix` | Local |
303| `spell-check` | `cspell` | spell-check.yml |
304| `spell-check:fix` | `cspell --show-suggestions` | Local |
305| `lint:frontmatter` | `Validate-MarkdownFrontmatter.ps1` | frontmatter-validation.yml |
306| `lint:md-links` | `Markdown-Link-Check.ps1` | markdown-link-check.yml |
307| `lint:links` | `Invoke-LinkLanguageCheck.ps1` | link-lang-check.yml |
308| `lint:yaml` | `Invoke-YamlLint.ps1` | yaml-lint.yml |
309| `lint:ps` | `Invoke-PSScriptAnalyzer.ps1` | ps-script-analyzer.yml |
310| `lint:collections-metadata` | `Validate-Collections.ps1` | plugin-validation.yml |
311| `lint:marketplace` | `Validate-Marketplace.ps1` | plugin-validation.yml |
312| `lint:version-consistency` | `Test-ActionVersionConsistency.ps1` | Local |
313| `lint:all` | Chains all linters | Local |
314| `format:tables` | `markdown-table-formatter` | table-format.yml |
315| `test:ps` | `Invoke-PesterTests.ps1` | pester-tests.yml |
316| `validate:skills` | `Validate-SkillStructure.ps1` | skill-validation.yml |
317| `validate:copyright` | `Test-CopyrightHeaders.ps1` | copyright-headers.yml |
318| `extension:prepare` | `Prepare-Extension.ps1` | extension-package.yml |
319| `extension:prepare:prerelease` | `Prepare-Extension.ps1 -Channel PreRelease` | extension-package.yml |
320| `extension:package` | `Package-Extension.ps1` | extension-package.yml |
321| `package:extension` | Alias for `extension:package` | extension-package.yml |
322| `extension:package:prerelease` | `Package-Extension.ps1 -PreRelease` | extension-package.yml |
323| `plugin:generate` | `Generate-Plugins.ps1` + post-process | plugin-package.yml |
324| `plugin:validate` | Alias for `lint:collections-metadata` | plugin-validation.yml |
325| `lint:py` | `ruff check` | python-lint.yml |
326| `lint:models` | `Validate-ModelReferences.ps1` | model-validation.yml |
327| `lint:ai-artifacts` | `Validate-AIArtifacts.ps1` | ai-artifact-validation.yml |
328| `lint:permissions` | `Test-WorkflowPermissions.ps1` | workflow-permissions-scan.yml |
329| `lint:dependency-pinning` | `Test-DependencyPinning.ps1` | dependency-pinning-scan.yml |
330| `test:py` | `pytest` | pytest-tests.yml |
331
332## Related Documentation
333
334* [Testing Architecture](testing.md) - PowerShell Pester test infrastructure
335* [Scripts README](https://github.com/microsoft/hve-core/blob/main/scripts/README.md) - Script organization and usage
336
337🤖 *Crafted with precision by ✨Copilot following brilliant human instruction, then carefully refined by our team of discerning human reviewers.*
338