Cve Mitigation

Use --fail-on critical or --fail-on high to get a non-zero exit code.

CI scan (leeway sbom)

CI uses leeway sbom export + leeway sbom scan against components:needs-vuln-scan. Leeway embeds both syft (SBOM generation) and grype (vulnerability matching) internally. The CI scan is slow (requires full build cache) and runs on every push to main and on PRs.

The daily scheduled run (build.yml, cron 0 0 * * *) checks vulnerability-stats.json for critical counts and sends Slack alerts via notify-scheduled-failure.

Read references/ci-scanning.md for the full CI pipeline details.

Trivy image scanning

A separate Trivy-based scan (scripts/trivy/trivy-scan-images.sh) scans built Docker images. It uses scripts/trivy/trivyignore.yaml for suppressions. This is distinct from the Go dependency scanning.

Triage

When a CVE is reported:

1. Check severity and CVSS score. Focus on Critical (CVSS ≥ 9.0) and High (CVSS ≥ 7.0) first.
2. Check if a fix exists. Use --only-fixed with grype. If no fix exists, document and defer.
3. Check exploitability in context. Not all CVEs are exploitable in Gitpod's usage. If a CVE is a false positive or not exploitable, add it to WORKSPACE.yaml under sbom.ignoreVulnerabilities with a vulnerability ID and a reason explaining why.
4. Check for breaking API changes. Some upgrades (e.g., runc v1.1.x → v1.2.x) change exported APIs. These require code migration, not just go get.

Remediation (Go dependencies)

Standard update flow

For each affected module:

cd components/<module>

# Update a specific dependency
go get google.golang.org/[email protected]

# Update transitive dependencies that need bumping
go get golang.org/x/net@latest

# Clean up
go mod tidy

# Verify it builds
go build ./...

Multi-module coordination

This repo has many Go modules that share dependencies via replace directives in go.mod. When updating a dependency:

1. Start with leaf modules (APIs, protocol packages) and work up to service modules.
2. Check replace directives. Many modules use replace to point to local paths:

   replace github.com/gitpod-io/gitpod/common-go => ../common-go // leeway
   

   If common-go bumps a dependency, all modules that replace-reference it will transitively pick up the new version.
3. Update all modules that directly depend on the vulnerable package. Use:

   grep -rl "google.golang.org/grpc" components/*/go.mod | sort
   

4. Don't forget dev/ and test/ modules. CI builds these too. They often lag behind.
5. Run go mod tidy in every updated module. Missing this causes CI failures.

OpenTelemetry replace directives

Some modules (notably image-builder-bob) use replace directives to pin OpenTelemetry versions. When upgrading grpc, the OTel SDK version requirements may change. Update the replace directives to match:

replace (
    go.opentelemetry.io/otel => go.opentelemetry.io/otel v1.39.0
    go.opentelemetry.io/otel/sdk => go.opentelemetry.io/otel/sdk v1.39.0
    // ... etc
)

Deprecated package migration

When a package is deprecated and replaced (e.g., dgrijalva/jwt-go → golang-jwt/jwt/v5):

1. Update go.mod: remove old dependency, add new one.
2. Update import paths in .go files.
3. Verify API compatibility — most maintained forks keep the same API surface.
4. Run go build ./... and go test ./... to confirm.

Packages with breaking changes

Some upgrades cannot be done with a simple version bump. Document these in the PR under "Not addressed (requires follow-up)" with the specific breaking change. Example: opencontainers/runc v1.1.x → v1.2.x made libcontainer/cgroups/ebpf functions unexported.

Suppressing false positives

Two suppression mechanisms exist:

WORKSPACE.yaml (leeway sbom scan)

Add entries under sbom.ignoreVulnerabilities in WORKSPACE.yaml: