Verity
Start free

Verity docs

Verity is CI-native: every test runs in your pipeline and posts results to Verity's ingestion API. One platform covers UI, API, mobile, performance, and security — each an additive engine you can turn on independently. We never host your browsers for paid usage, and never receive your secrets.

The fastest path: open the dashboard → a project → CI Setup, toggle the tests you want, and copy the generated config for your provider.

1 · Quickstart

Create a project in the dashboard and copy its ingest token into your CI secrets as VERITY_INGEST_TOKEN. Then add the config from CI Setup (it's generated for your exact selection). That's it — PRs get a fast Quick check, merges get the Full regression.

2 · Choosing which tests to run

You don't need all engines. In CI Setup, toggle any of: ui, api (flow), contract, mobile, perf, security. The engines input controls which steps run; the default ui keeps the classic behavior. Every engine maps its native output into one unified results schema, so the dashboard, PR comments, and predictions work the same regardless of what you enable.

3 · GitHub Actions

The first-party Action wraps everything. CI Setup generates this for your selection:

# .github/workflows/verity.yml
- uses: qashift/verity-action@v1
  with:
    tier: fast            # 'fast' on PRs, 'full' on merge
    token: ${{ secrets.VERITY_INGEST_TOKEN }}
    engines: ui,api,perf,security   # pick what you need

4 · GitLab CI

No GitHub Action needed — GitLab fetches the same self-contained runners via curl and reports to the same dashboard. Store VERITY_INGEST_TOKEN as a masked CI/CD variable. CI Setup generates the full .gitlab-ci.yml; the core is:

verity:fast:
  image: mcr.microsoft.com/playwright:v1.49.1-noble
  services: ["docker:dind"]
  script:
    - curl -Ls https://raw.githubusercontent.com/qashift/verity-action/v1/verity.mjs -o verity.mjs
    - curl -Ls https://raw.githubusercontent.com/qashift/verity-action/v1/verity-engines.mjs -o verity-engines.mjs
    - node verity.mjs run --tier fast --token "$VERITY_INGEST_TOKEN"

5 · Jenkins

Same approach via a Docker agent. Add VERITY_INGEST_TOKEN as a Jenkins credential (secret text); the agent needs docker run for the security/contract steps. CI Setup generates the full Jenkinsfile.

5b · CLI

Prefer not to use a CI integration? The CLI runs the same pipeline from any shell. Install it, then run the tier you want — it reruns failures to separate flaky from broken and posts results to Verity.

npm i -D @qashifthq/verity

# Quick check — diff-aware subset (fast, every PR)
npx verity run --tier fast --token $VERITY_INGEST_TOKEN

# Full regression — the entire suite (on merge)
npx verity run --tier full --token $VERITY_INGEST_TOKEN

There's also a local MCP server (@qashifthq/verity-mcp) so an AI IDE can propose, generate, and run tests in your repo. See the Local / MCP tab in your project.

6 · UI testing (Playwright)

AI-generated Playwright specs with grounded locators, run in your CI. Failures are rerun to separate flaky from broken; broken locators are self-healed (below). The fast tier is a diff-aware Quick check; only the Full regression tier can block a merge.

7 · API testing

Verity tests each endpoint two complementary ways and merges them so you see API health, not tooling:

  • Example journeys — Playwright request tests for real flows (create → read → update), generated from your OpenAPI spec.
  • Contract checks — Verity exercises every operation with automatically generated inputs and reports only the minimal reproducing failure.

The dashboard groups results by endpoint; a failure both methods catch is shown once and marked double-verified. Contract testing needs a reachable base URL — for internal APIs it runs only inside your CI.

8 · Mobile

AI-generated mobile app flows (YAML) in tests/verity/mobile/, run on an Android emulator or iOS simulator you provision in your workflow (cost is yours). Broken selectors self-heal from the screen's accessibility-tree dump — the mobile equivalent of a DOM snapshot — delivered as a PR diff. Real-device runs via a cloud device farm are available (Scale/Enterprise), gated behind a flag.

9 · Performance

Verity's performance engine is a load test for your API, not a website crawler. It generates concurrent traffic against the endpoints/flows Verity builds from your OpenAPI spec and measures latency percentiles (p50/p95/p99) and error rate under load — it does smoke, soak, and spike by shaping phases. For page-load performance, enable Core Web Vitals (the front-end half).

Key terms

  • VU (virtual user) — one simulated user running a scenario.
  • arrivalRate — new VUs started per second; your load knob.
  • ramp — increase arrivalRate over time to find the breaking point.
  • p95 — watch this, not the average; it reflects your slowest real users.

Optimal config: smoke (every PR) stays tiny (arrivalRate 1–2, ~30s) — it catches “this PR made the API slower,” never capacity. Full load (merge) should start near real peak traffic and ramp; healthy = error rate < 1% and a stable p95 as load rises. One CI runner caps at a few hundred req/s — use distributed mode for more.

10 · Security

Dynamic analysis scans your running app at a URL — baseline on PRs, full active scan on merge. Static analysis scans your code (diff on PRs, full repo on schedule) in an isolated container. Findings are surfaced as severity-ranked issues, not pass/fail tests, and never block the build.

How to read them: work top-down — fix Critical/High before release, schedule Medium, batch Low/Info. SAST points at a file:line (precise — fix at source); DAST points at a URL (baseline can be noisy — confirm first). Follow the CWE link for the bug class and standard fix; a finding that disappears next scan is resolved.

11 · Self-healing

When a locator breaks, Verity sends the old selector + surrounding context (DOM for web, accessibility tree for mobile) to the healing model, which proposes a corrected locator. It is never written into your source automatically — you review and apply the diff (or merge the PR). One delivery path for every discipline.

12 · Predictions

  • Regression-prone module flagging (Live) — high-regression-rate files are flagged in PR comments.
  • Flake vs. real-failure disambiguation (Live) — each failure shows its historical flake rate.
  • Risk correlation & deploy confidence (Calibrating) — switch on after ~20 deploys of history.

13 · Ingestion API

The Action, CLI, and engine adapters all call this. The contract is stable; new disciplines use additive optional fields (findings, perf, per-result discipline/layer).

POST /api/ingest
Authorization: Bearer $VERITY_INGEST_TOKEN

{
  "tier": "fast",
  "commit_sha": "abc123",
  "branch": "feature/x",
  "results": [
    { "spec_path": "tests/checkout.spec.ts", "attempts": [false, true, true] }
  ]
  // security runs add "findings": [...]; performance adds "perf": {...}
}

attempts is the per-rerun pass/fail array: mixed = flaky, all-false = real failure, all-true = pass.

14 · Roles & team

Invite teammates from Team (they must have a Verity account first). Three roles:

  • Owner — full control + billing. One per org.
  • Admin — manage projects, members, and CI.
  • Member — read-only access to projects, tests, runs, and all dashboards.

Access is enforced by row-level security: members only ever see their own org's data.

15 · Cloud Sandbox

To get the feel without wiring CI, generation and a capped number of runs execute on our infra. It's intentionally limited — the product lives in your CI, where runs are unlimited.

16 · Licenses & open source

The core engine (@qashift/verity-core) is MIT-licensed. Every test engine Verity drives is best-in-class open source, invoked in isolation (never linked into our code), so you own the test files outright and there's no lock-in. The hosted dashboard, predictions, and team management are proprietary.