Use when migrating Kibana Functional Test Runner (FTR) tests to Scout, including decisions about UI vs API tests, mapping FTR services/page objects/hooks to Scout fixtures, and splitting loadTestFile patterns.
Migrate FTR tests to Scout by deciding whether a test should be UI or API, mapping FTR concepts to Scout fixtures, and rewriting loadTestFile-based suites into standalone Scout specs. Core principle: keep UI tests focused on user interactions and move data validation to API tests.
test/scout path).loadTestFile() indexes.test() runs in a fresh browser context: if an FTR suite used multiple it() blocks as one journey, combine into one test() + test.step(). Do login/navigation in beforeEach (avoid page/browserAuth/pageObjects in beforeAll).describe, and don't use *.describe.configure().parallel_tests/: ingest via parallel_tests/global.setup.ts + globalSetupHook (don't use esArchiver in spec files).@kbn/scout vs @kbn/scout-security/@kbn/scout-oblt/@kbn/scout-search) and import expect from /ui or /api.test/scout/**/* into the plugin root tsconfig.json and add Scout kbn_references (like discover_enhanced), or keep dedicated test/scout/{ui,api}/tsconfig.json files. Only the latter forbids relative imports into server/ / public/.uiSettings / scoutSpace.uiSettings and (when needed) apiServices.core.settings(...).browserAuth (UI), requestAuth (API key), samlAuth (cookie / cookieHeader), plus custom roles. Avoid FTR-style role mutation. For Scout API tests, see Scout API auth (cookieHeader vs API key) under step 4.<module-root>/test/scout*/ui/{tests,parallel_tests}/**/*.spec.ts<module-root>/test/scout*/api/{tests,parallel_tests}/**/*.spec.tsui/parallel_tests/ + spaceTest when the flow can be space-isolated (state is scoped to a Kibana space) and should run in parallel; otherwise use ui/tests/ + test. See Scout parallelism for details on when to choose parallel vs sequential.api/tests/ (sequential). Use api/parallel_tests/ + parallel.playwright.config.ts only when the test is safe to run in parallel (no shared state) and you need the speedup.scoutSpace.id).For available tag helpers and their meaning, see Deployment tags.
FTR deployment-agnostic configs often load the same files under both stateful and serverless. In Scout, do not assume tags.deploymentAgnostic is the right default for every migrated spec. Instead:
x-pack/solutions/observability|security|search/...): use explicit solution targets (e.g. [...tags.stateful.classic, ...tags.serverless.observability.complete]) so CI only runs where that solution is present. Match sibling specs in the same module.src/platform/**, x-pack/platform/**): tags.deploymentAgnostic is appropriate when the original intent was "run everywhere."API and UI specs should both carry tags that match the intended run-tests / CI targets; see step 9.
describe/it -> test.describe/test or apiTest.describe/apiTest (but don't assume 1:1 it -> test).before/after -> test.beforeAll/test.afterAll.beforeEach/afterEach -> test.beforeEach/test.afterEach.describe; use test.step() inside a test for structure).describe blocks, split into multiple Scout specs (one describe per file).describe blocks: if the FTR file has nested describes, prefer splitting into separate Scout spec files. However, if the file is small and the nested describes are lightweight, flatten them into a single test.describe with individual test(...) blocks using test.step(...) for sub-structure instead of creating many tiny spec files.FTR often has separate but near-identical test files under test/*api_integration*/ (stateful) and test/serverless/ (or similar directories). Before migrating each file individually, compare them: if the test flow is identical or almost identical, combine into a single Scout spec with tags covering both deployment targets (e.g. [...tags.stateful.classic, ...tags.serverless.observability.complete]). Extract any deployment-specific differences into conditional helpers or small branching within the spec. Only keep separate specs when the flows genuinely diverge.
it blocks are sometimes steps (not full test cases)In FTR it's common for multiple it(...) blocks in one describe(...) to behave like a single user journey (shared browser state across its).
In Scout (Playwright), each test(...) runs with a fresh browser context, so you usually can't preserve that state across multiple tests.
Guideline:
it(...) blocks as sequential steps of one flow, combine them into a single test(...) and convert the step boundaries into test.step(...).it(...) block is already an independent test case, keep it as its own test(...) and ensure it sets up its own preconditions.Minimal sketch:
// FTR: multiple `it`s continue in the same browser context
it('create entity', async () => {});
it('edit entity', async () => {}); // continues...
// Scout: combine into one test and use `test.step` for debuggability
test('create and edit entity', async () => {
await test.step('create entity', async () => {});
await test.step('edit entity', async () => {});
});
supertest calls with Scout apiClient (endpoint under test) + requestAuth/samlAuth (auth). FTR stateful tests often use supertest with an implicit admin role—don't carry that over blindly. Research whether a lower default role like editor or viewer is sufficient; comparing the roles used in the serverless version of the same test (if one exists) is a good starting point.pageObjects, browserAuth, apiServices, kbnClient, esArchiver).apiServices/kbnClient for setup/teardown and verifying side effects.before/beforeEach/after/afterEach and verify it is still correct for Scout: replace FTR-specific APIs with their Scout equivalents, remove unnecessary calls (e.g. FTR service initialization that Scout fixtures handle automatically), and add any missing setup or cleanup that the FTR suite neglected. Ensure every resource created in beforeAll/beforeEach has matching cleanup in afterAll/afterEach—FTR suites frequently lack proper teardown. Place kbnClient.savedObjects.cleanStandardList() (or scoutSpace.savedObjects.cleanStandardList()) in afterAll, not beforeAll; beforeAll cleanup masks missing teardown and hides leaked state from previous runs.beforeAll/afterAll.uiSettings / scoutSpace.uiSettings, and (when needed) apiServices.core.settings(...).@kbn/scout vs @kbn/scout-<solution>), and import expect from /ui or /api.@kbn/rison and add it to test/scout*/ui/tsconfig.json kbn_references.cookieHeader vs API key)For general Scout API auth patterns (requestAuth, samlAuth, common headers, code examples), see Authentication in Scout API tests.
FTR mapping: FTR roleScopedSupertest with useCookieHeader: true / withInternalHeaders maps to samlAuth + cookieHeader merged with common headers on apiClient requests. FTR supertest with API key auth maps to requestAuth.getApiKey(...) + apiKeyHeader.
FTR migration gotchas (not covered in the general doc):
core.security.authc.apiKeys.create (nested API keys) often fail with HTTP 500 when the incoming request uses an API key. Use samlAuth.asInteractiveUser('admin') (or the role FTR used with cookies) for those routes.kibana: [] privileges) and expect 404 from scoped saved-object access: use samlAuth.asInteractiveUser(customRoleDescriptor) + cookieHeader. requestAuth.getApiKeyForCustomRole(...) can resolve 200 for the same role shape because API-key privilege resolution differs from an interactive session.loadTestFile target becomes its own Scout spec.test/scout*/ui/fixtures/helpers.ts (or API helpers in API fixtures).fixtures/constants.ts for reuse across tests and page objects.parallel_tests/ ingestion, use parallel_tests/global.setup.ts + globalSetupHook (no esArchiver in spec files).Import the fixture from @kbn/scout-synthtrace (not from @kbn/scout / @kbn/scout-oblt alone). Merge it into your module's apiTest in test/scout*/api/fixtures/index.ts:
import { apiTest as baseApiTest, mergeTests } from '@kbn/scout-oblt'; // or '@kbn/scout' for platform-only modules
import { synthtraceFixture } from '@kbn/scout-synthtrace';
export const apiTest = mergeTests(baseApiTest, synthtraceFixture);
Specs then receive worker fixtures such as logsSynthtraceEsClient (index / clean) for @kbn/synthtrace-client generators (log, timerange, etc.).
Add the same Scout kbn_references on whichever tsconfig.json includes the Scout API files: either the plugin root tsconfig.json or test/scout*/api/tsconfig.json. Typical API set: @kbn/scout-oblt (or @kbn/scout), @kbn/scout-synthtrace, @kbn/synthtrace-client (add @kbn/synthtrace only if types require it). UI-only synthtrace: same @kbn/scout-synthtrace import in fixtures.
See TypeScript layout in the scout-create-scaffold skill for full Pattern A / Pattern B details (what to add to tsconfig.json, kbn_references, and the yarn kbn bootstrap / type_check steps).
Choosing: Prefer Pattern A when migrating FTR tests that already imported registration constants or server helpers. Prefer Pattern B when you want minimal plugin compile cost and can keep imports boundary-safe.
If the module uses Pattern B, treat the Scout API directory as isolated:
server/ / public/ just to reuse a string constant—use api/fixtures/constants.ts or move the constant to common/ if both prod and tests should share it.@kbn/scout* / @kbn/synthtrace-client per that folder's kbn_references.FTR migration tip: FTR often imported server files because tests sat in the plugin program. Pattern A preserves that. Pattern B matches "thin" e2e deps—duplicate small literals or use fixtures when adding server/ to the Scout tsconfig graph is wrong.
it block needs a Scout equivalent.my_component.test.tsx alongside my_component.tsx).loadTestFile entries from any stateful and serverless FTR configs/index files.describe.skip to avoid duplicate coverage.node scripts/type_check --project <plugin-root>/tsconfig.json. For Pattern B, run node scripts/type_check --project <plugin>/test/scout/api/tsconfig.json (and UI project if present). Use full node scripts/type_check when shared types changed broadly. Huge TS6059 / TS6307 counts under a Scout-only project usually mean Pattern B + forbidden server/ relatives—switch to Pattern A or fix imports (step 6).node scripts/scout.js run-tests --arch stateful --domain classic --testFiles <path> and
node scripts/scout.js run-tests --arch serverless --domain observability_complete --testFiles <path> (adjust serverless domain).test/scout_<configSet>/..., run-tests auto-detects the server config set from the Playwright config path.start-server has no Playwright config to inspect, so pass --serverConfigSet <configSet> when your tests require a custom config set.tags.deploymentAgnostic (see Tags when the FTR suite was "deployment agnostic" under step 2).scout-best-practices-reviewer skill on all new/changed Scout spec files.blocker and major findings before finalizing; discuss minor and nit items as appropriate.test.step(...) inside a single test(...) when an FTR suite used multiple it(...) blocks as one journey.spaceTest + scoutSpace; avoid hardcoded saved object IDs and make names unique (often suffix with scoutSpace.id).globalSetupHook in parallel_tests/global.setup.ts to ingest shared data once.page.addInitScript(...) before navigation to set localStorage/cookies (skip tours/onboarding).@kbn/rison and add @kbn/rison to kbn_references on the tsconfig.json that includes the Scout UI files (plugin root under Pattern A, or test/scout/ui/tsconfig.json under Pattern B).data-test-subj attributes when selectors are unstable.loadTestFile suites into separate Scout specs.test/scout*/{ui,api}/{tests,parallel_tests}.tests/ with parallel_tests/).@kbn/scout).tags.deploymentAgnostic for specs under a solution plugin/package when the FTR suite was only “deployment agnostic” in the sense of shared stateful+serverless observability (or security/search) configs—those jobs still differ from the broad deploymentAgnostic tag set; use explicit tags.stateful.* + tags.serverless.<solution> instead (see step 2).expect from the wrong entrypoint (use /ui for UI, /api for API).esArchiver in parallel_tests/ spec files (ingest in parallel_tests/global.setup.ts instead).describe blocks or *.describe.configure() (split into separate specs, or flatten small files into test + test.step—see step 3).test(...) blocks (fresh browser context per test).expect-expect requires assertions in the test body; page objects should return state, not assert).test(...) blocks into a single spec file. Keep specs focused: 4–5 short scenarios or 2–3 long scenarios per file. Oversized specs create bottlenecks in parallel execution.requestAuth.getApiKey('admin') for internal routes whose handlers create nested API keys—often HTTP 500; use samlAuth.asInteractiveUser and merge cookieHeader (see step 4).getApiKeyForCustomRole for FTR parity on scoped saved-object / RBAC assertions that used cookie + custom role—prefer samlAuth.asInteractiveUser(customRoleDescriptor) + cookieHeader so outcomes match FTR (e.g. 404 vs 200).test/scout*/api/ into server/ / public/ (e.g. server/saved_objects/...). Fix by Pattern A (test/scout/**/* in the plugin tsconfig + Scout kbn_references) or duplicate constants in api/fixtures/constants.ts (step 6).test/scout/**/* to include or omitting @kbn/scout-oblt / synthtrace kbn_references—Scout files won't typecheck in check_types.