Analyze an FTR test directory and produce a structured Markdown migration plan for Scout. Use when planning an FTR-to-Scout migration, triaging FTR suites for Scout readiness, or when asked to create a migration plan before writing code. Does not write test code; outputs an architectural blueprint for an executor skill or human.
Analyze an FTR test directory end-to-end and produce a single Markdown file (migration_plan.md) that contains the key architectural decisions needed to migrate those tests to Scout. The planner does not write test code: it front-loads the analysis and high-level decisions so an executor (human or skill) can implement with minimal ambiguity.
Key principle: go deep on FTR analysis (read every file, understand every pattern, flag every issue). Stay at the decision level for Scout, as the executor skill already knows Scout's APIs, fixtures, and patterns. The planner's job is to say what was found and what should happen, not how to implement it in Scout code.
loadTestFile, configs, services, page objects).Before starting, collect or confirm:
x-pack/platform/test/functional/apps/dashboardx-pack/platform/test/functional/config.base.ts). Walk up from the test directory if not provided.x-pack/platform/plugins/shared/dashboard). If unknown, infer from the plugin that owns the FTR tests.Recursively read the FTR directory. For each file, record:
index (has loadTestFile), test (has describe/it), config, page_object, service, helper, archive, or fixtureit(...) / describe(...) blocksAlso read thoroughly:
kbnTestServer.serverArgs, esTestCluster.serverArgs, security.roles, security.defaultRoles, apps, testFiles, and any services/pageObjects registrations. Note config inheritance chains (which base config does it extend?).index.ts file that uses loadTestFile: capture shared before/after hooks and their setup logic. Note what state they create and whether downstream tests depend on it.existOrFail, missingOrFail, expect inside helpers).For every test file, decide:
| Decision | Criteria |
|---|---|
| Keep as UI test | Tests user flows, navigation, rendering that require a real browser and running server |
| Convert to API test | Asserts exact data values, validates API responses, or checks backend logic through the UI |
| Convert to unit test (RTL/Jest) | Tests isolated component logic: loading states, conditional rendering, table structure, tooltip content, form validation, hover behaviors |
| Drop | Duplicates existing coverage, tests deprecated features, or is an artifact of FTR limitations |
| Defer | Depends on capabilities that don't yet exist in Scout (flag explicitly what's missing) |
For each decision, write a one-line justification.
File splitting: when a single FTR file tests multiple roles or unrelated flows, recommend splitting it into separate specs (one role + one flow per file). List the proposed splits.
Rate each test file:
it blocks, no custom roles, no archives, straightforward page objectsit blocks, custom roles, archive loading, moderate page object usageSort the full inventory by complexity (simple first).
For each test (or group), decide whether it can run in parallel or must run sequentially:
Document which tests can share the same parallel pool and which need isolation. Explain why, referencing the specific state or mutation that drives the decision.
For each archive, data fixture, or setup pattern found:
kibanaServer.uiSettings.replace, uiSettings.update, uiSettings.delete call, which tests use them, and whether they use replace-all semantics (wipes all settings) vs selective setsuperuser or the FTR default role when a narrower role would suffice. For each, note what privileges the test actually exercisesrun_as, API-key-based auth, certificate auth, or any non-standard FTR authThe executor will decide the specific Scout auth API to use; the planner just provides the complete role inventory with privilege definitions and usage context.
existOrFail, missingOrFail, or expect calls that should move to specs)test/scout trees? Note: exists / exists-but-in-wrong-scope / missingfind.byCssSelector(...), find.byClassName(...), or text-based lookups. Note where data-test-subj attributes are missing in source code and need to be addedkbnTestServer.serverArgs and esTestCluster.serverArgs across all relevant configs (including inherited base configs)src/platform/packages/shared/kbn-scout/src/servers/configs/config_sets/For each test group, determine where it should run:
Cross-reference with existing FTR tags (@skipServerless, @skipStateful, etc.) and .buildkite/ftr_*_configs.yml to preserve CI coverage. Flag tests that currently only run in one environment but could run in both.
Cloud portability: flag FTR tests that make non-portable assumptions (hardcoded localhost URLs, local file paths, node topology assumptions, or cluster settings unavailable on Elastic Cloud).
Scan every test file for patterns that need attention during migration:
| Smell | What to flag |
|---|---|
| try/catch swallowing errors | try { ... } catch { } or catch blocks that don't rethrow |
| Conditional test logic | if/else inside it() blocks that change assertions based on runtime state |
| Global loading indicator waits | waitForSelector('globalLoadingIndicator') or similar global spinners |
| Hardcoded timeouts | await new Promise(r => setTimeout(r, ...)), browser.sleep(...) |
| Shared mutable state | Variables mutated across it() blocks relying on execution order |
Sequential journey as separate it blocks | Multiple it() blocks that form a single user journey (shared browser state) |
| Duplicate test cases | Multiple it() blocks testing the same behavior with minor variations |
| Missing cleanup | Setup in before/beforeEach without corresponding teardown |
| Retry wrappers | retry.try(...), retry.waitFor(...) around assertions |
| UI-based setup/teardown | before/after hooks that navigate pages or click through UI to create/delete test data instead of using APIs |
| Onboarding/tour dismissals | browser.setLocalStorageItem(...), manual tour-dismiss clicks, getting-started bypasses |
| Brittle CSS selectors | find.byCssSelector(...), find.byClassName(...), text-based lookups without data-test-subj |
| Over-privileged execution | Tests running as superuser that don't need elevated privileges |
For each smell, note the file and relevant context.
Group tests into ordered batches that can be migrated independently:
For each batch:
autopilot (executor can handle end-to-end), guided (needs a few human decisions or source code changes), or hands-on (significant manual work or missing infrastructure)data-test-subj in source code)Output the plan to migration_plan.md in the target Scout module root. Use the template in references/output_template.md.
NEEDS VERIFICATION rather than guessing.