How kitloom is tested: 4,700 tests, two admins and a video lesson for every screen

A CMS that takes over a WordPress site is trusted with years of somebody's content, orders and users. It is also trusted with an AI agent that writes to that site. So "it works on my machine" is not enough, and neither is a demo. Here is what checks kitloom before a release, with the actual numbers as of this week.

Every package tests itself

kitloom is 167 Composer packages: the core, every module with its Filament and Nova admins, and the themes. 164 of them have a test suite of their own, inside the package. A buyer who installs only some modules still gets their tests, and each package's split repository carries them too.

Together the packages hold 4,412 tests. The largest suites are where the most can go wrong:

Package	Tests	What they cover
Shop	426	Cart, taxes, coupons, shipping, stock, orders in WooCommerce's tables, the account pages
Site	362	Rewrite rules, the main query, canonical redirects, the template hierarchy, sessions and forms
WordPress data model	200	Posts, terms, users, meta and options, serialized and JSON values, visibility
Languages	185	Translations laid over the original, per-language URLs and slugs, conversion from Polylang and WPML
SEO	107	Titles and meta in Rank Math's format, sitemaps, robots.txt, instant indexing
Media	106	Uploads, image sizes, the library, old /wp-content/uploads/ links

The tests come in a few kinds:

• Tests on the WordPress tables. Rows are written exactly as WordPress leaves them, and the tests check that kitloom reads them the same way. They run in an in-memory database, so they are fast.
• Tests on real plugin data. 58 fixture files hold rows exactly as the plugins wrote them: every field type as Secure Custom Fields stores it, Rank Math's serialized redirect rules, WooCommerce's orders, and more. A module is tested against what the plugin really leaves behind, not against what we think it leaves.
• Tests of the whole application. These boot a real kitloom site and send requests to it, as a visitor and as each role.
• Tests of both admins. 102 test files drive Filament screens and 101 drive Nova screens: forms save, lists filter, a role that may not open a screen doesn't see it.
• Tests of the AI agent's tools. 21 modules test their MCP tools directly. On top of that, the agent server's own suite checks the rules every tool shares: a read-only key gets the reading tools only, the writers refuse what the key's user may not do, an author's agent sees only its own drafts, locked settings are reported as locked, and "delete" means the trash. Control over the agent is kitloom's main promise, so it is tested where it is made.
• Tests of the words. 24 packages check that their translation files match the texts in the code, so a new message never ships untranslated.
• Tests of the themes. 14 themes run their demo in a test: every widget zone is filled, every menu location has a menu, and the pages draw.

The skeletons test how it fits together

A package can pass on its own and still clash with another one. So the two applications kitloom is installed from, the Filament skeleton and the Nova skeleton, run every package's suite inside a real application, with all the modules installed together. They also have tests of their own: 185 in the Filament skeleton and 143 in the Nova one. Those cover the installer, the wiring between modules and the menus of the admin.

Altogether that is over 4,700 tests. A full check runs them about 7,400 times, because the domain packages run once under each admin: 3,795 test runs in the Filament skeleton and 3,648 in the Nova one. Each suite runs in parallel, a process per CPU core, and takes about a minute. The suites pass on SQLite and on MySQL with the WordPress table prefix, which is how most real sites run.

Before the tests: the repository checks

tools/check.sh runs before every release. Before the tests it checks:

• that every kitloom package a package uses is declared in its composer.json;
• that namespaces match file paths;
• that the skeletons' own tests import only the core, so a module's test lives in the module;
• the license fields of every package;
• that every package requires the others at the same version;
• code style with Pint;
• static analysis with PHPStan, where new code may not add a single error to the baseline.

Compared with a real WooCommerce store, on the same data

Some behaviour can only be judged next to the original. For the shop, a separate suite runs 29 scenarios on a real WooCommerce store and on kitloom over the same data: coupons, taxes, rounding, shipping, stock. It compares the totals and the orders written. When a number differs, kitloom is fixed, not the scenario.

For your own site, the same idea is a script you run yourself. compare-sites.php reads your WordPress and kitloom address by address and reports every difference in status, redirect, title, canonical tag and content. See Look at your WordPress site in Laravel before you move it.

Every module describes what it does

Each package has a README, 166 of them. A module's README is not only for developers. It is the module's How it works page in the admin, so the person who configures a site reads the same text the code was checked against: what each screen does, where the data is kept, which WordPress plugin's data it reads, and what it does not do yet. The module guides at kitloom.dev are built from the same READMEs.

The video lessons test the admin once more

The video lessons are not screen recordings made by hand. Each one is a script, 90 of them across 34 modules and themes. It drives the real admin in a browser, as a person would: it signs in, opens the screen, fills the fields and saves, while a voice reads the narration. Every step has to find what it clicks or types into. If a button moved or a screen broke, the lesson stops with an error before any video is made. A check mode plays the same steps without recording, in seconds.

They found real problems. The first time the lessons were written, they turned up 31 admin defects the tests had not caught, such as a date format the settings form refused to save and a counter that showed a stale number. Each one went on the fix list. The 75 videos published today are the screens as they are now, including what an AI agent does through them.

Try it

None of this proves kitloom fits your site, but it's why you can check that yourself without risk. Open a sandbox with every module and theme in one click, Filament or Nova. Or look at your own WordPress site through kitloom with a read-only database user, as Migrate from WordPress shows.