Reliable Playwright Testing in AI-Assisted Development

AI can help write a Playwright test quickly. The hard part is making sure the test exercises the same interaction a person uses, against the application you meant to test, and waits for the right evidence.

A recent local run on a legacy ASP.NET MVC app exposed several easy-to-miss problems: the app was not running on the port Playwright expected; page navigation waited on resources that were slow to finish; the development login selector did not match the rendered button; and refreshing map data removed existing markers before the replacement request had succeeded. Fixing those issues produced a small, repeatable workflow—and a regression test for the actual behavior.

1. Start with the user-visible behavior

Write down the action and the observable result before asking AI to generate a test. For example:

• Action: save a new map point, which triggers a point-data refresh.
• Expected result: existing markers stay visible while the refresh is pending; after a successful response, the map displays the refreshed set.

This is more useful than a broad instruction such as “test the map.” It gives the test a clear assertion and makes it easier to spot when the test is only proving that a function was called.

2. Make the test target explicit

Check the app’s documented launch command, base URL, and required local services. Keep the Playwright base URL aligned with the server port. A browser test pointed at an app that is not running cannot tell you anything about the feature.

const baseUrl = process.env.MOZZIE_E2E_BASE_URL || 'http://localhost:54933';
await page.goto(`${baseUrl}/Mapping/FullPageTrapMap`, { waitUntil: 'commit' });

In this case IIS Express was available and the project built successfully, but the app was not listening on Playwright’s default port. Starting it on the expected port fixed the connection error. After the first request, the app also needed time to warm up and compile its views.

3. Wait for the control you need, not an arbitrary delay

A navigation lifecycle event is not always the best readiness condition for a test. Pages with external or deferred scripts can make DOMContentLoaded or full page load unreliable as a proxy for “the UI I need is ready.” Use waitUntil: 'commit' when you only need the navigation response to arrive, then wait for a meaningful element or application state:

await page.goto(url, { waitUntil: 'commit' });
await page.locator('#points-dropdown-wrapper .dropdown-toggle')
    .waitFor({ state: 'visible' });

Avoid fixed sleeps such as waitForTimeout(2000) for app readiness. They slow down fast runs and still fail on slower ones. Use Playwright’s locator waits, URL waits, or an explicit condition such as a loaded panel appearing.

4. Inspect the rendered page before guessing selectors

Markup can differ from what an AI assumes. The development login button in this app had no explicit type="submit" attribute, so a selector requiring that attribute found nothing even though the button was on the page. Inspect the live DOM and use a stable accessible name or a selector tied to the actual form:

const devLogin = page.locator('form[action="/Account/DevLogin"] button');
await devLogin.waitFor({ state: 'attached' });
await devLogin.click();

Prefer role and label selectors for user-facing controls where possible. When a CSS selector is necessary, confirm it matches the rendered HTML. An assertion that the test found one element is useful because a silent zero-match should fail with a clear explanation.

5. Control asynchronous work to expose timing bugs

To check what happens during a refresh, let the request remain pending and inspect the existing UI before completing it. This makes the loading window deterministic. In a browser test, a deferred request can be stubbed after the app has loaded:

const assert = require('node:assert/strict');
const removedMarkers = await page.evaluate(() => {
    let removed = 0;
    window.markers = [{ setMap: map => { if (map === null) removed++; } }];
    const pendingRequest = $.Deferred();
    const originalAjax = $.ajax;
    $.ajax = () => pendingRequest;
    window.GetPointData();
    $.ajax = originalAjax;
    return removed;
});
assert.equal(removedMarkers, 0);

The point refresh had been clearing all existing markers immediately, then requesting new data. That left a blank map during the request and also left it blank if the request failed. Moving marker replacement to the successful response path keeps the current map visible until there is new data to show.

Stub only the boundary needed for the behavior under test. Keep the real page, real click path, and real application code where practical. For flows that can write to a database, use a test database or intercept the write endpoint; do not create persistent records in a shared environment just to verify a client-side loading state.

6. Keep the test suite small, repeatable, and informative

• Build the app before browser testing so compile errors are caught early.
• Run the focused test while debugging, then run the full Playwright suite.
• Use a dedicated test account and isolated data where a test creates or edits records.
• Capture useful failures: the URL, relevant console errors, and a screenshot or trace when the cause is not obvious.
• Do not add retries to hide intermittent failures; find whether the app, test data, or waiting condition is unstable.

For this run, the practical order was: build the MVC project, start IIS Express on the configured port, wait for the specific UI, log in with the rendered development control, run the focused map regression test, then run the full Playwright suite. The map test and the existing browser tests all passed once navigation and login assumptions matched the application.

A clean AI-assisted testing loop

1. Describe the user action and expected visible result.
2. Ask AI to locate the actual route, controls, code path, and test setup before it edits anything.
3. Have it write one focused test that proves the behavior, including an important pending or failure state when timing matters.
4. Build and start the real local app using the repository’s configuration.
5. Run the focused test, inspect failures against the rendered page and server output, and correct assumptions.
6. Run the full suite and report exactly what passed, what failed, and any environment limitation.

AI is useful for tracing code and producing a first test draft. The browser, the application, and the test assertion are what establish whether the behavior works.