Playwright with Java Day 23: Cross-Browser Testing on Chromium, Firefox and WebKit in GitHub Actions

Where Day 22 left off

  • Day 22 uploaded and downloaded a file on the-internet.
  • Every test so far runs only on Chromium, the engine behind Chrome and Edge.
  • Your users also use Firefox, and every iPhone browser runs on Safari's engine.
  • Day 23 runs the same suite on Chromium, Firefox and WebKit.

Why one browser is not enough

  • Each engine draws pages and runs scripts in its own way.
  • Firefox runs on Gecko, its own engine, so it catches different bugs again.
  • A layout or date-picker bug can show up in one engine only.
  • Real-life example: the practicetestautomation.com login page should behave the same everywhere.
  • If a CSS change hides the Submit button only in WebKit, iPhone users can't log in.
  • Playwright ships all three engines, so you need no extra drivers.

Step 1: Pick the browser with a flag

  • Add one method to src/test/java/utils/Env.java.
public static String browser() {
    return System.getProperty("browser", "chromium").toLowerCase();
}
  • Chromium stays the default, so mvn test works exactly as before.
  • mvn test -Dbrowser=firefox switches the whole run.
  • It works with the Day 13 profiles too: mvn test -Pprod -Dbrowser=webkit.
  • browser() lowercases the value, so -Dbrowser=WebKit works as well.

Step 2: Launch the chosen browser in BaseFixture

  • Disclaimer: your Day 12 launch line may differ. Keep its options.
  • Add this method to BaseFixture and use it in the launch line.
private BrowserType browserType() {
    switch (Env.browser()) {
        case "firefox":
            return playwright.firefox();
        case "webkit":
            return playwright.webkit();
        default:
            return playwright.chromium();
    }
}

// in @BeforeClass, replacing playwright.chromium()
browser = browserType().launch(
        new BrowserType.LaunchOptions().setHeadless(Env.headless()));
  • A classic switch keeps the code valid on the Java 11 setting from Day 4.
  • Any unknown name falls back to Chromium instead of crashing.

Step 3: Install the browsers locally

  • Each engine is a separate download.
  • On a Linux laptop, add --with-deps so the system libraries come along.
  • Run this once: mvn exec:java -e -D exec.mainClass=com.microsoft.playwright.CLI -D exec.args="install firefox webkit"
  • Then try mvn test -Dbrowser=webkit and watch the suite run on WebKit.

Step 4: Run all three in GitHub Actions

  • Change the top of the Day 14 job and two of its steps.
  test:
    runs-on: ubuntu-latest
    timeout-minutes: 30
    strategy:
      fail-fast: false
      matrix:
        browser: [ chromium, firefox, webkit ]
    env:
      BROWSER: ${{ matrix.browser }}
    steps:
      # checkout, setup-java and build stay the same

      - name: Install the browser and its Linux libraries
        run: mvn -B exec:java -e -D exec.mainClass=com.microsoft.playwright.CLI -D exec.args="install --with-deps $BROWSER"

      - name: Run the suite
        env:
          PROFILE: ${{ inputs.profile || 'staging' }}
          LOGIN_USER: ${{ secrets.LOGIN_USER }}
          LOGIN_PASSWORD: ${{ secrets.LOGIN_PASSWORD }}
          SAUCE_USER: ${{ secrets.SAUCE_USER }}
          SAUCE_PASSWORD: ${{ secrets.SAUCE_PASSWORD }}
        run: mvn -B test -P"$PROFILE" -Dbrowser="$BROWSER" --no-transfer-progress
  • The matrix starts three jobs, one per browser, on three separate runners.
  • They run side by side, so the total time stays close to one run.
  • fail-fast: false lets Firefox and WebKit finish even if Chromium fails.
  • $BROWSER goes through an env variable, the same safe habit as PROFILE on Day 14.
  • Each matrix job uses its own minutes, so a private repo spends three times as many.
  • Add -${{ matrix.browser }} to every artifact name, like surefire-reports-${{ matrix.browser }}.

Mistakes that break this setup

  • Same artifact name in every matrix job: the second upload fails with a conflict.
  • Installing only Chromium: Firefox and WebKit fail with "Executable doesn't exist".
  • Leaving out fail-fast: false: one red browser cancels the other two.
  • Expecting WebKit to equal real Safari: it is the same engine, not the same app.

What carries forward

  • Every test class from Days 12 to 22 is unchanged.
  • Env has browser(), and BaseFixture launches whichever engine it names.
  • CI now runs the suite three times per push, once per engine.
  • Allure reports, traces and videos now come in three sets, one per browser.

Day 24 runs tests in parallel inside each job, safely.