Tests: migrate testing infrastructure to minimal dependencies

This is a complete rework of our testing infrastructure. The main goal is to modernize and drop deprecated or undermaintained dependencies (specifically, grunt, karma, and testswarm). We've achieved that by limiting our dependency list to ones that are unlikely to drop support any time soon. The new dependency list includes:

- `qunit` (our trusty unit testing library)
- `selenium-webdriver` (for spinning up local browsers)
- `express` (for starting a test server and adding middleware)
  - express middleware includes uses of `body-parser` and `raw-body`
- `yargs` (for constructing a CLI with pretty help text)
- BrowserStack (for running each of our QUnit modules separately in all of our supported browsers)
  - `browserstack-local` (for opening a local tunnel. This is the same package still currently used in the new Browserstack SDK)
  - We are not using any other BrowserStack library. The newest BrowserStack SDK does not fit our needs (and isn't open source). Existing libraries, such as `node-browserstack` or `browserstack-runner`, either do not quite fit our needs, are under-maintained and out-of-date, or are not robust enough to meet all of our requirements. We instead call the [BrowserStack REST API](https://github.com/browserstack/api) directly.

**BrowserStack**
- automatically retries individual modules in case of test failure(s)
- automatically attempts to re-establish broken tunnels
- automatically refreshes the page in case a test run has stalled
- Browser workers are reused when running isolated modules in the same browser
- runs all browsers concurrently and uses as many sessions as are available under the BrowserStack plan. It will wait for available sessions if there are none.
- supports filtering the available list of browsers by browser name, browser version, device, OS, and OS version (see `npm run test:unit -- --list-browsers` for more info). It will retrieve the latest matching browser available if any of those parameters are not specified. Supports latest and latest-\d+ in place of browser version.
- cleans up after itself (closes the local tunnel, stops the test server, etc.)
- Requires `BROWSERSTACK_USERNAME` and `BROWSERSTACK_ACCESS_KEY` environment variables.

**Selenium**
- supports running any local browser as long as the driver is installed, including support for headless mode in Chrome, FF, and Edge
- supports running `basic` tests on the latest [jsdom](https://github.com/jsdom/jsdom#readme), which can be seen in action in this PR (see `test:browserless`)
- Node tests will run as before in PRs and all non-dependabot branches, but now includes tests on real Safari in a GH actions macos image instead of playwright-webkit.
- can run multiple browsers and multiple modules concurrently

Other notes:
- Stale dependencies have been removed and all remaining dependencies have been upgraded with a few exceptions:
  - `sinon`: stopped supporting IE in version 10. But, `sinon` has been updated to 9.x.
  - `husky`: latest does not support Node 10 and runs on `npm install`. Needed for now until git builds are migrated to GitHub Actions.
  - `rollup`: latest does not support Node 10. Needed for now until git builds are migrated to GitHub Actions.
- BrowserStack tests are set to run on each `main` branch commit
- `debug` mode leaves Selenium browsers open whether they pass or fail and leaves browsers with test failures open on BrowserStack. The latter is to avoid leaving open too many sessions.
- This PR includes a workflow to dispatch BrowserStack runs on-demand
- The Node version used for most workflow tests has been upgraded to 20.x
- updated supportjQuery to 3.7.1

Run `npm run test:unit -- --help` for CLI documentation

Close gh-5427

.github/workflows/browserstack-dispatch.yml

@@ -0,0 +1,62 @@
+name: Browserstack (Manual Dispatch)
+
+on:
+  workflow_dispatch:
+    inputs:
+      module:
+        description: 'Module to test'
+        required: true
+        type: choice
+        options:
+          - 'basic'
+          - 'ajax'
+          - 'animation'
+          - 'attributes'
+          - 'callbacks'
+          - 'core'
+          - 'css'
+          - 'data'
+          - 'deferred'
+          - 'deprecated'
+          - 'dimensions'
+          - 'effects'
+          - 'event'
+          - 'manipulation'
+          - 'offset'
+          - 'queue'
+          - 'selector'
+          - 'serialize'
+          - 'support'
+          - 'traversing'
+          - 'tween'
+      browser:
+        description: 'Browser to test, in form of \"browser_[browserVersion | :device]_os_osVersion\"'
+        required: false
+        type: string
+        default: 'chrome__windows_11'
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    environment: browserstack
+    env:
+      BROWSERSTACK_USERNAME: ${{ secrets.BROWSERSTACK_USERNAME }}
+      BROWSERSTACK_ACCESS_KEY: ${{ secrets.BROWSERSTACK_ACCESS_KEY }}
+    steps:
+      - uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
+
+      - uses: actions/setup-node@60edb5dd545a775178f52524783378180af0d1f8 # v4.0.2
+        with:
+          node-version: 20
+
+      - name: Install dependencies
+        run: npm ci
+
+      - name: Build jQuery
+        run: npm run build:all
+
+      - name: Pretest script
+        run: npm run pretest
+
+      - name: Run tests
+        run: npm run test:unit -- -v --browserstack ${{ inputs.browser }} -m ${{ inputs.module }}

.github/workflows/browserstack.yml

@@ -0,0 +1,80 @@
+name: Browserstack
+
+on:
+  push:
+    branches:
+      - 3.x-stable
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    environment: browserstack
+    env:
+      BROWSERSTACK_USERNAME: ${{ secrets.BROWSERSTACK_USERNAME }}
+      BROWSERSTACK_ACCESS_KEY: ${{ secrets.BROWSERSTACK_ACCESS_KEY }}
+      NODE_VERSION: 20.x
+    name: ${{ matrix.BROWSER }}
+    concurrency:
+      group: ${{ github.workflow }} ${{ matrix.BROWSER }}
+    timeout-minutes: 30
+    strategy:
+      fail-fast: false
+      matrix:
+        BROWSER:
+          - 'IE_11'
+          - 'IE_10'
+          - 'IE_9'
+          - 'Safari_latest'
+          - 'Safari_latest-1'
+          - 'Chrome_latest'
+          - 'Chrome_latest-1'
+          - 'Opera_latest'
+          - 'Edge_latest'
+          - 'Edge_latest-1'
+          - 'Edge_18'
+          - 'Firefox_latest'
+          - 'Firefox_latest-1'
+          - 'Firefox_115'
+          - 'Firefox_102'
+          - 'Firefox_91'
+          - 'Firefox_78'
+          - 'Firefox_60'
+          - 'Firefox_48'
+          - '__iOS_17'
+          - '__iOS_16'
+          - '__iOS_15'
+          - '__iOS_14'
+          - '__iOS_13'
+          - '__iOS_12'
+          - '__iOS_11'
+          - '__iOS_10'
+          - '__iOS_9'
+          - '__iOS_8'
+    steps:
+      - name: Checkout
+        uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
+
+      - name: Use Node.js ${{ env.NODE_VERSION }}
+        uses: actions/setup-node@60edb5dd545a775178f52524783378180af0d1f8 # v4.0.2
+        with:
+          node-version: ${{ env.NODE_VERSION }}
+
+      - name: Cache
+        uses: actions/cache@13aacd865c20de90d75de3b17ebe84f7a17d57d2 # v4.0.0
+        with:
+          path: ~/.npm
+          key: ${{ runner.os }}-node-${{ env.NODE_VERSION }}-npm-lock-${{ hashFiles('**/package-lock.json') }}
+          restore-keys: |
+            ${{ runner.os }}-node-${{ env.NODE_VERSION }}-npm-lock-
+
+      - name: Install dependencies
+        run: npm install
+
+      - name: Build jQuery
+        run: npm run build:all
+
+      - name: Pretest script
+        run: npm run pretest
+
+      - name: Run tests
+        run: npm run test:unit -- -v --browserstack "${{ matrix.BROWSER }}" --run-id ${{ github.run_id }} --isolate --retries 3

.github/workflows/node.js.yml

@@ -1,4 +1,4 @@
-name: CI
+name: Node
 
 on:
   pull_request:
@@ -9,45 +9,49 @@ permissions:
   contents: read # to fetch code (actions/checkout)
 
 jobs:
-  build:
+  build-and-test:
     runs-on: ubuntu-latest
+    name: ${{ matrix.NPM_SCRIPT }} - ${{ matrix.NAME }} (${{ matrix.NODE_VERSION }})
     strategy:
       fail-fast: false
       matrix:
-        # Node.js 10 is required by jQuery infra
+        NAME: ["Node"]
-        # Do not remove 16.x until jsdom tests are re-enabled on newer Node.js versions.
+        NODE_VERSION: [18.x, 20.x]
-        NODE_VERSION: [10.x, 16.x, 18.x, 20.x]
         NPM_SCRIPT: ["test:browserless"]
         include:
-          - NAME: "Browser tests: full build, Chrome, Firefox & WebKit"
+          - NAME: "Node"
-            NODE_VERSION: "18.x"
+            NODE_VERSION: "20.x"
+            NPM_SCRIPT: "lint"
+          - NAME: "Chrome/Firefox"
+            NODE_VERSION: "20.x"
             NPM_SCRIPT: "test:browser"
-            BROWSERS: "ChromeHeadless,FirefoxHeadless,WebkitHeadless"
+          - NAME: "Chrome"
-          - NAME: "Browser tests: slim build, Chrome"
+            NODE_VERSION: "20.x"
-            NODE_VERSION: "18.x"
             NPM_SCRIPT: "test:slim"
-            BROWSERS: "ChromeHeadless"
+          - NAME: "Chrome"
-          - NAME: "Browser tests: no-deprecated build, Chrome"
+            NODE_VERSION: "20.x"
-            NODE_VERSION: "18.x"
             NPM_SCRIPT: "test:no-deprecated"
-            BROWSERS: "ChromeHeadless"
+          - NAME: "Chrome"
-          - NAME: "Browser tests: selector-native build, Chrome"
+            NODE_VERSION: "20.x"
-            NODE_VERSION: "18.x"
             NPM_SCRIPT: "test:selector-native"
-            BROWSERS: "ChromeHeadless"
+          - NAME: "Chrome"
-          - NAME: "Browser tests: AMD build, Chrome stable"
+            NODE_VERSION: "20.x"
-            NODE_VERSION: "18.x"
             NPM_SCRIPT: "test:amd"
-            BROWSERS: "ChromeHeadless"
+          - NAME: "Firefox ESR"
-          - NAME: "Browser tests: full build, Firefox ESR"
+            NODE_VERSION: "20.x"
-            NODE_VERSION: "18.x"
+            NPM_SCRIPT: "test:firefox"
-            NPM_SCRIPT: "test:browser"
+          - NAME: "Node 10 Build"
-            BROWSERS: "FirefoxHeadless"
+            NODE_VERSION: "10.x"
-
+            NPM_SCRIPT: "build:main"
     steps:
       - name: Checkout
         uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
 
+      - name: Use Node.js ${{ matrix.NODE_VERSION }}
+        uses: actions/setup-node@60edb5dd545a775178f52524783378180af0d1f8 # v4.0.2
+        with:
+          node-version: ${{ matrix.NODE_VERSION }}
+
       - name: Cache
         uses: actions/cache@13aacd865c20de90d75de3b17ebe84f7a17d57d2 # v4.0.0
         with:
@@ -56,31 +60,49 @@ jobs:
           restore-keys: |
             ${{ runner.os }}-node-${{ matrix.NODE_VERSION }}-npm-lock-
 
-      - name: Use Node.js ${{ matrix.NODE_VERSION }}
-        uses: actions/setup-node@8f152de45cc393bb48ce5d89d36b731f54556e65 # v4.0.0
-        with:
-          node-version: ${{ matrix.NODE_VERSION }}
-
       - name: Install firefox ESR
         run: |
-          export FIREFOX_SOURCE_URL='https://download.mozilla.org/?product=firefox-esr-latest&lang=en-US&os=linux64'
+          export FIREFOX_SOURCE_URL='https://download.mozilla.org/?product=firefox-esr-latest-ssl&lang=en-US&os=linux64'
           wget --no-verbose $FIREFOX_SOURCE_URL -O - | tar -jx -C ${HOME}
         if: contains(matrix.NAME, 'Firefox ESR')
 
       - name: Install dependencies
         run: npm install
 
-      - name: Install Playwright dependencies
+      - name: Build All for Linting
-        run: npx playwright-webkit install-deps
+        run: npm run build:all
-        if: matrix.NPM_SCRIPT == 'test:browser' && contains(matrix.BROWSERS, 'WebkitHeadless')
+        if: contains(matrix.NPM_SCRIPT, 'lint')
-
-      - name: Lint code
-        run: npm run build && npm run lint
-        if: matrix.NODE_VERSION == '18.x'
 
       - name: Run tests
-        env:
-          BROWSERS: ${{ matrix.BROWSERS }}
         run: |
           export PATH=${HOME}/firefox:$PATH
+          export FIREFOX_BIN=${HOME}/firefox/firefox
           npm run ${{ matrix.NPM_SCRIPT }}
+
+  safari:
+    runs-on: macos-latest
+    env:
+      NODE_VERSION: 20.x
+    name: test:safari - Safari
+    steps:
+      - name: Checkout
+        uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
+
+      - name: Use Node.js ${{ env.NODE_VERSION }}
+        uses: actions/setup-node@60edb5dd545a775178f52524783378180af0d1f8 # v4.0.2
+        with:
+          node-version: ${{ env.NODE_VERSION }}
+
+      - name: Cache
+        uses: actions/cache@13aacd865c20de90d75de3b17ebe84f7a17d57d2 # v4.0.0
+        with:
+          path: ~/.npm
+          key: ${{ runner.os }}-node-${{ env.NODE_VERSION }}-npm-lock-${{ hashFiles('**/package-lock.json') }}
+          restore-keys: |
+            ${{ runner.os }}-node-${{ env.NODE_VERSION }}-npm-lock-
+
+      - name: Install dependencies
+        run: npm install
+
+      - name: Run tests
+        run: npm run test:safari

.gitignore

@@ -19,3 +19,7 @@ npm-debug.log*
 
 /test/data/core/jquery-iterability-transpiled.js
 /test/data/qunit-fixture.js
+
+# Ignore BrowserStack files
+local.log
+browserstack.err

CONTRIBUTING.md

@@ -68,12 +68,6 @@ We *love* when people contribute back to the project by patching the bugs they f
 
 Create a fork of the jQuery repo on github at https://github.com/jquery/jquery
 
-Change directory to your web root directory, whatever that might be:
-
-```bash
-$ cd /path/to/your/www/root/
-```
-
 Clone your jQuery fork to work locally
 
 ```bash
@@ -98,29 +92,47 @@ Get in the habit of pulling in the "upstream" main to stay up to date as jQuery
 $ git pull upstream main
 ```
 
-Run the build script
+Install the necessary dependencies
 
 ```bash
-$ npm run build
+$ npm install
 ```
 
-Now open the jQuery test suite in a browser at http://localhost/test. If there is a port, be sure to include it.
+Build all jQuery files
+
+```bash
+$ npm run build:all
+```
+
+Start a test server
+
+```bash
+$ npm run test:server
+```
+
+Now open the jQuery test suite in a browser at http://localhost:3000/test/.
 
 Success! You just built and tested jQuery!
 
-
 ### Test Suite Tips...
 
 During the process of writing your patch, you will run the test suite MANY times. You can speed up the process by narrowing the running test suite down to the module you are testing by either double clicking the title of the test or appending it to the url. The following examples assume you're working on a local repo, hosted on your localhost server.
 
 Example:
 
-http://localhost/test/?module=css
+http://localhost:3000/test/?module=css
 
 This will only run the "css" module tests. This will significantly speed up your development and debugging.
 
 **ALWAYS RUN THE FULL SUITE BEFORE COMMITTING AND PUSHING A PATCH!**
 
+#### Change the test server port
+
+The default port for the test server is 3000. You can change the port by setting the `PORT` environment variable.
+
+```bash
+$ PORT=3001 npm run test:server
+```
 
 #### Loading changes on the test page
 
@@ -134,6 +146,29 @@ Alternatively, you can **load tests in AMD** to avoid the need for rebuilding al
 
 Click "Load with AMD" after loading the test page.
 
+#### Running the test suite from the command line
+
+You can also run the test suite from the command line.
+
+First, prepare the tests:
+
+```bash
+$ npm run pretest
+```
+
+Make sure jQuery is built (`npm run build:all`) and run the tests:
+
+```bash
+$ npm run test:unit
+```
+
+This will run each module in its own browser instance and report the results in the terminal.
+
+View the full help for the test suite for more info on running the tests from the command line:
+
+```bash
+$ npm run test:unit -- --help
+```
 
 ### Repo organization
 

Gruntfile.js

@@ -1,235 +0,0 @@
-"use strict";
-
-module.exports = function( grunt ) {
-	const nodeV16OrNewer = !/^v1[0-5]\./.test( process.version );
-	const nodeV17OrNewer = !/^v1[0-6]\./.test( process.version );
-	const customBrowsers = process.env.BROWSERS && process.env.BROWSERS.split( "," );
-
-	// Support: Node.js <16
-	// Skip running tasks that dropped support for old Node.js in these Node versions.
-	function runIfNewNode( task ) {
-		return nodeV16OrNewer ? task : "print_old_node_message:" + task;
-	}
-
-	if ( nodeV16OrNewer ) {
-		const playwright = require( "playwright-webkit" );
-		process.env.WEBKIT_HEADLESS_BIN = playwright.webkit.executablePath();
-	}
-
-	if ( !grunt.option( "filename" ) ) {
-		grunt.option( "filename", "jquery.js" );
-	}
-
-	grunt.initConfig( {
-		pkg: grunt.file.readJSON( "package.json" ),
-		testswarm: {
-			tests: [
-
-				// A special module with basic tests, meant for
-				// not fully supported environments like Android 2.3,
-				// jsdom or PhantomJS. We run it everywhere, though,
-				// to make sure tests are not broken.
-				"basic",
-
-				"ajax",
-				"animation",
-				"attributes",
-				"callbacks",
-				"core",
-				"css",
-				"data",
-				"deferred",
-				"deprecated",
-				"dimensions",
-				"effects",
-				"event",
-				"manipulation",
-				"offset",
-				"queue",
-				"selector",
-				"serialize",
-				"support",
-				"traversing",
-				"tween"
-			]
-		},
-		karma: {
-			options: {
-				customContextFile: "test/karma.context.html",
-				customDebugFile: "test/karma.debug.html",
-				customLaunchers: {
-					ChromeHeadlessNoSandbox: {
-						base: "ChromeHeadless",
-						flags: [ "--no-sandbox" ]
-					}
-				},
-				frameworks: [ "qunit" ],
-				middleware: [ "mockserver" ],
-				plugins: [
-					"karma-*",
-					{
-						"middleware:mockserver": [
-							"factory",
-							require( "./test/middleware-mockserver.js" )
-						]
-					}
-				],
-				client: {
-					qunit: {
-
-						// We're running `QUnit.start()` ourselves via `loadTests()`
-						// in test/jquery.js
-						autostart: false
-					}
-				},
-				files: [
-					"test/data/jquery-1.9.1.js",
-					"external/sinon/sinon.js",
-					"external/npo/npo.js",
-					"external/requirejs/require.js",
-					"test/data/testinit.js",
-					"test/jquery.js",
-
-					{
-						pattern: "external/**",
-						included: false,
-						served: true,
-						nocache: true
-					},
-					{
-						pattern: "dist/jquery.*",
-						included: false,
-						served: true,
-						nocache: true
-					},
-					{
-						pattern: "src/**",
-						included: false,
-						served: true,
-						nocache: true
-					},
-					{
-						pattern: "test/**/*.@(js|css|jpg|html|xml|svg)",
-						included: false,
-						served: true,
-						nocache: true
-					}
-				],
-				reporters: [ "dots" ],
-				autoWatch: false,
-
-				// 2 minutes; has to be longer than QUnit.config.testTimeout
-				browserNoActivityTimeout: 120e3,
-
-				concurrency: 3,
-				captureTimeout: 20 * 1000,
-				singleRun: true
-			},
-			main: {
-				browsers: customBrowsers ||
-					[ "ChromeHeadless", "FirefoxHeadless", "WebkitHeadless" ]
-			},
-			amd: {
-				browsers: customBrowsers || [ "ChromeHeadless" ],
-				options: {
-					client: {
-						qunit: {
-
-							// We're running `QUnit.start()` ourselves via `loadTests()`
-							// in test/jquery.js
-							autostart: false,
-
-							amd: true
-						}
-					}
-				}
-			},
-
-			jsdom: {
-				options: {
-					files: [
-						"test/data/jquery-1.9.1.js",
-						"test/data/testinit-jsdom.js",
-
-						// We don't support various loading methods like AMD,
-						// choosing a version etc. for jsdom.
-						"dist/jquery.js",
-
-						// A partial replacement for testinit.js#loadTests()
-						"test/data/testrunner.js",
-
-						// jsdom only runs basic tests
-						"test/unit/basic.js",
-
-						{
-							pattern: "test/**/*.@(js|css|jpg|html|xml|svg)",
-							included: false,
-							served: true
-						}
-					]
-				},
-				browsers: [ "jsdom" ]
-			},
-
-			// To debug tests with Karma:
-			// 1. Run 'grunt karma:chrome-debug' or 'grunt karma:firefox-debug'
-			//    (any karma subtask that has singleRun=false)
-			// 2. Press "Debug" in the opened browser window to start
-			//    the tests. Unlike the other karma tasks, the debug task will
-			//    keep the browser window open.
-			"chrome-debug": {
-				browsers: [ "Chrome" ],
-				singleRun: false
-			},
-			"firefox-debug": {
-				browsers: [ "Firefox" ],
-				singleRun: false
-			},
-			"ie-debug": {
-				browsers: [ "IE" ],
-				singleRun: false
-			}
-		}
-	} );
-
-	// Load grunt tasks from NPM packages
-	require( "load-grunt-tasks" )( grunt, {
-		pattern: [ "grunt-*" ]
-	} );
-
-	// Integrate jQuery specific tasks
-	grunt.loadTasks( "build/grunt-tasks" );
-
-	grunt.registerTask( "print_old_node_message", ( ...args ) => {
-		var task = args.join( ":" );
-		grunt.log.writeln( "Old Node.js detected, running the task \"" + task + "\" skipped..." );
-	} );
-
-	grunt.registerTask( "print_jsdom_message", () => {
-		grunt.log.writeln( "Node.js 17 or newer detected, skipping jsdom tests..." );
-	} );
-
-	grunt.registerTask( "authors", async function() {
-		const done = this.async();
-		const { getAuthors } = require( "./build/release/authors.js" );
-		const authors = await getAuthors();
-		console.log( authors.join( "\n" ) );
-		done();
-	} );
-
-	grunt.registerTask( "test:jsdom", [
-
-		// Support: Node.js 17+
-		// jsdom fails to connect to the Karma server in Node 17+.
-		// Until we figure out a fix, skip jsdom tests there.
-		nodeV17OrNewer ? "print_jsdom_message" : runIfNewNode( "karma:jsdom" )
-	] );
-
-	grunt.registerTask( "test", [
-		"test:jsdom"
-	] );
-
-	grunt.registerTask( "default", [
-		"test"
-	] );
-};

build/grunt-tasks/testswarm.js

@@ -1,62 +0,0 @@
-"use strict";
-
-module.exports = function( grunt ) {
-	grunt.registerTask( "testswarm", function( commit, configFile, projectName, browserSets,
-			timeout, testMode ) {
-		var jobName, config, tests,
-			testswarm = require( "testswarm" ),
-			runs = {},
-			done = this.async(),
-			pull = /PR-(\d+)/.exec( commit );
-
-		projectName = projectName || "jquery";
-		config = grunt.file.readJSON( configFile )[ projectName ];
-		browserSets = browserSets || config.browserSets;
-		if ( browserSets[ 0 ] === "[" ) {
-
-			// We got an array, parse it
-			browserSets = JSON.parse( browserSets );
-		}
-		timeout = timeout || 1000 * 60 * 15;
-		tests = grunt.config( [ this.name, "tests" ] );
-
-		if ( pull ) {
-			jobName = "Pull <a href='https://github.com/jquery/jquery/pull/" +
-				pull[ 1 ] + "'>#" + pull[ 1 ] + "</a>";
-		} else {
-			jobName = "Commit <a href='https://github.com/jquery/jquery/commit/" +
-				commit + "'>" + commit.substr( 0, 10 ) + "</a>";
-		}
-
-		if ( testMode === "basic" ) {
-			runs.basic = config.testUrl + commit + "/test/index.html?module=basic";
-		} else {
-			tests.forEach( function( test ) {
-				runs[ test ] = config.testUrl + commit + "/test/index.html?module=" + test;
-			} );
-		}
-
-		testswarm.createClient( {
-			url: config.swarmUrl
-		} )
-		.addReporter( testswarm.reporters.cli )
-		.auth( {
-			id: config.authUsername,
-			token: config.authToken
-		} )
-		.addjob(
-			{
-				name: jobName,
-				runs: runs,
-				runMax: config.runMax,
-				browserSets: browserSets,
-				timeout: timeout
-			}, function( err, passed ) {
-				if ( err ) {
-					grunt.log.error( err );
-				}
-				done( passed );
-			}
-		);
-	} );
-};

build/tasks/lib/verifyNodeVersion.js

@@ -1,10 +1,10 @@
 "use strict";
 
 const { version } = require( "process" );
-const nodeV16OrNewer = !/^v1[0-5]\./.test( version );
+const nodeV18OrNewer = !/^v1[0-7]\./.test( version );
 
 module.exports = function verifyNodeVersion() {
-	if ( !nodeV16OrNewer ) {
+	if ( !nodeV18OrNewer ) {
 		console.log( "Old Node.js detected, task skipped..." );
 		return false;
 	}

eslint.config.js

@@ -120,7 +120,7 @@ module.exports = [
 			"test/**"
 		],
 		ignores: [
-			"test/data/jquery-1.9.1.js",
+			"test/data/jquery-3.7.1.js",
 			"test/data/badcall.js",
 			"test/data/badjson.js",
 			"test/data/support/csp.js",
@@ -173,6 +173,29 @@ module.exports = [
 		}
 	},
 
+	{
+		files: [
+			"test/runner/**/*.js"
+		],
+		languageOptions: {
+			globals: {
+				...globals.node
+			},
+			sourceType: "module"
+		},
+		rules: {
+			...jqueryConfig.rules
+		}
+	},
+
+	{
+		files: [ "test/runner/listeners.js" ],
+		languageOptions: {
+			ecmaVersion: 5,
+			sourceType: "script"
+		}
+	},
+
 	{
 		files: [
 			"test/data/testrunner.js",
@@ -214,8 +237,7 @@ module.exports = [
 	{
 		files: [
 			"build/**",
-			"test/data/testinit.js",
+			"test/data/testinit.js"
-			"test/data/testinit-jsdom.js"
 		],
 		languageOptions: {
 			globals: {
@@ -232,8 +254,7 @@ module.exports = [
 	{
 		files: [
 			"build/**/*.js",
-			"test/data/testinit.js",
+			"test/data/testinit.js"
-			"test/data/testinit-jsdom.js"
 		],
 		languageOptions: {
 			sourceType: "commonjs"

package.json

@@ -11,7 +11,7 @@
     "build": "node ./build/command.js",
     "build:all": "node -e \"require('./build/tasks/build.js').buildDefaultFiles()\"",
     "build:main": "node -e \"require('./build/tasks/build.js').build()\"",
-    "jenkins": "npm run pretest && npm run test:browserless",
+    "jenkins": "npm run build:all",
     "lint:dev": "eslint --cache .",
     "lint:json": "jsonlint --quiet package.json",
     "lint": "concurrently -r \"npm:lint:dev\" \"npm:lint:json\"",
@@ -20,15 +20,19 @@
     "pretest": "npm run qunit-fixture && npm run babel:tests && npm run npmcopy",
     "qunit-fixture": "node build/tasks/qunit-fixture.js",
     "start": "nodemon --watch src -x \"npm run build:all\"",
-    "test:browserless": "npm run pretest && npm run test:node_smoke_tests && npm run test:promises_aplus && npm run test:jsdom",
+    "test:browser": "npm run pretest && npm run build:main && npm run test:unit -- -b chrome -b firefox -h",
-    "test:browser": "npm run pretest && npm run build:all && grunt karma:main",
+    "test:browserless": "npm run pretest && npm run build:all && node build/tasks/node_smoke_tests.js && node build/tasks/promises_aplus_tests.js && npm run test:unit -- -b jsdom -m basic",
-    "test:amd": "npm run pretest && npm run build:main && grunt karma:amd",
+    "test:jsdom": "npm run pretest && npm run build:main && npm run test:unit -- -b jsdom -m basic",
-    "test:jsdom": "npm run pretest && npm run build:main && grunt test:jsdom",
-    "test:no-deprecated": "npm run pretest && npm run build -- -e deprecated && grunt karma:main",
-    "test:selector-native": "npm run pretest && npm run build -- -e selector && grunt karma:main",
-    "test:slim": "npm run pretest && npm run build -- --slim && grunt karma:main",
     "test:node_smoke_tests": "npm run pretest && npm run build:all && node build/tasks/node_smoke_tests.js",
     "test:promises_aplus": "npm run build:main && node build/tasks/promises_aplus_tests.js",
+    "test:firefox": "npm run pretest && npm run build:main && npm run test:unit -- -v -b firefox -h",
+    "test:safari": "npm run pretest && npm run build:main && npm run test:unit -- -b safari",
+    "test:server": "node test/runner/server.js",
+    "test:amd": "npm run pretest && npm run build:main && npm run test:unit -- --amd -h",
+    "test:no-deprecated": "npm run pretest && npm run build -- -e deprecated && npm run test:unit -- -h",
+    "test:selector-native": "npm run pretest && npm run build -- -e selector && npm run test:unit -- -h",
+    "test:slim": "npm run pretest && npm run build -- --slim && npm run test:unit -- -h",
+    "test:unit": "node test/runner/command.js",
     "test": "npm run build:all && npm run lint && npm run test:browserless && npm run test:browser && npm run test:amd && npm run test:slim && npm run test:no-deprecated && npm run test:selector-native"
   },
   "homepage": "https://jquery.com",
@@ -51,45 +55,37 @@
   },
   "license": "MIT",
   "devDependencies": {
-    "@babel/cli": "7.22.10",
+    "@babel/cli": "7.23.9",
-    "@babel/core": "7.22.11",
+    "@babel/core": "7.23.9",
-    "@babel/plugin-transform-for-of": "7.22.5",
+    "@babel/plugin-transform-for-of": "7.23.6",
     "@prantlf/jsonlint": "14.0.3",
-    "bootstrap": "5.3.0",
+    "@swc/core": "1.4.2",
+    "@types/selenium-webdriver": "4.1.21",
+    "body-parser": "1.20.2",
+    "bootstrap": "5.3.3",
+    "browserstack-local": "1.5.5",
     "chalk": "5.3.0",
     "colors": "1.4.0",
     "commitplease": "3.2.0",
-    "concurrently": "8.2.0",
+    "concurrently": "8.2.2",
-    "core-js-bundle": "3.32.1",
+    "core-js-bundle": "3.36.0",
-    "eslint": "8.51.0",
+    "eslint": "8.56.0",
     "eslint-config-jquery": "3.0.2",
+    "exit-hook": "4.0.0",
     "express": "4.18.2",
-    "globals": "13.20.0",
+    "express-body-parser-error-handler": "1.0.7",
-    "grunt": "1.5.3",
+    "globals": "14.0.0",
-    "grunt-cli": "1.4.3",
-    "grunt-karma": "4.0.2",
     "husky": "8.0.3",
-    "jsdom": "19.0.0",
+    "jsdom": "24.0.0",
-    "karma": "6.4.1",
-    "karma-browserstack-launcher": "1.6.0",
-    "karma-chrome-launcher": "3.1.1",
-    "karma-firefox-launcher": "2.1.2",
-    "karma-ie-launcher": "1.0.0",
-    "karma-jsdom-launcher": "12.0.0",
-    "karma-qunit": "4.1.2",
-    "karma-webkit-launcher": "2.1.0",
-    "load-grunt-tasks": "5.1.0",
     "native-promise-only": "0.8.1",
     "nodemon": "3.0.1",
-    "playwright-webkit": "1.30.0",
     "promises-aplus-tests": "2.1.2",
     "q": "1.5.1",
-    "qunit": "2.9.2",
+    "qunit": "2.20.1",
-    "raw-body": "2.3.3",
+    "raw-body": "2.5.2",
     "requirejs": "2.3.6",
-    "sinon": "2.3.7",
+    "selenium-webdriver": "4.18.1",
-    "strip-json-comments": "2.0.1",
+    "sinon": "7.5.0",
-    "testswarm": "1.1.2",
     "uglify-js": "3.4.7",
     "yargs": "17.7.2"
   },

test/data/core/dynamic_ready.html

@@ -2,7 +2,7 @@
 <html>
 <head>
 	<meta charset="utf-8">
-	<script src="../jquery-1.9.1.js"></script>
+	<script src="../jquery-3.7.1.js"></script>
 	<script>var $j = jQuery.noConflict();</script>
 	<script src="../iframeTest.js"></script>
 </head>

test/data/testinit-jsdom.js

@@ -1,60 +0,0 @@
-"use strict";
-
-// Support: jsdom 13.2+
-// jsdom implements a throwing `window.scrollTo`.
-QUnit.config.scrolltop = false;
-
-const FILEPATH = "/test/data/testinit-jsdom.js";
-const activeScript = document.currentScript;
-const parentUrl = activeScript && activeScript.src ?
-	activeScript.src.replace( /[?#].*/, "" ) + FILEPATH.replace( /[^/]+/g, ".." ) + "/" :
-	"../";
-const supportjQuery = this.jQuery;
-
-// baseURL is intentionally set to "data/" instead of "".
-// This is not just for convenience (since most files are in data/)
-// but also to ensure that urls without prefix fail.
-// Otherwise it's easy to write tests that pass on test/index.html
-// but fail in Karma runner (where the baseURL is different).
-const baseURL = parentUrl + "test/data/";
-
-// Setup global variables before loading jQuery for testing .noConflict()
-supportjQuery.noConflict( true );
-window.originaljQuery = this.jQuery = undefined;
-window.original$ = this.$ = "replaced";
-
-/**
- * Add random number to url to stop caching
- *
- * Also prefixes with baseURL automatically.
- *
- * @example url("index.html")
- * @result "data/index.html?10538358428943"
- *
- * @example url("mock.php?foo=bar")
- * @result "data/mock.php?foo=bar&10538358345554"
- */
-this.url = function( value ) {
-	return baseURL + value + ( /\?/.test( value ) ? "&" : "?" ) +
-		new Date().getTime() + "" + parseInt( Math.random() * 100000, 10 );
-};
-
-// We only run basic tests in jsdom so we don't need to repeat the logic
-// from the regular testinit.js
-this.includesModule = function() {
-	return true;
-};
-
-// The file-loading part of testinit.js#loadTests is handled by
-// jsdom Karma config; here we just need to trigger relevant APIs.
-this.loadTests = function() {
-
-	// Delay the initialization until after all the files are loaded
-	// as in the JSDOM case we load them via Karma (see Gruntfile.js)
-	// instead of directly in testinit.js.
-	window.addEventListener( "load", function() {
-		window.__karma__.start();
-		jQuery.noConflict();
-		QUnit.start();
-	} );
-};

test/data/testinit.js

@@ -2,18 +2,12 @@
 
 "use strict";
 
-var FILEPATH = "/test/data/testinit.js",
+var parentUrl = window.location.protocol + "//" + window.location.host,
-	activeScript = [].slice.call( document.getElementsByTagName( "script" ), -1 )[ 0 ],
-	parentUrl = activeScript && activeScript.src ?
-		activeScript.src.replace( /[?#].*/, "" ) + FILEPATH.replace( /[^/]+/g, ".." ) + "/" :
-		"../",
 
 	// baseURL is intentionally set to "data/" instead of "".
 	// This is not just for convenience (since most files are in data/)
 	// but also to ensure that urls without prefix fail.
-	// Otherwise it's easy to write tests that pass on test/index.html
+	baseURL = parentUrl + "/test/data/",
-	// but fail in Karma runner (where the baseURL is different).
-	baseURL = parentUrl + "test/data/",
 	supportjQuery = this.jQuery,
 
 	// see RFC 2606
@@ -317,14 +311,9 @@ this.testIframe = function( title, fileName, func, wrapper, iframeStyles ) {
 };
 this.iframeCallback = undefined;
 
-// Tests are always loaded async
+QUnit.config.autostart = false;
-// except when running tests in Karma (See Gruntfile)
-if ( !window.__karma__ ) {
-	QUnit.config.autostart = false;
-}
 
-// Leverage QUnit URL parsing to detect testSwarm environment and "basic" testing mode
+// Leverage QUnit URL parsing to detect "basic" testing mode
-QUnit.isSwarm = ( QUnit.urlParams.swarmURL + "" ).indexOf( "http" ) === 0;
 QUnit.basicTests = ( QUnit.urlParams.module + "" ) === "basic";
 
 // Async test for module script type support
@@ -385,10 +374,17 @@ this.loadTests = function() {
 	// QUnit.config is populated from QUnit.urlParams but only at the beginning
 	// of the test run. We need to read both.
 	var amd = QUnit.config.amd || QUnit.urlParams.amd;
+	var jsdom = QUnit.config.jsdom || QUnit.urlParams.jsdom;
+
+	if ( jsdom ) {
+
+		// JSDOM doesn't implement scrollTo
+		QUnit.config.scrolltop = false;
+	}
 
 	// Directly load tests that need evaluation before DOMContentLoaded.
-	if ( !amd || document.readyState === "loading" ) {
+	if ( !jsdom && ( !amd || document.readyState === "loading" ) ) {
-		document.write( "<script src='" + parentUrl + "test/unit/ready.js'><\x2Fscript>" );
+		document.write( "<script src='" + parentUrl + "/test/unit/ready.js'><\x2Fscript>" );
 	} else {
 		QUnit.module( "ready", function() {
 			QUnit.skip( "jQuery ready tests skipped in async mode", function() {} );
@@ -396,7 +392,7 @@ this.loadTests = function() {
 	}
 
 	// Get testSubproject from testrunner first
-	require( [ parentUrl + "test/data/testrunner.js" ], function() {
+	require( [ parentUrl + "/test/data/testrunner.js" ], function() {
 
 		// Says whether jQuery positional selector extensions are supported.
 		// A full selector engine is required to support them as they need to
@@ -447,7 +443,7 @@ this.loadTests = function() {
 
 			if ( dep ) {
 				if ( !QUnit.basicTests || i === 1 ) {
-					require( [ parentUrl + "test/" + dep ], loadDep );
+					require( [ parentUrl + "/test/" + dep ], loadDep );
 
 				// Support: Android 2.3 only
 				// When running basic tests, replace other modules with dummies to avoid overloading
@@ -458,26 +454,13 @@ this.loadTests = function() {
 				}
 
 			} else {
-				if ( window.__karma__ && window.__karma__.start ) {
-					window.__karma__.start();
-				} else {
-					QUnit.load();
-				}
 
 				/**
 				 * Run in noConflict mode
 				 */
 				jQuery.noConflict();
 
-				// Load the TestSwarm listener if swarmURL is in the address.
+				QUnit.start();
-				if ( QUnit.isSwarm ) {
-					require( [ "https://swarm.jquery.org/js/inject.js?" + ( new Date() ).getTime() ],
-					function() {
-						QUnit.start();
-					} );
-				} else {
-					QUnit.start();
-				}
 			}
 		} )();
 	} );

test/index.html

@@ -10,7 +10,7 @@
 		We have to use previous jQuery as helper to ensure testability with
 		ajax-free builds (and non-interference when changing ajax internals)
 	-->
-	<script src="data/jquery-1.9.1.js"></script>
+	<script src="data/jquery-3.7.1.js"></script>
 
 	<script src="../external/qunit/qunit.js"></script>
 	<script src="../external/sinon/sinon.js"></script>
@@ -26,19 +26,18 @@
 	<script src="jquery.js"></script>
 
 	<script>
-		// Load tests if they have not been loaded
+		( function () {
-		// This is in a different script tag to ensure that
+			// Load tests if they have not been loaded
-		// jQuery is on the page when the testrunner executes
+			// This is in a different script tag to ensure that
-		// QUnit.config is populated from QUnit.urlParams but only at the beginning
+			// jQuery is on the page when the testrunner executes
-		// of the test run. We need to read both.
+			// QUnit.config is populated from QUnit.urlParams but only at the beginning
-		var amd = QUnit.config.amd || QUnit.urlParams.amd;
+			// of the test run. We need to read both.
+			var amd = QUnit.config.amd || QUnit.urlParams.amd;
 
-		// Workaround: Remove call to `window.__karma__.loaded()`
+			if ( !amd ) {
-		// in favour of calling `window.__karma__.start()` from `loadTests()`
+				loadTests();
-		// because tests such as unit/ready.js should run after document ready.
+			}
-		if ( !amd ) {
+		} )();
-			loadTests();
-		}
 	</script>
 </head>
 

test/jquery.js

@@ -3,11 +3,7 @@
 	/* global loadTests: false */
 
 	var config, src,
-		FILEPATH = "/test/jquery.js",
+		parentUrl = window.location.protocol + "//" + window.location.host,
-		activeScript = [].slice.call( document.getElementsByTagName( "script" ), -1 )[ 0 ],
-		parentUrl = activeScript && activeScript.src ?
-			activeScript.src.replace( /[?#].*/, "" ) + FILEPATH.replace( /[^/]+/g, ".." ) + "/" :
-			"../",
 		QUnit = window.QUnit,
 		require = window.require;
 
@@ -67,7 +63,7 @@
 
 	// Otherwise, load synchronously
 	} else {
-		document.write( "<script id='jquery-js' nonce='jquery+hardcoded+nonce' src='" + parentUrl + src + "'><\x2Fscript>" );
+		document.write( "<script id='jquery-js' nonce='jquery+hardcoded+nonce' src='" + parentUrl + "/" + src + "'><\x2Fscript>" );
 	}
 
 } )();

test/karma.context.html

@@ -1,43 +0,0 @@
-<!DOCTYPE html>
-<html lang="en" id="html">
-<head>
-	<meta charset="utf-8">
-	<title>CONTEXT</title>
-	<!-- Karma serves this page from /context.html. Other files are served from /base -->
-	<link rel="stylesheet" href="/base/test/data/testsuite.css" />
-</head>
-<body id="body">
-	<div id="qunit"></div>
-
-	<!-- Start: jQuery Test HTML -->
-	<!-- this iframe is outside the #qunit-fixture so it won't waste time by constantly reloading; the tests are "safe" and clean up after themselves -->
-	<iframe id="loadediframe" name="loadediframe" style="display:none;" src="/base/test/data/iframe.html"></iframe>
-	<div id="qunit-fixture"></div>
-	<!-- End: jQuery Test HTML -->
-
-	<!-- Start: Karma boilerplate -->
-	<script src="/context.js"></script>
-	<script>
-		%CLIENT_CONFIG%
-		window.__karma__.setupContext(window);
-
-		%MAPPINGS%
-	</script>
-	%SCRIPTS%
-	<!-- End: Karma boilerplate -->
-
-	<script src="/base/test/data/qunit-fixture.js"></script>
-	<script>
-		// QUnit.config is populated from QUnit.urlParams but only at the beginning
-		// of the test run. We need to read both.
-		var amd = QUnit.config.amd || QUnit.urlParams.amd;
-
-		// Workaround: Remove call to `window.__karma__.loaded()`
-		// in favour of calling `window.__karma__.start()` from `loadTests()`
-		// because tests such as unit/ready.js should run after document ready.
-		if ( !amd ) {
-			loadTests();
-		}
-	</script>
-</body>
-</html>

test/karma.debug.html

@@ -1,45 +0,0 @@
-<!DOCTYPE html>
-<html lang="en" id="html">
-<head>
-%X_UA_COMPATIBLE%
-	<title>DEBUG</title>
-	<meta charset="utf-8">
-	<!-- Karma serves this page from /context.html. Other files are served from /base -->
-	<link rel="stylesheet" href="/base/external/qunit/qunit.css" />
-	<link rel="stylesheet" href="/base/test/data/testsuite.css" />
-</head>
-<body id="body">
-	<div id="qunit"></div>
-
-	<!-- Start: jQuery Test HTML -->
-	<!-- this iframe is outside the #qunit-fixture so it won't waste time by constantly reloading; the tests are "safe" and clean up after themselves -->
-	<iframe id="loadediframe" name="loadediframe" style="display:none;" src="/base/test/data/iframe.html"></iframe>
-	<div id="qunit-fixture"></div>
-	<!-- End: jQuery Test HTML -->
-
-	<!-- Start: Karma boilerplate -->
-	<script src="/context.js"></script>
-	<script src="/debug.js"></script>
-	<script>
-		%CLIENT_CONFIG%
-
-		%MAPPINGS%
-	</script>
-	%SCRIPTS%
-	<!-- End: Karma boilerplate -->
-
-	<script src="/base/test/data/qunit-fixture.js"></script>
-	<script>
-		// QUnit.config is populated from QUnit.urlParams but only at the beginning
-		// of the test run. We need to read both.
-		var amd = QUnit.config.amd || QUnit.urlParams.amd;
-
-		// Workaround: Remove call to `window.__karma__.loaded()`
-		// in favour of calling `window.__karma__.start()` from `loadTests()`
-		// because tests such as unit/ready.js should run after document ready.
-		if ( !amd ) {
-			loadTests();
-		}
-	</script>
-</body>
-</html>

test/middleware-mockserver.js

@@ -98,9 +98,14 @@ const mocks = {
 		);
 	},
 	json: function( req, resp ) {
+		const headers = {};
 		if ( req.query.header ) {
-			resp.writeHead( 200, { "content-type": "application/json" } );
+			headers[ "content-type" ] = "application/json";
 		}
+		if ( req.query.cors ) {
+			headers[ "access-control-allow-origin" ] = "*";
+		}
+		resp.writeHead( 200, headers );
 		if ( req.query.array ) {
 			resp.end( JSON.stringify(
 				[ { name: "John", age: 21 }, { name: "Peter", age: 25 } ]
@@ -238,7 +243,7 @@ const mocks = {
 		resp.writeHead( 200, {
 			"Content-Type": "text/html",
 			"Content-Security-Policy": "default-src 'self'; " +
-				"report-uri /base/test/data/mock.php?action=cspLog"
+				"report-uri /test/data/mock.php?action=cspLog"
 		} );
 		const body = fs.readFileSync( `${ __dirname }/data/csp.include.html` ).toString();
 		resp.end( body );
@@ -248,7 +253,7 @@ const mocks = {
 		resp.writeHead( 200, {
 			"Content-Type": "text/html",
 			"Content-Security-Policy": "script-src 'nonce-jquery+hardcoded+nonce'; " +
-				"report-uri /base/test/data/mock.php?action=cspLog"
+				"report-uri /test/data/mock.php?action=cspLog"
 		} );
 		const body = fs.readFileSync(
 			`${ __dirname }/data/csp-nonce${ testParam }.html` ).toString();
@@ -315,7 +320,7 @@ function MockserverMiddlewareFactory() {
 	 */
 	return function( req, resp, next ) {
 		const parsed = url.parse( req.url, /* parseQuery */ true );
-		let path = parsed.pathname.replace( /^\/base\//, "" );
+		let path = parsed.pathname;
 		const query = parsed.query;
 		const subReq = Object.assign( Object.create( req ), {
 			query: query,

test/runner/browsers.js

@@ -0,0 +1,25 @@
+// This list is static, so no requests are required
+// in the command help menu.
+
+import { getBrowsers } from "./browserstack/api.js";
+
+export const browsers = [
+	"chrome",
+	"ie",
+	"firefox",
+	"edge",
+	"safari",
+	"opera",
+	"yandex",
+	"IE Mobile",
+	"Android Browser",
+	"Mobile Safari",
+	"jsdom"
+];
+
+// A function that can be used to update the above list.
+export async function getAvailableBrowsers() {
+	const browsers = await getBrowsers( { flat: true } );
+	const available = [ ...new Set( browsers.map( ( { browser } ) => browser ) ) ];
+	return available.concat( "jsdom" );
+}

test/runner/browserstack/api.js

@@ -0,0 +1,332 @@
+/**
+ * Browserstack API is documented at
+ * https://github.com/browserstack/api
+ */
+
+import { createAuthHeader } from "./createAuthHeader.js";
+
+const browserstackApi = "https://api.browserstack.com";
+const apiVersion = 5;
+
+const username = process.env.BROWSERSTACK_USERNAME;
+const accessKey = process.env.BROWSERSTACK_ACCESS_KEY;
+
+// iOS has null for version numbers,
+// and we do not need a similar check for OS versions.
+const rfinalVersion = /(?:^[0-9\.]+$)|(?:^null$)/;
+const rlatest = /^latest-(\d+)$/;
+
+const rnonDigits = /(?:[^\d\.]+)|(?:20\d{2})/g;
+
+async function fetchAPI( path, options = {}, versioned = true ) {
+	if ( !username || !accessKey ) {
+		throw new Error(
+			"BROWSERSTACK_USERNAME and BROWSERSTACK_ACCESS_KEY environment variables must be set."
+		);
+	}
+	const init = {
+		method: "GET",
+		...options,
+		headers: {
+			authorization: createAuthHeader( username, accessKey ),
+			accept: "application/json",
+			"content-type": "application/json",
+			...options.headers
+		}
+	};
+	const response = await fetch(
+		`${ browserstackApi }/${ versioned ? `${ apiVersion }/` : "" }${ path }`,
+		init
+	);
+	if ( !response.ok ) {
+		console.log(
+			`\n${ init.method } ${ path }`,
+			response.status,
+			response.statusText
+		);
+		throw new Error( `Error fetching ${ path }` );
+	}
+	return response.json();
+}
+
+/**
+ * =============================
+ * Browsers API
+ * =============================
+ */
+
+function compareVersionNumbers( a, b ) {
+	if ( a != null && b == null ) {
+		return -1;
+	}
+	if ( a == null && b != null ) {
+		return 1;
+	}
+	if ( a == null && b == null ) {
+		return 0;
+	}
+	const aParts = a.replace( rnonDigits, "" ).split( "." );
+	const bParts = b.replace( rnonDigits, "" ).split( "." );
+
+	if ( aParts.length > bParts.length ) {
+		return -1;
+	}
+	if ( aParts.length < bParts.length ) {
+		return 1;
+	}
+
+	for ( let i = 0; i < aParts.length; i++ ) {
+		const aPart = Number( aParts[ i ] );
+		const bPart = Number( bParts[ i ] );
+		if ( aPart < bPart ) {
+			return -1;
+		}
+		if ( aPart > bPart ) {
+			return 1;
+		}
+	}
+
+	if ( rnonDigits.test( a ) && !rnonDigits.test( b ) ) {
+		return -1;
+	}
+	if ( !rnonDigits.test( a ) && rnonDigits.test( b ) ) {
+		return 1;
+	}
+
+	return 0;
+}
+
+function sortBrowsers( a, b ) {
+	if ( a.browser < b.browser ) {
+		return -1;
+	}
+	if ( a.browser > b.browser ) {
+		return 1;
+	}
+	const browserComparison = compareVersionNumbers(
+		a.browser_version,
+		b.browser_version
+	);
+	if ( browserComparison ) {
+		return browserComparison;
+	}
+	if ( a.os < b.os ) {
+		return -1;
+	}
+	if ( a.os > b.os ) {
+		return 1;
+	}
+	const osComparison = compareVersionNumbers( a.os_version, b.os_version );
+	if ( osComparison ) {
+		return osComparison;
+	}
+	const deviceComparison = compareVersionNumbers( a.device, b.device );
+	if ( deviceComparison ) {
+		return deviceComparison;
+	}
+	return 0;
+}
+
+export async function getBrowsers( { flat = false } = {} ) {
+	const query = new URLSearchParams();
+	if ( flat ) {
+		query.append( "flat", true );
+	}
+	const browsers = await fetchAPI( `/browsers?${ query }` );
+	return browsers.sort( sortBrowsers );
+}
+
+function matchVersion( browserVersion, version ) {
+	if ( !version ) {
+		return false;
+	}
+	const regex = new RegExp(
+		`^${ version.replace( /\\/g, "\\\\" ).replace( /\./g, "\\." ) }\\b`,
+		"i"
+	);
+	return regex.test( browserVersion );
+}
+
+export async function filterBrowsers( filter ) {
+	const browsers = await getBrowsers( { flat: true } );
+	if ( !filter ) {
+		return browsers;
+	}
+	const filterBrowser = ( filter.browser ?? "" ).toLowerCase();
+	const filterVersion = ( filter.browser_version ?? "" ).toLowerCase();
+	const filterOs = ( filter.os ?? "" ).toLowerCase();
+	const filterOsVersion = ( filter.os_version ?? "" ).toLowerCase();
+	const filterDevice = ( filter.device ?? "" ).toLowerCase();
+
+	const filteredWithoutVersion = browsers.filter( ( browser ) => {
+		return (
+			( !filterBrowser || filterBrowser === browser.browser.toLowerCase() ) &&
+			( !filterOs || filterOs === browser.os.toLowerCase() ) &&
+			( !filterOsVersion || matchVersion( browser.os_version, filterOsVersion ) ) &&
+			( !filterDevice || filterDevice === ( browser.device || "" ).toLowerCase() )
+		);
+	} );
+
+	if ( !filterVersion ) {
+		return filteredWithoutVersion;
+	}
+
+	if ( filterVersion.startsWith( "latest" ) ) {
+		const groupedByName = filteredWithoutVersion
+			.filter( ( b ) => rfinalVersion.test( b.browser_version ) )
+			.reduce( ( acc, browser ) => {
+				acc[ browser.browser ] = acc[ browser.browser ] ?? [];
+				acc[ browser.browser ].push( browser );
+				return acc;
+			}, Object.create( null ) );
+
+		const filtered = [];
+		for ( const group of Object.values( groupedByName ) ) {
+			const latest = group[ group.length - 1 ];
+
+			// Mobile devices do not have browser version.
+			// Skip the version check for these,
+			// but include the latest in the list if it made it
+			// through filtering.
+			if ( !latest.browser_version ) {
+
+				// Do not include in the list for latest-n.
+				if ( filterVersion === "latest" ) {
+					filtered.push( latest );
+				}
+				continue;
+			}
+
+			// Get the latest version and subtract the number from the filter,
+			// ignoring any patch versions, which may differ between major versions.
+			const num = rlatest.exec( filterVersion );
+			const version = parseInt( latest.browser_version ) - ( num ? num[ 1 ] : 0 );
+			const match = group.findLast( ( browser ) => {
+				return matchVersion( browser.browser_version, version.toString() );
+			} );
+			if ( match ) {
+				filtered.push( match );
+			}
+		}
+		return filtered;
+	}
+
+	return filteredWithoutVersion.filter( ( browser ) => {
+		return matchVersion( browser.browser_version, filterVersion );
+	} );
+}
+
+export async function listBrowsers( filter ) {
+	const browsers = await filterBrowsers( filter );
+	console.log( "Available browsers:" );
+	for ( const browser of browsers ) {
+		let message = `    ${ browser.browser }_`;
+		if ( browser.device ) {
+			message += `:${ browser.device }_`;
+		} else {
+			message += `${ browser.browser_version }_`;
+		}
+		message += `${ browser.os }_${ browser.os_version }`;
+		console.log( message );
+	}
+}
+
+export async function getLatestBrowser( filter ) {
+	if ( !filter.browser_version ) {
+		filter.browser_version = "latest";
+	}
+	const browsers = await filterBrowsers( filter );
+	return browsers[ browsers.length - 1 ];
+}
+
+/**
+ * =============================
+ * Workers API
+ * =============================
+ */
+
+/**
+ * A browser object may only have one of `browser` or `device` set;
+ * which property is set will depend on `os`.
+ *
+ * `options`: is an object with the following properties:
+ *   `os`: The operating system.
+ *   `os_version`: The operating system version.
+ *   `browser`: The browser name.
+ *   `browser_version`: The browser version.
+ *   `device`: The device name.
+ *   `url` (optional): Which URL to navigate to upon creation.
+ *   `timeout` (optional): Maximum life of the worker (in seconds). Maximum value of `1800`. Specifying `0` will use the default of `300`.
+ *   `name` (optional): Provide a name for the worker.
+ *   `build` (optional): Group workers into a build.
+ *   `project` (optional): Provide the project the worker belongs to.
+ *   `resolution` (optional): Specify the screen resolution (e.g. "1024x768").
+ *   `browserstack.local` (optional): Set to `true` to mark as local testing.
+ *   `browserstack.video` (optional): Set to `false` to disable video recording.
+ *   `browserstack.localIdentifier` (optional): ID of the local tunnel.
+ */
+export function createWorker( options ) {
+	return fetchAPI( "/worker", {
+		method: "POST",
+		body: JSON.stringify( options )
+	} );
+}
+
+/**
+ * Returns a worker object, if one exists, with the following properties:
+ *   `id`: The worker id.
+ *   `status`: A string representing the current status of the worker.
+ *     Possible statuses: `"running"`, `"queue"`.
+ */
+export function getWorker( id ) {
+	return fetchAPI( `/worker/${ id }` );
+}
+
+export async function deleteWorker( id ) {
+	return fetchAPI( `/worker/${ id }`, { method: "DELETE" } );
+}
+
+export function getWorkers() {
+	return fetchAPI( "/workers" );
+}
+
+/**
+ * Stop all workers
+ */
+export async function stopWorkers() {
+	const workers = await getWorkers();
+
+	// Run each request on its own
+	// to avoid connect timeout errors.
+	console.log( `${ workers.length } workers running...` );
+	for ( const worker of workers ) {
+		try {
+			await deleteWorker( worker.id );
+		} catch ( error ) {
+
+			// Log the error, but continue trying to remove workers.
+			console.error( error );
+		}
+	}
+	console.log( "All workers stopped." );
+}
+
+/**
+ * =============================
+ * Plan API
+ * =============================
+ */
+
+export function getPlan() {
+	return fetchAPI( "/automate/plan.json", {}, false );
+}
+
+export async function getAvailableSessions() {
+	try {
+		const [ plan, workers ] = await Promise.all( [ getPlan(), getWorkers() ] );
+		return plan.parallel_sessions_max_allowed - workers.length;
+	} catch ( error ) {
+		console.error( error );
+		return 0;
+	}
+}

test/runner/browserstack/browsers.js

@@ -0,0 +1,199 @@
+import chalk from "chalk";
+import { getBrowserString } from "../lib/getBrowserString.js";
+import { createWorker, deleteWorker, getAvailableSessions } from "./api.js";
+
+const workers = Object.create( null );
+
+/**
+ * Keys are browser strings
+ * Structure of a worker:
+ * {
+ * 	 debug: boolean, // Stops the worker from being cleaned up when finished
+ *   id: string,
+ *   lastTouch: number, // The last time a request was received
+ *   url: string,
+ *   browser: object, // The browser object
+ *   options: object // The options to create the worker
+ * }
+ */
+
+// Acknowledge the worker within the time limit.
+// BrowserStack can take much longer spinning up
+// some browsers, such as iOS 15 Safari.
+const ACKNOWLEDGE_INTERVAL = 1000;
+const ACKNOWLEDGE_TIMEOUT = 60 * 1000 * 5;
+
+const MAX_WORKER_RESTARTS = 5;
+
+// No report after the time limit
+// should refresh the worker
+const RUN_WORKER_TIMEOUT = 60 * 1000 * 2;
+
+const WORKER_WAIT_TIME = 30000;
+
+export function touchBrowser( browser ) {
+	const fullBrowser = getBrowserString( browser );
+	const worker = workers[ fullBrowser ];
+	if ( worker ) {
+		worker.lastTouch = Date.now();
+	}
+}
+
+async function waitForAck( worker, { fullBrowser, verbose } ) {
+	delete worker.lastTouch;
+	return new Promise( ( resolve, reject ) => {
+		const interval = setInterval( () => {
+			if ( worker.lastTouch ) {
+				if ( verbose ) {
+					console.log( `\n${ fullBrowser } acknowledged.` );
+				}
+				clearTimeout( timeout );
+				clearInterval( interval );
+				resolve();
+			}
+		}, ACKNOWLEDGE_INTERVAL );
+
+		const timeout = setTimeout( () => {
+			clearInterval( interval );
+			reject(
+				new Error(
+					`${ fullBrowser } not acknowledged after ${
+						ACKNOWLEDGE_TIMEOUT / 1000 / 60
+					}min.`
+				)
+			);
+		}, ACKNOWLEDGE_TIMEOUT );
+	} );
+}
+
+async function ensureAcknowledged( worker, restarts ) {
+	const fullBrowser = getBrowserString( worker.browser );
+	const verbose = worker.options.verbose;
+	try {
+		await waitForAck( worker, { fullBrowser, verbose } );
+		return worker;
+	} catch ( error ) {
+		console.error( error.message );
+		await cleanupWorker( worker, { verbose } );
+		await createBrowserWorker(
+			worker.url,
+			worker.browser,
+			worker.options,
+			restarts + 1
+		);
+	}
+}
+
+export async function createBrowserWorker( url, browser, options, restarts = 0 ) {
+	if ( restarts > MAX_WORKER_RESTARTS ) {
+		throw new Error(
+			`Reached the maximum number of restarts for ${ chalk.yellow(
+				getBrowserString( browser )
+			) }`
+		);
+	}
+	const verbose = options.verbose;
+	while ( ( await getAvailableSessions() ) <= 0 ) {
+		if ( verbose ) {
+			console.log( "\nWaiting for available sessions..." );
+		}
+		await new Promise( ( resolve ) => setTimeout( resolve, WORKER_WAIT_TIME ) );
+	}
+
+	const { debug, runId, tunnelId } = options;
+	const fullBrowser = getBrowserString( browser );
+
+	const worker = await createWorker( {
+		...browser,
+		url: encodeURI( url ),
+		project: "jquery",
+		build: `Run ${ runId }`,
+
+		// This is the maximum timeout allowed
+		// by BrowserStack. We do this because
+		// we control the timeout in the runner.
+		// See https://github.com/browserstack/api/blob/b324a6a5bc1b6052510d74e286b8e1c758c308a7/README.md#timeout300
+		timeout: 1800,
+
+		// Not documented in the API docs,
+		// but required to make local testing work.
+		// See https://www.browserstack.com/docs/automate/selenium/manage-multiple-connections#nodejs
+		"browserstack.local": true,
+		"browserstack.localIdentifier": tunnelId
+	} );
+
+	browser.debug = !!debug;
+	worker.url = url;
+	worker.browser = browser;
+	worker.restarts = restarts;
+	worker.options = options;
+	touchBrowser( browser );
+	workers[ fullBrowser ] = worker;
+
+	// Wait for the worker to show up in the list
+	// before returning it.
+	return ensureAcknowledged( worker, restarts );
+}
+
+export async function setBrowserWorkerUrl( browser, url ) {
+	const fullBrowser = getBrowserString( browser );
+	const worker = workers[ fullBrowser ];
+	if ( worker ) {
+		worker.url = url;
+	}
+}
+
+/**
+ * Checks that all browsers have received
+ * a response in the given amount of time.
+ * If not, the worker is restarted.
+ */
+export async function checkLastTouches() {
+	for ( const [ fullBrowser, worker ] of Object.entries( workers ) ) {
+		if ( Date.now() - worker.lastTouch > RUN_WORKER_TIMEOUT ) {
+			const options = worker.options;
+			if ( options.verbose ) {
+				console.log(
+					`\nNo response from ${ chalk.yellow( fullBrowser ) } in ${
+						RUN_WORKER_TIMEOUT / 1000 / 60
+					}min.`
+				);
+			}
+			await cleanupWorker( worker, options );
+			await createBrowserWorker(
+				worker.url,
+				worker.browser,
+				options,
+				worker.restarts
+			);
+		}
+	}
+}
+
+export async function cleanupWorker( worker, { verbose } ) {
+	for ( const [ fullBrowser, w ] of Object.entries( workers ) ) {
+		if ( w === worker ) {
+			delete workers[ fullBrowser ];
+			await deleteWorker( worker.id );
+			if ( verbose ) {
+				console.log( `\nStopped ${ fullBrowser }.` );
+			}
+			return;
+		}
+	}
+}
+
+export async function cleanupAllBrowsers( { verbose } ) {
+	const workersRemaining = Object.values( workers );
+	const numRemaining = workersRemaining.length;
+	if ( numRemaining ) {
+		await Promise.all(
+			workersRemaining.map( ( worker ) => deleteWorker( worker.id ) )
+		);
+		if ( verbose ) {
+			console.log(
+				`Stopped ${ numRemaining } browser${ numRemaining > 1 ? "s" : "" }.`
+			);
+		}
+	}
+}

test/runner/browserstack/buildBrowserFromString.js

@@ -0,0 +1,20 @@
+export function buildBrowserFromString( str ) {
+	const [ browser, versionOrDevice, os, osVersion ] = str.split( "_" );
+
+	// If the version starts with a colon, it's a device
+	if ( versionOrDevice && versionOrDevice.startsWith( ":" ) ) {
+		return {
+			browser,
+			device: versionOrDevice.slice( 1 ),
+			os,
+			os_version: osVersion
+		};
+	}
+
+	return {
+		browser,
+		browser_version: versionOrDevice,
+		os,
+		os_version: osVersion
+	};
+}

test/runner/browserstack/createAuthHeader.js

@@ -0,0 +1,7 @@
+const textEncoder = new TextEncoder();
+
+export function createAuthHeader( username, accessKey ) {
+	const encoded = textEncoder.encode( `${ username }:${ accessKey }` );
+	const base64 = btoa( String.fromCodePoint.apply( null, encoded ) );
+	return `Basic ${ base64 }`;
+}

test/runner/browserstack/local.js

@@ -0,0 +1,34 @@
+import browserstackLocal from "browserstack-local";
+
+export async function localTunnel( localIdentifier, opts = {} ) {
+	const tunnel = new browserstackLocal.Local();
+
+	return new Promise( ( resolve, reject ) => {
+
+		// https://www.browserstack.com/docs/local-testing/binary-params
+		tunnel.start(
+			{
+				"enable-logging-for-api": "",
+				localIdentifier,
+				...opts
+			},
+			async( error ) => {
+				if ( error || !tunnel.isRunning() ) {
+					return reject( error );
+				}
+				resolve( {
+					stop: function stopTunnel() {
+						return new Promise( ( resolve, reject ) => {
+							tunnel.stop( ( error ) => {
+								if ( error ) {
+									return reject( error );
+								}
+								resolve();
+							} );
+						} );
+					}
+				} );
+			}
+		);
+	} );
+}

test/runner/browserstack/queue.js

@@ -0,0 +1,90 @@
+import chalk from "chalk";
+import { getBrowserString } from "../lib/getBrowserString.js";
+import { checkLastTouches, createBrowserWorker, setBrowserWorkerUrl } from "./browsers.js";
+
+const TEST_POLL_TIMEOUT = 1000;
+
+const queue = [];
+
+export function getNextBrowserTest( reportId ) {
+	const index = queue.findIndex( ( test ) => test.id === reportId );
+	if ( index === -1 ) {
+		return;
+	}
+
+	// Remove the completed test from the queue
+	const previousTest = queue[ index ];
+	queue.splice( index, 1 );
+
+	// Find the next test for the same browser
+	for ( const test of queue.slice( index ) ) {
+		if ( test.fullBrowser === previousTest.fullBrowser ) {
+
+			// Set the URL for our tracking
+			setBrowserWorkerUrl( test.browser, test.url );
+			test.running = true;
+
+			// Return the URL for the next test.
+			// listeners.js will use this to set the browser URL.
+			return { url: test.url };
+		}
+	}
+}
+
+export function retryTest( reportId, maxRetries ) {
+	const test = queue.find( ( test ) => test.id === reportId );
+	if ( test ) {
+		test.retries++;
+		if ( test.retries <= maxRetries ) {
+			console.log(
+				`Retrying test ${ reportId } for ${ chalk.yellow(
+					test.options.modules.join( ", " )
+				) }...`
+			);
+			return test;
+		}
+	}
+}
+
+export function addBrowserStackRun( url, browser, options ) {
+	queue.push( {
+		browser,
+		fullBrowser: getBrowserString( browser ),
+		id: options.reportId,
+		url,
+		options,
+		retries: 0,
+		running: false
+	} );
+}
+
+export async function runAllBrowserStack() {
+	return new Promise( async( resolve, reject )=> {
+		while ( queue.length ) {
+			try {
+				await checkLastTouches();
+			} catch ( error ) {
+				reject( error );
+			}
+
+			// Run one test URL per browser at a time
+			const browsersTaken = [];
+			for ( const test of queue ) {
+				if ( browsersTaken.indexOf( test.fullBrowser ) > -1 ) {
+					continue;
+				}
+				browsersTaken.push( test.fullBrowser );
+				if ( !test.running ) {
+					test.running = true;
+					try {
+						await createBrowserWorker( test.url, test.browser, test.options );
+					} catch ( error ) {
+						reject( error );
+					}
+				}
+			}
+			await new Promise( ( resolve ) => setTimeout( resolve, TEST_POLL_TIMEOUT ) );
+		}
+		resolve();
+	} );
+}

test/runner/command.js

@@ -0,0 +1,131 @@
+import yargs from "yargs/yargs";
+import { browsers } from "./browsers.js";
+import { getPlan, listBrowsers, stopWorkers } from "./browserstack/api.js";
+import { buildBrowserFromString } from "./browserstack/buildBrowserFromString.js";
+import { modules } from "./modules.js";
+import { run } from "./run.js";
+
+const argv = yargs( process.argv.slice( 2 ) )
+	.version( false )
+	.strict()
+	.command( {
+		command: "[options]",
+		describe: "Run jQuery tests in a browser"
+	} )
+	.option( "module", {
+		alias: "m",
+		type: "array",
+		choices: modules,
+		description:
+			"Run tests for a specific module. " +
+			"Pass multiple modules by repeating the option. " +
+			"Defaults to all modules."
+	} )
+	.option( "browser", {
+		alias: "b",
+		type: "array",
+		choices: browsers,
+		description:
+			"Run tests in a specific browser." +
+			"Pass multiple browsers by repeating the option." +
+			"If using BrowserStack, specify browsers using --browserstack." +
+			"Only the basic module is supported on jsdom.",
+		default: [ "chrome" ]
+	} )
+	.option( "headless", {
+		alias: "h",
+		type: "boolean",
+		description:
+			"Run tests in headless mode. Cannot be used with --debug or --browserstack.",
+		conflicts: [ "debug", "browserstack" ]
+	} )
+	.option( "amd", {
+		type: "boolean",
+		description: "Run tests using jQuery's source, which is written with AMD modules."
+	} )
+	.option( "concurrency", {
+		alias: "c",
+		type: "number",
+		description:
+			"Run tests in parallel in multiple browsers. " +
+			"Defaults to 8 in normal mode. In browserstack mode, " +
+			"defaults to the maximum available under your BrowserStack plan."
+	} )
+	.option( "debug", {
+		alias: "d",
+		type: "boolean",
+		description:
+			"Leave the browser open for debugging. Cannot be used with --headless.",
+		conflicts: [ "headless" ]
+	} )
+	.option( "verbose", {
+		alias: "v",
+		type: "boolean",
+		description: "Log additional information."
+	} )
+	.option( "retries", {
+		alias: "r",
+		type: "number",
+		description: "Number of times to retry failed tests in BrowserStack.",
+		implies: [ "browserstack" ]
+	} )
+	.option( "run-id", {
+		type: "string",
+		description: "A unique identifier for this run."
+	} )
+	.option( "isolate", {
+		type: "boolean",
+		description: "Run each module by itself in the test page. This can extend testing time."
+	} )
+	.option( "browserstack", {
+		type: "array",
+		description:
+			"Run tests in BrowserStack.\n" +
+			"Requires BROWSERSTACK_USERNAME and BROWSERSTACK_ACCESS_KEY environment variables.\n" +
+			"The value can be empty for the default configuration, or a string in the format of\n" +
+			"\"browser_[browserVersion | :device]_os_osVersion\" (see --list-browsers).\n" +
+			"Pass multiple browsers by repeating the option.\n" +
+			"The --browser option is ignored when --browserstack has a value.\n" +
+			"Otherwise, the --browser option will be used, " +
+			"with the latest version/device for that browser, on a matching OS."
+	} )
+	.option( "list-browsers", {
+		type: "string",
+		description:
+			"List available BrowserStack browsers and exit.\n" +
+			"Leave blank to view all browsers or pass " +
+			"\"browser_[browserVersion | :device]_os_osVersion\" with each parameter " +
+			"separated by an underscore to filter the list (any can be omitted).\n" +
+			"\"latest\" can be used in place of \"browserVersion\" to find the latest version.\n" +
+			"\"latest-n\" can be used to find the nth latest browser version.\n" +
+			"Use a colon to indicate a device.\n" +
+			"Examples: \"chrome__windows_10\", \"safari_latest\", " +
+			"\"Mobile Safari\", \"Android Browser_:Google Pixel 8 Pro\".\n" +
+			"Use quotes if spaces are necessary."
+	} )
+	.option( "stop-workers", {
+		type: "boolean",
+		description:
+			"WARNING: This will stop all BrowserStack workers that may exist and exit," +
+			"including any workers running from other projects.\n" +
+			"This can be used as a failsafe when there are too many stray workers."
+	} )
+	.option( "browserstack-plan", {
+		type: "boolean",
+		description: "Show BrowserStack plan information and exit."
+	} )
+	.help().argv;
+
+if ( typeof argv.listBrowsers === "string" ) {
+	listBrowsers( buildBrowserFromString( argv.listBrowsers ) );
+} else if ( argv.stopWorkers ) {
+	stopWorkers();
+} else if ( argv.browserstackPlan ) {
+	console.log( await getPlan() );
+} else {
+	run( {
+		...argv,
+		browsers: argv.browser,
+		modules: argv.module
+	} );
+}

test/runner/createTestServer.js

@@ -0,0 +1,64 @@
+import bodyParser from "body-parser";
+import express from "express";
+import bodyParserErrorHandler from "express-body-parser-error-handler";
+import fs from "fs";
+import mockServer from "../middleware-mockserver.js";
+
+export async function createTestServer( report ) {
+	const nameHTML = await fs.promises.readFile( "./test/data/name.html", "utf8" );
+	const indexHTML = await fs.promises.readFile( "./test/index.html", "utf8" );
+	const app = express();
+
+	// Redirect home to test page
+	app.get( "/", ( _req, res ) => {
+		res.redirect( "/test/" );
+	} );
+
+	// Redirect to trailing slash
+	app.use( ( req, res, next ) => {
+		if ( req.path === "/test" ) {
+			const query = req.url.slice( req.path.length );
+			res.redirect( 301, `${ req.path }/${ query }` );
+		} else {
+			next();
+		}
+	} );
+
+	// Add a script tag to the index.html to load the QUnit listeners
+	app.use( /\/test(?:\/index.html)?\//, ( _req, res ) => {
+		res.send( indexHTML.replace(
+			"</head>",
+			"<script src=\"/test/runner/listeners.js\"></script></head>"
+		) );
+	} );
+
+	// Bind the reporter
+	app.post( "/api/report", bodyParser.json( { limit: "50mb" } ), ( req, res ) => {
+		if ( report ) {
+			const response = report( req.body );
+			if ( response ) {
+				res.json( response );
+				return;
+			}
+		}
+		res.sendStatus( 204 );
+	} );
+
+	// Handle errors from the body parser
+	app.use( bodyParserErrorHandler() );
+
+	// Hook up mock server
+	app.use( mockServer() );
+
+	// Serve static files
+	app.post( "/test/data/name.html", ( _req, res ) => {
+		res.send( nameHTML );
+	} );
+
+	app.use( "/dist", express.static( "dist" ) );
+	app.use( "/src", express.static( "src" ) );
+	app.use( "/test", express.static( "test" ) );
+	app.use( "/external", express.static( "external" ) );
+
+	return app;
+}

test/runner/jsdom.js

@@ -0,0 +1,55 @@
+import jsdom from "jsdom";
+
+const { JSDOM } = jsdom;
+
+const windows = Object.create( null );
+
+export async function runJSDOM( url, { reportId, verbose } ) {
+	const virtualConsole = new jsdom.VirtualConsole();
+	virtualConsole.sendTo( console );
+	virtualConsole.removeAllListeners( "clear" );
+
+	const { window } = await JSDOM.fromURL( url, {
+		resources: "usable",
+		runScripts: "dangerously",
+		virtualConsole
+	} );
+	if ( verbose ) {
+		console.log( "JSDOM window opened.", reportId );
+	}
+	windows[ reportId ] = window;
+
+	return new Promise( ( resolve ) => {
+		window.finish = resolve;
+	} );
+}
+
+export function cleanupJSDOM( reportId, { verbose } ) {
+	const window = windows[ reportId ];
+	if ( window ) {
+		if ( window.finish ) {
+			window.finish();
+		}
+		window.close();
+		delete windows[ reportId ];
+		if ( verbose ) {
+			console.log( "Closed JSDOM window.", reportId );
+		}
+	}
+}
+
+export function cleanupAllJSDOM( { verbose } ) {
+	const windowsRemaining = Object.keys( windows ).length;
+	if ( windowsRemaining ) {
+		if ( verbose ) {
+			console.log(
+				`Cleaning up ${ windowsRemaining } JSDOM window${
+					windowsRemaining > 1 ? "s" : ""
+				}...`
+			);
+		}
+		for ( const id in windows ) {
+			cleanupJSDOM( id, { verbose } );
+		}
+	}
+}

test/runner/lib/buildTestUrl.js

@@ -0,0 +1,29 @@
+import { generateModuleId } from "./generateHash.js";
+
+export function buildTestUrl( modules, { amd, browserstack, jsdom, port, reportId } ) {
+	if ( !port ) {
+		throw new Error( "No port specified." );
+	}
+
+	const query = new URLSearchParams();
+	for ( const module of modules ) {
+		query.append( "moduleId", generateModuleId( module ) );
+	}
+
+	if ( amd ) {
+		query.append( "amd", "true" );
+	}
+
+	if ( jsdom ) {
+		query.append( "jsdom", "true" );
+	}
+
+	if ( reportId ) {
+		query.append( "reportId", reportId );
+	}
+
+	// BrowserStack supplies a custom domain for local testing,
+	// which is especially necessary for iOS testing.
+	const host = browserstack ? "bs-local.com" : "localhost";
+	return `http://${ host }:${ port }/test/?${ query }`;
+}

test/runner/lib/generateHash.js

@@ -0,0 +1,50 @@
+import crypto from "node:crypto";
+
+export function generateHash( string ) {
+	const hash = crypto.createHash( "md5" );
+	hash.update( string );
+
+	// QUnit hashes are 8 characters long
+	// We use 10 characters to be more visually distinct
+	return hash.digest( "hex" ).slice( 0, 10 );
+}
+
+/**
+ * A copy of the generate hash function from QUnit,
+ * used to generate a hash for the module name.
+ *
+ * QUnit errors on passing multiple modules to the
+ * module query parameter. We need to know
+ * the hash for each module before loading QUnit
+ * in order to pass multiple moduleId parameters instead.
+ */
+export function generateModuleId( module, browser ) {
+
+	// QUnit normally hashes the test name, but
+	// we've repurposed this function to generate
+	// report IDs for module/browser combinations.
+	// We still use it without the browser parameter
+	// to get the same module IDs as QUnit to pass
+	// multiple ahead-of-time in the query string.
+	const str = module + "\x1C" + browser;
+	let hash = 0;
+
+	for ( let i = 0; i < str.length; i++ ) {
+		hash = ( hash << 5 ) - hash + str.charCodeAt( i );
+		hash |= 0;
+	}
+
+	let hex = ( 0x100000000 + hash ).toString( 16 );
+	if ( hex.length < 8 ) {
+		hex = "0000000" + hex;
+	}
+
+	return hex.slice( -8 );
+}
+
+export function printModuleHashes( modules ) {
+	console.log( "Module hashes:" );
+	modules.forEach( ( module ) => {
+		console.log( `  ${ module }: ${ generateModuleId( module ) }` );
+	} );
+}

test/runner/lib/getBrowserString.js

@@ -0,0 +1,49 @@
+const browserMap = {
+	chrome: "Chrome",
+	edge: "Edge",
+	firefox: "Firefox",
+	ie: "IE",
+	jsdom: "JSDOM",
+	opera: "Opera",
+	safari: "Safari"
+};
+
+export function browserSupportsHeadless( browser ) {
+	browser = browser.toLowerCase();
+	return (
+		browser === "chrome" ||
+		browser === "firefox" ||
+		browser === "edge"
+	);
+}
+
+export function getBrowserString(
+	{
+		browser,
+		browser_version: browserVersion,
+		device,
+		os,
+		os_version: osVersion
+	},
+	headless
+) {
+	browser = browser.toLowerCase();
+	browser = browserMap[ browser ] || browser;
+	let str = browser;
+	if ( browserVersion ) {
+		str += ` ${ browserVersion }`;
+	}
+	if ( device ) {
+		str += ` for ${ device }`;
+	}
+	if ( os ) {
+		str += ` on ${ os }`;
+	}
+	if ( osVersion ) {
+		str += ` ${ osVersion }`;
+	}
+	if ( headless && browserSupportsHeadless( browser ) ) {
+		str += " (headless)";
+	}
+	return str;
+}

test/runner/lib/prettyMs.js

@@ -0,0 +1,18 @@
+/**
+ * Pretty print a time in milliseconds.
+ */
+export function prettyMs( time ) {
+	const minutes = Math.floor( time / 60000 );
+	const seconds = Math.floor( time / 1000 );
+	const ms = Math.floor( time % 1000 );
+
+	let prettyTime = `${ ms }ms`;
+	if ( seconds > 0 ) {
+		prettyTime = `${ seconds }s ${ prettyTime }`;
+	}
+	if ( minutes > 0 ) {
+		prettyTime = `${ minutes }m ${ prettyTime }`;
+	}
+
+	return prettyTime;
+}

test/runner/listeners.js

@@ -0,0 +1,99 @@
+( function() {
+
+	"use strict";
+
+	// Get the report ID from the URL.
+	var match = location.search.match( /reportId=([^&]+)/ );
+	if ( !match ) {
+		return;
+	}
+	var id = match[ 1 ];
+
+	// Adopted from https://github.com/douglascrockford/JSON-js
+	// Support: IE 11+
+	// Using the replacer argument of JSON.stringify in IE has issues
+	// TODO: Replace this with a circular replacer + JSON.stringify + WeakSet
+	function decycle( object ) {
+		var objects = [];
+
+		// The derez function recurses through the object, producing the deep copy.
+		function derez( value ) {
+			if (
+				typeof value === "object" &&
+				value !== null &&
+				!( value instanceof Boolean ) &&
+				!( value instanceof Date ) &&
+				!( value instanceof Number ) &&
+				!( value instanceof RegExp ) &&
+				!( value instanceof String )
+			) {
+
+				// Return a string early for elements
+				if ( value.nodeType ) {
+					return value.toString();
+				}
+
+				if ( objects.indexOf( value ) > -1 ) {
+					return;
+				}
+
+				objects.push( value );
+
+				if ( Array.isArray( value ) ) {
+
+					// If it is an array, replicate the array.
+					return value.map( derez );
+				} else {
+
+					// If it is an object, replicate the object.
+					var nu = Object.create( null );
+					Object.keys( value ).forEach( function( name ) {
+						nu[ name ] = derez( value[ name ] );
+					} );
+					return nu;
+				}
+			}
+			return value;
+		}
+		return derez( object );
+	}
+
+	function send( type, data ) {
+		var json = JSON.stringify( {
+			id: id,
+			type: type,
+			data: data ? decycle( data ) : undefined
+		} );
+		var request = new XMLHttpRequest();
+		request.open( "POST", "/api/report", true );
+		request.setRequestHeader( "Content-Type", "application/json" );
+		request.send( json );
+		return request;
+	}
+
+	// Send acknowledgement to the server.
+	send( "ack" );
+
+	QUnit.on( "testEnd", function( data ) {
+		send( "testEnd", data );
+	} );
+
+	QUnit.on( "runEnd", function( data ) {
+
+		// Reduce the payload size.
+		// childSuites is large and unused.
+		data.childSuites = undefined;
+
+		var request = send( "runEnd", data );
+		request.onload = function() {
+			if ( request.status === 200 && request.responseText ) {
+				try {
+					var json = JSON.parse( request.responseText );
+					window.location = json.url;
+				} catch ( e ) {
+					console.error( e );
+				}
+			}
+		};
+	} );
+} )();

test/runner/modules.js

@@ -0,0 +1,24 @@
+export const modules = [
+	"basic",
+
+	"ajax",
+	"animation",
+	"attributes",
+	"callbacks",
+	"core",
+	"css",
+	"data",
+	"deferred",
+	"deprecated",
+	"dimensions",
+	"effects",
+	"event",
+	"manipulation",
+	"offset",
+	"queue",
+	"selector",
+	"serialize",
+	"support",
+	"traversing",
+	"tween"
+];

test/runner/package.json

@@ -0,0 +1,3 @@
+{
+  "type": "module"
+}

test/runner/reporter.js

@@ -0,0 +1,54 @@
+import chalk from "chalk";
+import { getBrowserString } from "./lib/getBrowserString.js";
+import { prettyMs } from "./lib/prettyMs.js";
+
+export function reportTest( test, reportId, { browser, headless } ) {
+	if ( test.status === "passed" ) {
+
+		// Write to console without newlines
+		process.stdout.write( "." );
+		return;
+	}
+
+	let message = `Test ${ test.status } on ${ chalk.yellow(
+		getBrowserString( browser, headless )
+	) } (${ chalk.bold( reportId ) }).`;
+	message += `\n${ chalk.bold( `${ test.suiteName }: ${ test.name }` ) }`;
+
+	// Prefer failed assertions over error messages
+	if ( test.assertions.filter( ( a ) => !!a && !a.passed ).length ) {
+		test.assertions.forEach( ( assertion, i ) => {
+			if ( !assertion.passed ) {
+				message += `\n${ i + 1 }. ${ chalk.red( assertion.message ) }`;
+				message += `\n${ chalk.gray( assertion.stack ) }`;
+			}
+		} );
+	} else if ( test.errors.length ) {
+		for ( const error of test.errors ) {
+			message += `\n${ chalk.red( error.message ) }`;
+			message += `\n${ chalk.gray( error.stack ) }`;
+		}
+	}
+
+	console.log( "\n\n" + message );
+
+	if ( test.status === "failed" ) {
+		return message;
+	}
+}
+
+export function reportEnd( result, reportId, { browser, headless, modules } ) {
+	console.log(
+		`\n\nTests for ${ chalk.yellow( modules.join( ", " ) ) } on ${ chalk.yellow(
+			getBrowserString( browser, headless )
+		) } finished in ${ prettyMs( result.runtime ) } (${ chalk.bold( reportId ) }).`
+	);
+	console.log(
+		( result.status !== "passed" ?
+			`${ chalk.red( result.testCounts.failed ) } failed. ` :
+			"" ) +
+			`${ chalk.green( result.testCounts.total ) } passed. ` +
+			`${ chalk.gray( result.testCounts.skipped ) } skipped.`
+	);
+	return result.testCounts;
+}

test/runner/run.js

@@ -0,0 +1,337 @@
+import chalk from "chalk";
+import { asyncExitHook, gracefulExit } from "exit-hook";
+import { getLatestBrowser } from "./browserstack/api.js";
+import { buildBrowserFromString } from "./browserstack/buildBrowserFromString.js";
+import { localTunnel } from "./browserstack/local.js";
+import { reportEnd, reportTest } from "./reporter.js";
+import { createTestServer } from "./createTestServer.js";
+import { buildTestUrl } from "./lib/buildTestUrl.js";
+import { generateHash, printModuleHashes } from "./lib/generateHash.js";
+import { getBrowserString } from "./lib/getBrowserString.js";
+import { cleanupAllJSDOM, cleanupJSDOM } from "./jsdom.js";
+import { modules as allModules } from "./modules.js";
+import { cleanupAllBrowsers, touchBrowser } from "./browserstack/browsers.js";
+import {
+	addBrowserStackRun,
+	getNextBrowserTest,
+	retryTest,
+	runAllBrowserStack
+} from "./browserstack/queue.js";
+import { addSeleniumRun, runAllSelenium } from "./selenium/queue.js";
+
+const EXIT_HOOK_WAIT_TIMEOUT = 60 * 1000;
+
+/**
+ * Run modules in parallel in different browser instances.
+ */
+export async function run( {
+	amd,
+	browsers: browserNames,
+	browserstack,
+	concurrency,
+	debug,
+	headless,
+	isolate,
+	modules = [],
+	retries = 0,
+	runId,
+	verbose
+} ) {
+	if ( !browserNames || !browserNames.length ) {
+		browserNames = [ "chrome" ];
+	}
+	if ( !modules.length ) {
+		modules = allModules;
+	}
+	if ( headless && debug ) {
+		throw new Error(
+			"Cannot run in headless mode and debug mode at the same time."
+		);
+	}
+
+	if ( verbose ) {
+		console.log( browserstack ? "Running in BrowserStack." : "Running locally." );
+	}
+
+	const errorMessages = [];
+	const pendingErrors = {};
+
+	// Convert browser names to browser objects
+	let browsers = browserNames.map( ( b ) => ( { browser: b } ) );
+	const tunnelId = generateHash(
+		`${ Date.now() }-${ modules.join( ":" ) }-${ ( browserstack || [] )
+			.concat( browserNames )
+			.join( ":" ) }`
+	);
+
+	// A unique identifier for this run
+	if ( !runId ) {
+		runId = tunnelId;
+	}
+
+	// Create the test app and
+	// hook it up to the reporter
+	const reports = Object.create( null );
+	const app = await createTestServer( ( message ) => {
+		switch ( message.type ) {
+			case "testEnd": {
+				const reportId = message.id;
+				const report = reports[ reportId ];
+				touchBrowser( report.browser );
+				const errors = reportTest( message.data, reportId, report );
+				pendingErrors[ reportId ] ??= Object.create( null );
+				if ( errors ) {
+					pendingErrors[ reportId ][ message.data.name ] = errors;
+				} else {
+					delete pendingErrors[ reportId ][ message.data.name ];
+				}
+				break;
+			}
+			case "runEnd": {
+				const reportId = message.id;
+				const report = reports[ reportId ];
+				touchBrowser( report.browser );
+				const { failed, total } = reportEnd(
+					message.data,
+					message.id,
+					reports[ reportId ]
+				);
+				report.total = total;
+
+				cleanupJSDOM( reportId, { verbose } );
+
+				// Handle failure
+				if ( failed ) {
+					const retry = retryTest( reportId, retries );
+					if ( retry ) {
+						return retry;
+					}
+					errorMessages.push( ...Object.values( pendingErrors[ reportId ] ) );
+					return getNextBrowserTest( reportId );
+				}
+
+				// Handle success
+				if (
+					pendingErrors[ reportId ] &&
+					Object.keys( pendingErrors[ reportId ] ).length
+				) {
+					console.warn( "Detected flaky tests:" );
+					for ( const [ , error ] in Object.entries( pendingErrors[ reportId ] ) ) {
+						console.warn( chalk.italic( chalk.gray( error ) ) );
+					}
+					delete pendingErrors[ reportId ];
+				}
+				return getNextBrowserTest( reportId );
+			}
+			case "ack": {
+				const report = reports[ message.id ];
+				touchBrowser( report.browser );
+				break;
+			}
+			default:
+				console.warn( "Received unknown message type:", message.type );
+		}
+	} );
+
+	// Start up local test server
+	let server;
+	let port;
+	await new Promise( ( resolve ) => {
+
+		// Pass 0 to choose a random, unused port
+		server = app.listen( 0, () => {
+			port = server.address().port;
+			resolve();
+		} );
+	} );
+
+	if ( !server || !port ) {
+		throw new Error( "Server not started." );
+	}
+
+	if ( verbose ) {
+		console.log( `Server started on port ${ port }.` );
+	}
+
+	function stopServer() {
+		return new Promise( ( resolve ) => {
+			server.close( () => {
+				if ( verbose ) {
+					console.log( "Server stopped." );
+				}
+				resolve();
+			} );
+		} );
+	}
+
+	async function cleanup() {
+		console.log( "Cleaning up..." );
+
+		if ( tunnel ) {
+			await tunnel.stop();
+			if ( verbose ) {
+				console.log( "Stopped BrowserStackLocal." );
+			}
+		}
+
+		await cleanupAllBrowsers( { verbose } );
+		cleanupAllJSDOM( { verbose } );
+	}
+
+	asyncExitHook(
+		async() => {
+			await stopServer();
+			await cleanup();
+		},
+		{ wait: EXIT_HOOK_WAIT_TIMEOUT }
+	);
+
+	// Start up BrowserStackLocal
+	let tunnel;
+	if ( browserstack ) {
+		if ( headless ) {
+			console.warn(
+				chalk.italic(
+					"BrowserStack does not support headless mode. Running in normal mode."
+				)
+			);
+			headless = false;
+		}
+
+		// Convert browserstack to browser objects.
+		// If browserstack is an empty array, fall back
+		// to the browsers array.
+		if ( browserstack.length ) {
+			browsers = browserstack.map( ( b ) => {
+				if ( !b ) {
+					return browsers[ 0 ];
+				}
+				return buildBrowserFromString( b );
+			} );
+		}
+
+		// Fill out browser defaults
+		browsers = await Promise.all(
+			browsers.map( async( browser ) => {
+
+				// Avoid undici connect timeout errors
+				await new Promise( ( resolve ) => setTimeout( resolve, 100 ) );
+
+				const latestMatch = await getLatestBrowser( browser );
+				if ( !latestMatch ) {
+					console.error(
+						chalk.red( `Browser not found: ${ getBrowserString( browser ) }.` )
+					);
+					gracefulExit( 1 );
+				}
+				return latestMatch;
+			} )
+		);
+
+		tunnel = await localTunnel( tunnelId );
+		if ( verbose ) {
+			console.log( "Started BrowserStackLocal." );
+
+			printModuleHashes( modules );
+		}
+	}
+
+	function queueRun( modules, browser ) {
+		const fullBrowser = getBrowserString( browser, headless );
+		const reportId = generateHash( `${ modules.join( ":" ) } ${ fullBrowser }` );
+		reports[ reportId ] = { browser, headless, modules };
+
+		const url = buildTestUrl( modules, {
+			amd,
+			browserstack,
+			jsdom: browser.browser === "jsdom",
+			port,
+			reportId
+		} );
+
+		const options = {
+			debug,
+			headless,
+			modules,
+			reportId,
+			runId,
+			tunnelId,
+			verbose
+		};
+
+		if ( browserstack ) {
+			addBrowserStackRun( url, browser, options );
+		} else {
+			addSeleniumRun( url, browser, options );
+		}
+	}
+
+	for ( const browser of browsers ) {
+		if ( isolate ) {
+			for ( const module of modules ) {
+				queueRun( [ module ], browser );
+			}
+		} else {
+			queueRun( modules, browser );
+		}
+	}
+
+	try {
+		console.log( `Starting Run ${ runId }...` );
+		if ( browserstack ) {
+			await runAllBrowserStack( { verbose } );
+		} else {
+			await runAllSelenium( { concurrency, verbose } );
+		}
+	} catch ( error ) {
+		console.error( error );
+		if ( !debug ) {
+			gracefulExit( 1 );
+		}
+	} finally {
+		console.log();
+		if ( errorMessages.length === 0 ) {
+			let stop = false;
+			for ( const report of Object.values( reports ) ) {
+				if ( !report.total ) {
+					stop = true;
+					console.error(
+						chalk.red(
+							`No tests were run for ${ report.modules.join(
+								", "
+							) } in ${ getBrowserString( report.browser ) }`
+						)
+					);
+				}
+			}
+			if ( stop ) {
+				return gracefulExit( 1 );
+			}
+			console.log( chalk.green( "All tests passed!" ) );
+
+			if ( !debug || browserstack ) {
+				gracefulExit( 0 );
+			}
+		} else {
+			console.error( chalk.red( `${ errorMessages.length } tests failed.` ) );
+			console.log(
+				errorMessages.map( ( error, i ) => `\n${ i + 1 }. ${ error }` ).join( "\n" )
+			);
+
+			if ( debug ) {
+				console.log();
+				if ( browserstack ) {
+					console.log( "Leaving browsers with failures open for debugging." );
+					console.log(
+						"View running sessions at https://automate.browserstack.com/dashboard/v2/"
+					);
+				} else {
+					console.log( "Leaving browsers open for debugging." );
+				}
+				console.log( "Press Ctrl+C to exit." );
+			} else {
+				gracefulExit( 1 );
+			}
+		}
+	}
+}

test/runner/selenium/createDriver.js

@@ -0,0 +1,81 @@
+import { Builder, Capabilities, logging } from "selenium-webdriver";
+import Chrome from "selenium-webdriver/chrome.js";
+import Edge from "selenium-webdriver/edge.js";
+import Firefox from "selenium-webdriver/firefox.js";
+import { browserSupportsHeadless } from "../lib/getBrowserString.js";
+
+// Set script timeout to 10min
+const DRIVER_SCRIPT_TIMEOUT = 1000 * 60 * 10;
+
+export default async function createDriver( { browserName, headless, verbose } ) {
+	const capabilities = Capabilities[ browserName ]();
+	const prefs = new logging.Preferences();
+	prefs.setLevel( logging.Type.BROWSER, logging.Level.ALL );
+	capabilities.setLoggingPrefs( prefs );
+
+	let driver = new Builder().withCapabilities( capabilities );
+
+	const chromeOptions = new Chrome.Options();
+	chromeOptions.addArguments( "--enable-chrome-browser-cloud-management" );
+
+	// Alter the chrome binary path if
+	// the CHROME_BIN environment variable is set
+	if ( process.env.CHROME_BIN ) {
+		if ( verbose ) {
+			console.log( `Setting chrome binary to ${ process.env.CHROME_BIN }` );
+		}
+		chromeOptions.setChromeBinaryPath( process.env.CHROME_BIN );
+	}
+
+	const firefoxOptions = new Firefox.Options();
+
+	if ( process.env.FIREFOX_BIN ) {
+		if ( verbose ) {
+			console.log( `Setting firefox binary to ${ process.env.FIREFOX_BIN }` );
+		}
+
+		firefoxOptions.setBinary( process.env.FIREFOX_BIN );
+	}
+
+	const edgeOptions = new Edge.Options();
+	edgeOptions.addArguments( "--enable-chrome-browser-cloud-management" );
+
+	// Alter the edge binary path if
+	// the EDGE_BIN environment variable is set
+	if ( process.env.EDGE_BIN ) {
+		if ( verbose ) {
+			console.log( `Setting edge binary to ${ process.env.EDGE_BIN }` );
+		}
+		edgeOptions.setEdgeChromiumBinaryPath( process.env.EDGE_BIN );
+	}
+
+	if ( headless ) {
+		chromeOptions.addArguments( "--headless=new" );
+		firefoxOptions.addArguments( "--headless" );
+		edgeOptions.addArguments( "--headless=new" );
+		if ( !browserSupportsHeadless( browserName ) ) {
+			console.log(
+				`Headless mode is not supported for ${ browserName }.` +
+					"Running in normal mode instead."
+			);
+		}
+	}
+
+	driver = await driver
+		.setChromeOptions( chromeOptions )
+		.setFirefoxOptions( firefoxOptions )
+		.setEdgeOptions( edgeOptions )
+		.build();
+
+	if ( verbose ) {
+		const driverCapabilities = await driver.getCapabilities();
+		const name = driverCapabilities.getBrowserName();
+		const version = driverCapabilities.getBrowserVersion();
+		console.log( `\nDriver created for ${ name } ${ version }` );
+	}
+
+	// Increase script timeout to 10min
+	await driver.manage().setTimeouts( { script: DRIVER_SCRIPT_TIMEOUT } );
+
+	return driver;
+}

test/runner/selenium/queue.js

@@ -0,0 +1,70 @@
+// Build a queue that runs both browsers and modules
+// in parallel when the length reaches the concurrency limit
+// and refills the queue when one promise resolves.
+
+import chalk from "chalk";
+import { getBrowserString } from "../lib/getBrowserString.js";
+import { runSelenium } from "./runSelenium.js";
+import { runJSDOM } from "../jsdom.js";
+
+const promises = [];
+const queue = [];
+
+const SELENIUM_WAIT_TIME = 100;
+
+// Limit concurrency to 8 by default in selenium
+// BrowserStack defaults to the max allowed by the plan
+// More than this will log MaxListenersExceededWarning
+const MAX_CONCURRENCY = 8;
+
+export function addSeleniumRun( url, browser, options ) {
+	queue.push( { url, browser, options } );
+}
+
+export async function runAllSelenium( { concurrency = MAX_CONCURRENCY, verbose } ) {
+	while ( queue.length ) {
+		const next = queue.shift();
+		const { url, browser, options } = next;
+
+		const fullBrowser = getBrowserString( browser, options.headless );
+		console.log(
+			`\nRunning ${ chalk.yellow( options.modules.join( ", " ) ) } tests ` +
+				`in ${ chalk.yellow( fullBrowser ) } (${ chalk.bold( options.reportId ) })...`
+		);
+
+		// Wait enough time between requests
+		// to give concurrency a chance to update.
+		// In selenium, this helps avoid undici connect timeout errors.
+		await new Promise( ( resolve ) => setTimeout( resolve, SELENIUM_WAIT_TIME ) );
+
+		if ( verbose ) {
+			console.log( `\nTests remaining: ${ queue.length + 1 }.` );
+		}
+
+		let promise;
+		if ( browser.browser === "jsdom" ) {
+			promise = runJSDOM( url, options );
+		} else {
+			promise = runSelenium( url, browser, options );
+		}
+
+		// Remove the promise from the list when it resolves
+		promise.then( () => {
+			const index = promises.indexOf( promise );
+			if ( index !== -1 ) {
+				promises.splice( index, 1 );
+			}
+		} );
+
+		// Add the promise to the list
+		promises.push( promise );
+
+		// Wait until at least one promise resolves
+		// if we've reached the concurrency limit
+		if ( promises.length >= concurrency ) {
+			await Promise.any( promises );
+		}
+	}
+
+	await Promise.all( promises );
+}

test/runner/selenium/runSelenium.js

@@ -0,0 +1,30 @@
+import createDriver from "./createDriver.js";
+
+export async function runSelenium(
+	url,
+	{ browser },
+	{ debug, headless, verbose } = {}
+) {
+	if ( debug && headless ) {
+		throw new Error( "Cannot debug in headless mode." );
+	}
+
+	const driver = await createDriver( {
+		browserName: browser,
+		headless,
+		verbose
+	} );
+
+	try {
+		await driver.get( url );
+		await driver.executeScript(
+`return new Promise( ( resolve ) => {
+	QUnit.on( "runEnd", resolve );
+} )`
+		);
+	} finally {
+		if ( !debug || headless ) {
+			await driver.quit();
+		}
+	}
+}

test/runner/server.js

@@ -0,0 +1,13 @@
+import { createTestServer } from "./createTestServer.js";
+
+const port = process.env.PORT || 3000;
+
+async function runServer() {
+	const app = await createTestServer();
+
+	app.listen( { port, host: "0.0.0.0" }, function() {
+		console.log( `Open tests at http://localhost:${ port }/test/` );
+	} );
+}
+
+runServer();

test/server.js

@@ -1,21 +0,0 @@
-const express = require( "express" );
-const mockServer = require( "./middleware-mockserver" );
-const fs = require( "fs" );
-
-const nameHTML = fs.readFileSync( "./test/data/name.html", "utf8" );
-const app = express();
-
-app.use( mockServer() );
-
-app.post( "/test/data/name.html", function( _req, res ) {
-	res.send( nameHTML );
-} );
-
-app.use( "/dist", express.static( "dist" ) );
-app.use( "/src", express.static( "src" ) );
-app.use( "/test", express.static( "test" ) );
-app.use( "/external", express.static( "external" ) );
-
-app.listen( 3000, function() {
-  console.log( "Server is running on port 3000" );
-} );

test/unit/ajax.js

@@ -307,16 +307,7 @@ QUnit.module( "ajax", {
 					assert.strictEqual( xhr.getResponseHeader( "List-Header" ), "Item 1, Item 2", "List header received" );
 				}
 
-				if ( isAndroid && QUnit.isSwarm ) {
+				assert.strictEqual( xhr.getResponseHeader( "constructor" ), "prototype collision (constructor)", "constructor header received" );
-
-					// Support: Android 4.0-4.3 on BrowserStack only
-					// Android Browser versions provided by BrowserStack fail this test
-					// while locally fired emulators don't, even when they connect
-					// to TestSwarm. Just skip the test there to avoid a red build.
-					assert.ok( true, "BrowserStack's Android fails the \"prototype collision (constructor)\" test" );
-				} else {
-					assert.strictEqual( xhr.getResponseHeader( "constructor" ), "prototype collision (constructor)", "constructor header received" );
-				}
 				assert.strictEqual( xhr.getResponseHeader( "__proto__" ), null, "Undefined __proto__ header not received" );
 			}
 		};
@@ -1698,23 +1689,15 @@ QUnit.module( "ajax", {
 			jQuery.each(
 				{
 					"If-Modified-Since": {
-						url: "mock.php?action=ims",
+						url: "mock.php?action=ims"
-						qunitMethod: "test"
 					},
 					"Etag": {
-						url: "mock.php?action=etag",
+						url: "mock.php?action=etag"
-
-						// Support: TestSwarm
-						// TestSwarm is now proxied via Cloudflare which cuts out
-						// headers relevant for ETag tests, failing them. We're still
-						// running those tests in Karma on Chrome & Firefox (including
-						// Firefox ESR).
-						qunitMethod: QUnit.isSwarm ? "skip" : "test"
 					}
 				},
 				function( type, data ) {
 					var url = baseURL + data.url + "&ts=" + ifModifiedNow++;
-					QUnit[ data.qunitMethod ]( "jQuery.ajax() - " + type +
+					QUnit.test( "jQuery.ajax() - " + type +
 							" support" + label, function( assert ) {
 						assert.expect( 4 );
 						var done = assert.async();

test/unit/animation.js

@@ -13,10 +13,10 @@ var oldRaf = window.requestAnimationFrame,
 // This module tests jQuery.Animation and the corresponding 1.8+ effects APIs
 QUnit.module( "animation", {
 	beforeEach: function() {
-		window.requestAnimationFrame = null;
+		this.sandbox = sinon.createSandbox();
-		this.sandbox = sinon.sandbox.create();
 		this.clock = this.sandbox.useFakeTimers( startTime );
 		this._oldInterval = jQuery.fx.interval;
+		window.requestAnimationFrame = null;
 		jQuery.fx.step = {};
 		jQuery.fx.interval = 10;
 		jQuery.Animation.prefilters = [ defaultPrefilter ];
@@ -93,7 +93,7 @@ QUnit.test( "Animation.prefilter( fn ) - calls prefilter after defaultPrefilter"
 		assert.expect( 1 );
 
 		var prefilter = this.sandbox.stub(),
-			defaultSpy = this.sandbox.spy( jQuery.Animation.prefilters, 0 );
+			defaultSpy = this.sandbox.spy( jQuery.Animation.prefilters, "0" );
 
 		jQuery.Animation.prefilter( prefilter );
 
@@ -107,7 +107,7 @@ QUnit.test( "Animation.prefilter( fn, true ) - calls prefilter before defaultPre
 		assert.expect( 1 );
 
 		var prefilter = this.sandbox.stub(),
-			defaultSpy = this.sandbox.spy( jQuery.Animation.prefilters, 0 );
+			defaultSpy = this.sandbox.spy( jQuery.Animation.prefilters, "0" );
 
 		jQuery.Animation.prefilter( prefilter, true );
 
@@ -135,7 +135,7 @@ QUnit.test( "Animation.prefilter - prefilter return hooks", function( assert ) {
 			assert.equal( arguments[ 2 ], this.opts, "third param opts" );
 			return ourAnimation;
 		} ),
-		defaultSpy = sandbox.spy( jQuery.Animation.prefilters, 0 ),
+		defaultSpy = sandbox.spy( jQuery.Animation.prefilters, "0" ),
 		queueSpy = sandbox.spy( function( next ) {
 			next();
 		} ),
@@ -168,7 +168,7 @@ QUnit.test( "Animation.prefilter - prefilter return hooks", function( assert ) {
 	assert.equal( TweenSpy.callCount, 0, "Returning something never creates tweens" );
 
 	// Test overridden usage on queues:
-	prefilter.reset();
+	prefilter.resetHistory();
 	element = jQuery( "<div>" )
 		.css( "height", 50 )
 		.animate( props, 100 )
@@ -200,7 +200,7 @@ QUnit.test( "Animation.prefilter - prefilter return hooks", function( assert ) {
 	// ourAnimation.stop.reset();
 	assert.equal( prefilter.callCount, 3, "Got the next animation" );
 
-	ourAnimation.stop.reset();
+	ourAnimation.stop.resetHistory();
 
 	// do not clear queue, gotoEnd
 	element.stop( false, true );

test/unit/core.js

@@ -1,6 +1,6 @@
 QUnit.module( "core", {
 	beforeEach: function() {
-		this.sandbox = sinon.sandbox.create();
+		this.sandbox = sinon.createSandbox();
 	},
 	afterEach: function() {
 		this.sandbox.restore();
@@ -1563,7 +1563,7 @@ QUnit[ includesModule( "deferred" ) ? "test" : "skip" ]( "jQuery.readyException
 
 	var message;
 
-	this.sandbox.stub( window, "setTimeout", function( fn ) {
+	this.sandbox.stub( window, "setTimeout" ).callsFake( function( fn ) {
 		try {
 			fn();
 		} catch ( error ) {
@@ -1586,7 +1586,7 @@ QUnit[ includesModule( "deferred" ) ? "test" : "skip" ]( "jQuery.readyException
 
 	var done = assert.async();
 
-	this.sandbox.stub( jQuery, "readyException", function( error ) {
+	this.sandbox.stub( jQuery, "readyException" ).callsFake( function( error ) {
 		assert.strictEqual(
 			error.message,
 			"Error in jQuery ready",

test/unit/effects.js

@@ -17,10 +17,10 @@ var oldRaf = window.requestAnimationFrame,
 
 QUnit.module( "effects", {
 	beforeEach: function() {
-		window.requestAnimationFrame = null;
+		this.sandbox = sinon.createSandbox();
-		this.sandbox = sinon.sandbox.create();
 		this.clock = this.sandbox.useFakeTimers( 505877050 );
 		this._oldInterval = jQuery.fx.interval;
+		window.requestAnimationFrame = null;
 		jQuery.fx.step = {};
 		jQuery.fx.interval = 10;
 	},
@@ -690,15 +690,8 @@ QUnit.test( "stop()", function( assert ) {
 	this.clock.tick( 100 );
 } );
 
-// In IE9 inside testswarm this test doesn't work properly
 ( function() {
-	var type = "test";
+	QUnit.test( "stop() - several in queue", function( assert ) {
-
-	if ( QUnit.isSwarm && /msie 9\.0/i.test( window.navigator.userAgent ) ) {
-		type = "skip";
-	}
-
-	QUnit[ type ]( "stop() - several in queue", function( assert ) {
 		assert.expect( 5 );
 
 		var nw, $foo = jQuery( "#foo" );

test/unit/event.js

@@ -2224,7 +2224,7 @@ QUnit.test( "window resize", function( assert ) {
 } );
 
 QUnit.test( "focusin bubbles", function( assert ) {
-	assert.expect( 2 );
+	assert.expect( 3 );
 
 	var input = jQuery( "<input type='text' />" ).prependTo( "body" ),
 		order = 0;
@@ -2240,14 +2240,13 @@ QUnit.test( "focusin bubbles", function( assert ) {
 		assert.equal( 0, order++, "focusin on the element first" );
 	} );
 
-// Removed since DOM focus is unreliable on test swarm
 	// DOM focus method
-//	input[ 0 ].focus();
+	input[ 0 ].focus();
 
 	// To make the next focus test work, we need to take focus off the input.
 	// This will fire another focusin event, so set order to reflect that.
-//	order = 1;
+	order = 1;
-//	jQuery( "#text1" )[ 0 ].focus();
+	jQuery( "#text1" )[ 0 ].focus();
 
 	// jQuery trigger, which calls DOM focus
 	order = 0;
@@ -2274,13 +2273,12 @@ QUnit.test( "focus does not bubble", function( assert ) {
 		assert.ok( true, "focus on the element" );
 	} );
 
-// Removed since DOM focus is unreliable on test swarm
 	// DOM focus method
-//	input[ 0 ].focus();
+	input[ 0 ].focus();
 
 	// To make the next focus test work, we need to take focus off the input.
 	// This will fire another focusin event, so set order to reflect that.
-//	jQuery( "#text1" )[ 0 ].focus();
+	jQuery( "#text1" )[ 0 ].focus();
 
 	// jQuery trigger, which calls DOM focus
 	input.trigger( "focus" );
@@ -2695,15 +2693,7 @@ testIframe(
 		// Remove body handler manually since it's outside the fixture
 		jQuery( "body" ).off( "focusin.iframeTest" );
 
-		setTimeout( function() {
+		setTimeout( done, 50 );
-
-			// DOM focus is unreliable in TestSwarm
-			if ( QUnit.isSwarm && !focus ) {
-				assert.ok( true, "GAP: Could not observe focus change" );
-			}
-
-			done();
-		}, 50 );
 	}
 );
 
@@ -2726,11 +2716,6 @@ QUnit.test( "focusin on document & window", function( assert ) {
 
 	input[ 0 ].blur();
 
-	// DOM focus is unreliable in TestSwarm
-	if ( QUnit.isSwarm && counter === 0 ) {
-		assert.ok( true, "GAP: Could not observe focus change" );
-	}
-
 	assert.strictEqual( counter, 2,
 		"focusout handlers on document/window fired once only" );
 
@@ -3023,20 +3008,8 @@ QUnit.test( "preventDefault() on focusin does not throw exception", function( as
 			"Preventing default on focusin throws no exception" );
 
 		done();
-		done = null;
 	} );
 	input.trigger( "focus" );
-
-	// DOM focus is unreliable in TestSwarm; set a simulated event workaround timeout
-	setTimeout( function() {
-		if ( !done ) {
-			return;
-		}
-		input[ 0 ].addEventListener( "click", function( nativeEvent ) {
-			jQuery.event.simulate( "focusin", this, jQuery.event.fix( nativeEvent ) );
-		} );
-		input[ 0 ].click();
-	}, QUnit.config.testTimeout / 4 || 1000 );
 } );
 
 QUnit.test( ".on('focus', fn) on a text node doesn't throw", function( assert ) {
@@ -3218,12 +3191,6 @@ QUnit.test( "Check order of focusin/focusout events", function( assert ) {
 
 	// cleanup
 	input.off();
-
-	// DOM focus is unreliable in TestSwarm
-	if ( !focus ) {
-		assert.ok( true, "GAP: Could not observe focus change" );
-		assert.ok( true, "GAP: Could not observe focus change" );
-	}
 } );
 
 QUnit.test( "focus-blur order (trac-12868)", function( assert ) {
@@ -3255,12 +3222,6 @@ QUnit.test( "focus-blur order (trac-12868)", function( assert ) {
 	assert.equal( document.activeElement, $radio[ 0 ], "radio has focus" );
 	$text.trigger( "focus" );
 
-	// DOM focus is unreliable in TestSwarm
-	if ( order === 0 ) {
-		assert.ok( true, "GAP: Could not observe focus change" );
-		assert.ok( true, "GAP: Could not observe focus change" );
-	}
-
 	assert.equal( document.activeElement, $text[ 0 ], "text has focus" );
 
 	// Run handlers without native method on an input
@@ -3308,18 +3269,6 @@ QUnit.test( "Event handling works with multiple async focus events (gh-4350)", f
 
 	// gain focus
 	input.trigger( "focus" );
-
-	// DOM focus is unreliable in TestSwarm
-	setTimeout( function() {
-		if ( QUnit.isSwarm && remaining === 3 ) {
-			assert.ok( true, "GAP: Could not observe focus change" );
-			assert.ok( true, "GAP: Could not observe focus change" );
-			assert.ok( true, "GAP: Could not observe focus change" );
-			setTimeout( function() {
-				done();
-			} );
-		}
-	} );
 } );
 
 // Support: IE <=9 - 11+

test/unit/selector.js

@@ -573,7 +573,7 @@ QUnit.test( "attributes - existence", function( assert ) {
 	assert.t( "On any element", "#qunit-fixture *[title]", [ "google" ] );
 	assert.t( "On implicit element", "#qunit-fixture [title]", [ "google" ] );
 	assert.t( "Boolean", "#select2 option[selected]", [ "option2d" ] );
-	assert.t( "For attribute on label", "form label[for]", [ "label-for" ] );
+	assert.t( "For attribute on label", "#qunit-fixture form label[for]", [ "label-for" ] );
 } );
 
 QUnit.test( "attributes - equals", function( assert ) {

test/unit/support.js

@@ -238,30 +238,6 @@ testIframe(
 				sortDetached: true,
 				sortStable: true
 			},
-			webkit: {
-				ajax: true,
-				boxSizingReliable: true,
-				checkClone: true,
-				checkOn: true,
-				clearCloneStyle: true,
-				cssHas: true,
-				cors: true,
-				createHTMLDocument: true,
-				disconnectedMatch: true,
-				getById: true,
-				noCloneChecked: true,
-				option: true,
-				optSelected: true,
-				pixelBoxStyles: true,
-				pixelPosition: true,
-				radioValue: true,
-				reliableMarginLeft: true,
-				reliableTrDimensions: true,
-				scope: true,
-				scrollboxSize: true,
-				sortDetached: true,
-				sortStable: true
-			},
 			firefox_60: {
 				ajax: true,
 				boxSizingReliable: true,
@@ -531,18 +507,6 @@ testIframe(
 		expected = expectedMap.ios_15_4_16_3;
 	} else if ( /\b(?:iphone|ipad);.*(?:iphone)? os \d+_/i.test( userAgent ) ) {
 		expected = expectedMap.ios;
-	} else if ( typeof URLSearchParams !== "undefined" &&
-
-		// `karma-webkit-launcher` adds `test_browser=Playwright` to the query string.
-		// The normal way of using user agent to detect the browser won't help
-		// as on macOS Playwright doesn't specify the `Safari` token but on Linux
-		// it does.
-		// See https://github.com/google/karma-webkit-launcher#detected-if-safari-or-playwright-is-used
-		new URLSearchParams( document.referrer || window.location.search ).get(
-			"test_browser"
-		) === "Playwright"
-	) {
-		expected = expectedMap.webkit;
 	} else if ( /\bversion\/(?:15|16\.[0123])(?:\.\d+)* safari/i.test( userAgent ) ) {
 		expected = expectedMap.safari_16_3;
 	} else if ( /\bversion\/\d+(?:\.\d+)+ safari/i.test( userAgent ) ) {

test/unit/tween.js

@@ -10,7 +10,7 @@ var oldRaf = window.requestAnimationFrame;
 QUnit.module( "tween", {
 	beforeEach: function() {
 		window.requestAnimationFrame = null;
-		this.sandbox = sinon.sandbox.create();
+		this.sandbox = sinon.createSandbox();
 		this.clock = this.sandbox.useFakeTimers( 505877050 );
 		this._oldInterval = jQuery.fx.interval;
 		jQuery.fx.step = {};