This is a complete rework of our testing infrastructure. The main goal is to modernize and drop deprecated or undermaintained dependencies (specifically, grunt, karma, and testswarm). We've achieved that by limiting our dependency list to ones that are unlikely to drop support any time soon. The new dependency list includes:
- `qunit` (our trusty unit testing library)
- `selenium-webdriver` (for spinning up local browsers)
- `express` (for starting a test server and adding middleware)
- express middleware includes uses of `body-parser` and `raw-body`
- `yargs` (for constructing a CLI with pretty help text)
- BrowserStack (for running each of our QUnit modules separately in all of our supported browsers)
- `browserstack-local` (for opening a local tunnel. This is the same package still currently used in the new Browserstack SDK)
- We are not using any other BrowserStack library. The newest BrowserStack SDK does not fit our needs (and isn't open source). Existing libraries, such as `node-browserstack` or `browserstack-runner`, either do not quite fit our needs, are under-maintained and out-of-date, or are not robust enough to meet all of our requirements. We instead call the [BrowserStack REST API](https://github.com/browserstack/api) directly.
**BrowserStack**
- automatically retries individual modules in case of test failure(s)
- automatically attempts to re-establish broken tunnels
- automatically refreshes the page in case a test run has stalled
- Browser workers are reused when running isolated modules in the same browser
- runs all browsers concurrently and uses as many sessions as are available under the BrowserStack plan. It will wait for available sessions if there are none.
- supports filtering the available list of browsers by browser name, browser version, device, OS, and OS version (see `npm run test:unit -- --list-browsers` for more info). It will retrieve the latest matching browser available if any of those parameters are not specified. Supports latest and latest-\d+ in place of browser version.
- cleans up after itself (closes the local tunnel, stops the test server, etc.)
- Requires `BROWSERSTACK_USERNAME` and `BROWSERSTACK_ACCESS_KEY` environment variables.
**Selenium**
- supports running any local browser as long as the driver is installed, including support for headless mode in Chrome, FF, and Edge
- supports running `basic` tests on the latest [jsdom](https://github.com/jsdom/jsdom#readme), which can be seen in action in this PR (see `test:browserless`)
- Node tests will run as before in PRs and all non-dependabot branches, but now includes tests on real Safari in a GH actions macos image instead of playwright-webkit.
- can run multiple browsers and multiple modules concurrently
Other notes:
- Stale dependencies have been removed and all remaining dependencies have been upgraded with a few exceptions:
- `sinon`: stopped supporting IE in version 10. But, `sinon` has been updated to 9.x.
- `husky`: latest does not support Node 10 and runs on `npm install`. Needed for now until git builds are migrated to GitHub Actions.
- `rollup`: latest does not support Node 10. Needed for now until git builds are migrated to GitHub Actions.
- BrowserStack tests are set to run on each `main` branch commit
- `debug` mode leaves Selenium browsers open whether they pass or fail and leaves browsers with test failures open on BrowserStack. The latter is to avoid leaving open too many sessions.
- This PR includes a workflow to dispatch BrowserStack runs on-demand
- The Node version used for most workflow tests has been upgraded to 20.x
- updated supportjQuery to 3.7.1
Run `npm run test:unit -- --help` for CLI documentation
Close gh-5427
Remove a redundant Express-targeted `resp.set` call from the `script` handler
in `middleware-mockserver.js` as was done on the `main` branch.
Closes gh-5413
Ref gh-5397
Close gh-5330
- lint
- npmcopy
- build, minify, and process for distribution.
- new custom build command using yargs
- compare size of minified/gzip built files
- pretest scripts, including qunit-fixture, babel transpilation, and npmcopy
- node smoke tests
- promises aplus tests
- new watch task using nodemon, which runs `npm run build:all` on `src` changes.
Also:
- upgraded husky and added the new lint command
- updated lint config to use new "flat" config format.
See https://eslint.org/docs/latest/use/configure/configuration-files-new
- Temporarily disabled one lint rule until flat config is
supported by eslint-plugin-import.
See https://github.com/import-js/eslint-plugin-import/issues/2556
- committed package-lock.json
- updated all test scripts to use the new build
- added an express test server that uses middleware-mockserver
this can be used to run tests without karma
- build-all-variants is now build:all
- run pretest script in jenkins
---------
Co-authored-by: Michał Gołębiowski-Owczarek <m.goleb@gmail.com>
The `test/middleware-mockserver.js` file used to have the same ESLint
settings applied as other test files that are directly run in tested
browsers. Now it shares settings of other Node.js files.
The file is now also written using modern JS, leveraging ES2018.
Closes gh-5196
(cherry picked from commit ce90a48450)
Only allow alphanumeric characters & underscores for callback parameters.
The change is done both for the PHP server as well as the Node.js-based version.
This is only test code so we're not fixing any security issue but it happens
often enough that the whole jQuery repository directory structure is deployed
onto the server with PHP enabled that it makes is easy to introduce security
issues if this cleanup is not done.
Ref gh-4764
Closes gh-4871
(cherry picked from a70274632d)
Two issues are fixed in testing for responses with a script Content-Type not
getting auto-executed unless an explicit `dataType: "script"` is provided:
* the test is now using a correct "text/javascript" Content-Type; it was using
"text/html" until now which doesn't really check if the fix works
* the Node.js based version of the tests didn't account for an empty `header`
query string parameter
Closes gh-4824
Ref gh-2432
Ref gh-2588
Ref 39cdb8c9aa
(cherry picked from commit d38528b17a)
This aligns the Node.js server with the previous PHP one in sending `mock.php`
as a callback if there's no `callback` parameter in the query string which is
triggered by a recently added test. This prevents the request crashing on that
Node.js server and printing a JS error:
```
TypeError: Cannot read property '1' of null
```
Closes gh-4764
Ref gh-4754
(cherry picked from commit df6858df2e)
The "jQuery.ajax() - JSONP - Same Domain" test is firing a request with
a duplicate "callback" parameter, something like (simplified):
```
mock.php?action=jsonp&callback=jQuery_1&callback=jQuery_2
```
There was a difference in how the PHP & Node.js implementations of the jsonp
action in the mock server handled situations like that. The PHP implementation
was using the latest parameter while the Node.js one was turning it into an
array but the code didn't handle this situation. Because of how JavaScript
stringifies arrays, while the PHP implementation injected the following code:
```js
jQuery_2(payload)
```
the Node.js one was injecting the following one:
```js
jQuery_1,jQuery_2(payload)
```
This is a comma expression in JavaScript; it so turned out that in the majority
of cases both callbacks were identical so it was more like:
```js
jQuery_1,jQuery_1(payload)
```
which evaluates to `jQuery_1(payload)` when `jQuery_1` is defined, making the
test go as expected. In many cases, though, especially on Travis, the callbacks
were different, triggering an `Uncaught ReferenceError` error & requiring
frequent manual re-runs of Travis builds.
This commit fixes the logic in the mock Node.js server, adding special handling
for arrays.
Closes gh-4687
(cherry picked from commit 7b0864d053)
The script transport used to evaluate fetched script sources which is
undesirable for unsuccessful HTTP responses. This is different to other data
types where such a convention was fine (e.g. in case of JSON).
(cherry picked from 50871a5a85)
Fixes gh-4250
Fixes gh-4655
Closes gh-4379
Also, run `grunt npmcopy` to sync the "external" directory with dependencies
from package.json. For example, the Sinon library version didn't match.
Ref gh-4234
Closes gh-4297
- getResponseHeader(key) combines all header values for the provided key into a
single result where values are concatenated by ', '. This does not happen for
IE11 since multiple values for the same header are returned on separate lines.
This makes the function only return the last value of the header for IE11.
- Updated ajax headers test to better cover Object.prototype collisions
Close gh-4173
Fixes gh-3403
- Update QUnit to 1.23.1
- Remove unused dl#dl from test/index.html
- Remove unused map#imgmap from test/index.html
- Ensure all urls to data use baseURI
- Add the 'grunt karma:main' task
- customContextFile & customDebugFile
- Add 'npm run jenkins' script
Close gh-3744
Fixes gh-1999
