This is a complete rework of our testing infrastructure. The main goal is to modernize and drop deprecated or undermaintained dependencies (specifically, grunt, karma, and testswarm). We've achieved that by limiting our dependency list to ones that are unlikely to drop support any time soon. The new dependency list includes:
- `qunit` (our trusty unit testing library)
- `selenium-webdriver` (for spinning up local browsers)
- `express` (for starting a test server and adding middleware)
- express middleware includes uses of `body-parser` and `raw-body`
- `yargs` (for constructing a CLI with pretty help text)
- BrowserStack (for running each of our QUnit modules separately in all of our supported browsers)
- `browserstack-local` (for opening a local tunnel. This is the same package still currently used in the new Browserstack SDK)
- We are not using any other BrowserStack library. The newest BrowserStack SDK does not fit our needs (and isn't open source). Existing libraries, such as `node-browserstack` or `browserstack-runner`, either do not quite fit our needs, are under-maintained and out-of-date, or are not robust enough to meet all of our requirements. We instead call the [BrowserStack REST API](https://github.com/browserstack/api) directly.
**BrowserStack**
- automatically retries individual modules in case of test failure(s)
- automatically attempts to re-establish broken tunnels
- automatically refreshes the page in case a test run has stalled
- Browser workers are reused when running isolated modules in the same browser
- runs all browsers concurrently and uses as many sessions as are available under the BrowserStack plan. It will wait for available sessions if there are none.
- supports filtering the available list of browsers by browser name, browser version, device, OS, and OS version (see `npm run test:unit -- --list-browsers` for more info). It will retrieve the latest matching browser available if any of those parameters are not specified. Supports latest and latest-\d+ in place of browser version.
- cleans up after itself (closes the local tunnel, stops the test server, etc.)
- Requires `BROWSERSTACK_USERNAME` and `BROWSERSTACK_ACCESS_KEY` environment variables.
**Selenium**
- supports running any local browser as long as the driver is installed, including support for headless mode in Chrome, FF, and Edge
- supports running `basic` tests on the latest [jsdom](https://github.com/jsdom/jsdom#readme), which can be seen in action in this PR (see `test:browserless`)
- Node tests will run as before in PRs and all non-dependabot branches, but now includes tests on real Safari in a GH actions macos image instead of playwright-webkit.
- can run multiple browsers and multiple modules concurrently
Other notes:
- Stale dependencies have been removed and all remaining dependencies have been upgraded with a few exceptions:
- `sinon`: stopped supporting IE in version 10. But, `sinon` has been updated to 9.x.
- `husky`: latest does not support Node 10 and runs on `npm install`. Needed for now until git builds are migrated to GitHub Actions.
- `rollup`: latest does not support Node 10. Needed for now until git builds are migrated to GitHub Actions.
- BrowserStack tests are set to run on each `main` branch commit
- `debug` mode leaves Selenium browsers open whether they pass or fail and leaves browsers with test failures open on BrowserStack. The latter is to avoid leaving open too many sessions.
- This PR includes a workflow to dispatch BrowserStack runs on-demand
- The Node version used for most workflow tests has been upgraded to 20.x
- updated supportjQuery to 3.7.1
Run `npm run test:unit -- --help` for CLI documentation
Close gh-5427
Build was already happening in scripts like `test:browser` but those scripts
were missing `pretest`, meaning that running `npm install && npm test:browser`
may have failed if `pretest` wasn't run before or if its results were out of
date.
Even worse, with such stale data some tests may erroneously succeed.
This also removes a separate `pretest` step from GitHub Actions as it's no
longer needed.
Closes gh-5338
(cherry picked from commit 1ad66aeb6d)
Close gh-5330
- lint
- npmcopy
- build, minify, and process for distribution.
- new custom build command using yargs
- compare size of minified/gzip built files
- pretest scripts, including qunit-fixture, babel transpilation, and npmcopy
- node smoke tests
- promises aplus tests
- new watch task using nodemon, which runs `npm run build:all` on `src` changes.
Also:
- upgraded husky and added the new lint command
- updated lint config to use new "flat" config format.
See https://eslint.org/docs/latest/use/configure/configuration-files-new
- Temporarily disabled one lint rule until flat config is
supported by eslint-plugin-import.
See https://github.com/import-js/eslint-plugin-import/issues/2556
- committed package-lock.json
- updated all test scripts to use the new build
- added an express test server that uses middleware-mockserver
this can be used to run tests without karma
- build-all-variants is now build:all
- run pretest script in jenkins
---------
Co-authored-by: Michał Gołębiowski-Owczarek <m.goleb@gmail.com>
Summary of the changes:
* expand `node_smoke_tests` to test the full & slim builds
* run `compare_size` on all built minified files; don't run it anymore on
unminified files where they don't provide lots of value
The main goal of this change is to make it easier to compare sizes of both the
full & slim builds between the `3.x-stable` & `main` branches.
Closes gh-5291
Ref gh-5255
(partially cherry-picked from commit 8be4c0e4f8)
Bootstrap 5 includes the following CSS on the page:
```css
*,
*::before,
*::after {
box-sizing: border-box;
}
```
That threw our `reliableTrDimensions` support test off. This change fixes the
support test and adds a unit test ensuring support test values on a page
including Bootstrap 5 CSS are the same as on a page without it.
Fixes gh-5270
Closes gh-5279
Ref gh-5278
`CSS.supports( "selector(...)" )` has different semantics than selectors passed
to `querySelectorAll`. Apart from the fact that the former returns `false` for
unrecognized selectors and the latter throws, `qSA` is more forgiving and
accepts some invalid selectors, auto-correcting them where needed - for
example, mismatched brackers are auto-closed. This behavior difference is
breaking for many users.
To add to that, a recent CSSWG resolution made `:is()` & `:where()` the only
pseudos with forgiving parsing; browsers are in the process of making `:has()`
parsing unforgiving.
Taking all that into account, we go back to our previous try-catch approach
without relying on `CSS.supports( "selector(...)" )`. The only difference
is we detect forgiving parsing in `:has()` and mark the selector as buggy.
The PR also updates `playwright-webkit` so that we test against a version
of WebKit that already has non-forgiving `:has()`.
Fixes gh-5194
Closes gh-5207
Ref gh-5206
Ref gh-5098
Ref gh-5107
Ref w3c/csswg-drafts#7676
So far, we've been running browser tests on GitHub Actions in Chrome
and Firefox. Regular Safari is not available in GitHub Actions but
Playwright WebKit comes close to a dev version of Safari.
With this change, our GitHub CI & local test runs will invoke tests on
all actively developed browser engines on all PRs.
Also, our GitHub Actions browser tests are now running on Node.js 18.
Detection of the Playwright WebKit browser in support unit tests is done
by checking if the `test_browser` query parameter is set to `"Playwright"`;
this is a `karma-webkit-launcher` feature. Detecting that browser via
user agent as we normally do is hard as the UA on Linux is very similar
to a real Safari one but it actually uses a newer version of the engine.
In addition, we now allow to pass custom browsers when one needs it;
e.g., to run the tests in all three engines on Linux/macOS, run:
```
grunt && BROWSERS=ChromeHeadless,FirefoxHeadless,WebkitHeadless grunt karma:main
```
Closes gh-5190
(cherry picked from commit b02a257f98)
This commit removes Sizzle from jQuery, inlining its code & removing obsolete
workarounds where applicable.
The Sizzle AUTHORS.txt file has been merged with the jQuery one - people are
sorted by their first contributions to either of the two repositories.
The main `selector` module can be disabled in favor of `selector-native`
via:
grunt custom:-selector
For backwards compatibility, the legacy `sizzle` alias is also supported (it
will be dropped in jQuery `4.0.0`):
grunt custom:-selector
Sizzle tests have been ported to jQuery ones. Ones that are not compatible
with the `selector-native` module are disabled if the regular selector module
is excluded.
Backwards compatibility is still kept for all `Sizzle` utils - they continue to be
available under `jQuery.find` - but the primary implementation is now attached
directly to jQuery.
Some selector utils shared by `selector` & `selector-native` have been
extracted & deduplicated. `jQuery.text` and `jQuery.isXMLDoc` have been
moved to the `core` module.
The commit reduces the gzipped jQuery size by 851 bytes compared to the
`3.x-stable` branch.
Closes gh-5113
Ref gh-4395
Ref gh-4406
This adds testing on Node.js 17 in addition to the currently tested 10, 12, 14
and 16 versions.
Also, update Grunt & `karma-*` packages.
Testing in Karma on jsdom is broken in Node 17 at the moment; until we find
a fix, this change disables such testing on Node 17 or newer.
Node smoke tests & promises aplus tests are disabled on Node.js 10 as they
depend on jsdom and the latest jsdom version doesn't run properly on Node 10.
Closes gh-5023
(cherry picked from commit 2525cffc42)
The `grunt-karma` version we used required Karma 3.x and it's now enforced
in npm 7. Update a few Karma & Grunt-related packages to resolve the issue.
Closes gh-4810
This commit fixes unit tests for the following builds:
1. The no-deprecated build: `custom:-deprecated`
2. The current slim build: `custom:-ajax,-effects`
3. The 4.0 (#4553) slim build: `custom:-ajax,-callbacks,-deferred,-effects`
It also adds separate Travis jobs for the no-deprecated & slim builds.
Apart from that, add intuitive names to Travis jobs. Otherwise it's hard to see
at a glance that a particular job is running on Firefox ESR, for example.
Ref gh-4577
Ref gh-4596
Closes gh-4600
While we have absolutely no style-related expectations to our minified file,
we do care that it's valid ES 5.1. This is now verified.
Also, update grunt-eslint as a newer ESLint version is required to be able
to use "extends" inside of the "overrides" section.
Fixes gh-3075
Closes gh-4594
Ref gh-4598
Also, run such a suite in CI to make sure modules are working as expected
when used directly.
(partially cherry picked from 341c6d1b5a)
(partially cherry picked from 437f389a24)
Closes gh-4595
Ref gh-4550
Ref gh-4574
This commit requires all function parameters to be used, not just the last one.
In cases where that's not possible as we need to match an external API, there's
an escape hatch of prefixing an unused argument with `_`.
This change makes it easier to catch unused AMD dependencies and unused
parameters in internal functions the API of which we may change at will, among
other things.
Unused AMD dependencies have been removed as part of this commit.
Closes gh-4381
(cherry-picked from 438b1a3e8a)
So far, we've been testing that jQuery element iteration works with polyfilled
Symbol & transpiled for-of via a Node test with jsdom with the Symbol global
removed. Unfortunately, jsdom now requires Symbol to be present for its internal
functionality so such a test is no longer possible. Instead, it's been migrated
to an iframe test with transpiled JavaScript.
This PR also enables us to use ECMAScript 2017 or newer in Node.js code.
Closes gh-4305
The only packages not fully updated are:
- qunitjs & karma-qunit as that's a QUnit 2.x update that will require some
changes and we'll do that later
- jsdom as we need to first rewrite the test with the Symbol polyfill - newer
jsdom versions don't work with such a hacked Symbol instance
- sinon as the v2 -> v7 upgrade requires to update our unit tests
- uglify-js & grunt-contrib-uglify as latest uglify-js versions slightly worsen
the minified gzipped size
Closes gh-4227
Closes gh-4228
Closes gh-4230
Closes gh-4232
