This is a complete rework of our testing infrastructure. The main goal is to modernize and drop deprecated or undermaintained dependencies (specifically, grunt, karma, and testswarm). We've achieved that by limiting our dependency list to ones that are unlikely to drop support any time soon. The new dependency list includes:
- `qunit` (our trusty unit testing library)
- `selenium-webdriver` (for spinning up local browsers)
- `express` (for starting a test server and adding middleware)
- express middleware includes uses of `body-parser` and `raw-body`
- `yargs` (for constructing a CLI with pretty help text)
- BrowserStack (for running each of our QUnit modules separately in all of our supported browsers)
- `browserstack-local` (for opening a local tunnel. This is the same package still currently used in the new Browserstack SDK)
- We are not using any other BrowserStack library. The newest BrowserStack SDK does not fit our needs (and isn't open source). Existing libraries, such as `node-browserstack` or `browserstack-runner`, either do not quite fit our needs, are under-maintained and out-of-date, or are not robust enough to meet all of our requirements. We instead call the [BrowserStack REST API](https://github.com/browserstack/api) directly.
## BrowserStack Runner
- automatically retries individual modules in case of test failure(s)
- automatically attempts to re-establish broken tunnels
- automatically refreshes the page in case a test run has stalled
- runs all browsers concurrently and uses as many sessions as are available under the BrowserStack plan. It will wait for available sessions if there are none.
- supports filtering the available list of browsers by browser name, browser version, device, OS, and OS version (see `npm run test:unit -- --list-browsers` for more info). It will retrieve the latest matching browser available if any of those parameters are not specified.
- cleans up after itself (closes the local tunnel, stops the test server, etc.)
- Requires `BROWSERSTACK_USERNAME` and `BROWSERSTACK_ACCESS_KEY` environment variables.
## Selenium Runner
- supports running any local browser as long as the driver is installed, including support for headless mode in Chrome, FF, and Edge
- supports running `basic` tests on the latest [jsdom](https://github.com/jsdom/jsdom#readme), which can be seen in action in this PR (see `test:browserless`)
- Node tests will run as before in PRs and all non-dependabot branches, but now includes tests on real Safari in a GH actions macos image instead of playwright-webkit.
- can run multiple browsers and multiple modules concurrently
Other notes:
- Stale dependencies have been removed and all remaining dependencies have been upgraded with a few exceptions:
- `sinon`: stopped supporting IE in version 10. But, `sinon` has been updated to 9.x.
- `husky`: latest does not support Node 10 and runs on `npm install`. Needed for now until git builds are migrated to GitHub Actions.
- `rollup`: latest does not support Node 10. Needed for now until git builds are migrated to GitHub Actions.
- BrowserStack tests are set to run on each `main` branch commit
- `debug` mode leaves Selenium browsers open whether they pass or fail and leaves browsers with test failures open on BrowserStack. The latter is to avoid leaving open too many sessions.
- This PR includes a workflow to dispatch BrowserStack runs on-demand
- The Node version used for most workflow tests has been upgraded to 20.x
- updated supportjQuery to 3.7.1
Run `npm run test:unit -- --help` for CLI documentation
Close gh-5418
Bootstrap 5 includes the following CSS on the page:
```css
*,
*::before,
*::after {
box-sizing: border-box;
}
```
That threw our `reliableTrDimensions` support test off. This change fixes the
support test and adds a unit test ensuring support test values on a page
including Bootstrap 5 CSS are the same as on a page without it.
Fixes gh-5270
Closes gh-5278
Ref gh-5279
Chrome 112 & Safari 16.4 introduce two changes:
* `:has()` is non-forgiving
* `CSS.supports( "selector(...)" )` parses everything in a non-forgiving way
We no longer care about the latter but the former means the `cssHas` support
test now passes.
Closes gh-5225
`CSS.supports( "selector(...)" )` has different semantics than selectors passed
to `querySelectorAll`. Apart from the fact that the former returns `false` for
unrecognized selectors and the latter throws, `qSA` is more forgiving and
accepts some invalid selectors, auto-correcting them where needed - for
example, mismatched brackers are auto-closed. This behavior difference is
breaking for many users.
To add to that, a recent CSSWG resolution made `:is()` & `:where()` the only
pseudos with forgiving parsing; browsers are in the process of making `:has()`
parsing unforgiving.
Taking all that into account, we go back to our previous try-catch approach
without relying on `CSS.supports( "selector(...)" )`. The only difference
is we detect forgiving parsing in `:has()` and mark the selector as buggy.
The PR also updates `playwright-webkit` so that we test against a version
of WebKit that already has non-forgiving `:has()`.
Fixes gh-5194
Closes gh-5206
Ref gh-5098
Ref gh-5107
Ref w3c/csswg-drafts#7676
Co-authored-by: Richard Gibson <richard.gibson@gmail.com>
So far, we've been running browser tests on GitHub Actions in Chrome
and Firefox. Regular Safari is not available in GitHub Actions but
Playwright WebKit comes close to a dev version of Safari.
With this change, our GitHub CI & local test runs will invoke tests on
all actively developed browser engines on all PRs.
Also, our GitHub Actions browser tests are now running on Node.js 18.
Detection of the Playwright WebKit browser in support unit tests is done
by checking if the `test_browser` query parameter is set to `"Playwright"`;
this is a `karma-webkit-launcher` feature. Detecting that browser via
user agent as we normally do is hard as the UA on Linux is very similar
to a real Safari one but it actually uses a newer version of the engine.
In addition, we now allow to pass custom browsers when one needs it;
e.g., to run the tests in all three engines on Linux/macOS, run:
```
grunt && BROWSERS=ChromeHeadless,FirefoxHeadless,WebkitHeadless grunt karma:main
```
Closes gh-5190
Firefox 106 adjusted to the spec mandating that `CSS.supports("selector(...)")`
uses non-forgiving parsing which makes it pass the relevant support test.
Closes gh-5141
Re-introduce the `selector-native` similar to the one on the `3.x-stable`
branch. One difference is since the `main` branch inlined Sizzle, some
selector utils can be shared between the main `selector` module and
`selector-native`.
The main `selector` module can be disabled in favor of `selector-native`
via:
grunt custom:-selector
Other changes:
* Tests: Fix Safari detection - Chrome Headless has a different user
agent than Safari and a browser check in selector tests didn't take
that into account.
* Tests: Run selector-native tests in `npm test`
* Selector: Fix querying on document fragments
Ref gh-4395
Closes gh-5085
jQuery has followed the following logic for selector handling for ages:
1. Modify the selector to adhere to scoping rules jQuery mandates.
2. Try `qSA` on the modified selector. If it succeeds, use the results.
3. If `qSA` threw an error, run the jQuery custom traversal instead.
It worked fine so far but now CSS has a concept of forgiving selector lists that
some selectors like `:is()` & `:has()` use. That means providing unrecognized
selectors as parameters to `:is()` & `:has()` no longer throws an error, it will
just return no results. That made browsers with native `:has()` support break
selectors using jQuery extensions inside, e.g. `:has(:contains("Item"))`.
Detecting support for selectors can also be done via:
```js
CSS.supports( "selector(SELECTOR_TO_BE_TESTED)" )
```
which returns a boolean. There was a recent spec change requiring this API to
always use non-forgiving parsing:
https://github.com/w3c/csswg-drafts/issues/7280#issuecomment-1143852187
However, no browsers have implemented this change so far.
To solve this, two changes are being made:
1. In browsers supports the new spec change to `CSS.supports( "selector()" )`,
use it before trying `qSA`.
2. Otherwise, add `:has` to the buggy selectors list.
Fixes gh-5098
Closes gh-5107
Ref w3c/csswg-drafts#7676
Introduces a new test API, `includesModule`. The method returns whether
a particular module like "ajax" or "deprecated" is included in the current
jQuery build; it handles the slim build as well. The util was created so that
we don't treat presence of particular APIs to decide whether to run a test as
then if we accidentally remove an API, the tests would still not fail.
Fixes gh-5069
Closes gh-5046
The GitHub UI treats `#NUMBER` as referring to its own issues which is confusing
when in jQuery source it's usually referring to the old deprecated Trac instance
at https://bugs.jquery.com. This change replaces all such Trac references with
`trac-NUMBER`.
A few of the references came with the Sizzle integration and referred to the
Sizzle GitHub bug tracker. Those have been replaced with full links instead.
A new entry describing issue reference conventions has been added to README.
Closes gh-4993
Firefox incorrectly (or perhaps correctly) includes table borders in computed
dimensions, but they are the only one. Workaround this by testing for it and
falling back to offset properties
Fixes gh-4529
Closes gh-4808
Drop support for Edge Legacy: the non-Chromium, EdgeHTML-based Microsoft
Edge version. Also, restrict some workarounds that were applied
unconditionally in all browsers to run only in IE now. This slightly
increases the size but reduces the performance burden on modern browsers
that don't need the workarounds.
Also, clean up some comments & remove some obsolete workarounds.
Fixes gh-4568
Closes gh-4792
The `:scope` pseudo-class[1] has surprisingly good browser support: Chrome,
Firefox & Safari have supported if for a long time; only IE & Edge lack support.
This commit leverages this pseudo-class to get rid of the ID hack in most cases.
Adding a temporary ID may cause layout thrashing which was reported a few times
in [the past.
We can't completely eliminate the ID hack in modern browses as sibling selectors
require us to change context to the parent and then `:scope` stops applying to
what we'd like. But it'd still improve performance in the vast majority of
cases.
[1] https://developer.mozilla.org/en-US/docs/Web/CSS/:scope
Fixes gh-4453
Closes gh-4454
Ref gh-4332
Ref jquery/sizzle#405
Latest versions of all browsers now implement focusin & focusout natively
and they all converged on a common event order so it doesn't make much sense
for us to normalize it to a different order anymore.
Note that it means we no longer guarantee that focusin fires before focus
and focusout before blur.
Fixes gh-4300
Closes gh-4362
Also, run `grunt npmcopy` to sync the "external" directory with dependencies
from package.json. For example, the Sinon library version didn't match.
Ref gh-4234
Closes gh-4297
Current width/height cssHook reads the computed position style even if not
necessary as the browser passes the scrollboxSize support test. That has been
changed.
This commit also makes the scrollboxSize support test in line with all others
(i.e. only return true or false) and changes the variable name in the hook
to make the code clearer.
Fixes gh-4185
Closes gh-4187
The user agent of the iPad with iOS 11.3 on BrowserStack is missing the "iPhone"
part in the "iPhone OS 11_3" part. This commit makes the iOS regex accept such
(probably?) malformed UAs.
A whitelist is available so that some tests can be allowed to always succeed.
This is used only for ajax for now as it can be manually disabled in IE but
is enabled by default.
Closes gh-4052
- Update QUnit to 1.23.1
- Remove unused dl#dl from test/index.html
- Remove unused map#imgmap from test/index.html
- Ensure all urls to data use baseURI
- Add the 'grunt karma:main' task
- customContextFile & customDebugFile
- Add 'npm run jenkins' script
Close gh-3744
Fixes gh-1999
Support comments that mentioned only Safari < 7 were checked & updated
to account for bugs existing in newer versions as well; Safari 6 support
test results were removed.
Refs gh-2482
