Latest Results
Reject an empty host for http and friends in WHATWG mode (#1970)
<!-- Thank you for your contribution! -->
## What do these changes do?
The WHATWG host parser fails on an empty host for every special scheme
except `file`, so `http://`, `http:///`, `http://?q` and `http://#` are
invalid URLs for the WHATWG URL Standard (and for Node's `URL`). WHATWG
mode accepted them. The root cause was printing: a URL of these schemes
without an authority, such as `http:?q` (kept as a reference to a base
for `join()`), was printed with `//` as `http://?q`, so the parser had
to accept `http://?q` to read it back.
WHATWG mode now prints such a URL without `//` (`http:`, `http:/`,
`http:?q`), compares it the same way, and rejects an empty authority for
`http`, `https`, `ws`, `wss` and `ftp` when parsing a string and when an
RFC 3986 mode URL is moved into WHATWG mode. `file` URLs still always
print `//`. RFC 3986 mode is unchanged.
This is a prerequisite for applying RFC 9110 section 4.2.1 (an http URI
with an empty host is invalid) in the RFC 3986 oracle of the conformance
report. With that oracle change alone, `http://?` and `http://#` would
land in "yarl differs, RFC 3986+3987 and WHATWG agree".
Conformance report: `http://?` and `http://#` moved from "WHATWG mode
differs from WHATWG, RFC mode follows RFC 3986+3987" to "yarl follows
each standard in its mode". `http:` against base
`https://example.org/foo/bar` stays in "WHATWG mode differs from
WHATWG", with yarl WHATWG mode now giving `http:` instead of `http://`
(WHATWG fails there; that is the join of a special-scheme reference
against a base with another scheme and is left for a later change). No
other row changed and "yarl differs, RFC 3986+3987 and WHATWG agree"
stays empty.
Existing tests updated because they pinned the old behaviour:
`tests/test_url_special_authority.py::test_kept_without_authority`
(WHATWG column for `http:`, `http:/`, `http:?q`; the `http://`,
`http:///` and `http:///?q` cases moved to the new module as rejected),
`tests/test_url_empty_components.py::test_special_scheme_authority_by_mode[http:/]`
and `test_equality_follows_str` (`http:/` against `http:///`),
`tests/test_url.py::test_empty_authority` (now uses `file:///`) and the
`("http:///", "..", "http:///")` vector of
`tests/test_url.py::test_join_cpython_urljoin`, which has an invalid
base in WHATWG mode. No strict xfail covered these cases.
## Are there changes in behavior for the user?
Yes, in WHATWG mode: `URL("http://")`, `URL("http:///")`,
`URL("http://?q")` and the same for `https`, `ws`, `wss` and `ftp` raise
`ValueError`, and `str(URL("http:?q"))` is `http:?q` instead of
`http://?q`.
## Is it a substantial burden for the maintainers to support this?
No. The check sits in the branch for a URL without a netloc, and
`_str_empty`/`_cmp_empty` test `scheme == "file"` instead of membership
in `SPECIAL_SCHEMES`.
## Related issue number
N/A
## Checklist
- [x] I think the code is well written
- [x] Unit tests for the changes exist
- [x] Documentation reflects the changes
- [ ] If you provide code modification, please add yourself to
`CONTRIBUTORS.txt`
- [x] Add a new news fragment into the `CHANGES/` folder
<details>
<summary>Agent run details (optional, for reviewers)</summary>
Tests: `PYTHONPATH=$PWD python -m pytest ./tests ./yarl`: 3449 passed,
55 xfailed. With `YARL_NO_EXTENSIONS=1`: 3066 passed, 7 skipped, 53
xfailed. The new `tests/test_url_special_empty_host.py` (37 cases) fails
28 cases against an `upstream/master` copy of `yarl`.
Coverage: `PYTHONPATH=$PWD python -m coverage run --source=yarl,tests -m
pytest ./tests ./yarl -n0 --no-cov`: every changed line and every line
of the new and edited test modules covered; the 5 remaining misses in
`yarl/_url.py` are outside the diff.
Conformance: `PYTHONPATH=$PWD python tools/conformance/compare.py
--check` passes. Section diff against master, case by case: `http://?`
and `http://#` changed section; `http:` on `https://example.org/foo/bar`
changed only its yarl WHATWG value.
WHATWG cross-check with Node `URL`: `http://`, `http:///`, `http:////`,
`http://?`, `http://#`, `http://?q`, `http:///?q`, `http://#f`,
`HTTP://?`, `https://`, `ws://`, `wss://?q`, `ftp://#f` fail in both;
`file://` is `file:///` and `sc://`, `sc://?q` are kept in both;
`http:`, `http:?q`, `http:#f` against `http://h/a?x` give
`http://h/a?x`, `http://h/a?q`, `http://h/a?x#f` in both.
Docs: `make doc-spelling` and `make -C docs doctest` (238 tests, 0
failures) pass.
Lint: `pre-commit run` on the changed files passes, including mypy.
Benchmarks: callgrind instruction counts, Python 3.13.2,
`PYTHONHASHSEED=0 YARL_NO_EXTENSIONS=1`, every body in
`tests/test_url_benchmarks.py` (91 bodies), per call as the difference
between 51 and 11 iterations, against `git archive upstream/master
yarl`. Only `test_human_repr` moved by more than 1%: -1.3%, rechecked
with 21 and 61 iterations at -1.5% (cheaper `_str_empty` check).
Everything else within +-0.7% on the first run and +-0.3% on the
recheck.
</details>
Drafted with Claude Code (Claude Opus 5.5); reviewed by @asvetlov. Reject an empty host for http and friends in WHATWG mode (#1970)
<!-- Thank you for your contribution! -->
## What do these changes do?
The WHATWG host parser fails on an empty host for every special scheme
except `file`, so `http://`, `http:///`, `http://?q` and `http://#` are
invalid URLs for the WHATWG URL Standard (and for Node's `URL`). WHATWG
mode accepted them. The root cause was printing: a URL of these schemes
without an authority, such as `http:?q` (kept as a reference to a base
for `join()`), was printed with `//` as `http://?q`, so the parser had
to accept `http://?q` to read it back.
WHATWG mode now prints such a URL without `//` (`http:`, `http:/`,
`http:?q`), compares it the same way, and rejects an empty authority for
`http`, `https`, `ws`, `wss` and `ftp` when parsing a string and when an
RFC 3986 mode URL is moved into WHATWG mode. `file` URLs still always
print `//`. RFC 3986 mode is unchanged.
This is a prerequisite for applying RFC 9110 section 4.2.1 (an http URI
with an empty host is invalid) in the RFC 3986 oracle of the conformance
report. With that oracle change alone, `http://?` and `http://#` would
land in "yarl differs, RFC 3986+3987 and WHATWG agree".
Conformance report: `http://?` and `http://#` moved from "WHATWG mode
differs from WHATWG, RFC mode follows RFC 3986+3987" to "yarl follows
each standard in its mode". `http:` against base
`https://example.org/foo/bar` stays in "WHATWG mode differs from
WHATWG", with yarl WHATWG mode now giving `http:` instead of `http://`
(WHATWG fails there; that is the join of a special-scheme reference
against a base with another scheme and is left for a later change). No
other row changed and "yarl differs, RFC 3986+3987 and WHATWG agree"
stays empty.
Existing tests updated because they pinned the old behaviour:
`tests/test_url_special_authority.py::test_kept_without_authority`
(WHATWG column for `http:`, `http:/`, `http:?q`; the `http://`,
`http:///` and `http:///?q` cases moved to the new module as rejected),
`tests/test_url_empty_components.py::test_special_scheme_authority_by_mode[http:/]`
and `test_equality_follows_str` (`http:/` against `http:///`),
`tests/test_url.py::test_empty_authority` (now uses `file:///`) and the
`("http:///", "..", "http:///")` vector of
`tests/test_url.py::test_join_cpython_urljoin`, which has an invalid
base in WHATWG mode. No strict xfail covered these cases.
## Are there changes in behavior for the user?
Yes, in WHATWG mode: `URL("http://")`, `URL("http:///")`,
`URL("http://?q")` and the same for `https`, `ws`, `wss` and `ftp` raise
`ValueError`, and `str(URL("http:?q"))` is `http:?q` instead of
`http://?q`.
## Is it a substantial burden for the maintainers to support this?
No. The check sits in the branch for a URL without a netloc, and
`_str_empty`/`_cmp_empty` test `scheme == "file"` instead of membership
in `SPECIAL_SCHEMES`.
## Related issue number
N/A
## Checklist
- [x] I think the code is well written
- [x] Unit tests for the changes exist
- [x] Documentation reflects the changes
- [ ] If you provide code modification, please add yourself to
`CONTRIBUTORS.txt`
- [x] Add a new news fragment into the `CHANGES/` folder
<details>
<summary>Agent run details (optional, for reviewers)</summary>
Tests: `PYTHONPATH=$PWD python -m pytest ./tests ./yarl`: 3449 passed,
55 xfailed. With `YARL_NO_EXTENSIONS=1`: 3066 passed, 7 skipped, 53
xfailed. The new `tests/test_url_special_empty_host.py` (37 cases) fails
28 cases against an `upstream/master` copy of `yarl`.
Coverage: `PYTHONPATH=$PWD python -m coverage run --source=yarl,tests -m
pytest ./tests ./yarl -n0 --no-cov`: every changed line and every line
of the new and edited test modules covered; the 5 remaining misses in
`yarl/_url.py` are outside the diff.
Conformance: `PYTHONPATH=$PWD python tools/conformance/compare.py
--check` passes. Section diff against master, case by case: `http://?`
and `http://#` changed section; `http:` on `https://example.org/foo/bar`
changed only its yarl WHATWG value.
WHATWG cross-check with Node `URL`: `http://`, `http:///`, `http:////`,
`http://?`, `http://#`, `http://?q`, `http:///?q`, `http://#f`,
`HTTP://?`, `https://`, `ws://`, `wss://?q`, `ftp://#f` fail in both;
`file://` is `file:///` and `sc://`, `sc://?q` are kept in both;
`http:`, `http:?q`, `http:#f` against `http://h/a?x` give
`http://h/a?x`, `http://h/a?q`, `http://h/a?x#f` in both.
Docs: `make doc-spelling` and `make -C docs doctest` (238 tests, 0
failures) pass.
Lint: `pre-commit run` on the changed files passes, including mypy.
Benchmarks: callgrind instruction counts, Python 3.13.2,
`PYTHONHASHSEED=0 YARL_NO_EXTENSIONS=1`, every body in
`tests/test_url_benchmarks.py` (91 bodies), per call as the difference
between 51 and 11 iterations, against `git archive upstream/master
yarl`. Only `test_human_repr` moved by more than 1%: -1.3%, rechecked
with 21 and 61 iterations at -1.5% (cheaper `_str_empty` check).
Everything else within +-0.7% on the first run and +-0.3% on the
recheck.
</details>
Drafted with Claude Code (Claude Opus 5.5); reviewed by @asvetlov. Latest Branches
0%
Samin061:scheme-grammar-validate -8%
Kayvan-Zahiri:fix/strip-zone-from-host-header-1862 0%
asvetlov:rfc9110-oracle-2 © 2026 CodSpeed Technology