Get the parse status of URLs

The status is a single value collapsed from three independent facts — URL syntax, admission policy, and the Public Suffix List annotation — so it is a lossy view of them. The guaranteed loss: a structural syntax failure and a policy rejection both report "error". Call get_parse_verdicts when you need to tell those apart, or to read the PSL result as a typed annotation state rather than as a warning. Nothing here is deprecated — the layered accessor is purely additive.

Usage

get_parse_status(
  url,
  protocol_handling = "keep",
  www_handling = "none",
  subdomain_levels_to_keep = NULL,
  source = c("all", "private", "icann"),
  scheme_policy = c("infer", "require"),
  scheme_acceptance = c("web", "general"),
  url_standard = NULL
)

Arguments

url

A character vector of URLs to be parsed.

protocol_handling

A character string specifying how to handle protocols. Defaults to "keep". Regardless of this option, rurl only processes authority-based URLs whose scheme is one of http, https, ftp, or ftps; a scheme-bearing input with any other scheme (e.g. mailto:, tel:, ws:) yields parse_status = "error". Scheme inference (below) also requires the input to be host-shaped: a scheme-less string that is not a host (e.g. "asdfghjkl", "12345", "/path") or is a non-canonical IP literal (integer/hex/octal/short forms, or leading-zero octets like "192.168.010.1") is rejected as "error" rather than having a scheme fabricated for it.

"keep": If a supported scheme exists (http, https, ftp, ftps), it's used. If no scheme and the input is host-shaped, "http://" is added; otherwise the input is not a URL and yields "error".
"none": If a supported scheme exists, it's used. If no scheme, then no scheme is used (scheme component will be NA).
"strip": Any existing scheme is removed (scheme component will be NA).
"http": The scheme is forced to be "http".
"https": The scheme is forced to be "https".

www_handling

A character string specifying how to handle "www" and www[number] prefixes in the host. Defaults to "none".

"none": (Default) Leaves the host's www prefix (or lack thereof) untouched.
"strip": Removes any "www." or www[number]. prefix.
"keep": Ensures the host starts with "www.". If it has www[number]., it's normalized to "www.". If no www prefix, "www." is added. An empty input host remains empty.
"if_no_subdomain": If the host is a bare registered domain (e.g., "example.com"), "www." is added. If the host already has a "www." or www[number]. prefix, it is normalized to "www." (e.g., "www1.example.com" becomes "www.example.com"; "www1.sub.example.com" becomes "www.sub.example.com"). If a non-www subdomain exists (e.g., "sub.example.com" or the normalized "www.sub.example.com"), the host is not further altered. An empty input host remains empty.

subdomain_levels_to_keep

An integer or NULL. Determines how many levels of subdomains are kept, in addition to any 'www.' prefix handled by www_handling.

NULL: (Default) No specific subdomain stripping is performed beyond www_handling.
0: All subdomains are stripped. If www_handling preserved or added 'www.', it remains (e.g., 'www.sub.example.com' becomes 'www.example.com'; 'sub.example.com' becomes 'example.com').
N > 0: Keeps up to N levels of subdomains, counted from right-to-left (closest to the registered domain), in addition to any 'www.' prefix. E.g., if N=1, 'three.two.one.example.com' becomes 'one.example.com'; 'www.three.two.one.example.com' (post www_handling) becomes 'www.one.example.com'.

source

Which PSL source to use: "all", "private", or "icann". Warning statuses such as warning-no-tld, warning-invalid-tld, and warning-public-suffix depend on which PSL section is consulted, so pass source = "icann" to use only ICANN-managed TLDs.

scheme_policy

Controls whether scheme-less, host-shaped input is accepted (an input-acceptance axis, distinct from protocol_handling, which only controls how the scheme is presented, and from url_standard, which controls interpretation). Defaults to "infer".

"infer": (Default) Fabricate http:// for scheme-less host-shaped input (e.g. example.com parses as http://example.com), a browser-omnibox-style affordance. This is the historical behavior.
"require": Reject scheme-less input — a scheme-less host-shaped value becomes parse_status = "error" rather than gaining a fabricated scheme. Use this for a strict, pure-parser posture. Note this governs only bare host input; scheme-relative //host input is governed separately by scheme_relative_handling.

scheme_acceptance

Which scheme tokens may enter parsing (a scheme-acceptance axis, distinct from scheme_policy, which governs scheme-less input, and from url_standard, which governs interpretation). Defaults to "web".

"web": (Default) Only the curated web-scheme allowlist (http/https/ftp/ftps/file) is admitted; a scheme-bearing input outside it is parse_status = "error". This is the historical, byte-for-byte compatible behavior.
"general": Admit any syntactically valid scheme token and parse opaque (mailto:x), non-special (foo://host), and RFC-generic URLs. Requires an explicit url_standard ("rfc3986" or "whatwg"), which decides the interpretation; general with url_standard = NULL is an error. Non-special / opaque hosts receive no www-stripping, no domain/TLD derivation, and are never run through the IDNA/punycode helpers. A non-special scheme with no // is an opaque path: it has no authority, so host, user, port and the domain/tld columns are all NA and the entire remainder is the path (query/fragment are still split off). This includes mailto: — the recipient's @ never re-triggers authority parsing. To decompose a mailto: recipient, use the accessors (get_host() / get_domain() / get_user(), ADR 0012 D7) or get_mailto_recipients(); those deliberately return a recipient's parts where this table presents NA, because a recipient domain is extraction metadata, not the URL's authority.

url_standard

Optional top-level standard profile: NULL (default), "rfc3986", or "whatwg". With NULL the behavior is exactly what the individual low-level options select (fully backward compatible). When set, it selects a coherent set of standard-conformant behaviors for the axes it governs — path percent/dot handling, the host IPv4/reg-name model, and case_handling — so callers do not have to hand-assemble the low-level knobs. Passing a governed low-level knob (path_normalization or case_handling) with a value the selected profile would not choose is an error; passing the value the profile would pick is accepted (only case_handling = "lower_host" is accepted under a selector — "keep", "lower", and "upper" all conflict, since "lower" also lowercases the path, which neither standard sanctions). Added as the last argument so existing positional calls keep their meaning; always pass it by name. Under "whatwg" the selector additionally recognizes a literal backslash as a path separator for WHATWG-special schemes (http/https/ftp) and nulls default ports in parse output; use port_handling = "strip_default" for spec-style clean URL port rendering. See resolve_url for url_standard-governed reference resolution. The selector does not govern whether port_handling may be set (it is a standalone editorial knob), nor does it govern path_encoding (an orthogonal path-presentation knob that layers on any profile), IDNA rendering, or query handling.

Value

A character vector with the parse status of each URL: one of "ok", "ok-ftp", "ok-scheme-relative", "warning-no-tld", "warning-invalid-tld", "warning-public-suffix", "warning-userinfo" (a scheme-less input carrying userinfo, e.g. "user@example.com"), or "error". See safe_parse_url for the full semantics.

Examples

get_parse_status(
  c("http://example.com", "ftp://example.com", "mailto:user@example.com")
)
#> [1] "ok"     "ok-ftp" "error" 
get_parse_status(c("http://example.com", "not-a-url"))
#> [1] "ok"    "error"
get_parse_status("http://example.com", source = "icann")
#> [1] "ok"

Usage

Arguments

Value

See also

Examples