Filters an edge list by registrable domain and/or host rules. Rows are kept only when both endpoints satisfy the keep/ignore logic. Ignore rules always override keep rules. When keep rules are provided, `drop_third_party = TRUE` removes URLs outside the keep lists.
This function is intended as a **pre-processing step** before calling [pagerank()]. For example, to scope a PageRank analysis to a single site or exclude CDN / tracking domains.
**Ordering relative to folding:** when `pagerank()` calls this filter internally (via its `keep_domains` / `exclude_domains` / `keep_hosts` / `exclude_hosts` arguments), the filter runs *after* redirect and canonical folding, so it scopes the post-fold (canonical) namespace. If an out-of-scope canonical/redirect rewrites the crawled domain/host onto a different one, filtering on the crawled value inside `pagerank()` matches nothing. To domain-scope the **crawled input** instead, call `filter_links_by_domain()` on the edge list yourself *before* folding (i.e. before passing it to `pagerank()`).
Arguments
- edge_list_df
A data frame representing the edge list, with at least two URL columns.
- from_col
Name of the source URL column. Default `"from"`.
- to_col
Name of the target URL column. Default `"to"`.
- keep_domains
Character vector of registrable domains to keep (e.g., `"example.com"`). Subdomains are included when their registrable domain matches.
- keep_hosts
Character vector of specific hosts to keep (e.g., `"www.example.com"`). Only exact host matches are kept.
- ignore_domains
Character vector of registrable domains to drop.
- ignore_hosts
Character vector of specific hosts to drop.
- drop_third_party
Logical. When keep lists are provided and this is `TRUE` (default), URLs outside the keep lists are dropped. When `FALSE`, only explicitly ignored URLs are dropped.
- return_report
Logical. If `TRUE`, returns a list with the filtered data frame and a filter report. Default `FALSE`.
- psl_section
Public Suffix List section used to derive registrable domains, passed to `rurl::get_domain()` / `rurl::safe_parse_urls()`. One of `"all"` (default, ICANN + private suffixes), `"icann"`, or `"private"`. Affects domain-based (not host-based) keep/ignore matching; e.g. under `"icann"`, `user.github.io` has registrable domain `github.io`, while under `"all"` it is `user.github.io`.
- rurl_params
A list of `rurl` canonicalization arguments overriding pagerankr's profile per key, used when extracting hosts/domains from both the edge URLs and the keep/ignore values. Pass the **same** profile used to clean the graph so the comparison keys are derived identically. The host-relevant knobs are `host_encoding` (`"keep"`/`"idna"`/`"unicode"` — IDN folding; e.g. `"idna"` makes `münchen.de` and `xn–mnchen-3ya.de` match), `www_handling`, `subdomain_levels_to_keep`, `case_handling`, and `protocol_handling`. Registrable-domain matching is encoding-independent. When called from [pagerank()], this is forwarded automatically.
Value
If `return_report = FALSE` (default), the filtered data frame (preserving all columns). If `TRUE`, a list with elements `filtered_df` and `report`.
Examples
links <- data.frame(
from = c(
"http://www.example.com/a", "http://example.com/b",
"http://cdn.tracker.com/c"
),
to = c(
"http://example.com/b", "http://help.example.com/d",
"http://www.example.com/a"
)
)
# Keep only example.com edges
filter_links_by_domain(links, keep_domains = "example.com")
#> from to
#> 1 http://www.example.com/a http://example.com/b
#> 2 http://example.com/b http://help.example.com/d
# Ignore a specific subdomain
filter_links_by_domain(links, ignore_hosts = "cdn.tracker.com")
#> from to
#> 1 http://www.example.com/a http://example.com/b
#> 2 http://example.com/b http://help.example.com/d
# Get a report of what was filtered
result <- filter_links_by_domain(links,
keep_domains = "example.com",
return_report = TRUE
)
result$report
#> $rows_before
#> [1] 3
#>
#> $rows_after
#> [1] 2
#>
#> $rows_dropped
#> [1] 1
#>
#> $keep_domains
#> [1] "example.com"
#>
#> $keep_hosts
#> character(0)
#>
#> $ignore_domains
#> character(0)
#>
#> $ignore_hosts
#> character(0)
#>