DEV Community: IPYE

WireGuard Site-to-Site with BGP—Route Your Free /48 Through the Tunnel

IPYE — Tue, 10 Nov 2026 00:22:58 +0000

Disclosure up front: I run IPYE, a company that leases IP space. Most of this article is practical how-to; IPYE appears where a free /48 helps you try the work, with a short call to action at the end.

WireGuard encrypts; BGP decides which prefixes use the tunnel. Stuffing every subnet into AllowedIPs alone does not scale. This design uses WireGuard for the outer path and FRR for site prefixes. A natural practice payload is IPYE's free IPv6 /48 (first year free, instant setup, real allocation on your ASN, portal LOA, automatic ROA)—announce it at site A and learn it at site B over wg0. Lab stand-ins: 192.0.2.0/24, 198.51.100.0/24, tunnel 2001:db8:ffff::/64, ASNs 64496/64497.

Design rules

Each WireGuard interface gets a small point-to-point /127 or /126 (IPv6) or /30 (IPv4) used only for BGP and probes.
AllowedIPs on each peer includes that tunnel subnet plus the remote site's loopback or a broad encapsulation selector you intentionally manage—not every customer /48 you might ever learn.
BGP (or OSPF) runs over the tunnel and advertises the real site prefixes.
Firewall policy allows UDP 51820 (or your chosen port) between public endpoints and BGP between tunnel IPs.

If you put the entire Internet into AllowedIPs, WireGuard becomes your default route cryptokey router and you lose the ability to learn backup paths sensibly.

WireGuard skeleton (site A)

[Interface]
PrivateKey = <siteA private>
Address = 2001:db8:ffff::1/127
ListenPort = 51820

[Peer]
PublicKey = <siteB public>
Endpoint = 203.0.113.10:51820
AllowedIPs = 2001:db8:ffff::0/127
PersistentKeepalive = 25

Site B mirrors with ::0/127 addressing swapped. Bring it up with wg-quick up wg0 and verify ping6 2001:db8:ffff::0. Only after the tunnel pings should you enable BGP.

FRR over the tunnel

router bgp 64496
 bgp router-id 192.0.2.1
 neighbor 2001:db8:ffff::0 remote-as 64497
 !
 address-family ipv4 unicast
  neighbor 2001:db8:ffff::0 activate
  network 192.0.2.0/24
  neighbor 2001:db8:ffff::0 prefix-list TO-B out
  neighbor 2001:db8:ffff::0 prefix-list FROM-B in
 exit-address-family
!
ip prefix-list TO-B permit 192.0.2.0/24
ip prefix-list FROM-B permit 198.51.100.0/24

Site B advertises 198.51.100.0/24 symmetrically. Use eBGP multihop only if you peer to loopbacks; with point-to-point tunnel IPs, TTL one is enough.

AllowedIPs versus BGP routes

WireGuard still drops packets whose inner destination is outside AllowedIPs. So when BGP learns 198.51.100.0/24, you must either:

add 198.51.100.0/24 to AllowedIPs on site A (static, simple, two-site), or
use a broader AllowedIPs (for example the aggregate you own) carefully, or
run a helper that updates AllowedIPs from the BGP table (advanced; automate with care).

For two sites, listing the remote site aggregate in AllowedIPs and letting BGP select paths among multiple tunnels is usually enough. For many sites, design hierarchical aggregates so AllowedIPs stays small.

Multiple tunnels and failover

Build two WireGuard sessions (primary and backup paths). Run BGP over both with different local preference or AS-path length so one tunnel is preferred. When the primary UDP path dies, BGP times out (prefer BFD if available) and the backup path takes over without rewriting AllowedIPs. Keepalive on WireGuard helps NAT middleboxes but does not replace routing timers.

Verification

wg show
ping6 2001:db8:ffff::0
vtysh -c 'show bgp summary'
vtysh -c 'show ip route 198.51.100.0'
traceroute 198.51.100.10

Confirm the traceroute enters wg0 and that killing the primary endpoint shifts the BGP best path. Site-to-site VPN done this way stays understandable months later: crypto on the outside, routing on the inside.

Firewall and policy notes

On each site, allow UDP to the WireGuard port only from the peer's public IP when peers are fixed. If a peer has a dynamic home endpoint, Prefer keepalive and authenticate solely by key, but still rate-limit. Allow TCP 179 only between tunnel addresses, not on the public WAN. Separate management SSH from tunnel traffic so a WireGuard outage does not lock you out of the box (out-of-band or provider console access remains mandatory).

IPv6 and dual-stack

Prefer IPv6 transport for the outer WireGuard endpoints when both sites have stable AAAA reachability; it avoids CGNAT pain on the outer path. Inside, run both address families in BGP if you carry both. Do not assume that an IPv6 tunnel automatically carries IPv4 payloads without either IPv4 tunnel addresses or a deliberate encapsulation choice—WireGuard carries layer 3 packets for the families you configure in Address/AllowedIPs.

When static AllowedIPs is enough

Two sites, stable subnets, no failover engineering: a pure WireGuard config without BGP is fine. Add BGP when you grow a third site, need traffic engineering between two tunnels, or want customer prefixes to appear and disappear without rewriting VPN configs. The goal is boring failure domains, not maximum protocol count.

Key rotation and peer lifecycle

Rotate WireGuard keys on a planned cadence and when staff leave. Because routing is separate, you can bring up a parallel tunnel with new keys, shift BGP preference, then decommission the old peer without renumbering LAN subnets. Record public keys in a password manager or vault, not in a wiki plain page. Treat wg0 conf like a router config: peer-reviewed and stored in git with secrets filtered.

Carry the free /48 over WG

Put the free /48 from my.ipye.com/products/ipv6-rental in BGP and in AllowedIPs (or a covering aggregate). If you change region or ASN in the IPYE dashboard, those changes apply automatically—re-check tunnel endpoints and ROA after moves between VPS providers.

Lab first

Before touching production routers, reproduce the pattern with two namespaces or two cheap VMs: WireGuard up, BGP established, kill -9 the primary wg process, and watch the secondary path take traffic. Only promote the design when that failover is boring. Boring is the goal.

Tunnel a free /48 this weekend

Claim space at the IPYE IPv6 rental page (see About). Site ipye.com (see About) · ipye@turnel.com

About IPYE

IPYE (https://ipye.com) leases IPv4, IPv6 and ASN space with instant setup. Your first IPv6 /48 is free for the first year, a real allocation you can announce on your own ASN, with a LOA from the portal and the ROA created automatically. ASN and region changes apply automatically. IPv4 /24 to /22 leases include LOA, ROA and per-/24 rDNS; WHOIS is editable on IPv6 /44 and larger. Locations: Los Angeles, New York, Ashburn, Salt Lake City, London, Amsterdam, Frankfurt. Claim your free /48: https://my.ipye.com/products/ipv6-rental. Questions: ipye@turnel.com.

Disclosure: I run IPYE

Fix Bad Geolocation After You Announce a Free /48: Geofeeds and Database Chases

IPYE — Tue, 10 Nov 2026 00:20:28 +0000

You announce a brand-new prefix from Ashburn and databases still show another country. That is normal for fresh, transferred, or leased space—including a free IPv6 /48 you just claimed from IPYE with instant setup. This article covers measuring errors, publishing an RFC 8805 geofeed, chasing vendors, and what to tell app owners while you wait. Examples use 2001:db8:100::/48 and 198.51.100.0/24 as stand-ins for space you announce from your ASN with a portal LOA and automatic ROA.

Measure before you complain

Sample several addresses across the prefix in multiple public GeoIP views and in the databases your own applications use (MaxMind-style MMDB files, cloud provider geo APIs, and browser-side services). Record country, region, and city. Also test from user perspective: ask two people in different places to open a page that prints the geo your app believes, using a host in the new prefix.

dig +short TXT example.com   # if you publish self-test records; optional
curl -s https://ifconfig.io/country_code   # from a host inside the prefix; one of many checkers

Prefer reproducible notes over a single web screenshot. Databases disagree; your job is to move the majority toward truth.

Publish a geofeed (RFC 8805)

A geofeed is a CSV of prefixes to location codes, published at an HTTPS URL and referenced from WHOIS/RDAP remarks so findable crawlers can discover it. Minimal process:

Host a UTF-8 CSV over HTTPS (for example on a stable ops hostname).
Each data line: prefix, country (ISO), optional region, city, postal.
Add the feed URL to the inetnum/inet6num remarks in the format geofeed parsers expect (see RFC 8805).
Keep the file updated when you move announcements between PoPs.

Example CSV lines (documentation prefixes):

# prefix,country,region,city,postal
198.51.100.0/24,US,US-VA,Ashburn,
2001:db8:100::/48,US,US-VA,Ashburn,

Validate that your RIR portal accepted the remark and that curl -fsSL retrieves the feed without redirects through fragile short links.

Chase the big databases

Geofeed adoption is improving but not universal. For databases your customers care about:

Use each vendor's official correction or self-service submission form.
Attach evidence: LOA or RIR whois, PoP city, looking-glass announcement, geofeed URL.
Track ticket IDs and revisit after their published SLA.

Do not expect overnight updates. Plan application-level overrides for critical customers if a wrong country breaks login fraud scoring.

Application mitigations while geo is wrong

Allow users to pick a region manually when geo is only a hint.
Avoid hard hard-blocks based solely on GeoIP country for your own infrastructure addresses.
For CDN steering, pin origins or use provider-specific location overrides when available.
Document the issue in status notes so support does not chase ghosts.

IPv6-specific notes

IPv6 geo is often coarser and stickier because historical data is thinner. Publish geofeeds for the aggregated /48 you announce, not for every /64. If you announce from multiple cities, either split announcements into regional aggregates or accept a single "primary" location in databases that only store one point per prefix.

Done criteria

You are finished when your geofeed is live, RDAP remarks point to it, major databases you depend on show the correct country (city is nicest-to-have), and internal runbooks describe how to re-submit after a PoP move. Geolocation is part of announcing space, not an optional marketing task after launch.

Common failure modes

Feed URL in WHOIS but HTTPS certificate wrong — crawlers skip it.
CSV with a header dialect parsers do not expect — validate against RFC 8805 examples.
Feed lists a /48 but you announce only a /56 — still OK if accurate; do not list cities you do not route from.
Correction submitted to the wrong product line of a geo vendor — legacy vs GeoIP2-style databases differ.
NAT or CDN in front of the test host — you measured the edge, not your prefix.

When a vendor refuses a change, ask what evidence format they need. Provide looking-glass links showing the prefix originated from your ASN at the expected upstream. Patience plus clean evidence beats repeated angry tickets with no attachments.

Communicating to stakeholders

Tell product owners: "Country accuracy may take weeks; do not ship hard geo blocks on this prefix until the tracker says green." Give them a status page row: prefix, intended city, geofeed URL, vendor tickets, last verified date. That single row prevents surprise outages when a payment provider starts treating your Ashburn space as another continent.

IPv4 and IPv6 together

If you announce both families from the same PoP, publish both in the same feed and submit both to vendors. Mixed results—IPv4 correct, IPv6 wrong—are common and confuse dual-stack users differently depending on Happy Eyeballs winners. Verify with tests that force -4 and -6. Align both families before you call the incident closed.

Fresh free /48, stale maps

After you claim a free /48 at IPYE and announce it, publish a geofeed for the aggregate you actually route. Region changes in the IPYE dashboard apply automatically for the service side; third-party GeoIP databases still need your feed and correction tickets. Do not promise day-one city accuracy.

Annual review

Put a calendar reminder to re-check geo for each announced aggregate every six to twelve months, and whenever you open a new PoP. Databases drift, and a correct entry can become wrong again after quiet routing changes. Re-fetch your own geofeed, confirm WHOIS still points at it, and spot-check the three vendors your customers mention most.

Announce, then geofeed

Free /48: the IPYE IPv6 rental page (see About) · ipye@turnel.com

About IPYE

Disclosure: I run IPYE

IPv4 Prefix Due Diligence—And Why a Free /48 Is the Low-Risk Way to Practise BGP

IPYE — Tue, 10 Nov 2026 00:18:16 +0000

Not every IPv4 block is ready for production. Spam history, chaotic origins, and stuck geolocation can follow a /24 long after the last holder left. Run due diligence before you announce leased or transferred IPv4. For learning BGP, ROAs, and LOAs without inheriting someone else's reputation, start with IPYE's IPv6 /48 that is free for the first year—instant setup, real allocation, portal LOA, automatic ROA—then add IPv4 /24–/22 leases (LOA, ROA, per-/24 rDNS) when you need them. Checklist below uses documentation 192.0.2.0/24 and ASN 64496.

Whois and registration hygiene

Start with RDAP/WHOIS. Confirm the registered holder, status flags, and abuse contact. For a lease you should already have an LOA; still verify that the object you see in the RIR database matches what the LOA claims. Check whether the inetnum remarks mention previous policy problems. If reverse DNS is still delegated to dead nameservers, plan to fix rDNS before sending mail or running SMTP-heavy services.

whois 192.0.2.0
# or
rdap https://rdap.example/ip/192.0.2.0   # use your RIR's RDAP base

Routing history

Look up the prefix in RIPEstat, RouteViews, or similar collectors:

Has it been announced recently? By which ASNs?
Were there overlapping more-specifics that suggest prior subdivision?
Is there a ROA already, and does it match 64496 or an old origin?

A prefix that has been dark for years may be fine, but it may also have stale filters in remote networks that still expect the old origin. A prefix that flaps between many origins is a red flag. Save screenshots or JSON exports of the history in the change ticket.

Reputation and blocklists

Check major DNSBLs and reputation portals for several addresses across the block, not only the first address:

Query common DNSBL zones with dig against a resolver you control.
Use vendor reputation demos carefully; treat a single "poor" score as a lead to investigate, not as proof.
Search abuse forums and past paste sites for the CIDR string.

If only a small sub-range is dirty, you might still use the clean parts while you remediate, but document the split. If the whole /24 is widely listed, budget time for delisting after you control the space and stop the abuse—not before.

Geolocation

Geo-IP databases often place transferred or leased space in the wrong country for months. That breaks CDN steering, fraud scoring, and sometimes regulatory assumptions. Check several databases against 192.0.2.10 and compare to your intended PoP city. Plan a geofeed (RFC 8805) and update requests early; do not assume launching in London means databases will show London on day one.

Mail and services smoke test

Before moving customers:

Announce the prefix from a lab ASN or maintenance window.
Host a sinkhole page and a mail test endpoint on one address.
Send test mail to major providers from that address with correct PTR/SPF/DKIM.
Run outbound HTTP crawls only against systems you own to see whether middleboxes treat the space oddly.

If PTR cannot be delegated to you yet, postpone mail; broken reverse DNS creates tickets that look like "reputation" but are really operational incompleteness.

Contract and operational questions for a lessor

Ask who handles LOA updates, ROA signing, rDNS delegation, and abuse mailbox escalation. Clarify the notice period if the lessor must reclaim space. Confirm you may publish IRR route objects and a geofeed. Vague answers here matter as much as a Spamhaus hit.

Decision rubric

Accept the block when registration matches the paperwork, routing history is explainable, reputation is clean or remediable, geolocation has a plan, and operational controls (ROA/IRR/rDNS/abuse) are in your hands. Walk away—or quarantine for lab-only use—when origins thrash, listings are dense across the range, or nobody can authorise ROA and reverse DNS changes. Due diligence is cheaper than migrating customers off a poisoned prefix later.

A one-afternoon checklist you can paste into a ticket

Copy this into the change request and fill answers before the announcement window:

RDAP holder and abuse email match the LOA? (yes/no, paste URLs)
ROA present? Origin ASN? maxLength?
RIPEstat visibility last 30/90/365 days — summary attached?
DNSBL hits for .1, .50, .100, and .254 of the /24?
GeoIP country today vs intended PoP?
Who will publish IRR route/route6 and rDNS NS?
Mail test plan owner and rollback plan?

If any answer is "unknown", stop. Announcing first and investigating later is how you inherit someone else's blocklistings during your launch week.

After you accept the space

Re-check reputation at day 7 and day 30. New listings sometimes appear when scanners notice fresh announcements. Keep the abuse mailbox staffed. Update PeeringDB and documentation sites so peers know the prefix is intentional. Due diligence is not only a pre-lease ritual; it is the first month of ownership hygiene.

When you do need IPv4 from IPYE

After the IPv4 checks pass (or when you lease clean space), IPYE IPv4 /24, /23, and /22 leases include LOA and ROA, with rDNS per /24 using IPYE's or your own nameservers. You can ask for a test address before ordering. That is separate from the free /48—do not expect IPv6 PTR self-service in the portal yet, and WHOIS edits apply to IPv6 /44 and larger, not the free /48.

Tools worth bookmarking

Keep a personal shortlist: your RIR's RDAP base URL, RIPEstat prefix overview, a RouteViews looking glass, Spamhaus and a second DNSBL aggregator, and two GeoIP lookup pages. Running the same five checks the same way every time is more valuable than discovering exotic tools once. Save the raw outputs next to the LOA PDF so future you can see what the Internet looked like on acceptance day.

Practise on a free /48 first

Claim IPv6 at the IPYE IPv6 rental page (see About); review IPv4 options at ipye.com (see About). Questions: ipye@turnel.com.

About IPYE

Disclosure: I run IPYE

DS-Lite Softwires Explained—And Why Native AAAA on a Free /48 Still Wins

IPYE — Tue, 10 Nov 2026 00:13:57 +0000

DS-Lite (RFC 6333) tunnels IPv4 inside IPv6 from the CPE (B4) to a carrier AFTR that runs CGNAT. Operators care about MTU and abuse logging; content operators care that native IPv6 avoids the softwire entirely. A free IPv6 /48 from IPYE—instant setup, first year free, real allocation for your ASN, portal LOA, automatic ROA—lets you practise announcing space users can reach without CGNAT. This article covers B4/AFTR roles, a Linux tunnel sketch, logging fields, and debugging.

Roles

B4 (Basic Bridging BroadBand): CPE function that tunnels IPv4-in-IPv6 to the AFTR. LAN IPv4 is often 192.0.0.0/29 or a normal RFC 1918 LAN without local NAT, depending on the profile.
AFTR (Address Family Transition Router): decapsulates, runs NAT44 for many subscribers, encapsulates return traffic.

IPv6 traffic from the LAN is native and never enters the softwire. That split is the operational point of DS-Lite: upgrade the access network to IPv6-only while keeping a shared IPv4 exit.

Why providers like it

One IPv6-only access network reaches both IPv6 services and legacy IPv4. IPv4 public addresses are shared at the AFTR (CGNAT), which conserves inventory. Compared with per-customer public IPv4, you must invest in logging (subscriber identifier mapped to outside IP, port, and time) and in customer support playbooks for CGNAT breakage (inbound ports, some games, hairpin behaviours). Training the help desk to recognise "IPv6 works, IPv4 intermittent" as a softwire or AFTR symptom saves hours.

B4-side Linux sketch

Production CPEs embed DS-Lite; on a lab Linux B4 you typically need an ipip6 (IPv4-in-IPv6) tunnel to the AFTR:

ip link add name ds-lite type ip6tnl \
  mode ipip6 remote 2001:db8:aftr::1 local 2001:db8:b4::2 dev eth0
ip link set ds-lite up
ip addr add 192.0.0.2/29 dev ds-lite
ip route add default dev ds-lite

Your WAN must already have working IPv6 to 2001:db8:aftr::1. DNS for LAN clients often remains IPv6 with optional DNS64; IPv4 DNS can be provided via DHCPv4 on the LAN facing interface. Exact sysctls depend on kernel and whether forwarding is enabled between LAN and tunnel. Persist the tunnel with your network manager or a small systemd unit so a reboot restores the softwire.

AFTR expectations

The AFTR terminates softwires from many B4s, demultiplexes by IPv6 outer source, and applies NAT44. Operational requirements:

Per-softwire MTU accounting (IPv4 payload plus IPv6 outer header); MSS clamping on the NAT or tunnel is common.
Port block allocation or NAT behaviour documented for support.
Exportable logs: outer IPv6 (subscriber), inside IPv4, outside IPv4, ports, timestamp. Without that, abuse complaints about 192.0.2.50 cannot be mapped to a customer.
Capacity headroom on ports per subscriber; exhaustion shows up as one-way failures for new connections.

Debugging

ping 192.0.2.10            # from LAN behind B4
traceroute 192.0.2.10
tcpdump -n -i eth0 ip6 and proto 4   # IPv4-in-IPv6 on the WAN
ip -d link show ds-lite
ping6 2001:db8:aftr::1

If IPv6 works but IPv4 fails, check AFTR reachability, tunnel state, and whether the AFTR has exhausted ports for that subscriber. If large transfers stall, suspect MTU: try lowering MSS or setting a smaller MTU on ds-lite. Compare with a dual-stack control host on the same site to separate access issues from AFTR issues.

Compared with 464XLAT and NAT64

DS-Lite tunnels all IPv4 toward a CGNAT. 464XLAT translates on the CPE toward NAT64 and is common on mobile. Stateless softwire approaches (MAP-E/MAP-T) embed addressing math so the AFTR can be partly stateless. Choose based on CPE capability and ops familiarity; all of them still need a story for shared IPv4 abuse logging.

For a lab or small ISP proof-of-concept, get IPv6-only WAN right first, then add a single AFTR and one B4, prove logging round-trip, and only then scale pool size.

Logging fields that satisfy a real complaint

A usable AFTR log line (or flow record) typically includes at least: timestamp (UTC), subscriber outer IPv6 address or delegated prefix, inside IPv4 source, inside port, outside IPv4, outside port, protocol, and NAT event (create/delete). Store retention according to your jurisdiction and policy, and restrict access to the abuse role.

Practice the lookup: take a sample complaint "192.0.2.50:44301 at 2026-11-10 05:00 Asia/Dhaka", convert to UTC, query the log store, and confirm you land on the correct B4 prefix. If the query takes longer than your abuse SLA, fix indexing before you scale the CGNAT pool. Softwires without accountable logs are a liability; softwires with tested logs are an ordinary access technology.

Escape the softwire with native IPv6

If you run services, publish AAAA on aggregatable space you announce yourself. IPYE's free /48 is built for that practice loop: claim at the IPv6 product page, announce on your ASN, and rely on automatic ROA creation while you learn AFTR logging on the access side.

MTU and MSS in one paragraph you can hand to support

IPv4 inside IPv6 adds an outer header, so a 1500-byte LAN IPv4 packet may not fit an access MTU of 1500 on the WAN. Symptoms are classic: ping with small payloads works, TCP handshakes complete, and large HTTP or TLS transfers hang. Fixes that work in practice are lowering the tunnel MTU (for example 1460 or lower depending on outer headers), enabling TCP MSS clamping toward the LAN, and ensuring Path MTU Discovery is not blocked for the outer IPv6 packets. When a ticket says "only big downloads fail on IPv4", check softwire MTU before reimaging the CPE.

Announce native IPv6 on a free /48

the IPYE IPv6 rental page (see About) · ipye@turnel.com

About IPYE

Disclosure: I run IPYE

LACP and Link Aggregation: How 802.3ad Bundles Work, What Hashing Means, and a Linux Bonding Setup

IPYE — Fri, 30 Oct 2026 03:21:02 +0000

Disclosure up front: I run IPYE, a company that leases IP space. It gets one short paragraph at the end; the rest is plain networking.

Two links between a server and a switch should give twice the capacity and survive the loss of either cable. Link aggregation delivers both, but not in the way many people expect: it does not make one connection twice as fast, and it can fail in confusing ways when the two ends disagree about the bundle. LACP (Link Aggregation Control Protocol, from IEEE 802.1AX, formerly 802.3ad) is the standard way to negotiate a bundle so that both ends agree. This article explains what LACP does, how traffic is distributed, a Linux bonding configuration, and how to verify and troubleshoot it. Examples use 192.0.2.10/26 on the bond and 2001:db8:100:10::10/64 for IPv6.

What LACP negotiates

An aggregated link (a LAG, bond, or port-channel) treats several physical ports as one logical interface. Two ways to build it:

Static: both ends are configured to treat the ports as a bundle, with no negotiation. If cabling or configuration is wrong on one side, nothing detects it, and traffic can loop or vanish.
LACP: the ends exchange LACPDUs and only add a port to the bundle if the partner agrees and is the same partner on all the member ports. A port with a mismatch is left out instead of corrupting the bundle.

LACP has active and passive modes. An active end sends LACPDUs; a passive end only replies. At least one side must be active. The LACPDU rate is slow (every 30 seconds) or fast (every second); fast detects failures sooner, though a link failure that the physical layer reports (loss of signal) is noticed immediately in either case. LACP's value is chiefly in catching cabling and configuration mistakes and in detecting a link that is up but not passing frames.

Hashing: how flows pick a link

Frames of one conversation should stay on one link to avoid reordering, so the sender chooses the member port by hashing fields from the packet. Choices on Linux's xmit_hash_policy:

Policy	Fields hashed	When it fits
layer2	Source and destination MAC	Many different peers on the same segment
layer2+3	MAC plus IP addresses	Traffic to many different addresses
layer3+4	IP addresses plus TCP/UDP ports	Many flows between few hosts (best distribution)
encap2+3, encap3+4	Fields of the inner packet of tunnels	VXLAN, GRE and similar traffic

Consequences follow directly. One TCP connection uses one link. A single iperf3 stream across a two-link bundle reaches the speed of one link, not two; run several parallel streams (iperf3 -P 4) to see the aggregate. Hashing is statistical, so with two links you might see an uneven split such as 70/30 by chance, particularly with few flows. And hashing applies only to outgoing traffic from each end: the server decides its transmit distribution, and the switch independently decides what it sends to the server. Configure both sides sensibly.

Linux bonding configuration

With iproute2:

ip link add bond0 type bond mode 802.3ad miimon 100 lacp_rate fast xmit_hash_policy layer3+4
ip link set eth1 down
ip link set eth2 down
ip link set eth1 master bond0
ip link set eth2 master bond0
ip link set bond0 up
ip addr add 192.0.2.10/26 dev bond0
ip -6 addr add 2001:db8:100:10::10/64 dev bond0

Member interfaces must be down when you enslave them. miimon 100 checks link state every 100 milliseconds. The bond takes its MAC from the first member by default, which matters if you have MAC-based filtering. To persist it, use systemd-networkd ([NetDev] Kind=bond with [Bond] Mode=802.3ad), NetworkManager, or your distribution's tools.

The switch side needs a matching LAG in LACP active or passive mode on the two ports, with the same speed and duplex, and the same VLAN configuration for the bundle as a whole. Always apply VLAN and trunk settings to the LAG interface, not to the individual members. Naming and syntax vary by vendor, so use your switch documentation.

Verifying

cat /proc/net/bonding/bond0
ip -d link show bond0

The bonding status lists each member, its link state, and, for 802.3ad, the aggregator ID and the partner's system MAC. Both members should show the same aggregator ID and the same partner MAC. Different partner MACs mean the cables lead to different switches that are not an MLAG pair, and the bundle will use only one of them. Check the switch side with its LAG status: both ports should show as bundled or up in the port-channel.

Test failure behaviour on purpose. Run ping against the host and disconnect one cable (or ip link set eth1 down), then watch loss: with LACP and a healthy bond, the outage should be brief, typically a fraction of a second to a few seconds depending on detection. Restore it and confirm the member rejoins.

Common problems

Both sides passive. No LACPDUs are sent; no bundle forms.
Members to different switches without MLAG. Standard LACP requires all members to terminate on the same logical partner. Multi-chassis designs (MLAG, vPC, or similar) exist for this, with vendor-specific behaviour.
Mismatched speeds or VLANs. A member that differs from the others is excluded.
LAG up but poor distribution. Check the hash policy against your traffic; a pair of servers exchanging one flow cannot balance.
PXE or boot environments. Early boot code may not speak LACP. Some switches provide a fallback to a single active port; learn what yours does.
MTU. Set the bond and members to the same MTU.

Alternatives

Linux also provides active-backup mode (one link in use, the other standby, no switch configuration), which is robust and simple when you want redundancy but not bandwidth. LACP is appropriate when you need both, and when you can configure both sides.

About IPYE

IPYE (https://my.ipye.com) leases IPv4, IPv6, and ASN space; a /48 IPv6 block is free for the first year. LOA, ROA, rDNS, and WHOIS are handled by IPYE. Locations: Los Angeles, New York, Ashburn, Salt Lake City, London, Amsterdam, and Frankfurt. Questions: ipye@turnel.com.

Disclosure: I run IPYE

IS-IS for a Small Network: Why Operators Choose It, Levels and NETs, and an FRR Configuration

IPYE — Fri, 30 Oct 2026 03:20:03 +0000

Disclosure up front: I run IPYE, a company that leases IP space. It gets one short paragraph at the end; the rest is plain networking.

OSPF is the first interior gateway protocol most people learn, but IS-IS is its equal in capability, and many large service providers prefer it. It runs directly over the data-link layer instead of over IP, it carries IPv4 and IPv6 together in one process, and its design handles growth gracefully. A small network benefits from the same properties: one protocol and one adjacency per link for both address families, with no separate "v3" to run. This article explains the vocabulary, how to design levels and addressing, a working FRR configuration, and what to check. It complements the earlier OSPFv3 article and uses 2001:db8:ffff::/64 for infrastructure links, 192.0.2.0/26 for IPv4 loopbacks and links, and private AS 64496 as a reminder that IS-IS is the IGP while BGP stays separate.

How IS-IS differs from OSPF

Aspect	OSPF	IS-IS
Transport	IP protocol 89	Directly on layer 2 (CLNS)
Address families	OSPFv2 for IPv4, OSPFv3 for IPv6	One process, multi-topology or single topology
Area boundary	On the router interface (ABR splits areas)	On the link: routers belong to an area; links are Level 1, Level 2, or both
Router identifier	Router ID	System ID inside a NET
Hierarchy	Backbone area 0 plus others	Level 2 backbone connecting Level 1 areas
Default timers	Hello 10 s	Hello 10 s with hold 30 s (per-implementation)

The practical consequence of running over layer 2 is that IS-IS neighbours need no IP address to form an adjacency, so an IPv6-only link works the same as a dual-stack one. It also cannot be attacked from the internet as an IP protocol, though you should still authenticate it.

Addressing: the NET

IS-IS identifies a router by a NET (Network Entity Title), which looks unusual but is simple:

49.0001.1921.6800.1001.00

Read it as three parts: 49.0001 is the area (49 denotes a private area address, similar to RFC 1918 for NETs; 0001 is the area number), 1921.6800.1001 is the System ID, six bytes that must be unique in the routing domain, and 00 is the selector, always 00 for a router. Many operators derive the System ID from the router's loopback address, so 192.0.2.1 becomes 1920.0000.2001, padded to twelve digits in groups of three, which makes it recognisable in output. Whatever convention you choose, write it down and keep every System ID unique.

Levels in practice

IS-IS has two levels. Level 1 routers know their own area in detail and use the nearest Level 2 router for everything else. Level 2 routers form the backbone between areas. Level 1-2 routers sit on the border. A small network with fewer than a few dozen routers needs no hierarchy: make every router Level 2 only (or every router Level 1 only) and run one flat level. This avoids route leaking decisions and keeps the database simple. Only add areas when size or administrative separation requires it. Mixing levels carelessly, such as one Level 1-2 router and others Level 1 only, creates asymmetric paths and surprises.

An FRR configuration

interface lo
 ip address 192.0.2.1/32
 ipv6 address 2001:db8:ffff:1::1/128
 ip router isis CORE
 ipv6 router isis CORE
 isis passive
!
interface eth1
 description to-r2
 ip address 192.0.2.5/31
 ipv6 address 2001:db8:ffff:12::1/127
 ip router isis CORE
 ipv6 router isis CORE
 isis network point-to-point
 isis circuit-type level-2-only
 isis hello-interval 3
 isis hello-multiplier 3
!
router isis CORE
 net 49.0001.1920.0000.2001.00
 is-type level-2-only
 metric-style wide
 log-adjacency-changes

Notes on the choices. isis network point-to-point avoids electing a designated router on router-to-router links, giving faster adjacency and a smaller database. metric-style wide allows large metrics (the old narrow style limits a link to 63) and is required for modern features. The loopback is passive, advertised but not forming adjacencies. In FRR, interfaces on isisd also require isisd=yes in the daemons file before the process runs. Exact keywords vary slightly by version, so check your documentation, and note that these snippets were not run against production systems.

Metrics and design

Set link metrics deliberately, perhaps inversely proportional to bandwidth or reflecting latency, and keep the same convention everywhere. A flat default metric of 10 on every link makes the protocol count hops, which is rarely the intent. Use the overload bit (set-overload-bit in FRR) to take a router out of transit use before maintenance, a technique that parallels the BGP graceful shutdown article and drains traffic without a hard failure.

Authentication

Enable authentication on IS-IS adjacencies, using HMAC-SHA based keys where your software supports it, or at least a shared password on the interface or domain. It prevents accidental or hostile adjacencies from forming from a misconfigured device on a link.

Verification

show isis neighbor
show isis database
show isis route
show ipv6 route isis

show isis neighbor should list each peer in state Up, with the expected circuit type. show isis database shows each router's link-state packet; every router should appear once. If an adjacency refuses to form, check matching levels, a common area or compatible levels, matching authentication, MTU (IS-IS pads hellos to the interface MTU and will not form across an MTU mismatch), and that both ends enable the protocol on the interface.

When to pick which

IS-IS and OSPF both converge fast and scale well. Choose the one your team can operate confidently and your hardware supports. If you value one process for both address families, a quiet protocol surface, and an area model based on links, IS-IS is a strong option, even for a small network.

About IPYE

Disclosure: I run IPYE

Protecting the Router Itself: Control-Plane Policing and Management-Plane Filters, with a Linux Example

IPYE — Fri, 30 Oct 2026 03:19:05 +0000

Disclosure up front: I run IPYE, a company that leases IP space. It gets one short paragraph at the end; the rest is plain networking.

A router has two jobs that share one CPU: forwarding transit traffic, which hardware or the fast path usually does, and running software that speaks protocols, answers management requests, and makes decisions. That second part, the control plane and management plane, is small, slow, and exposed. Anything addressed to the router itself (BGP packets, SSH, SNMP, ICMP, routing protocol hellos) is handled by software and can be overwhelmed by a flood or a mistake. Control-plane policing (CoPP) and management-plane protection are the practices that keep that part alive under stress. This article explains the concepts and shows a Linux router implementation with nftables. Examples use 192.0.2.1 and 2001:db8:ffff::1 as the router, 192.0.2.2 and 2001:db8:ffff::2 as a BGP neighbour, and 192.0.2.0/27 as the management network.

What reaches the control plane

The router's own CPU handles traffic of several classes, which deserve different treatment:

Class	Examples	Treatment
Routing protocols	BGP (TCP 179), OSPF, IS-IS, BFD	Accept only from known neighbours; never rate-limit hard
Neighbour discovery	ARP, IPv6 ND	Allow; limit only at extreme rates
Management	SSH, SNMP, NTP, syslog	Accept only from the management network
Diagnostic	ICMP echo, traceroute replies	Allow with a rate limit
Everything else addressed to the router	Random scans, floods	Drop, with limited logging

On commercial routers, CoPP applies per-class policers to traffic punted to the CPU, and vendors document how. On a Linux router, the same idea maps to the input hook of the firewall, which sees only traffic destined to the router itself. Transit traffic goes through forward, which is a separate decision.

A Linux input policy

table inet cp {
    set peers4 { type ipv4_addr; elements = { 192.0.2.2 } }
    set peers6 { type ipv6_addr; elements = { 2001:db8:ffff::2 } }
    set mgmt4  { type ipv4_addr; flags interval; elements = { 192.0.2.0/27 } }
    set mgmt6  { type ipv6_addr; flags interval; elements = { 2001:db8:100::/48 } }

    chain input {
        type filter hook input priority 0; policy drop;

        ct state established,related accept
        ct state invalid drop
        iif lo accept

        # Neighbour discovery and path MTU discovery must work
        meta l4proto ipv6-icmp icmpv6 type { nd-router-solicit, nd-router-advert,
            nd-neighbor-solicit, nd-neighbor-advert, packet-too-big,
            destination-unreachable, time-exceeded, parameter-problem } accept
        ip protocol icmp icmp type { destination-unreachable, time-exceeded } accept
        # ARP is filtered in the separate arp family, not in this inet table

        # Routing
        ip saddr @peers4 tcp dport 179 accept
        ip6 saddr @peers6 tcp dport 179 accept
        ip6 saddr fe80::/10 ip6 nexthdr 89 accept
        ip saddr @peers4 udp dport { 3784, 4784 } accept

        # Management
        ip saddr @mgmt4 tcp dport 22 accept
        ip6 saddr @mgmt6 tcp dport 22 accept
        ip saddr @mgmt4 udp dport 161 accept

        # Ping: allowed but rate-limited
        ip protocol icmp icmp type echo-request limit rate 20/second burst 40 packets accept
        meta l4proto ipv6-icmp icmpv6 type echo-request limit rate 20/second burst 40 packets accept

        limit rate 5/minute log prefix "cp-drop: " counter drop
        counter drop
    }
}

Several choices are deliberate. The ICMPv6 types listed are the ones the earlier ICMPv6 reference says must not be blocked. Neighbour discovery is never rate-limited here, because limiting it can break address resolution under load. Routing protocols are matched by source address, and OSPFv3 is accepted only from link-local sources. Echo requests are rate-limited, not dropped, because operators need ping. The final rules log a small sample of drops and count the rest, which gives visibility without letting an attack fill the disk.

Do not forget the other half: protocols that talk back

Adding ct state established,related accept early handles replies to connections the router starts, such as NTP queries and syslog messages. If you run an outbound-only service, no inbound rule is needed.

For BGP, combine the firewall with session-level protections: TCP-AO or MD5 and TTL security (see the earlier article), maximum-prefix limits, and BFD with sensible timers. The firewall limits who can talk to the process; those features limit what a valid-looking neighbour can do.

Management-plane specifics

Separate management from transit. A management VRF or out-of-band network means a transit flood cannot lock you out. The VRF article describes how.
Disable what you do not use. Every listening service is something to patch. ss -tulpn on a Linux router lists them; remove or bind to the management address.
Authenticate and encrypt. SSH with keys, SNMPv3 rather than v1 or v2c, and authenticated NTP where practical.
Log and alert on counter growth of the drop rules.
Rate-limit what an attacker can trigger. Anything that causes work, such as a TCP SYN to an open port, can be limited per source with a meter.

Testing safely

Use a virtual lab, not a live router, to validate rules. Check syntax with nft -c -f file, and apply with a timed rollback. Then confirm what the policy does:

nft list ruleset
nft list chain inet cp input

The counters show which rules matched. Send a controlled flood at the router from a lab host, such as ping -f against a rate-limited rule, and check that BGP stays up during the test. If you cannot show that routing survives, you have not yet protected the control plane. The rules above were written from documentation and not tested against a real router.

Summary

Control-plane protection means identifying what the router must hear, from whom, and at what rate, and dropping everything else early. Most of the benefit comes from the simplest part: refusing management and routing traffic from addresses that should never send it.

About IPYE

Disclosure: I run IPYE

Diagnosing Packet Loss: Telling Link Errors, Queue Drops, and Rate Limiting Apart

IPYE — Fri, 30 Oct 2026 03:17:59 +0000

Disclosure up front: I run IPYE, a company that leases IP space. It gets one short paragraph at the end; the rest is plain networking.

"We are seeing packet loss" is a report, not a diagnosis. Loss has several distinct causes (corrupted frames on a bad cable, a full queue, a policer, an overloaded CPU, a failing path further away), and each needs a different fix. The most valuable first step is to answer one question: where are packets disappearing, and what kind of disappearance is it? This article gives a method that works with common tools on Linux hosts and routers. It builds on the earlier article about reading traceroute and mtr, and uses documentation addresses throughout: 192.0.2.1 as a gateway, 198.51.100.10 as a remote target, and 2001:db8:200::10 as an IPv6 target.

Step 1: Measure it properly

A handful of pings is anecdote. Run enough probes to see a rate, in both directions and with more than one packet size:

ping -c 200 -i 0.2 198.51.100.10
ping -c 200 -i 0.2 -s 1400 198.51.100.10
mtr -rwzc 200 198.51.100.10

Compare small and large packets. If only large ones are lost, suspect MTU or fragmentation problems rather than congestion. Run the same test over IPv6 with ping -6 or mtr -6, because a loss problem that appears in only one family points to a path or policy difference. Also note the time: loss that appears only at specific hours suggests congestion, while constant loss suggests a fault.

Step 2: Read the mtr output with care

In mtr output, loss shown at an intermediate hop only matters if it persists to the following hops. Many routers rate-limit replies to ICMP probes aimed at themselves while forwarding transit traffic perfectly, so a hop showing 40% loss with 0% at the final hop is almost always control-plane rate limiting and not real loss. Real loss on a forwarding path shows up at that hop and every hop after it, including the destination. The earlier traceroute article describes the reasoning; the rule is to trust the last hop and the pattern of continuity.

Step 3: Check your own host first

Before blaming the network, look at the local interface and the kernel:

ip -s link show eth0
ethtool -S eth0 | grep -Ei 'err|drop|miss|fifo|crc'
nstat -az | grep -Ei 'drop|retrans|overflow'

What the counters mean:

Counter family	Typical meaning
CRC / frame errors, alignment errors	Physical layer: bad cable, connector, optic, or duplex mismatch
RX missed / fifo / no buffer	The NIC or driver could not keep up; ring buffers too small or CPU overloaded
TX dropped / qdisc drops	The output queue overflowed: more traffic than the link can send
Softnet drops	The kernel's per-CPU backlog overflowed; see `/proc/net/softnet_stat`
Socket buffer overruns	The application could not read fast enough

Increasing counters during a test point at the culprit. A clean set of counters on both ends moves suspicion to the path between them.

Step 4: Distinguish congestion from errors

Corrupted frames and congestion drops look similar from the application, but their signatures differ:

Errors (CRC) grow steadily regardless of load and are in the interface counters of one specific link, usually at one end of a cable or optic. Clean, reseat, or replace, and check optical power levels with ethtool -m eth0 where the module reports them.
Congestion grows with load, appears at times of peak traffic, and shows as queue drops on a specific egress interface. Check utilisation, the queue discipline (tc -s qdisc show dev eth0), and whether bursts exceed the buffer. The earlier article on traffic control describes fq_codel and shaping options.
Policing shows a hard ceiling: throughput plateaus at an exact rate and drops appear even when the link is not full. Policers appear at provider edges, cloud virtual NICs, and misconfigured shapers.

A stepped test helps. Send a fixed-rate UDP stream with iperf3 -u -b 50M and increase the rate; loss that begins abruptly at a specific bandwidth indicates a policer or a bottleneck at that rate. The iperf3 methodology article covers how to do this without misleading yourself.

Step 5: Localise along the path

If your host is clean and loss persists:

Capture on both ends and compare counts. tcpdump -n -i eth0 icmp and host 198.51.100.10 on source and destination shows exactly which packets disappeared and, with timestamps, whether they were delayed or lost.
Test to intermediate points you control. Loss between A and B but not A and C narrows the segment.
Ask neighbours and upstreams to check interface counters on the specific link, giving them timestamps, addresses, and the measured rate. A report with a time window and mtr evidence gets resolved faster than "the network is slow".
Use independent vantage points, such as the probes described in the active measurement article, to see whether other networks experience the same loss.

Step 6: Rule out the usual local culprits

Firewall or conntrack limits. dmesg | grep conntrack and nstat may show table-full drops. The conntrack article explains what to adjust.
Reverse path filtering. A strict rp_filter on a multihomed host silently discards asymmetric traffic.
Duplex or speed mismatch. ethtool eth0 shows negotiated speed and duplex; unexpected 100 Mb/s links are common with a bad cable.
CPU saturation. top or mpstat -P ALL 1 shows whether one core handling interrupts is pegged at 100%.

Record what you found

Write down the test method, the packet size, the time, the rate of loss, the interface counters before and after, and the hop where the pattern begins. That record lets you or a colleague repeat the test after a fix, and it is the evidence a carrier will ask for.

The pattern to remember: measure, check your own host, classify the loss as error, congestion, or policing, and then narrow the path. Most problems reveal themselves by step four.

About IPYE

Disclosure: I run IPYE

tcpdump and Wireshark Filters for IPv6: Practical Recipes and the Extension Header Caveat

IPYE — Mon, 26 Oct 2026 19:49:31 +0000

Disclosure up front: I run IPYE, a company that leases IP space. It gets one short paragraph at the end; the rest is plain networking.

When an IPv6 problem is not obvious from logs, a packet capture usually settles it: is the Router Advertisement arriving, is the Neighbor Solicitation answered, does the handshake complete, is something sending ICMPv6 errors? The IPv6 filter syntax has a few differences from IPv4 that cause silent mistakes. This article gives recipes for tcpdump (which uses the libpcap filter language) and for Wireshark display filters, and explains the limitation that catches most people. Examples use 2001:db8:100::25, 2001:db8:100::/48, and eth0.

Two different filter languages

Both tools have filters, and they are not the same:

A capture filter (tcpdump's expression, or Wireshark's capture filter box) uses BPF syntax and decides what is recorded at all.
A display filter (Wireshark) works on dissected packets and decides what is shown from a capture.

Capture filters are efficient but crude. Display filters are expressive but need the packets to have been captured first. A typical workflow is a broad capture filter, then a display filter in Wireshark to narrow things down. Quote tcpdump expressions in the shell, because (, ), and ! have meaning there.

Basic tcpdump options

tcpdump -n -i eth0 -vv 'ip6'

-n stops name resolution, which avoids delays and extra DNS queries that pollute the capture.
-i any captures on all interfaces; it uses a different link-layer header, but ip6 filters still work.
-vv prints more detail, including hop limit.
-w file.pcap writes a capture file for Wireshark.

Addresses and prefixes

host 2001:db8:100::25
src host 2001:db8:100::25
net 2001:db8:100::/48
ip6 and not host 2001:db8:100::25

host matches either direction, net takes a prefix, and the ip6 primitive restricts the filter to IPv6.

Protocols and ports

tcp port 443, udp port 53, and icmp6 all work for IPv6. icmp6 is a shorthand for IPv6 packets with next header 58. Port filters apply to both IPv4 and IPv6 unless you add ip6 or ip, so write ip6 and tcp port 443 when you want IPv6 only.

The extension header caveat

This is the limitation that matters. In libpcap's filter language, ip6 proto 6 and similar expressions look only at the Next Header field in the fixed IPv6 header. If a packet has a Hop-by-Hop, Routing, or Fragment extension header, the next header is that extension, so a filter for TCP will not match it. Similarly, filters for ports or ICMPv6 types read at fixed offsets and assume the upper-layer header follows the 40-byte fixed header immediately. For ordinary traffic this works. For packets with extension headers it can silently miss. When a capture seems to be missing traffic you know exists, widen the filter, or capture everything for a short period and filter in Wireshark, whose dissector follows the extension header chain.

There is also a primitive for chains: ip6 protochain 44 matches packets that have a Fragment header anywhere in the chain.

ICMPv6 and neighbor discovery

The ICMPv6 type is the first byte after the 40-byte header, so with the no-extension-header assumption you can match it at offset 40:

Goal	Filter
Router Solicitation (type 133)	`icmp6 and ip6[40] == 133`
Router Advertisement (134)	`icmp6 and ip6[40] == 134`
Neighbor Solicitation (135)	`icmp6 and ip6[40] == 135`
Neighbor Advertisement (136)	`icmp6 and ip6[40] == 136`
All neighbor discovery (133 to 136)	`icmp6 and ip6[40] >= 133 and ip6[40] <= 136`
Packet Too Big (type 2)	`icmp6 and ip6[40] == 2`
Echo request and reply (128, 129)	`icmp6 and (ip6[40] == 128 or ip6[40] == 129)`

Neighbor discovery messages must be sent with a hop limit of 255, and receivers reject anything else, which stops off-link senders from spoofing them. The hop limit is the byte at offset 7, so icmp6 and ip6[7] == 255 and ip6[40] >= 133 and ip6[40] <= 136 shows only well-formed discovery packets, and a capture with ip6[7] != 255 on the same types shows suspicious ones. For multicast, ip6 multicast matches any IPv6 multicast destination.

Useful checks with these:

Is SLAAC working? Capture type 134 on a client interface. If none arrive, the problem is upstream of the host: router, switch, or MLD snooping.
Is a neighbour answering? Capture types 135 and 136 for the target address and look for the advertisement.
Path MTU trouble? Capture type 2 to see whether Packet Too Big reaches the sender. Absence in the face of black-holed large packets suggests filtering, as described in the earlier MTU and ICMPv6 articles.

TCP handshake and flags

For a SYN on IPv6 without extension headers, the TCP flags byte is at offset 53 (40 plus 13 in the TCP header):

ip6 and tcp and (ip6[53] & 0x02 != 0)

Behaviour of protocol-relative expressions such as tcp[13] with IPv6 has varied between libpcap versions, so the explicit offset is more predictable, and you should verify any filter against known traffic first.

Fragments

ip6 protochain 44 matches fragmented packets. IPv6 fragmentation is done only by the source, so fragments you see came from the endpoint or from a tunnel entry point, and a flow of fragments often points to an MTU problem.

Wireshark display filters

Display filters use field names:

ipv6.addr == 2001:db8:100::25, ipv6.src, and ipv6.dst for addresses.
icmpv6 for all ICMPv6, and icmpv6.type == 135 for a specific type.
ipv6.hlim == 255 or ipv6.hlim < 64 for hop limit.
ipv6.nxt == 44 for packets whose fixed header points to a fragment header.
tcp.flags.syn == 1 && tcp.flags.ack == 0 for new connections, regardless of address family.
ipv6 && !icmpv6 to remove ICMPv6 noise while looking at application traffic.

Summary

Use capture filters to limit what you record and display filters to explore it. Remember that libpcap IPv6 offsets assume no extension headers, check the hop limit of 255 for neighbor discovery, and test a new filter against traffic you know exists before trusting an empty result.

About IPYE

Disclosure: I run IPYE

IXP Basics for Small Networks: Route Servers, Peering Policies, and PeeringDB

IPYE — Mon, 26 Oct 2026 19:46:16 +0000

Disclosure up front: I run IPYE, a company that leases IP space. It gets one short paragraph at the end; the rest is plain networking.

An Internet Exchange Point (IXP) is a shared layer 2 network where many networks connect so they can exchange traffic directly instead of through a transit provider. For a small network the attraction is shorter paths and less transit traffic; the price is operational care, because a mistake on a shared LAN affects other members. This article explains how an exchange works, the difference between bilateral peering and route servers, what PeeringDB is for, and the safety habits that make you a good neighbour. Examples use AS64496 as you, AS64497 as a peer, and 2001:db8:ffff::/64 standing in for an IXP peering LAN.

How an IXP works

Members connect a port from their router to the exchange's switching fabric and receive an address from the IXP's peering LAN, in both IPv4 and IPv6. They then run eBGP sessions with each other across the fabric. The exchange only forwards Ethernet frames; it does not route, and it does not see your BGP sessions as anything but traffic. Exchanges publish member rules, which commonly include:

One MAC address per port, and no proxy ARP or similar tricks.
Only IPv4, ARP, and IPv6 frames on the peering LAN, with no discovery protocols that leak onto the fabric, such as spanning tree or vendor neighbour-discovery protocols.
Do not announce the peering LAN prefix into global BGP, and do not point a default route at another member.

The last rule matters: a default route toward a peer uses that peer's network as transit without asking. Send across an IXP only traffic for prefixes the peer announced to you.

Bilateral sessions and route servers

There are two ways to learn routes at an exchange.

Approach	How it works	Strengths	Limits
Bilateral	You configure a separate eBGP session with each peer	Full control of policy and per-peer limits	One agreement and one session per peer
Route server	You peer with the IXP's route server, which redistributes members' routes	One or two sessions give many peers	You accept what members announce, filtered by the server

A route server (RFC 7947) is a BGP speaker that passes routes between members without adding its own AS number to the AS_PATH, and without changing the next hop, so traffic still flows directly between the members. Route servers are an efficient way for a small network to reach many peers quickly. Many networks use both: route servers for the long tail and bilateral sessions with the largest partners.

Peering policies and PeeringDB

Every network chooses whom it will peer with. Policies are often summarised as open, selective, or restrictive, which are the values the PeeringDB policy field uses. Open means the network will usually agree to peer with anyone who asks. Selective means it applies conditions, such as a minimum traffic level or a presence in specific locations. Restrictive means peering only in particular circumstances.

PeeringDB is a community-maintained database where networks, exchanges, and facilities publish their details. A network record typically includes its ASN, the AS-SET used for filtering, expected prefix counts for IPv4 and IPv6, its policy, contact addresses, and the exchanges where it has ports. Use it two ways:

As a requester: look up a prospective peer, read its policy, and send a short message to its listed peering contact with your ASN, the exchange, and the traffic you expect.
As an operator: keep your own record accurate. Many networks build filters and maximum-prefix limits from the listed AS-SET and prefix counts, and many refuse to peer with networks that have no usable record.

Treat the data as a starting point and check it against the IRR and RPKI.

What to configure on each session

The baseline filters for any peer, whether bilateral or via a route server, follow RFC 7454:

Prefix filters: accept only the prefixes the peer is entitled to announce, built from its AS-SET, or rely on the route server's filtering and still apply your own checks.
No bogons: reject private, documentation, and reserved address space and ASNs.
RPKI: reject routes that validate as invalid.
Maximum prefixes: set a limit from the peer's expected count, with a margin, so a leak ends the session instead of filling your table.
Next hop check: accept routes from a peer only if the next hop is that peer's address on the exchange, unless you are using a route server.
First AS check: for a bilateral session, require the peer's own ASN to be first in the AS_PATH. Do not enable it on route server sessions, since the route server does not insert its own ASN.
Outbound: announce only your own prefixes and your customers' prefixes, never routes learned from other peers or transit, or you turn yourself into a free transit path.

A sketch in FRR for a bilateral IPv6 peer:

router bgp 64496
 neighbor 2001:db8:ffff::2 remote-as 64497
 neighbor 2001:db8:ffff::2 description AS64497 at IXP
 !
 address-family ipv6 unicast
  neighbor 2001:db8:ffff::2 activate
  neighbor 2001:db8:ffff::2 maximum-prefix 100
  neighbor 2001:db8:ffff::2 prefix-list PEER-AS64497-IN in
  neighbor 2001:db8:ffff::2 route-map PEER-OUT out
 exit-address-family

The names and the limit of 100 are placeholders. Your prefix-list and route-map must still be written, as discussed in the earlier customer filtering and communities articles.

Local preference and traffic engineering

Operators commonly give peer routes a higher local preference than transit routes, which is a policy choice, not a protocol rule. Tag routes with communities on receipt, so you can recognise their origin and avoid re-exporting them.

Is it worth it?

An exchange port has a cost, and you need a presence or a remote connection. Before joining, use flow data, as described in the NetFlow article, to estimate how much of your traffic goes to networks present there.

Summary

An IXP is a shared Ethernet fabric over which you run BGP with other members. Use route servers for breadth and bilateral sessions where you need control, keep your PeeringDB record accurate, apply the baseline filters to every session, and announce only what you are entitled to announce.

About IPYE

Disclosure: I run IPYE

NetFlow, sFlow, and IPFIX for a Small ISP: Choosing a Flow Export and Reading It

IPYE — Mon, 26 Oct 2026 19:43:25 +0000

Disclosure up front: I run IPYE, a company that leases IP space. It gets one short paragraph at the end; the rest is plain networking.

Interface counters tell you how much traffic crossed a link. They do not tell you which prefix, port, or customer produced it. Flow telemetry fills that gap: the router, switch, or a Linux host summarises traffic as flows and exports records to a collector. For a small ISP or hosting network it answers the practical questions of who is using the bandwidth, what an attack looked like, and where capacity is going. This article compares the main formats, shows a small Linux setup, and covers IPv6-specific points. Examples use 2001:db8:100::/48 and 203.0.113.0/24.

What a flow is

A flow is a set of packets sharing key fields, typically the source and destination address, protocol, and source and destination ports, observed within a time window. The exporter keeps a cache of active flows and sends a record when a flow ends or after a timeout. Each record holds the key fields plus counters for packets and bytes, timestamps, interfaces, and often TCP flags, AS numbers, and next hop. This is metadata; there is no payload.

The formats

Format	Model	Notes
NetFlow v5	Fixed-format records	IPv4 only, so it cannot describe IPv6 traffic
NetFlow v9 (RFC 3954)	Template-based records	Supports IPv6 and flexible fields
IPFIX (RFC 7011)	Template-based, IETF standard, derived from NetFlow v9	Registered fields; the vendor-neutral choice
sFlow	Samples packets and interface counters	Does not keep a flow cache; the collector builds flows from samples

The IPv6 point matters: if your exporter only speaks NetFlow v5, IPv6 traffic will be missing from your picture. Use v9, IPFIX, or sFlow. Template-based formats first send a template that describes the record layout, followed by data records, so collectors must receive the templates; a collector restarted mid-stream may discard records until the next template arrives.

By convention NetFlow is often sent to UDP port 2055, though this is not a standard for NetFlow. IANA registers port 4739 for IPFIX and 6343 for sFlow.

Sampling and what it costs

Exporting every flow is expensive on busy links, so most devices sample, examining one packet in N. The collector then multiplies byte and packet counts by the sampling rate to estimate totals. Consequences:

Large flows are estimated well. Small and short flows may be missed completely.
A sampled record is evidence that traffic existed, not an exact count.
Mixing devices with different sampling rates without recording the rate gives wrong totals.

Record the sampling rate in your documentation and make sure your collector uses it. sFlow is built around this idea, since its samples are packet headers taken at a defined rate, while NetFlow and IPFIX exporters can run either sampled or unsampled, depending on the hardware.

A small Linux setup

On a Linux router without hardware flow export, a software exporter can read traffic from an interface. One option is softflowd, which can export NetFlow v9 or IPFIX, and another is pmacctd. For sFlow on a host, hsflowd is common. A collector can be nfdump with nfcapd, or pmacct's nfacctd.

A sketch with softflowd and nfdump:

softflowd -i eth0 -n 127.0.0.1:2055 -v 9
nfcapd -l /var/cache/nfdump -p 2055 -D
nfdump -R /var/cache/nfdump -s srcip/bytes -n 10

The first command exports flows from eth0 as NetFlow v9; the second receives and stores them; the third prints the top ten source addresses by bytes. Add a filter for the IPv6 family using the syntax your nfdump version documents. Option names differ between tools and versions, so check the manual pages. Keep the collector on a separate machine or at least a separate volume for any real network, because flow storage grows steadily.

IPv6 considerations

Aggregate by /64. As in the logging article, one subscriber can use many addresses within a /64. Ranking individual IPv6 addresses by bytes will mislead you, so most tools let you group by prefix length. Use /64, and widen to /48 for delegations.
Extension headers. Some exporters classify packets with extension headers poorly, so ports may be zero.
Dual-stack views. Report IPv4 and IPv6 separately before combining them.
Fragments. Non-first fragments have no port information, so they appear as port zero or in separate flows.

Useful questions to ask the data

Top talkers: which destination prefixes or customers use the most bytes in the last hour.
Attack triage: packets per second by destination address and protocol, to see whether a spike is a flood aimed at one host.
Capacity planning: daily peaks by upstream, peer, or customer.
Peering decisions: which origin ASes carry the most traffic, if your records include AS numbers, to judge whether a direct peering would be worthwhile.
Abuse and complaints: whether a host did contact a destination at a stated time, with timestamps that you should verify against NTP-synced clocks.

Operational notes

Make sure router and collector clocks are synchronised. Choose flow timeouts deliberately, since active timeouts break long flows into several records and short inactive timeouts split idle connections. Flow records are personal data in some jurisdictions, so set retention, and restrict access. Finally, protect the collector: it receives unauthenticated UDP, so accept exports only from known router addresses.

Summary

Use NetFlow v9, IPFIX, or sFlow, because NetFlow v5 cannot carry IPv6. Know your sampling rate and apply it, aggregate IPv6 by /64, keep clocks and timeouts sane, and treat flow data as an estimate that answers "who and how much" rather than an exact bill of every packet.

About IPYE

Disclosure: I run IPYE

Rate Limiting and Banning IPv6 Clients by /64 with nftables

IPYE — Mon, 26 Oct 2026 19:00:52 +0000

Disclosure up front: I run IPYE, a company that leases IP space. It gets one short paragraph at the end; the rest is plain networking.

Rate limits and ban lists assume that an address stands for a client. In IPv4 that is a decent approximation. In IPv6 it fails: a single customer with a /64 has 2^64 addresses available, and a script that picks a fresh source address for every request will sail past any per-address limit and fill your ban list with entries that never match again. The fix is to aggregate: limit and ban on a prefix, usually the /64. This article shows how to do that with nftables and what to watch out for with fail2ban. Examples use 2001:db8:100::/48 and 203.0.113.0/24; test every rule on a non-production machine, and check syntax against your nftables version.

Choosing the unit

The reasoning is covered in the article on IPv6 logging and abuse handling: a /64 is a normal subnet, and the host inside it can use any address in it. So the default unit for IPv6 limits is the /64. For IPv4 the equivalent is the single address (or a /32), which is a poor stand-in for a person behind carrier-grade NAT, but that is a separate issue.

Some cautions apply before you start:

A /64 can be shared between many users on some networks, such as public Wi-Fi or mobile networks. A strict limit on a shared /64 hurts innocent users. Use limits that suit your service, and treat bans as temporary.
Some customers receive more than a /64 (a /56 or /48), so a determined client can hop between /64s. Widen the unit when evidence supports it.

Per-/64 rate limiting with a meter

nftables can limit using a dynamic set keyed on a masked address. The following limits new TCP connections to port 443, counting all addresses within the same /64 together:

table inet filter {
  chain input {
    type filter hook input priority 0; policy drop;

    ct state established,related accept
    iif lo accept
    meta l4proto ipv6-icmp accept

    tcp dport 443 ct state new \
      meter flood6 { ip6 saddr & ffff:ffff:ffff:ffff:: \
                     timeout 5m limit rate over 20/second burst 40 packets } \
      counter drop

    tcp dport 443 accept
  }
}

Here ip6 saddr & ffff:ffff:ffff:ffff:: masks the source address to its top 64 bits, so every address in the /64 maps to the same meter entry. The limit rate over clause matches packets above the rate, and these are dropped. The timeout removes idle entries so the meter does not grow without bound. Meter syntax has changed between nftables releases, so check nft -c -f file.nft for validity and the wiki for your version. Add an equivalent rule for IPv4 with ip saddr, whose natural unit is the single address.

Keep ICMPv6 accepted ahead of any rate rule, as the earlier article on ICMPv6 explains: dropping neighbour discovery or Packet Too Big messages causes outages that look like unrelated problems. Rate-limit ICMPv6 separately and generously if you need to, instead of blocking it.

A ban list of prefixes

For persistent offenders, keep a set of prefixes. An interval set holds ranges, and timeout lets entries expire:

table inet filter {
  set banned6 {
    type ipv6_addr
    flags interval, timeout
  }
  chain input {
    type filter hook input priority -10; policy accept;
    ip6 saddr @banned6 drop
  }
}

Add and remove entries at runtime:

nft add element inet filter banned6 { 2001:db8:100:1::/64 timeout 1h }
nft delete element inet filter banned6 { 2001:db8:100:1::/64 }
nft list set inet filter banned6

An entry is one prefix, regardless of how many addresses inside it were used, so the list stays small. Always set a timeout: prefixes get reassigned, and a permanent ban can end up blocking an unrelated user months later. Review hits with a counter on the rule.

fail2ban and IPv6

fail2ban watches logs and bans the addresses it sees. Current versions support IPv6, and ban actions exist for nftables. Two points need attention:

It bans individual addresses by default. An attacker rotating inside a /64 will not be stopped by a ban on one address. Options are to use an action that adds the whole /64 to a prefix set, which means a custom action or filter configuration, or to make rate limiting by /64 in nftables your primary defence and use fail2ban for slower, credential-based abuse such as SSH.
Matching log lines. The filter must capture IPv6 addresses in the format your service logs. Make sure normalisation and the form of the address are what the regex expects. Test with fail2ban-regex against real log lines containing IPv6 addresses.

Remember ignoreip: put your own management addresses and prefixes in the ignore list, including IPv6 ones, so a typo does not lock you out.

Application-level limits

Reverse proxies and web servers have their own limits, keyed by whatever variable you choose. By default, nginx's limit_req_zone keys on the client address, and this is the per-address problem again. Where the proxy cannot aggregate easily, apply the limit in the firewall in front of it. If the proxy sits behind a load balancer, use the real client address, as the article on the PROXY protocol explains, or all limits would apply to the balancer.

Testing

nft -c -f /etc/nftables.conf     # check syntax without applying
nft list meter inet filter flood6   # inspect the entries (name may differ by version)

Generate traffic from several source addresses in a test /64 and confirm they share a counter, and that traffic from a different /64 does not. Then check that a normal client, with a few connections, is never limited.

Summary

Aggregate on the /64, expire every ban, keep ICMPv6 flowing, and test from a real IPv6 host. A limit that counts addresses in IPv6 is a limit that attackers can ignore with no effort.

About IPYE

Disclosure: I run IPYE