Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Networking (Crossbow)

illumos does not bolt virtual networking on as an afterthought — it ships an integrated network-virtualization stack in the kernel called Crossbow. Crossbow turns the network stack into something you carve up the way ZFS carves up storage: virtual NICs, virtual switches, and bandwidth/flow controls, all first-class and all administered with a small set of commands. It is the plumbing under per-zone networking.

On illumos generally Crossbow has been part of the platform since OpenSolaris (2009) and is mature and load-bearing — every zone’s network comes through it. On solnix Crossbow is inherited from the illumos gate; the dladm/flowadm/ipadm commands below are the genuine illumos commands and work on any illumos host. What is not yet built on solnix is a declarative solnix.networking.* Nix module that renders these from config — that is a roadmap item, like the SMF module. Today you configure networking imperatively with the commands below, the same as on any illumos system.

For the Linux admin: what Crossbow replaces

If you came from Linux, you assembled virtual networking from several independent tools. Crossbow unifies that same functionality — and it did so years earlier:

You’d reach for (Linux)On illumos (Crossbow)
ip link add … type veth / tapdladm create-vnic (a virtual NIC)
brctl / ip link add … type bridgedladm create-etherstub (a virtual switch)
ip link set … master br0VNICs created over an etherstub share it
network namespaces (ip netns)zones get their own VNICs
tc (traffic shaping / rate limits)dladm set-linkprop maxbw / flowadm
ip addr / ip routeipadm (IP addresses and interface config)
VLAN subinterfaces (ip link … type vlan)dladm create-vlan
bonding / LACPdladm create-aggr (link aggregation)

The mental model is close, but Crossbow is one coherent stack rather than a half-dozen loosely related tools, and the datalink/IP split (dladm for links, ipadm for IP) is cleaner than Linux’s overloaded ip.

The three command families

illumos splits network administration by layer:

  • dladm — the datalink layer (layer 2): physical NICs, VNICs, etherstubs, VLANs, aggregations, and per-link properties (MTU, MAC, bandwidth caps).
  • ipadm — the IP layer (layer 3): addresses, interface configuration, and tunables. (This replaced the old ifconfig.)
  • flowadmflows: policy on classified traffic (by port, protocol, address), including bandwidth limits and priorities finer-grained than a whole-link cap.

List every datalink and its state:

dladm show-link
dladm show-phys          # physical NICs only
dladm show-vnic          # virtual NICs only

Create a VNIC over a physical NIC (this is the workhorse — every zone gets one):

pfexec dladm create-vnic -l net0 vnic0

Create an etherstub (a virtual switch with no physical uplink) and hang VNICs off it — this is how you build an isolated internal network between zones, with no external connectivity unless you add it:

pfexec dladm create-etherstub stub0
pfexec dladm create-vnic -l stub0 vnic_a
pfexec dladm create-vnic -l stub0 vnic_b
# vnic_a and vnic_b can now talk to each other, and nothing else

Set a bandwidth cap on a link (the simple, whole-link rate limit):

pfexec dladm set-linkprop -p maxbw=100M vnic0    # cap vnic0 at 100 Mbit/s
dladm show-linkprop -p maxbw vnic0               # check it

Other common link operations:

pfexec dladm create-vlan -l net0 -v 42 vlan42    # VLAN tag 42 on net0
pfexec dladm create-aggr -l net0 -l net1 aggr0   # LACP aggregation of two NICs
pfexec dladm set-linkprop -p mtu=9000 net0       # jumbo frames
pfexec dladm delete-vnic vnic0                   # tear a VNIC down

ipadm — IP configuration

Once a datalink exists (physical or virtual), ipadm gives it an IP:

ipadm show-if                                    # list IP interfaces
pfexec ipadm create-if vnic0                     # bring vnic0 up as an IP if
pfexec ipadm create-addr -T static \
  -a 192.0.2.10/24 vnic0/v4                       # static IPv4 address
pfexec ipadm create-addr -T dhcp vnic0/dhcp      # or DHCP
ipadm show-addr                                  # list all addresses
pfexec ipadm set-prop -p forwarding=on ipv4      # enable IP forwarding

flowadm — flows and finer bandwidth control

A flow classifies a subset of a link’s traffic and applies policy to it — more surgical than the whole-link maxbw:

# Cap only HTTPS traffic on vnic0 to 50 Mbit/s
pfexec flowadm add-flow -l vnic0 \
  -a transport=tcp,local_port=443 -p maxbw=50M https-flow
flowadm show-flow                                # list flows
flowadm show-flowprop https-flow                 # its properties
pfexec flowadm remove-flow https-flow            # remove it

Flows are how you say “throttle backups but not interactive traffic” or “give this service priority” without touching the rest of the link.

How Crossbow underpins zones

Crossbow is why zones get real, isolated network stacks rather than shared plumbing. The usual pattern:

  1. Create a VNIC per zone (dladm create-vnic -l net0 zoneA0).
  2. Assign that VNIC to the zone in its zonecfg (an anet/net resource).
  3. Inside the zone, that VNIC looks like a dedicated physical NIC — the zone has its own IP, routing, and firewall (ipfilter), fully isolated from the global zone and other zones.

Put the zones’ VNICs on an etherstub instead of the physical NIC and you have a private virtual network between zones with no external exposure — the illumos equivalent of a Docker bridge network or a set of namespaces on a veth bridge, but built into the OS and older than either.

Firewalling

Packet filtering is a separate subsystem, ipfilter (ipf/ipnat), configurable globally and per zone. It is the illumos analog of iptables/nftables. Crossbow shapes and switches; ipfilter filters and NATs.

solnix status, honestly

  • The Crossbow stack itself is real and inherited from the gate — it works on any illumos host today.
  • Declarative solnix networking is not built yet. There is no solnix.networking.* module rendering dladm/ipadm/flowadm state from Nix; that waits on a bootable solnix system (see How solnix is built) and follows the same pattern as the planned SMF module. Until then, configure networking with the commands above.

Further reading