This describes how to connect an Oxide rack to additional upstream physical networks through the API. It assumes that the rack is already accessible through a primary network, and the goal is to connect the rack to additional physical networks.
The example in this guide will walk through taking a rack that is only connected via a single switch port on a single switch, to a rack that is connected via a switch port on each switch. The diagram below shows two oxide switches connected to an upstream router. The new configuration added in this example is highlighted in yellow.
Initial Configuration
This guide will use the oxide CLI. To get started we’re going to look at the
current network configuration of the rack.
$ oxide system networking switch-port-settings show switch0/qsfp0 ============= Autoneg Fec Speed false None Speed100G Address Lot VLAN 198.51.101.2/30 initial-infra None BGP Peer Config Export Import Communities Connect Retry Delay Open Enforce First AS Hold Time Idle Hold Time Keepalive Local Pref Md5 Auth Min TTL MED Remote ASN VLAN Destination Nexthop Vlan Preference 0.0.0.0/0 198.51.101.1/32 0 0
This shows that we only have one port connected on one switch.
Expanding Connectivity
Expanding connectivity in this example involves creating a new switch port
settings object and associating that settings object with a switch port. In this
example we’ll be configuring port qsfp0 on switch1.
Before configuring network settings, we’ll need to grab the ID of the rack we’ll be working with. In this example we assume a single rack deployment for simplicity and just grab the first rack ID.
export rack=`oxide system hardware rack list | jq -r .[0].id` echo $rack
The first thing to do is create a link. This will create a settings object
called switch1-qsfp0 that we’ll apply in the next step.
$ oxide system networking link add \ --rack $rack \ --switch switch1 \ --port qsfp0 \ --fec rs \ --speed 100g
oxide
system networking link add --help to see if any are important for your
deployment.Next, associate the link with a physical port.
$ oxide system hardware switch-port apply-settings \ --rack-id $rack \ --switch-location switch1 \ --port qsfp0 \ --port-settings switch1-qsfp0
Next, add an address to this switch port settings object.
$ oxide system networking addr add \ --rack $rack \ --switch switch1 \ --port qsfp0 \ --addr 198.51.101.6/30 \ --lot initial-infra
Finally, we’ll set up a static default route on the new link. This is just an example and setting more specific routes may be more applicable in other situations.
$ oxide system networking route set \ --rack $rack \ --switch switch1 \ --port qsfp0 \ --destination 0.0.0.0/0 \ --nexthop 198.51.101.5
Now we can run the switch-port-settings show command from the beginning of the
guide and see the new port configuration.
$ oxide system networking switch-port-settings show switch1/qsfp0 ============= Autoneg Fec Speed false Rs Speed100G Address Lot VLAN 198.51.101.6/30 initial-infra None BGP Peer Config Export Import Communities Connect Retry Delay Open Enforce First AS Hold Time Idle Hold Time Keepalive Local Pref Md5 Auth Min TTL MED Remote ASN VLAN Destination Nexthop Vlan Preference 0.0.0.0/0 198.51.101.5/32 0 0 switch0/qsfp0 ============= Autoneg Fec Speed false None Speed100G Address Lot VLAN 198.51.101.2/30 initial-infra None BGP Peer Config Export Import Communities Connect Retry Delay Open Enforce First AS Hold Time Idle Hold Time Keepalive Local Pref Md5 Auth Min TTL MED Remote ASN VLAN Destination Nexthop Vlan Preference 0.0.0.0/0 198.51.101.1/32 0 0
oxide
system hardware switch-port show-status.