1. Getting Started
In the first phase of the rack configuration, the Rack Setup Service (RSS) is performed interactively via an Oxide-provided laptop connecting to the “Technician Port” of the rack, via a jumpbox in most cases. With the assistance of Oxide technicians, you will locate the technician port IP addresses and interface names to both of the rack switches.
First, ssh into the wicket
captive shell against one of the Switch 0 tech ports:
ssh wicket@${IP_ADDRESS}%${PORT_NAME}
This should give you an Oxide splash screen and land you in wicket showing a graphical display of the rack.
Since the rack does not have network access at this point in the process, the RSS covers only the minimum configuration necessary to:
Validate component end-to-end connectivity
Update rack software to the latest versions located in the technician laptop
Upload an SSL certificate
Set up recovery account credentials
Configure basic networking such as upstream DNS, NTP, VLAN and routing information
2. Validate component connectivity
The RSS will communicate with the Management Gateway Service (MGS) to retrieve information about sled position and identity from the Service Processors (SP) in the Gimlet on each sled.
To view the sled and switch information:
On the left pane of the wicket UI, you can use the up and down arrows (or
j
/k
a la vim) to select a screen.Select
OVERVIEW
and pressTab
to move focus into the rack. For every sled displayed, you can pressEnter
to see its details.On the sled detail screen, you can use left and right arrows (or
h
/l
) to move left and right.
Confirm that the number of sleds with a "Yes" SP Ignition state matches the expected count (16 for a half-rack, 32 for a full rack), and that the two switches and PSC all have Ignition on.
3. Update rack software
3.1. Upload software to the rack
Close wicket via Ctrl-C
. Upload the software image zip file by executing the following command:
ssh wicket@${IP_ADDRESS}%${PORT_NAME} upload < tuf-mupdate.zip
Once the upload is complete, you will see a "successfully uploaded repository to wicketd" message and you will be returned to your shell. Next, ssh back into wicket:
ssh wicket@${IP_ADDRESS}%${PORT_NAME}
3.2. Execute sled updates
You will initiate software update one sled at a time, each of which should take about 20 minutes. You can have multiple sleds execute the process in parallel without waiting for the other ones to finish.
To initiate a sled update,
On the left pane of the wicket UI, select
UPDATE
and pressTab
to move focus into the rack.Arrow down to the target sled. You can press the right/left arrows to expand or collapse the short list of versions (there should be only one version available during the first rack install).
Press
Enter
and this should take you to another pane with the versions listed at the top, and the bottom should say "Update ready: pressCtrl-U
to start".Press
Ctrl-U
, then pressY
on the popup to confirm you want to start the update. The bottom pane will be replaced by a list of steps that will be performed.
At any time, you can move up and down the list (via up/down/j
/k
) and press enter
to see details about the step. The sleds will be rebooted automatically after update. Here is an example of the update step details:
3.3. Execute Switch 1 and PSC updates
The following steps can be done in parallel with the sled updates above and should take no more than 10 minutes each:
Select
SWITCH 1
- the switch that is not connecting to the current technician port - and initiate update in the same way as how it is done for sleds.Select
PSC 0
and initiate update as well.
3.4. Update Switch 0 and its adjacent sled
Once the updates for sleds, switch 1, and PSC invoked above have been completed successfully, exit from wicket and disconnect from Switch 0.
Next, connect to one of the technician ports on Switch 1. Ssh into wicket and select the sled that was excluded from update previously. Follow the same steps to initiate software update for this sled.
After the sled comes back up, select SWITCH 0
and initiate update. Upon completion, the rack should have all the latest software and is ready for configuration and setup.
4. Configure Rack Settings
On the left pane of the wicket UI, select RACK SETUP
. The current rack status displayed on the right pane will be "Uninitialized" at this point.
While keeping the wicket UI open, start another terminal session and ssh into the setup command shell:
ssh wicket@${IP_ADDRESS}%${PORT_NAME} setup
This should bring up the list of available subcommands:
Usage: wicket setup <COMMAND> get-config Get the current rack configuration as a TOML template set-config Set the current rack configuration from a filled-in TOML template reset-config Reset the configuration to its original (empty) state. set-password Set the password for the recovery user of the recovery silo upload-cert Upload a certificate chain upload-key Upload the private key of a certificate chain help Print this message or the holp of the given subcommand(s)
In this second terminal window, you will make use of the commands above to enter or upload the necessary rack configurations.
4.1. Upload SSL certificate
Oxide Console and API will be hosted under the domain name controlled by your organization. In this step, you will upload the certificate and key files that correspond to the subdomain delegated to the Oxide Rack.
Execute the upload-cert
subcommand to import the SSL certificate chain file:
ssh wicket@${IP_ADDRESS}%${PORT_NAME} setup upload-cert < ${CERT-CHAIN}.pem
and then upload-key
to import the key file:
ssh wicket@${IP_ADDRESS}%${PORT_NAME} setup upload-key < ${CERT-KEY}.pem
4.2. Set Recovery User Password
The RSS will create a built-in silo for setup and recovery purposes. This is an ordinary silo backed by the local-only identity provider, with a system user named "recovery". This user has the privileges to create other silos and modify mutable pieces of their identity provider configuration.
Execute the following subcommand to enter the password for the recovery
user:
ssh wicket@${IP_ADDRESS}%${PORT_NAME} setup set-password
4.3. Configure basic networking
In this step, you will configure the endpoints of boundary services that integrate with the Oxide Rack. You will supply the information in the form of a text file in toml
format.
To begin the configuration, retrieve the toml template via
ssh wicket@${IP_ADDRESS}%${PORT_NAME} setup get-config > rack.toml
The content of the file should look like this:
# DNS name for the DNS zone delegated to the rack for external DNS. external_dns_zone_name = " " # External NTP servers; e.g., "ntp.eng.oxide.computer". ntp_servers = [ ] # External DNS servers; e.g., "1.1.1.1", "9.9.9.9". dns_servers = [ ] # Ranges of the service IP pool which may be used for internal services. # # Elements of this list should be of the form: # # { first = "first_ip", last = "last_ip" } # # where `last_ip` is equal to or higher than `first_ip`; e.g., # # { first = "172.20.26.1", last = "172.20.26.10" } internal_services_ip_pool_ranges = [ ] # List of sleds to initialize. # # Confirm this list contains all expected sleds before continuing! bootstrap_sleds = [ (list of sleds auto-discovered from the rack will be displayed here) ] # VLAN and routing information [rack_network_config] gateway_ip = " " infra_ip_first = " " infra_ip_last = " " uplink_port = " " uplink_port_speed = " " uplink_port_fec = " " uplink_ip = " "
internal_services_ip_pool_ranges
are used for Control Plane DNS and API services. The pool range(s) must cover 16 or more IP addresses.Use a text editor such as vim
to edit the toml file. Upon completing the configuration data entry, you can upload the file via
ssh wicket@${IP_ADDRESS}%${PORT_NAME} setup set-config < rack.toml
The configurations should be refreshed automatically in the wicket UI with the uploaded data. If everything looks correct, proceed to the next step; else, edit the configurations with reset-config
, get-config
, and set-config
as needed.
To continue with rack setup, press Ctrl-K
. The process may take about 30 minutes or longer. Once the initialization has completed, the Current rack status
will become "Initialized". Here is an example of the final state:
If certain misconfigurations are found after the rack has been initialized, you can reset the rack with Ctrl-R Ctrl-R
to remove the control plane and any VM instances, and repeat rack initialization after correcting the settings.
Next: Log in the web console to complete the rest of the rack setup.