User Resource Management
Silos
Silos are the top-level containers for grouping users, projects, and resources. As the name suggests, silos are isolated from each other. Resources in a silo cannot be shared with any other silos.
Every Oxide rack includes a Recovery silo which is reserved for built-in users and resources. Rack administrators can create one or more silos based on their usage model. Silos can be backed by an external or local identity provider, as defined by the identity_mode property of the silo:
SamlJit: Users are authenticated via SAML using an external identity provider (IdP). The IdP is the source of truth about users and groups. The control plane’s information about users and groups gets imported and updated only during user authentication (i.e., on a "just-in-time" basis). User discovery is not supported at this time for pre-provisioning users prior to their first-time login to the rack.LocalOnly: Users are authenticated by username and password. The identity data is managed through Oxide API and resides only in the rack. IAM roles can only be granted to local users on a per-user basis as groups are not supported for this type of silos.
The identity_mode property is set when the silo is created and cannot be changed afterwards.
While each silo is linked to a single realm of users, the same provider can be configured to back multiple silos. In other words, it is possible to have the same user onboarded to different silos that are set up for different purposes, e.g.
Silo DNS Endpoints
As part of the on-premises network integration, a DNS domain is delegated to the Oxide rack at the time of initialization. This allows the rack external endpoints to be accessible using DNS names and secured with TLS certificates.
Each silo has a unique console and API endpoint with the naming convention $silo_dns_name.sys.$delegated_domain. The silo_dns_name is set to the silo’s name by default but is customizable. For this reason, silo names must comply with the domain name standard RFC 1035, i.e. they may contain only the English-language letters a-z, the digits 0-9, and hyphens; hyphens cannot be at the beginning and at the end.
Projects
Projects provide another level of resource and access isolation within each silo. They serve as containers for managing compute, storage, and network resources. Resources such as VM instances and disks are always scoped under specific projects and cannot be shared with other projects. Internal network traffic among VM instances in the same project is bound within the virtual private network (VPC) for that project.
Silo users can be members of one or more projects. Their access permissions are governed by both the silo-level and project-level access policies.
Like silos, project names are also governed by RFC 1035 to enable the use of DNS names for VM instances in the future versions of Oxide Rack.
Identity & Access Management (IAM) Policy
Access to projects and resources may be granted on a per-group or per-user basis. There are three built-in roles with different levels of permissions:
| Role | Description |
|---|---|
admin | has all permissions on the target object |
collaborator | has |
viewer | has |
Here is an example of IAM policies with different access permissions for different members of a project:
IAM roles granted at the silo level provide cascading permissions to all child objects scoped within the silo. For example, when a group is assigned the "viewer" role in a silo, group members can view all the projects of the silo and their resources. If a particular user within the group is also given the "collaborator" role, the person gains additional permissions to act on the projects.
Rack Network Management
Integrating an Oxide rack into the on-premises network involves the allocation of IP addresses and configuration of rack switches. There are five basic elements on a rack switch that may be configured:
Ports carry physical layer (L1).
Links carry link layer (L2) information.
Interfaces carry internet layer (L3) information and typically belong to a port.
Addresses are assigned to interfaces.
Routes determine forwarding behavior across the switch.
Address Lots
Usable addresses are communicated to the Oxide Rack through address lots. An address lot contains an array of address ranges. Address lots are named elements with a resource id. Address lots allow operators to associate an intended use to a collection of addresses. To this end address lots have types. Currently there are two types, infra and pool. The infra type indicates that an address lot is to be used for the network infrastructure connecting the rack to upstream networks. The pool type indicates that an address lot is to be bound to an IP pool. For example, for a rack on the public Internet with the address prefix 203.0.113.0/24, an administrator may choose to break this prefix up as follows into lots.
| name | type | block |
|---|---|---|
network infrastructure |
| 203.0.113.2 - 203.0.113.3 |
control plane |
| 203.0.113.4 - 203.0.113.10 |
instances |
| 203.0.113.11 - 203.0.113.100 |
Once these address blocks are created. Addresses within these blocks may be used for things like rack switch addresses and ip-pools. The Oxide control plane tracks the use of these addresses to prevent inconsistencies such as assigning the same address twice or trying to use an address that is not in an address block.
Switch Ports
Port objects are associated with physical switch ports. There are different types of ports, e.g. faceplate data ports, backplane ports. Ports provide a configuration interface for physical-layer properties such as phy lane geometry and power levels supplied to transceivers.
Naming:
qsfp0-31: data ports on the faceplate of the rack switchtech0-1: RJ-45 technician portsrear0-31: backplane portsint0: internal 0 for the CPU portmgmt0-34: vsc7448 portsmanagement ports 0-31: service processors.
management ports 32-33: power controllers 0 and 1
management port 34: adjacent sidecar
Switch Links
Links provide a configuration interface for link-layer properties such as MAC addresses. The number of links per port depends on the phy geometry that has been configured for the port. Ports that support modular interfaces such as QSFP have multiple data lanes that connect the switch ASIC to the internal side of a transceiver. Some transceivers support multiple independent physical cables on the external side of the transceiver. This is called a breakout cable. The phy geometry configuration of a port defines how the internal transceiver data lanes are grouped into layer-2 link elements. In the case of QSFP, the Q stands for quad. Typical breakout configurations include 1xQSFP (no breakout), 2xQSFP (2 lanes per QSFP link, delivering half the available bandwidth to each), 4xSFP (1 lane per SFP link, delivering a quarter of the available bandwidth to each). Thus in the case of QSFP, there may be 1, 2 or 4 links per port with 4, 2 and 1 lanes per link respectively.
Naming:
If the phy geometry only specifies one link, the link has the same name as the port. If the phy geometry specifies multiple links, the link is named as follows "<port-name>:<index>" where index ranges from 0 to one less than the
number of links specified in the phy geometry. For example the following enumerates all possible link configurations for the port qsfp7.
| Phy Geometry | Links |
|---|---|
|
|
|
|
|
|
Switch Interfaces
Interfaces carry internet layer information and typically belong to a port. They come in three flavors, loopback (lo), link and vlan. Loopback interfaces are not associated with any link and are typically created to hold router IPs. There is one loopback interface per switch that cannot be manually created or destroyed. A link interface is automatically created for every link object and there is only one per link. Link interfaces cannot be manually created or destroyed, they simply exist as a companion to a link. Vlan interfaces are associated with a link, and are configured with a VLAN id (VID) to only send and receive traffic on that VID. There can be an arbitrary number of vlan interfaces per link, but they all must have distinct VIDs. These interfaces exist primarily to support routing protocols over VLAN segmented networks. Traffic delivered to routing daemons from the vlan interface has the VLAN tag removed. Traffic sent from routing daemons through a vlan interface has the VLAN tag added. As the product evolves, more interface types such as vxlan, Geneve, gre etc. may be added.
Naming:
Loopback interface is named
loas there is only one per switch currently. When there is a need to create multiple loopback interfaces in the future, users can supply a suffix to uniquely identify them.Vlan interfaces always start with the prefix "vlan". Users may also supply a suffix for a vlan interface.
Switch Addresses and Routes
An arbitrary number of IPv4 or IPv6 addresses may be added to any type of interface. Adding an address to a non-loopback interface automatically creates a route whose destination is the subnet prefix of the address. This route is called an on-link route, meaning that it does not have a nexthop gateway. Layer-2 frame information for packets egressing along this type of route is acquired from ARP or NDP neighbor tables for IPv4 and IPv6 respectively. On-link routes may not be manually created or destroyed, they belong to an address and share a lifetime with that address.
Interface routes map a destination prefix to an interface. They depend on IPv6 router advertisements to work. The target interface must have an IPv6 link-local address, and the switch must have received a router advertisement on that address. The switch uses the mac address of the router it received an advertisement from, to form the L2 frame of packets egressing along an interface route. Note that interface routes can have either an IPv4 or IPv6 destination prefix.
Nexthop routes associate a destination prefix with a nexthop address. The nexthop address must be reachable through an existing chain of routes.