Key Concepts

Silos are the top-level containers for grouping users, projects, and resources. As the name suggests, silos are isolated from each other, providing strict tenancy separation between groups of users. Each silo has its own resource limits and access policies as well as its own subdomain for the web console and API. Resources in a silo cannot be shared with any other silos.

Every Oxide rack includes a Recovery silo which is reserved for built-in users and resources. Rack administrators can create one or more silos based on their usage model. Silos can be backed by an external or local identity provider, as defined by the identity_mode property of the silo:

The identity_mode property is set when the silo is created and cannot be changed afterwards.

While each silo is linked to a single realm of users, the same provider can be configured to back multiple silos. In other words, it is possible to have the same user onboarded to different silos that are set up for different purposes, e.g.

Projects provide another level of resource and access isolation within each silo. They serve as containers for managing compute, storage, and network resources. Resources such as VM instances and disks are always scoped under specific projects and cannot be shared with other projects. Internal network traffic among VM instances in the same project is bound within the virtual private network (VPC) for that project.

Silo users can be members of one or more projects. Their access permissions are governed by both the silo-level and project-level access policies.

Like silos, project names are also governed by RFC 1035 to enable the use of DNS names for VM instances in the future versions of Oxide Rack.

Access to projects and resources may be granted on a per-group or per-user basis. There are three built-in roles with different levels of permissions:

Here is an example of IAM policies with different access permissions for different members of a project:

IAM roles granted at the silo level provide cascading permissions to all child objects scoped within the silo. For example, when a group is assigned the "viewer" role in a silo, group members can view all the projects of the silo and their resources. If a particular user within the group is also given the "collaborator" role, the person gains additional permissions to act on the projects.

Instances are virtual machines that run on the Oxide platform. Each VM can be attached to up to eight virtual disks, one private network interface, and multiple external network interfaces.

Instances that have one or more network interfaces get outbound internet access through NAT service on the rack. They can be optionally attached to temporary or permanent external IP addresses to allow them to communicate with external hosts.

Disks are persistent volumes that can be managed independently from VM instances. A snapshot is a lightweight point-in-time copy of a disk that can be used to create a reusable image shared within the project or with other projects.

Images and snapshots let you create a new disk based on an existing one, providing the mechanisms for disk cloning and templating. See the Images and Disks and Snapshots guides for more information.

IP pools are collections of external IPs you can assign to silos. When a pool is linked to a silo, users in that silo can allocate temporary or permanent external IPs from the pool for their instances.

Temporary external IPs are known as ephemeral IPs. They are allocated and deallocated during instance creation and termination respectively. Permanent external IPs are known as floating IPs. They exist independently of instances as project-scoped resources and can be attached to and detached from instances within the project as needed.

See the Networking and Floating IPs guides to understand more about VM networking.

VPCs are private RFC1918 networks that isolate sets of instances from each other. Instances within a VPC subnet can talk to each other using private IP addresses (if firewall rules allow it) but traffic between VPCs must go through external IPs.

Usable addresses are communicated to the Oxide Rack through address lots. An address lot contains an array of address ranges. Address lots are named elements with a resource id. Address lots allow operators to associate an intended use to a collection of addresses. To this end address lots have types. Currently there are two types, infra and pool. The infra type indicates that an address lot is to be used for the network infrastructure connecting the rack to upstream networks. The pool type indicates that an address lot is to be bound to an IP pool. For example, for a rack on the public Internet with the address prefix 203.0.113.0/24, an administrator may choose to break this prefix up as follows into lots.

Once these address blocks are created. Addresses within these blocks may be used for things like rack switch addresses and ip-pools. The Oxide control plane tracks the use of these addresses to prevent inconsistencies such as assigning the same address twice or trying to use an address that is not in an address block.

Port objects are associated with physical switch ports. There are different types of ports, e.g., faceplate data ports, backplane ports. Ports provide a configuration interface for physical-layer properties such as phy lane geometry and power levels supplied to transceivers.

Naming:

Links provide a configuration interface for link-layer properties such as MAC addresses. The number of links per port depends on the phy geometry that has been configured for the port. Ports that support modular interfaces such as QSFP have multiple data lanes that connect the switch ASIC to the internal side of a transceiver. Some transceivers support multiple independent physical cables on the external side of the transceiver. This is called a breakout cable. The phy geometry configuration of a port defines how the internal transceiver data lanes are grouped into layer-2 link elements. In the case of QSFP, the Q stands for quad. Typical breakout configurations include 1xQSFP (no breakout), 2xQSFP (2 lanes per QSFP link, delivering half the available bandwidth to each), 4xSFP (1 lane per SFP link, delivering a quarter of the available bandwidth to each). Thus in the case of QSFP, there may be 1, 2 or 4 links per port with 4, 2 and 1 lanes per link respectively.

Naming:

If the phy geometry only specifies one link, the link has the same name as the port. If the phy geometry specifies multiple links, the link is named as follows "<port-name>:<index>" where index ranges from 0 to one less than the number of links specified in the phy geometry. For example the following enumerates all possible link configurations for the port qsfp7.

Interfaces carry internet layer information and typically belong to a port. They come in three flavors, loopback (lo), link and vlan. Loopback interfaces are not associated with any link and are typically created to hold router IPs. There is one loopback interface per switch that cannot be manually created or destroyed. A link interface is automatically created for every link object and there is only one per link. Link interfaces cannot be manually created or destroyed, they simply exist as a companion to a link. Vlan interfaces are associated with a link, and are configured with a VLAN id (VID) to only send and receive traffic on that VID. There can be an arbitrary number of vlan interfaces per link, but they all must have distinct VIDs. These interfaces exist primarily to support routing protocols over VLAN segmented networks. Traffic delivered to routing daemons from the vlan interface has the VLAN tag removed. Traffic sent from routing daemons through a vlan interface has the VLAN tag added. As the product evolves, more interface types such as vxlan, Geneve, gre etc. may be added.

Naming:

An arbitrary number of IPv4 or IPv6 addresses may be added to any type of interface. Adding an address to a non-loopback interface automatically creates a route whose destination is the subnet prefix of the address. This route is called an on-link route, meaning that it does not have a nexthop gateway. Layer-2 frame information for packets egressing along this type of route is acquired from ARP or NDP neighbor tables for IPv4 and IPv6 respectively. On-link routes may not be manually created or destroyed, they belong to an address and share a lifetime with that address.

Interface routes map a destination prefix to an interface. They depend on IPv6 router advertisements to work. The target interface must have an IPv6 link-local address, and the switch must have received a router advertisement on that address. The switch uses the mac address of the router it received an advertisement from, to form the L2 frame of packets egressing along an interface route. Note that interface routes can have either an IPv4 or IPv6 destination prefix.

Nexthop routes associate a destination prefix with a nexthop address. The nexthop address must be reachable through an existing chain of routes.