Rack Maintenance
What happens when there is a power or network outage?
Power outage:
The rack hardware is designed to boot up on its own once the power supply is resumed. All user interfaces, i.e., API and Web Console, will be accessible again after the sleds and switches have come back up. No manual intervention or rack configuration data loss is expected.
Applications running on the rack instances may or may not fully recover automatically, depending on their running state prior to the power outage and the design of those applications. It is the responsibility of the instance owners to ensure instances are properly shut down prior to scheduled outage to prevent data loss.
Network outage:
Network connectivity from/to the rack and the VM instances should resume upon the restoration of uplink connection. No manual intervention or data loss is expected.
How do I prepare the rack for scheduled power maintenance?
We recommend shutting down instances on the rack preemptively and bringing them back up after the maintenance. Consider automating some of these operations with the instance_stop and instance_start API.
The LED is off for one of the drives/sleds. Should I attempt to reseat it?
No. It is possible that the component connection is intact but the LED itself is faulty. Please contact Oxide Support for investigation and refrain from making any hot-plug/unplug or reboot attempts.
Network Configuration and IP Address Management
Why are users getting "No external IP addresses available" error when most instances have only private NICs?
External IP addresses are allocated to instances for both inbound and outbound access outside the rack. All instances make use of NAT for outbound access. The NAT service also consumes IP addresses from the rack IP pool. Unlike inbound external IP addresses that are assigned one per instance, each NAT IP is shared by up to four instances and the address is not exposed to users.
To rectify the "No external IP addresses available" error, the fleet administrator can add one or more IP address ranges to the default IP pool using ip_pool_range_add.
How do I modify the uplink IP addresses?
No operator API is available to reconfigure uplink settings at this time. Please contact Oxide Support for assistance.
What do I do if our DNS or NTP server endpoints have changed?
No operator API is available to reconfigure DNS or NTP configurations at this time. Please contact Oxide Support for assistance.
Password and SSL Certificate Management
How do I rotate a silo TLS certificate?
The ability to replace a TLS certificate is limited to users with the silo admin role via Oxide API.
Follow the steps here to replace TLS certificates.
How do I set/invalidate local user passwords?
The ability to set or reset password is limited to users with the fleet admin role at this time via the local_idp_user_set_password API. The requester must provide the fleet administrator with the specific user_id which can be obtained using user_list or current_user_view.
To change the password for a user, set the mode to password in the request.
To invalidate the password to revoke access, set the mode to login_disallowed in the request.
How do I update the "recovery" user password?
Recovery user is a built-in user created during rack setup time. While the account should not be altered, it can be managed just like any regular local user. You can therefore update the password of the recovery user in the same way as local users using local_idp_user_set_password.
Access Management
How do I offboard a user from a silo?
Users managed in an identity provider:
In the identity provider system, remove the user account from the realm that is associated with the user’s silo.
Users managed locally on the rack:
As a user with the fleet
adminrole, use local_idp_user_delete to completely remove the user from the system.Fleet administrators may also use local_idp_user_set_password to revoke the user’s login but the change takes effect on the Web Console only and does not impact device token-based access.
Please note that there is no operator API for device token invalidation at this time (follow "Known Behavior and Limitations" in the Release Notes section for further updates). If the user being offboarded has been granted any fleet, silo or project roles directly in IAM, the corresponding silo or project administrators should also delete the user from the IAM role assignment to limit their access.
How do I offboard a user from a project?
If the user is given access to the project via the identity provider group membership, you can simply disassociate the user from the groups being granted the project IAM roles.
If the user is given access directly in IAM, the project administrator can remove them from the project IAM role assignment.
How do I modify identity provider configurations such as silo admin mapping?
Identity provider configurations are not modifiable to prevent silo membership from going out of sync with the identity provider system. The only way to modify the "silo admin" role mapping is to delete and re-create the silo.