Menu
My Datacenter Evicted Me

My Datacenter Evicted Me

Linus Tech Tips

2,158,073 views • 3 days ago Save 10 min 4 min read

Video Summary

Facing a 40% price hike on cloud infrastructure, the Floatplane team has taken matters into their own hands, migrating their video platform to a new co-location data center. This ambitious move involves setting up their own network, servers, and storage, a stark contrast to their previous reliance on rented cloud services.

The team is installing high-speed networking equipment, including two 100-gigabit switches for redundancy, and out-of-band management switches for remote access. They're also deploying a multi-blade server system running XCP-NG to pool resources, alongside powerful AMD EPYC servers boasting up to 1.5 terabytes of RAM. This self-hosted infrastructure aims to provide greater control, potential cost savings, and improved performance for their video platform.

Short Highlights

  • Cloud Infrastructure Cost Surge: Floatplane's monthly cloud bill increased by 40%, prompting a move to self-hosting.
  • Co-location Facility: The team is setting up operations in CoLogix's VAN3, a large network-neutral data center.
  • Networking Setup: Installation includes redundant 100-gigabit switches and out-of-band management switches.
  • Server Deployment: A multi-blade server system with XCP-NG and powerful AMD EPYC servers are being deployed.
  • Redundant Power and Connectivity: The facility offers redundant power feeds and multiple internet connections.
  • Direct Connectivity: Plans include a direct fiber link between the data center and the office, eliminating VPN reliance.

Key Details

Setting Up in a Co-location Center [00:00:00]

  • Floatplane is migrating its video platform from rented cloud infrastructure to a self-managed co-location data center.
  • The decision was driven by a 40% increase in their monthly cloud hosting bill.
  • The new facility, CoLogix's VAN3, offers redundant power, high-speed networking, and security.

    "We're here between a Walmart super center and a Canadian tire on the edge of Vancouver with a trunk full of servers, because sometimes it's just better to do it yourself."

Network Infrastructure Installation [00:01:00]

  • The initial setup focuses on installing networking equipment, including two 100-gigabit switches for redundancy.
  • These switches are configured to pull air from the back and push it to the front for optimal airflow.
  • The team is also installing out-of-band (OOB) switches for remote management, independent of the main network.

    "So yes, this one is the first switch. You will have seen this switch before in a previous video. So this is the secondary switch, all 100 gig, 32 ports."

Server Deployment and Virtualization [00:03:00]

  • A multi-blade server, previously used but now repurposed, is being installed for Floatplane.
  • Software called XCP-NG will be used to pool the resources of multiple blades and servers into a single virtualized environment.
  • This setup allows for redundancy, with capacity for three total nodes running services and one for redundancy.

    "What we're going to be doing is installing a software called XCPNG, which allows us to pool all the resources together."

High-Performance Hardware and Backup Systems [00:04:00]

  • Two powerful nodes are being deployed, each featuring a 92-core CPU and 1.5 terabytes of RAM.
  • A sixth node, described as worth a car, will handle backups.
  • A UniFi UDM Pro is being used for the out-of-band network, supporting WireGuard for remote connectivity.

    "Here's our fifth node, a 92 core CPU from AMD. Oh, actually it has two. So a hundred, actually too many."

Connectivity and Peering [00:05:00]

  • The team plans a direct fiber link between the data center and their office for a local area network (LAN) connection, eliminating the need for VPNs.
  • They are utilizing BiDi connections with Vanex, a local internet exchange, to optimize fiber usage.
  • This setup will allow direct peering with partners like Cloudflare, reducing reliance on the public internet.

    "So technically, from the office, we'll be able to connect to the network directly as if it was at the office."

Advanced Cabling and Future Migrations [00:07:00]

  • MPO cables are being used, which contain multiple fiber strands to transport multiple signals, allowing 100 gig ports to be broken down into 10 gig ports.
  • The team plans to migrate their Kubernetes clusters and databases, currently hosted elsewhere, to this new data center.
  • This migration is expected to improve performance due to increased horsepower and bandwidth, with servers being upgraded from 25 gig to 100 gig connections.

    "This here is a MPO cable. The really nice thing is that it contains multiple strands of fiber."

Other People Also See