We had a call, and EcoBytes, IndieHosters and weho.st expressed interest in sharing a production cluster.
We need to define the desired architecture.
The building of the staging cluster would depend on that as the purpose of the staging cluster is to test features before production.
I have several questions:
- baremetal vs cloud
My guess was that we’ll use Hetzner cloud for the VMs, and the auction for the baremetal machines.
Apparently weho.st has experience with physical machines.
Even for Indie, we want at the end of the day to have our own machines, so it looks like a great opportunity.
On the other hand, I think the complexity to build on baremetal is higher, so it’ll take more time to setup.
- load balancer
If we go with hetzner, we can have failover IP and point our dns to this IP, and make sure this IP is always attached to a healthy host.
Do you have idea on how to achieve that on your baremetal?
- persistence layer
The idea was to experiement with rook, and try various failure mode to get a sense if it is reliable or not.
This means that nodes are generic, and we have both storage and compute on the same nodes, and provisioned with k8s.
If we see the need, we can separate the nodes from storage and compute.
And if we see that rook is not up to the job, we can make a ceph cluster.
Do you have strong opinions on this topic?
If we use our hardware, it would make sense to encrypt underlying disks. do you have experience with that?
(I mean, not encrypting disk, but how to boot the servers in a secure way)
Or maybe we could handle this at ceph level, I’ll see if that’s viable.
Then, I think we have to discuss what hardware, how do we do staging, buy the hardware, mount the staging cluster, and start to play!
If you have other doubts/questions, please let u know!