Yesterday was a rough day for me as a #sysadmin, but #ceph still maintained its perfect record of no data loss, even though I don't 💯 understand all the inner workings.

At this point, I'd say it is battle tested. I've had concurrent drive failures, and this latest issue where it couldn't find the DB device fors of 17 of the 21 drives!

I'm not sure I'd go with ceph again if I had to start over. It allows live migrating running VMs to anither host, which is a huge win for avoiding downtime and saving disk writes for migrating an entire hard drive (as opposed to just migrating the confents of memory). But having standalone disks would be way more performent in terms of both I/O and memory usage.

If I could get equal disk I/O performance, I'd choose ceph for any #SelfHosting. Maybe not for a #homelab, where nobody relies on any of the services, but certainly if you're using it in your daily life.

I still have a #bounty out of 300K sats for anyone who can get me 100MB/s random read inside a VM and 100 IOPS for the rados write benchmark.

What I actually want is for VMs to run at least as well as if they were on a standalone consumer grade hard drive, but I'm trying to provide concrete metrics so there's an objective metric of whether it goes "fast enough".