Matej Bašić
I am a Senior DevOps Engineer at ReversingLabs, where I build and run the Kubernetes and hypervisor clusters behind dynamic malware analysis. I am also a teaching assistant at Algebra Bernays University. I hold a master’s degree in cybersecurity from the Faculty of Information Studies in Novo Mesto, Slovenia, where I am now a doctoral student. I am a Red Hat Certified Architect and a Red Hat Certified Instructor.
My research asks how Kubernetes and the infrastructure under it behave under load and when parts of it fail. I measure that on labs I build and run; the current one is 11 VMs on vSphere, reset to one cold snapshot between runs.
The lab behind How Kubernetes behaves when its components disagree
23 fault scenarios run on these 11 VMs. Pick one of the examples the research page names to see which part of the lab it hits.
- etcd partition (crash-stop and timing)
- etcd sits outside the control-plane nodes so a scenario can slow it down, compact it or partition it on its own. It has five members so that a partition can split it 3 to 2 and still leave a majority.
- slow etcd writes (crash-stop and timing)
- One of the fourteen crash-stop and timing faults, with network partitions, etcd compaction and clock skew.
- etcd NOSPACE alarm (gray failure)
- etcd refuses writes after a NOSPACE alarm. In a gray failure every node stays Ready while something underneath has stopped.
- pod network cut (gray failure)
- The pod network is cut while the management network stays up. Every node stays Ready.
- containerd frozen (gray failure)
- containerd is frozen with SIGSTOP. Every node stays Ready.
- between runs (reset)
- All 11 VMs go back to one cold snapshot, taken with the fleet powered down. vCenter cannot snapshot several running VMs at the same instant, and etcd members restored from different moments can come back at different points in their logs.
matej.basic (at) outlook.com
CV available on request.