Building a system with multiple servers can be a daunting task, as achieving proper coordination between devices is often challenging. The distributed algorithms are complex and often turn out to be incorrect, leading to tricky data losses and split brains.
To simplify the lives of developers, distributed coordination systems such as Apache Zookeeper, etcd, and Consul were created. These systems provide APIs for solving common problems that arise during distributed systems design, including service discovery, leader election, barriers, and exactly-once message processing.
YTsaurus is a big data storage and processing system that recently became open-sourced. While initially created for working with big data, YTsaurus offers rich functionality for solving distributed coordination problems. In this talk, we will explore how YTsaurus can assist in building distributed systems and compare it with similar systems.
Session author's bio
I am a software engineer interested in distributed storage systems and efficient data processing. For the last four years, I’ve been lucky to participate in YTsaurus development working on metadata storage, distributed transactions, and multiple other things.
|Level of Difficulty||Intermediate|