Speaker
Description
In this talk we will see how Apache Spark can be run on top of Kubernetes, and how Canonical's Charmed Apache Spark solution makes this process easier by leveraging the capabilities of Juju to run and manage application workloads in Kubernetes. We'll see how Spark jobs can be run using the spark-client snap and leverage Charmed Apache Kyuubi for managed SQL workloads. Additionally, we’ll cover Spark History Server for job tracking and observability using the Canonical Observability Stack (COS). Through real-world use cases, attendees will see how all of these tools are consolidated together in the Charmed Apache Spark bundle and how these tools streamline and optimize big data processing.
What audience can learn
Attendees will learn how to deploy and manage Apache Spark on Kubernetes using Canonical's Charmed Apache Spark and Juju, simplifying big data processing. They'll also discover how to submit jobs with the spark-client snap, manage SQL workloads with Charmed Apache Kyuubi, and enhance monitoring with Spark History Server and the Canonical Observability Stack (COS).
Summary
This session will demonstrate deploying Apache Spark on Kubernetes using Canonical's Charmed Apache Spark and Juju. We'll cover job submission via the spark-client snap, managing SQL workloads with Charmed Apache Kyuubi, and utilizing Spark History Server alongside the Canonical Observability Stack
Things to know or prepare for this session
Apache Spark, Apache Kyuubi, Big Data, Juju, Kubernetes
Biography
Bikalpa is a software engineer and open-source enthusiast from Pokhara Nepal. He is currently working as a Software Engineer in the Data team in Canonical.
| Difficulty level | Intermediate |
|---|