Staff Software Engineer - Data Platform

Discord's mission is to bring people together around games. Tens of millions of people use Discord everyday to have meaningful interactions, whether that's playing their favorite games together, discussing the latest news on topics they care about, or just hanging out in their shared server space. Over 300 million people have signed up for Discord and we strive to continue building a product and place that connect people.

The Data Infrastructure team builds data pipelines to reliably collect and process billions of product activity events every day. We operate a petabyte-scale data warehouse and provide tools that empower the machine learning, data science, and business analysis teams at Discord. Our work enables user privacy and product features; and is foundational to making Discord a safer place for you to talk and hang out with your friends.

What you'll be doing

• Build the platform that empowers our Data Science, Machine Learning & Analytics teams; and powers product features used by millions of users every day.
• Manage and operate our entire data stack using modern technologies such as BigQuery, Cassandra, Apache Beam, Pub/Sub, and Airflow.
• Build efficient solutions on top of Google Cloud Platform using Python and Scala .

What you should have

• You have 10+ years of experience as a Software Engineer.
• Experience building and deploying large-scale distributed systems.
• Experience leading complex technical projects with successful outcomes
• Experience with open source and managed big data services like Kafka, Spark, Flink, Cassandra , Hadoop
• You can work with others to create solutions — and then evolve and generalize them.
• You can take a high-level goal and provide shippable code.

Bonus Points

• Experience with open source and managed big data services like Kafka, Spark, Flink, Cassandra/Scylla, Hadoop
• Experience with data products on Google Cloud Platform
• Experience with Kubernetes and Docker
• Experience with Airflow
• Can produce some real good SQL