This job might no longer be available.
Software Engineer, Data - Video Distribution
2 years ago
About Us
Launched in 2011, Twitch is a global community that comes together each day to create multiplayer entertainment: unique, live, unpredictable experiences created by the interactions of millions. We bring the joy of co-op to everything, from casual gaming to world-class esports to anime marathons, music, and art streams. Twitch also hosts TwitchCon, where we bring everyone together to celebrate, learn, and grow their personal interests and passions. We’re always live at Twitch. Stay up to date on all things Twitch on LinkedIn , Twitter and on our Blog .
About the Role
Twitch operates the largest dedicated live video CDN in the world, routinely delivering over 25Tbps to more than 5MM concurrent viewers. The Video Distribution team oversees the software that operates that CDN, which includes everything from custom edge software that delivers video on thousands of hosts worldwide, to graphing algorithms that manage our worldwide caching hierarchy, and to the complex and introspective algorithms that decide what server a user should be assigned when they start playback.
We are looking for a senior data engineer to help us transform the massive amounts of data our CDN platform generates into actionable insights, helping us better understand and improve the performance, reliability and scalability of our CDN. Our team is also responsible for delivering usable data to other teams, and works closely with video intelligence and product teams to empower them with data. You will report to the Software Development Manager.
You Will:
- Create and maintain large, complex data streaming pipelines that support multiple teams.
- Identify and design internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability , etc .
- Build the infrastructure required for optimal extraction, transformation, and provide insights using streaming technologies like Apache Flink or ClickHouse.
- Work with data and product teams to understand the video domain and bring team time insights into the performance of our video systems.
You Have:
- 5+ years of data engineering experience , software engineering, or other related roles.
- 2+ years professional programming experience
- 2+ years working with distributed, highly available systems.
- Experience with SQL and SQL optimization.
- Experience working with Big Data technologies such as Redshift, EMR, ClickHouse or Cassandra
Bonus Points
- You've made petabytes of data usable.
- You know Go and have used it in production-ready systems.
- You know Python, and have used it in production-ready systems.
- You've worked with Amazon Web Services in a production environment.
- You contribute to open source.
- You've built brilliant APIs/interfaces that other programmers love to use.
- You've served in a team lead capacity in the data or data infrastructure space
- You understand video streaming technologies, and have worked with live video.
Perks
- Medical, Dental, Vision & Disability Insurance
- 401(k)
- Maternity & Parental Leave
- Flexible PTO
- Amazon Employee Discount
- Monthly Contribution & Discounts for Wellness Related Activities & Programs (e.g., gym memberships , off-site massages , etc. )
Pursuant to the San Francisco Fair Chance Ordinance, we will consider for employment qualified applicants with arrest and conviction records.
We are an equal opportunity employer and value diversity at Twitch. We do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status.
Create Your Profile — Game companies can contact you with their relevant job openings.