Big Data Lead
15 days ago
WHAT MAKES US EPIC?
At the core of Epic’s success are talented, passionate people. Epic prides itself on creating a collaborative, welcoming, and creative environment. Whether it’s building award-winning games or crafting engine technology that enables others to make visually stunning interactive experiences, we’re always innovating.
Being Epic means being a part of a team that continually strives to do right by our community and users. We’re constantly innovating to raise the bar of engine and game development.
What We Do
We enable Epic’s online services teams to build, deploy, and manage services that are used by more than half a billion players around the world. Our mission is to provide world class tools and platforms to improve the experience of our developers and make it easier, faster, and safer to build, operate, and scale their applications. We operate at massive scale as one of the largest cloud computing users in the world.
What You'll Do
As a Lead Big Data Admin/Architect, you will help manage our big data platform infrastructure. You'll take responsibility for defining, running and optimizing our batch hardware running across several AWS EMR and traditional Hive/Spark clusters.
In this role, you will
- Support and augment our production and development Hadoop clusters
- Maintain and extend our Hive metastore as necessary
- Manage resource utilization across competing business priorities
- Monitor and tune cluster performance
- Identify opportunities to leverage components of the Hadoop ecosystems which are not yet part of our architecture
- Ensure a high level of security and auditing around our private and sensitive data
- Provide after-hours and weekend support for routine maintenance as well as troubleshooting outages
- Remain current on the latest Hadoop and distributed computing technologies
What we're looking for
- Deep knowledge of the low level workings of different big data execution engines: Spark, Tez, Hive, Trino/Presto, EMR
- Experience in performance tuning of auto scaling Hadoop clusters, Spark routines and Map/Reduce jobs
- Experience in security aspects of Hadoop and Linux such as File based ACLs, Kerberos, Ranger, and encryption
- Experience in independently performing root cause analysis of issues and presenting mitigation recommendations
- Experience in at least one scripting language (example bash, Perl etc.)
- Experience in debugging issues related to Java or Python applications
- Deep understanding of different storage/table formats and how they integrate with big data engines, e.g. Parquet, ORC, Iceberg
- Experience with any of the following is a plus: Snowflake, AWS RDS, MongoDB, any other RDBMS
EPIC JOB + EPIC BENEFITS = EPIC LIFE
Our intent is to cover all things that are medically necessary and improve the quality of life. We pay 100% of the premiums for both you and your dependents. Our coverage includes Medical, Dental, a Vision HRA, Long Term Disability, Life Insurance & a 401k with competitive match and profit share. We also offer a robust mental well-being program through Modern Health, which provides free therapy and coaching for employees & dependents.
Throughout the year we celebrate our employees with events and company-wide paid breaks. We offer unlimited PTO and sick time and recognize individuals for 7 years of employment with a paid sabbatical.
Create Your Profile — Game companies can contact you with their relevant job openings.
Epic Games spans across 19 countries with 55 studios and 4,500+ employees globally. For over 25 years, we’ve been making award-winning games and engine technology that empowers others to make visually stunning games and 3D content that bring environments to life like never before. Epic’s award-winning Unreal Engine technology not only provides game developers the ability to build high-fidelity, interactive experiences for PC, console, mobile, and VR, it is also a tool being embraced by content creators across a variety of industries such as media and entertainment, automotive, and architectural design. As we continue to build our Engine technology and develop remarkable games, we strive to build teams of world-class talent.
Like what you hear? Come be a part of something Epic!
Epic Games deeply values diverse teams and an inclusive work culture, and we are proud to be an Equal Opportunity employer. Learn more about our Equal Employment Opportunity (EEO) Policy here.