The Lead Data Engineer will support data lake developers and data warehouse engineers on various data initiatives and will ensure optimal data delivery architecture is consistent throughout ongoing projects. He/She must be self-directed and comfortable supporting the data engineering needs of multiple teams, systems and products. He/She will be re-designing the company’s data architecture along with other data architects to support our next generation products and data initiatives.
Responsibilities:
- Create and maintain optimal data pipeline architecture.
- Integrate large, complex data sets that meet business requirements.
- Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability.
- Evaluate/Review/Implement/Build the infrastructure required for optimal extraction, transformation, and loading of data from wide variety of data sources.
- Work with stakeholders at various levels to assist with data-related technical issues and support their data infrastructure needs.
- Create data tools for analytics and data scientist team members to help build and optimize our product and services.
- Work with data and analytics experts to strive for greater functionality in data ecosystem.
Qualifications:
- Advanced working SQL knowledge and experience working with relational databases as well as working familiarity with a variety of databases.
- Experience building and optimizing big data pipelines, architectures and data sets.
- Experience performing root cause analysis on internal & external data and processes and identify opportunities for improvement.
- Strong analytic skills related to working with both structured and unstructured datasets.
- Build processes supporting data transformation, data structures, metadata, dependency and workload management.
- Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores.
- Experience supporting and working with cross-functional teams in a dynamic environment.
Preferred Technical Skills:
- Candidate with 10+ years of experience in a Data Engineer role; Graduate degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field.