Senior Data Quality Engineer - Contractor
Toronto, ON; Sunnyvale, CA; Plantation, FL
13 days ago
This is a full-stack software development and systems operations role supporting collection, processing, analyzing and reporting of Magic Leap's Quality data. In this position, you will be part of a team with the mandate to gather, evaluate, and improve the quality of Magic Leap’s software stack. Individuals in this role will design and implement scalable algorithms and data models to manage Magic Leap’s Quality data processing for internal projects. You will lead, choose, and deploy commercial and open-source tools and technologies to manage quality data with an emphasis on operational self-service and continuous development. You will also harness collected data to devise visualization, debugging, and simulation tools.
Essential Duties and Responsibilities
Core duties and responsibilities include the following. Other duties may be assigned.
- Active development using Python and shell internal infrastructure projects and tests
- Design, build and launch extremely efficient and reliable data pipelines to track metrics resulting from the execution of non-functional tests
- Design, develop, deploy and maintain quality data collection and reporting systems including data converters, databases, web services, and graphical reports
- Evaluate and quantify various qualities of the software stack including source code, performance, reliability, power consumption, etc.
- Develop tools and scripts to automate the collection and conversion of both software testing data and metric data from active devices
- Devise visualization, debugging, and simulation tools to process and analyze collected data and represent it in a form that is simple and accessible to other teams and management
- Work closely with product and engineering teams to describe and validate expected software behavior and performance
- Encourage and enforce best practices to enable rapid new feature development while maintaining multiple functioning software products
- Work closely with Test, Software, Build and Integration engineers to develop tools for automation to continuously improve the release process and automation environment
- For larger data projects, the individual is expected to take ownership of project direction and provide technical team support
To perform this job successfully, an individual must be able to perform each essential duty in a satisfactory manner. The requirements listed below are representative of the knowledge, skill, and/or ability required. Reasonable accommodations may be made to enable individuals with disabilities to perform the essential functions.
- Language Skills: Ability to read and interpret documents such as safety rules, operating and maintenance instructions, and procedure manuals
- Reasoning Ability: Ability to define problems, collect data, establish facts, and draw valid conclusions
- Computer Skills:
- Python3 development, using best practices and following PEP8
- Experience of REST APIs (using, designing and development)
- Experience making data visualizations like graphs or charts
- Knowledge of how to effectively parse and manage raw data files like logs/JSON/CSV/Parquet/Avro/XML/etc. Especially for large datasets (>100GB)
- Design, deploy and maintain SQL or/and NoSQL databases (ideally one or more of both)
Data pipeline tools like Hadoop, Kafka, RabbitMQ, Spark, MapReduce, BigQuery, ELK, etc.
Knowledge of orchestration tools like Rancher, Mesos and Kubernetes
- Recent experience managing and building Android/AOSP projects
- Experience with software development and testing, and familiarity with Test-Driven and Behavior-Driven Development (TDD/BDD), Continuous Integration, and Continuous Deployment
- Interaction and instrumenting Jenkins for metrics collection from various sources (e.g. from code changes)
- Familiar with virtual machines such as VMWare and VirtualBox, and containers such as Docker and LXC in production environment
Education and/or Experience :
- Bachelor or Master degree; 5 or more years related experience and/or training; or equivalent combination of education and experience
- All your information will be kept confidential according to Equal Employment Opportunities guidelines