This job might no longer be available.

Principal Data Engineer, Defence

Improbable Defence is hiring its first Lead Data engineer. Excitingly, this is a brand new team so its remit is still being determined, but may well include:

* Storing and cataloguing data used in simulations (models, assets, configuration, etc) in a secure way.

* Versioning data, managing metadata, data classification

* Allowing data inputs to be queried to help our users rationalise about simulation outcomes

* Building scalable data pipelines and tools to generate new inputs and models from a variety of (often real-world) data source

* Understanding where data originated and where it is used.

Areas for Impact

• Help build a team including recruitment and mentoring, helping to shape the backlog in the process.
• Lead the team to design and develop the framework and tools that enable applied scientists to build pipelines to extract, transform, synthesise and integrate data from various sources.
• Provide tools for the automation and scheduling of data extraction from both static and live sources.
• Help to design and build the services needed for accessing and updating the data and models for use within a synthetic environment.
• Utilise standard data science tooling where possible to provide an intuitive experience for users with minimal onboarding time.
• Engineer for scalability and efficiency so that our customers can process large volumes of data in a timely way.
• Build secure and auditable systems.

We'd like to hear from you if you identify with the following:

• You are an experienced Data Engineer and team leader, capable of building a team from the ground up.
• You have experience building with scalable data processing implementations using multiple technologies (eg Hadoop, Apache Spark, Apache Airflow, Kafka, Mongo, Timescale, MinIO) and across multiple platforms (AWS, GCP, Azure, possibly custom environments too).
• You have experience forming and leading a team of engineers, interfacing with Product and the wider business.
• You are passionate about sharing knowledge, learning and collaborating with others.
• You are pragmatic and able to identify the most impactful work among competing requirements.
• Ideally, you have knowledge of Go, Python or C++, but we’re open-minded.

About Us

Improbable is determined to foster an environment where people can do their best work and feel like they belong. We believe a healthy culture, strong values and contribution from a diverse range of individuals will help us to achieve success.

We do not discriminate based on race, ethnicity, gender, ancestry, national origin, religion, sex, sexual orientation, gender identity, age disability, veteran status, genetic information, marital status or any other legally protected status.