Intermediate Data Engineer

The Company

A leading AI consulting firm specializing in leveraging data to drive business value. Known for delivering cutting-edge solutions in Natural Language Processing, GenAI, LLMs, Predictive Analytics, and Machine Learning, they help clients optimize operations, enhance customer experiences, and gain a competitive edge.

The Role 

• Building modern database structures storing terabytes of data used to develop and maintain machine learning models in a cloud environment.
• Collaborate with Software Engineering stakeholders and Leads to define and implement new data-powered solutions and deploy and maintain them in production.
• Supporting best practices in cataloguing, characterising, classifying and communicating accessible data and initiatives relative to data creation so teams can easily access the data.
• Creating and owning best practices in terms of data ‘organisation’, standards and versioning including in a Cloud environment.
• Own data privacy and security standards according to GDPR and apply these to production systems.
• Embracing modern DevOps principles and working in a setting where code is expected to be shared and peer-reviewed.

Experience 

• 3 years experience with data wrangling/data engineering, implementing, and supporting robust data transformation pipelines in a production Cloud environment.
• RDBMS systems for OLTP workloads, such as Postgres or MySQL.
• Implementing methods to ensure large-scale (big data) quality and security.
• Software design patterns and test-driven development.
• Experience working with big data technologies such as Spark, Hadoop, Presto, Hive, AWS Athena and Terraform.
• Dynamic languages, such as Python or R.
• Container technology, like Docker.
• Experience with JavaScript frameworks like Vue, Node.js.