Head of Data Engineering – Azure, Databricks, Private Markets
A global Private Equity firm are seeking an experienced Head of Data Engineering as part of a critical data transformation and modernisation programme, moving towards a data-first approach. They are currently in the process of replacing a legacy data-warehouse setup with a platform as a service model using Databricks hosted on Azure, which this role will be responsible for leading, taking line management responsibilities for their Data Engineering team.
You will take ownership for progressing their Databricks setup, from its current PoC/incubator phase through to production, ensuring a high level of optimisation and scalability to operate at a global scale.
This will also involve implementing a semantic layer within the platform for effective data management and organisation, as well as the development of associated data pipelines and real-time reporting & visualization capabilities.
Once the platform is in place, the business will then look to apply Data Science techniques with the aim of building a best-in-class data function that works in partnership with the Investment Team and actively provides deep, actionable insights to enable business growth.
The ideal candidate will demonstrate:
- A technical background in data engineering with experience of technologies including Python, Spark, Databricks and Azure cloud.
- Previous experience in managing the end-to-end build of an Azure Databricks platform
- Passionate about all aspects of data (data governance, data quality management etc.) and the positive impact ‘good data’ can bring to a business
- An understanding of Data Science techniques (ML, NLP, AI etc.) would be beneficial
- Demonstrable experience in setting up and managing high-performing technology teams, establishing and driving best practises and being able to still engage at a technical level
- Able to credibly and confidently engage with the business at all levels
- A background in Financial Services (Private Markets, Investment Banking, Investment Management) is critical
If you are a data enthusiast with experience in leading high-performing teams to develop modern data platforms, then this is a genuinely exciting time to join a growing business transitioning to a data-first approach.
