Lead Data Engineer

JOB ALERT FOR! We are currently recruiting for a LEAD DATA ENGINEER for one of our clients in the insurance industry.The Lead Data Engineer will be responsible for:Leading a team of developers and data engineers who build and deliver the components of modern Data platform – Data Lakehouse Architecture in Azure, leveraging data lake, delta Lake for data storage Utilizing Azure Data Factory for data integration Databricks for data transformation Data governance using purview Profisee for MDM Synapse and Power BI for end-user data analytics layer and reporting. In the pipeline – Co Pilot and Fabric.Key responsibilitiesLead data engineer responsible of:• Mixture of hands on and lead a newly established data engineering team through both strategic development projects and day-to-day data management (BAU) activities.o Defining and orchestrating tasks and deliverables in support of the project/scrum master. o Setting the standard of what good looks like.• Design, develop, and hold accountability for core data engineering services, encompassing the end-to-end data lifecycle journey from ingestion through to end user reporting. • Data acquisition from various data sources (API, XML, Json, parquet, flat file systems and relational data. • Skills in data modelling (both structured and unstructured data) working directly with the business & data architects and SME’s. • Administering CICD pipelines, completing peer/code reviews, ensuring the integrity of data, and holding accountability for access and security permission. • Design and develop metadata driven data ingestion (ETL/ELT) processes, data lake house architecture, data warehouse, data marts, reports, and dashboards; in-line with the company data management framework and enterprise data strategy. • Work in collaboration with Data Architects and act as a Senior contributor on the design, build, and management of the Data Platform; taking direct ownership of data engineering activities and responsibilities and ensure that all new data requirements are met using the most appropriate architecture components and engineering practices. • Design and develop models using delta lake both relational and dimensional to support reporting requirements. • Create reporting solutions in line with standards, architectural principles, and practices. • Work in an agile, proactive manner, delivering high quality solutions in agreed timescales. • Mentor and support data engineers/developers in the team. • Team leadership of both senior and general data engineers and BI developers. • Stay abreast of technology trends, state-of-the-art platforms, data engineering techniques/standards and industry trends to provide input on strategic roadmap including Fabric and AI copilot • Develop Proof of Concepts to prove that business requirements can be met by technical solutions.Highly Desirable Skills and Knowledge: • Microsoft Fabric – Data engineering, Lake house • Experience with Co Pilot • REST API Data sources (MongoDB, Cosmos DB) • Experience in design and developing MDM solutions ie MDS, Profisee • MS SSIS, MS SQL Server Reporting • Data Governance – Purview, Databricks Unity catalogue.