Senior Data Engineer

Senior Data Engineer (Databricks) – Remote – Leeds - 6 Months Contract (Initial) – Outside IR35We are seeking Senior Data engineers (Databricks) on a contract basis to assist our client as they migrate over from legacy to AWS. The main focus will be to centralise the organisations data warehouses and consolidating / centralising a number of BI tools. Senior Data Engineers should have extensive experience working with Databricks on recent projects and be exposed to the latest features and toolsets of DatabricksRole Responsibilities:Design and develop scalable data pipelines using Databricks.Implement data processing workflows in Python and SQL.Manage and optimise data storage solutions on AWS.Ensure data quality and integrity across all stages of data processing.Develop and maintain documentation for data engineering processes.Utilise Databricks features, including Delta Live Tables (DLT) to simplify streaming and ensure data quality.Work with asset bundles in Databricks for streamlined data operations.Key Skills:Proven experience as a Senior Data Engineer with a strong focus on Databricks.Proficiency in Python and SQL for data processing and analysis.Spark Python API / PySparkIn-depth knowledge of Databricks Delta Lake and Live Tables.Familiarity with Databricks Asset Bundles (DAB’s) for streamlining the development of complex data and analytics for the Databricks platform. (IaC)Strong understanding of data warehousing concepts and ETL processes.Excellent problem-solving skills and attention to detail.Ability to work independently in a remote setup with minimal supervision.Role details:Job role: Senior Data Engineer (Databricks) Job Type: ContractDuration: 6 months with view of extensionIR35 status: Outside IR35Rate: £450 - £500 Location: Remote based with ad hoc travel to LeedsTo discuss this exciting opportunity in more detail, please APPLY NOW for a no obligation chat with your VIQU Consultant. Additionally, you can contact Suzie Stone by exploring the VIQU IT Recruitment website.To be the first to hear about other exciting opportunities, technology and recruitment news, please also follow us at ‘VIQU IT Recruitment’ on LinkedIn, and Twitter: @VIQU_UK