Job Description

We are looking for Intermediate to Senior Data Engineers with strong experience in modern data platforms and streaming services to join our growing team. The ideal candidate will have in-depth expertise in Microsoft Fabric tools (including Delta Lake, Synapse, Databricks, and Data Factory) and hands-on experience with streaming services like Kafka, Flink, or Azure Data Analytics. You will play a key role in designing, implementing, and optimising data pipelines that drive our data architecture.Key Responsibilities:

• Design, build, and maintain data pipelines using Fabric tools such as Delta Lake, Synapse, Databricks, and Azure Data Factory.
• Develop and manage real-time streaming data pipelines using technologies such as Kafka, Flink, or Azure Data Analytics.
• Implement scalable and reliable ETL/ELT solutions to process large datasets.
• Collaborate with data scientists, analysts, and software engineers to design data solutions that meet business requirements.
• Optimise and monitor data pipelines for performance, scalability, and reliability.
• Ensure the integrity and security of data across the organisation's data infrastructure.
• Work with stakeholders to ensure proper data governance and regulatory compliance.
• Troubleshoot and resolve issues with data pipelines and infrastructure as they arise.
• Stay updated with the latest industry trends and technologies in data engineering and apply them to improve our data systems.

• Experience with Fabric tools such as Delta Lake, Azure Synapse, Databricks, and Azure Data Factory.
• Extensive knowledge of streaming services like Kafka, Flink, or Azure Data Analytics (strong priority).
• Proficiency in SQL and experience working with relational and non-relational databases.
• Strong experience with ETL/ELT processes and data modelling.
• Hands-on experience with big data technologies and cloud platforms (Azure preferred).
• Familiarity with data governance, data security, and best practices in handling large datasets.
• Ability to work in an Agile environment and collaborate with cross-functional teams.

• Bachelor's degree in Computer Science, Data Engineering, or a related field (or equivalent experience).
• Experience with Azure DevOps and CI/CD pipelines for data workflows.
• Knowledge of Python, Scala, or other scripting languages for data processing.
• Experience with real-time data processing and event-driven architecture.

PBT Group