Job Description

Data Engineering Lead (Cape Town)The Intelligent Debt Management (IDM) Group, South Africa's largest and multi-award-winning debt management company, has a position available for a Data Engineering Lead.This individual will join the Data Team and will be accountable for mostly (but not limited to) data engineering design and implementation across on premise and cloud based platforms.Qualifications:

• Bachelor's degree in Computer Science, Engineering, or a related field.
• Certifications in Azure, Azure Databricks Data Engineering or similar or equivalent practical experience.

Candidates must be able to demonstrate:

• Extensive proven experience as a data engineer or in a similar role, designing and implementing data architectures, data integration and ETL processes. Practical experience in Azure, Azure Data Factory and Azure Databricks is preferred.

• Proven experience in Python/PySpark and SQL/SparkSQL programming skills.

• At least 4 years of experience in Information Technology - specifically in data management.
• Strong knowledge of data management and storage solutions, including SQL Server Database, Azure Data Lake Storage or similar.
• Ability to lead and innovate in a fast-paced and evolving technology landscape.
• Strong communication and leadership abilities.
• Excellent attention to detail and problem solving skills related to data, data structures and algorithms.

Key responsibilities will include:

• Lead the design and implementation of scalable data solutions on the Microsoft Azure cloud platform, including Azure Databricks.
• Develop, maintain and optimise robust data pipelines for data ingestion, processing, and transformation to support analytics, machine learning and other data-driven services.

• Ensure data security, quality and integrity across all of the pipelines and related data processes.
• Define the mechanism through which skilled decentralized analytics teams can contribute to the engineering processes.
• Work closely with the Head of Data and enterprise architecture to ensure that the data engineering vision is aligned with the systems architecture and the data strategy.
• Collaborate with the data product owner, analysts, data scientists, and business stakeholders to understand data requirements and implement effective data workflows.
• Assist in data initiatives and support technical teams including architects, software developers, and data scientists.
• Provide leadership and mentoring to a small, skilled team of data engineers.

What's in it for you?Being South Africa's largest company of its kind - IDM is continuously growing, with no signs of stopping.One of our core values is 'growing people' and we take that very seriously. We have internal learning opportunities to better our people, and leaders who are experts in their fields and freely impart knowledge and are open to new ideas. As one of our people said, xe2x80x9cyou can speak to anyone here. Your rank doesn't matter. You are listened to.xe2x80x9dOur culture is one of collaboration, diversity, and helpfulness. That goes for our people internally and how we deal with our clients.At IDM, you can play a part in educating and assisting real South Africans get out of tough financial situations. You can make a difference with the work you do. Our people believe this too - one said, xe2x80x9cI wake up each day knowing I am going to make a difference.xe2x80x9dOther key benefits:

• We make sure our employees are mentally and physically healthy with Healthy Together campaigns throughout the year.
• We offer various rewards and awards as tokens of recognition for hard work.
• We offer company benefits - like a half day off on your birthday, medical aid, retirement annuity, and further study opportunities should you meet relevant requirements.
• We have a culture of growth, meaning you have access to move both upwards or across in the business if you want to.

IDM Group