Job Description

Intermediate Data Engineer
JohannesburgResponsibilities:

• Develop and manage data pipelines to support data scientists, analysts, actuaries, and other initiatives.
• Ensure accurate, efficient, and cost-effective development of robust data systems in line with the organization's data strategy.
• Implement data validation processes to maintain clean, high-quality data.
• Establish and enforce guidelines for data collection, integration, and retention.
• Monitor data sources, processing, and environment for inconsistencies or anomalies, and resolve data quality problems.
• Leverage Azure DevOps Repos for version control and foster a collaborative development environment.
• Implement CI/CD practices using Azure Pipelines to automate and streamline build and deployment processes.
• Ensure seamless integration between code repositories and deployment pipelines.
• Build and nurture relationships across the business to ensure alignment and commitment to data engineering objectives.
• Engage effectively with stakeholders to drive the success of data initiatives.
• Participate in assessment and development planning activities, formal and informal training, and coaching.
• Maintain an understanding of relevant technology, external regulation, and industry best practices through ongoing education and professional development.
• Up-skill in all facets of data engineering.

Skills:

• Expert knowledge in Azure data services including Azure Data Factory, Azure Databricks, Data Lake, SQL Databases, Azure DevOps Boards and Repos.
• Proficiency in large-scale data processing with Apache Spark.
• Knowledge of GCP and AWS is desirable but not essential.
• Proficiency in Python and SQL is imperative.
• Exposure to PySpark is advantageous.

Qualifications:

• Grade 12/SAQA Accredited Equivalent (Essential)
• 3-year Degree in Information Technology/Sciences/Informatics/Computer Sciences (Essential)
• Masters Degree in Information Technology/Sciences/Informatics/Computer Sciences (Advantageous)

Experience:

• 2 to 5 years of experience in data engineering, data science, or computer science (Essential)
• Experience in the Financial Services Industry, specifically Short-Term Insurance (Advantageous)

We appreciate your interest, however if you have not had any response within 2 weeks, please consider your application unsuccessful.
We also invite you to contact us to discuss your next career move in IT.IT Consultant

ExecutivePlacements.com