Senior Data Engineer
Cape Town, Western Cape, South Africa
Job Description
Responsibilities:
- Design and implement scalable data pipelines using Apache Spark, Scala, and Java
- Develop and maintain data architectures for batch and real-time processing
- Collaborate with cross-functional teams to identify and prioritize project requirements
- Optimize data processing performance, scalability, and reliability
- Ensure data quality, security, and compliance
- Mentor junior data engineers and contribute to knowledge sharing
- Stay up-to-date with industry trends and emerging technologies
- Relevant Degree
- 6+ years of experience in data engineering (Spark, Scala, Java)
- Apache Spark (Core, SQL, Streaming)
- Scala programming language
- Java (Java 8 and above)
- Gradle build tool
- Data architectures (batch, real-time, lambda)
- Data storage solutions
- Big Data technologies (Hadoop, Hive, Impala)
- Cloud platforms (AWS, Azure, Google Cloud)
- Agile development methodologies
- Version control systems (Git)
- Excellent problem-solving, analytical, and communication skills
- Apache Kafka, Apache Flink, or other streaming technologies
- NoSQL databases (MongoDB, Couchbase)
- Data science concepts (machine learning, R, Python)
- Containerization (Docker)
- Orchestration tools (Apache Airflow, Kubernetes)
- Certification in Spark, Scala, or Java development
- Familiarity with DevOps practices
If you have not had any response in two weeks, please consider the vacancy application unsuccessful. Your profile will be kept on our database for any other suitable roles / positionsFor more information contact:
LaReine Mulder
Specialist Recruitment Consultant
ExecutivePlacements.com
Beware of fraud agents! do not pay money to get a job
MNCJobs.co.za will not be responsible for any payment made to a third-party. All Terms of Use are applicable.
Related Jobs
Job Detail
-
Job IdJD1354142
-
IndustryNot mentioned
-
Total Positions1
-
Job Type:Full Time
-
Salary:Not mentioned
-
Employment StatusPermanent
-
Job LocationCape Town, Western Cape, South Africa
-
EducationNot mentioned