Senior Aws Data Engineer
Pretoria, GP, ZA, South Africa
Job Description
You will be part of a team responsible for building and maintaining Big Data Pipelines using Data Platforms.
Be a custodianof data and must ensure that data is shared in line with the information classification requirements on a need to know basis.
You will be expected to:
Stay up to date with the latest data engineering tools, technologies and industry trends.
Identify opportunities for process improvements and automation to enhance the efficiency and reliability of data pipelines.
Explore and evaluate new data engineering approaches and technologies to drive innovation within the organisation.
Mentor, train and upskill members in the team.
Experience in the following is required:
Terraform
Python 3x
SQL - Oracle/PostgreSQL
Py Spark
Boto3
ETL
Docker
Linux / Unix
Big Data
Powershell / Bash
Cloud Data
Basic understanding of AWS components such as:
Glue
CloudWatch
SNS
Athena
S3
Kinesis Streams (Kinesis, Kinesis Firehose)
Lambda
DynamoDB
Step Function
Param Store
Secrets Manager
Code Build/Pipeline
CloudFormation
Business Intelligence (BI) Experience
Technical data modelling and schema design ("not drag and drop")
Kafka
AWS EMR
Redshift
Experience working with Enterprise Collaboration tools such as Confluence, JIRA.
Experience developing technical documentation and artefacts.
Knowledge of data formats such as Parquet, AVRO, JSON, XML, CSV etc.
Experience working with Data Quality Tools such as Great Expectations.
Experience developing and working with REST API's is a bonus.
Basic experience in Networking and troubleshooting network issues.
Relevant IT Degree / Diploma and AWS Certifications.
8+years related experience.
Between 7 - 10 Years
Beware of fraud agents! do not pay money to get a job
MNCJobs.co.za will not be responsible for any payment made to a third-party. All Terms of Use are applicable.
Related Jobs
Job Detail
-
Job IdJD1398682
-
IndustryNot mentioned
-
Total Positions1
-
Job Type:Full Time
-
Salary:Not mentioned
-
Employment StatusPermanent
-
Job LocationPretoria, GP, ZA, South Africa
-
EducationNot mentioned