Data Engineer (spark)

Cape Town, Western Cape, South Africa

Job Description


SUMMARY:We are looking for an experienced Data Engineer to join our team and play a key role in building and optimizing scalable data solutions. As a Data Engineer, you will design, implement, and maintain data architectures using cutting-edge technologies like Spark, while collaborating with a dynamic team to deliver high-quality solut...POSITION INFO:We are looking for an experienced Data Engineer to join our team and play a key role in building and optimizing scalable data solutions. As a Data Engineer, you will design, implement, and maintain data architectures using cutting-edge technologies like Spark, while collaborating with a dynamic team to deliver high-quality solutions that meet business needs.What youll do:

  • Implement scalable data architectures using technologies like Spark.
  • Evaluate and recommend new technologies and approaches to improve the performance, scalability, and reliability of our software systems.
  • Build and maintain data pipelines and compute tiers, primarily operating on Spark (Hadoop knowledge is a plus).
  • Serve as a technical resource for team members and mentor junior engineers.
  • Collaborate with the team to deliver high-quality solutions that meet business requirements.
  • Ensure that code is well-designed, maintainable, and adheres to best practices and standards.
  • Play a key role in shaping the direction of
practices by working on a scrum-sized team empowered to organize and manage the teams operations. * Utilize project development tools like JIRA, Confluence, and GIT.
  • Write Python/SQL scripts to automate ETL processes.
  • Build and maintain operations tools for monitoring, notifications, trending, and analysis.
  • Define, create, test, and execute operations procedures.
  • Assist the DevOps Engineer in automating CI/CD practices.
Your expertise:
  • 4 to 8 years of experience in Data Engineering/Development.
  • Strong experience in Python, SQL, and Data Flow.
  • Hands-on experience with Spark development and a background in big data.
  • Experience with Linux is preferred.
  • Experience in AWS (AWS certification is a plus).
  • Solid understanding of data modeling and data architecture design.
Personal attributes:
  • Proven ability to work creatively and analytically in a problem-solving environment
  • Confidence to express ideas as part of a team
  • Excellent communication (written, oral) and interpersonal skills.
  • A Self-Starter capable of working under pressure
  • Curious and engaged in Continuous Learning and Professional Development
Other information applicable to the opportunity:
  • Contract Position (12 Months)
  • Location:xc3x82 Johannesburg orxc3x82 Cape Townxc3x82
Why work for us?Want to work for an organization that solves complex real-world problems with innovative software solutions? At iOCO, we believe anything is possible with modern technology, software, and development expertise. We are continuously pushing the boundaries of innovative solutions across multiple industries using an array of technologies.xc2xafYou will be part of a consultancy, working with some of the most knowledgeable minds in the industry on interesting solutions across different business domains.xc2xafOur culture of continuous learning will ensure that you will have all the opportunities, tools, and support to hone and grow your craft.xc2xafBy joining IOCO you will have an open invitation to our inspiring developer forums. A place where you will be able to connect and learn from and with your peers by sharing ideas, experiences, practices, and solutions.xc2xaf

ExecutivePlacements.com

Beware of fraud agents! do not pay money to get a job

MNCJobs.co.za will not be responsible for any payment made to a third-party. All Terms of Use are applicable.


Related Jobs

Job Detail

  • Job Id
    JD1361497
  • Industry
    Not mentioned
  • Total Positions
    1
  • Job Type:
    Full Time
  • Salary:
    Not mentioned
  • Employment Status
    Permanent
  • Job Location
    Cape Town, Western Cape, South Africa
  • Education
    Not mentioned