big data engineer jobs in Milnerton

PBT Group has an opportunity for a Kafka Big Data Engineer. As a Kafka Engineer, you will be responsible for the building, improvement and scaling of our streaming data platform. This role requires a combination of strong technical skills, deep understanding of distributed systems as well as excellent communication abilities. Kafka engineer is a big data engineer who specializes in developing...

Client Details:Role Responsibilities:Collecting, storing, processing, and analysing huge sets of data. Selecting and integrating any Big Data tools and frameworks required to provide requested capabilities. Implementing ETL process. Monitoring performance and advising any necessary infrastructure changes. Defining data retention policies. Preferred Qualifications:Relevant tertiary...

Overview Hire Resolve’s client is urgently seeking the expertise of an experienced Backend Data Engineer in Cape Town (Hybrid). Responsibilities Collaborate with cross-functional teams to define system requirements, design solutions, and ensure the delivery of high-quality software. Advocate for best practices in Data processing and API designs, systems architectures, automation, and system...

Job Description Are you a Senior Data Engineer seeking new career opportunities? Well we have the right job for you! A well-established IT company is seeking a Senior Data Engineer based in South Africa. Requirements: •5+ years’ experience in big data engineering •Skilled in Python (PySpark) and AWS. •Experienced in SQL, data modelling, ETL development, and Data warehousing. •Previous...

Job Description This is an opportunity for a Data Engineer to gain experiencearchitecting scalable & efficient data solutionssolve global issues in telecommunications, sustainable water management & energy. Working 100% Remotely | R1.2M – R1.4M per annum | Based in South Africa THE COMPANY This company is a leading global software, data, and AI solutions provider renowned for solving...

SQL Hadoop SQL MS data engineering data warehousing Python, Java, or Scala Analytical Machine Learning - Degree or diploma in IT or related technical discipline. - At least 3 years in a technical role with experience in data warehousing, and data engineering. - Proficiency in programming languages such as Python, Java, or Scala for data processing. - Proficiency in SQL for data processing...

Got Drive? Join AnyVan as we make moving anything, anywhere miles better and build a career that moves just as fast. Back in 2009, our CEO Angus saw half-empty vans everywhere and knew there had to be a better way! That's how AnyVan began. He set out to create the world's most efficient logistics technology and help halve the number of wasted miles by filling those empty vans. Since then we've...

This is a remote position. About our client: Our client develops and supports software and data solutions across a variety of industries. They want you to get ahead of the market and stay there. They offer a combination of plug and play products that can be integrated with existing systems and processes and can also be customised to client needs. Their capabilities extend to big data...

We are seeking a Senior Data Engineer to play a pivotal role in advancing Outsourced CFO's technology vision. This individual will be responsible for executing the vision for the organization’s technology solutions and systems, ensuring alignment with strategic goals. The role involves co-designing and implementing innovative technologies that drive operational excellence and support...

Job Summary: The Data Engineer will manage and analyze the company's data from battery systems, microgrids, and energy management solutions. This role includes designing databases, developing analytics/reporting systems, and applying machine learning (ML) for actionable insights. The Data Engineer will also integrate AI/ML into the EMS product to drive innovation in energy management. Key...

We partner with a wide network of innovative organisationsfrom agile start-ups to global enterpriseslooking for data professionals to design, analyse, and optimise solutions that inform critical business decisions. These roles offer exposure to modern data platforms, diverse technical challenges, and environments that encourage learning, innovation, and flexibility. Whether you thrive in...

Job Description Do you have exceptional ideas? Do you consider yourself determined and result-driven? We have just the role for you! We are looking for someone who can continuously improve quality and customer value as well as bring in passionate and exciting ideas. Key Requirements: •4-7 years’ experience •IT Degree or Diploma •Spark, ETL, Big Data •Hadoop, Airflow, Kafka •AWS, Azure or...

Youll combine deep technical expertise with leadership, guiding engineers, shaping architecture, and delivering mission-critical data solutions that power real business outcomes. If youre passionate about data engineering, ownership, and building systems that truly matter, this is the role for you. Key Responsibilities: - Data Engineering & Migration Leadership: - Design and deliver...

ENVIRONMENT: A Cape Town Travel Tech company that focuses on the global supply and revenue optimisation of travel ancillary products and services is looking for a hands-on Data Engineer to take ownership of- and optimise their entire data engineering function. As the sole data engineering expert in a small team, you'll design, build, and maintain robust data pipelines that power business...

PBT Group is currently offering an opportunity for an AWS Data Engineer with 2 to 5 years of relevant experience. The role of a Data Engineer involves constructing and maintaining data pipelines and datamarts, emphasizing scalability, repeatability, and security. Data Engineers play a pivotal role in facilitating the acquisition of data from diverse sources, ensuring its conformity to data...

ENVIRONMENT: A bold Marketing Consultancy for challengers is on the hunt for a Senior Data Engineer to join its Cape Town-based Media & Data team. This is not your typical data role — it's for someone who thrives on building systems that power breakthrough marketing strategies designed to disrupt the norm, outsmart competitors, and deliver outsized results. You'll take charge of architecting,...

PBT Group has an opportunity for a Data Streaming Engineer to build, enhance, and maintain real-time data pipeline. Work with various infrastructure and operations teams to maintain data infrastructure. As a senior engineer on the Data Streaming Platform team, you will be responsible for developing and maintaining data ingestion pipelines vital to the continued growth. You will collaborate with...

PBT Group has a requirement for an intermediate DevOps Data Engineer. Duties: - Design, build and operationalize large scale enterprise data solutions and applications using one or more of AWS data and analytics services in combination with 3rd parties – Glue, Step-functions, Kafka CC, PySpark, DynamoDB, Delta.io, RedShift, Lambda, DeltaLake, Python,. - Analyze, re-architect and re-platform...

PBT Group, leaders in Business Intelligence, has a vacancy for a Microsoft SQL Data Engineer. Duties: - Plan and analyse complex business requirements and implement technology enabled solutions to address multi-discipline business opportunities/problems. - Conduct planning, analysis and design activities in conjunction with other development specialists. - Participate in analysis of complex...

PBT Group is looking for a Platform Data Engineer to join a growing Platform Engineering team within a large financial services environment. The focus of this role is on building and enabling the data platform, rather than traditional reporting or analytics delivery. These engineers will play a key role in designing, building, and maintaining the Microsoft Fabric platform, with a strong emphasis...