Back to Search Results

Senior Data Engineer

Cape Town
Salary: R90 000.00 - R110 000.00 Per Month
Job Type: Permanent
Sectors: Technology
Reference: ITE006395/SAN

Apply before Nov 17 2024 | 16 Days left

Apply

External Application

Vacancy Details

Employer: Network Recruitment

Our client, a technology company with an international client base, is recruiting for a professional and self-motivated Senior Data Engineer to join their highly skilled team.

Senior Data Engineer
Fully remote, Cape Town

Qualifications:

BSc Degree in IT / technical related field

Requirements:

At least 5 years' experience with a minimum of 2 years' experience in building and optimising big data pipelines, architecture, and maintaining data sets
Experienced with Python, SQL (PostgreSQL, MS SQL), AWS, Azure or GCP, version control, CI/CD and GitHub
Knowledge and experience with Glue and PySpark is advantageous
Expertise in managing the data lifecycle
Proficiency in handling, processing, and designing large, unconnected datasets to meet analytical needs
Ability to maintain and optimize processes related to data transformation, data structures, metadata, dependencies, and workload management
Solid understanding of data management principles, including data quality assurance and governance
Strong analytical skills for working with unstructured datasets
Familiarity with message queuing, stream processing, and highly scalable big data storage systems
Strong attention to detail
Excellent communication and interpersonal skills

KPA’s:

Identify and implement efficiencies by automating manual processes, optimizing data delivery, and ensuring the scalability of technology
Design and implement enhancements and new features across data systems
Streamline CI/CD processes for data systems to improve the performance of data pipelines
Assemble large, complex datasets that meet both functional and non-functional business requirements
Expertise in ETL processes for data management
Develop and perform unit tests on data pipelines, ensuring data consistency
Create and maintain automated monitoring solutions
Support the infrastructure for reporting and analytics
Ensure data quality, governance, and the overall maintenance of data infrastructure systems, such as AWS and database security
Manage data warehouse and data lake metadata, maintain the data catalog, and provide user documentation for internal business users
Ensure best practices for databases, including collations, database engines, and indices, are implemented and maintained

Tools & Technologies:

Python
SQL
PySpark
AWS
Airflow
Docker
Terraform

Apply now!

For more IT jobs, please visit https://www.networkrecruitmentinternational.com/

If you have not had any response in two weeks, please consider the vacancy application unsuccessful. Your profile will be kept on our database for any other suitable roles / positions.

For more information contact:
Sanet Snooke
Recruitment Consultant
012 348 7559

Apply

148 people have viewed this job.

About Network Recruitment

Network Recruitment

Network Recruitment is a leading specialist recruitment company in the fields of IT, Finance and Engineering.

Since our establishment in 1987, we have been at the forefront of the specialist recruitment industry. Network Recruitment’s key to success is the focus on delivering superior quality service by developing and maintaining an exclusive focus on sourcing high-calibre skills for medium-sized to JSE-listed companies operating in these three niche sectors across South Africa.

Our experience in permanent and contract-based recruitment enables us to provide you with recruitment solutions that meet your long- and short-term needs.

Network Recruitment is a division of ADvTECH Resourcing, the recruitment arm of the national JSE-listed ADvTECH Limited Group a leader in quality education, training, skills development and placement services.