Summary of Role<\/p>\n
\tThe purpose of the Data Engineer is to leverage their data expertise and data related technologies, in line with the Data Architecture Roadmap, to advance technical thought leadership for the Enterprise, deliver fit for purpose data products, and support data initiatives. In addition, Data Engineers enhance the data infrastructure of the to enable advanced analytics, machine learning and artificial intelligence by providing clean, usable data to stakeholders. They also create data pipelines, Ingestion, provisioning, streaming, self-service, API and solutions around big data that support the strategy to become a data driven organisation.<\/p>\n
Responsibilities<\/p>\n
\tResponsible for the day to day maintenance of data pipelines, providing support to the data squads including performing data related tasks (such as data profiling, data cleaning, data configurations, data support, data validation, data quality assurance) in a Data Epic and assisting in basic data pipelines, data ingestion and supporting Data Engineers in their Epics.
\n\tData Infrastructure: Support and maintain the Data Infrastructure to ensure it is secure, available and reliable.
\n\tData Pipeline Build (Ingestion, Provisioning, Streaming and API).
\n\tMaintain data pipelines starting with Data Virtualisation, then progressing to Data Ingestion and finally to Data Provisioning.
\n\tEnsure that data pipelines are monitored and run successfully and configure and support minor changes in data pipelines.
\n\tData Visualisation: Create virtual data bases and assist in creating data extracts for the business in response to business needs.
\n\tDocumentation and Data Analysis: Collaborate with the Data Analyst to perform data profiling, data validation, and data documentation in support of Epics.
\n\tData Warehouse Monitoring and Support: Monitor data pipelines and infrastructure to provide first line support, resolve issues and ensure that the warehouse meets its SLA timelines for data availability and warehouse reliability.
\n\tCloud Monitoring and Support Services: Ensure cloud processes (Compute and Storage) are monitored and managed daily, to ensure that Cloud pipelines run successfully.
\n\tOperations: Support and run daily operational reports to ensure that all jobs ran successfully and that the data warehouse is maintained according to standard.
\n\tCollaboration: Work collaboratively with business stakeholders to gain business knowledge, understand data extracts and improve and optimise business queries.<\/p>\n
Qualifications<\/p>\n
\tMatric and a Bachelor\u2019s degree in Computer Science, Information Systems, Engineering, or related field.
\n\t7 years of experience in data engineering or a related discipline.
\n\tStrong experience with SQL and relational database technologies.
\n\tProficiency with ETL\/ELT tools and frameworks (e.g., Airflow, DBT, Azure Data Factory, AWS Glue).
\n\tExperience with cloud platforms (Azure, AWS, or GCP).
\n\tKnowledge of big data technologies (e.g., Spark, Hadoop) is advantageous.
\n\tExperience with Python or Scala for data processing.<\/p>\n
Skills<\/p>\n
\tSQL, Python (required)
\n\tCloud services (Azure\/AWS\/GCP)
\n\tData warehousing (Snowflake, Redshift, BigQuery, Synapse)
\n\tBig data frameworks (Spark, Databricks)
\n\tVersion control (Git)
\n\tContainerisation (Docker\/Kubernetes)<\/p>\n
go to method of application \u00bb<\/p>\n
Apply via company website ( ) or<\/p>\n
<\/p>\n","protected":false},"author":2,"featured_media":0,"template":"","meta":{"_promoted":"","_job_location":"","_application":"","_company_name":"Salt South Africa","_company_website":"","_company_tagline":"","_company_twitter":"","_company_video":"","_filled":0,"_featured":0,"_remote_position":0,"_job_salary":"","_job_salary_currency":"","_job_salary_unit":""},"job_listing_region":[11],"job-types":[12],"class_list":{"0":"post-34369","1":"job_listing","2":"type-job_listing","3":"status-publish","4":"hentry","5":"job_listing_region-gauteng","7":"job-type-full-time"},"jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/jobs.dataaxisnode.com\/southafrica\/wp-json\/wp\/v2\/job-listings\/34369","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/jobs.dataaxisnode.com\/southafrica\/wp-json\/wp\/v2\/job-listings"}],"about":[{"href":"https:\/\/jobs.dataaxisnode.com\/southafrica\/wp-json\/wp\/v2\/types\/job_listing"}],"author":[{"embeddable":true,"href":"https:\/\/jobs.dataaxisnode.com\/southafrica\/wp-json\/wp\/v2\/users\/2"}],"wp:attachment":[{"href":"https:\/\/jobs.dataaxisnode.com\/southafrica\/wp-json\/wp\/v2\/media?parent=34369"}],"wp:term":[{"taxonomy":"job_listing_region","embeddable":true,"href":"https:\/\/jobs.dataaxisnode.com\/southafrica\/wp-json\/wp\/v2\/job_listing_region?post=34369"},{"taxonomy":"job_listing_type","embeddable":true,"href":"https:\/\/jobs.dataaxisnode.com\/southafrica\/wp-json\/wp\/v2\/job-types?post=34369"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}