Summary
Date Posted: Monday, June 10 2024, Base Salary: Not Disclosed

JOB DETAILS:
DUTIES AND RESPONSIBILITIES
• To design, implement and manage big data collection and pre-processing of structured and unstructured data from various sources, such as databases, APIs, streaming platforms, and files;
• To analyse and handle large volumes of data and utilize frameworks like Apache Hadoop and Apache Spark to distribute data processing tasks across multiple nodes;
• Designing and maintaining robust Extract, Transform, Load (ETL) pipelines to ensure smooth data flow and integration from various sources;
• To optimize data processing pipelines for performance and cost-effectiveness, utilizing technologies such as Hadoop, Spark and other Open Source technologies;
• To integrate disparate datasets from different sources, formats and schemas, maintaining data lineage and metadata management;
• Apply the use of appropriate Machine Learning algorithms and models for extraction of useful information from large datasets to identify patterns, trends and relationships;
• To collaborate with cross-functional teams including data analysts, and business stakeholders to understand data requirements and ensure data accessibility and usability;
• To design and implement scalable data architectures and storage solutions to accommodate the volume, variety, and velocity of big data, leveraging technologies such as HDFS and OLAP (Online Analytical Processing) databases;
• To define data partitioning, indexing, and compression strategies to optimize storage efficiency and query performance;
• To establish and enforce data governance policies, standards, and best practices to ensure data privacy, security, and compliance with Laws and regulations;
• To implement access controls, encryption, and auditing mechanisms to protect sensitive data and mitigate risks of data breaches or unauthorized access;
• To monitor data pipelines and systems for performance, availability, and reliability, proactively identifying and resolving issues to minimize downtime and data loss;
• To conduct regular maintenance tasks such as data backups, system upgrades, and capacity planning to ensure the stability and scalability of the infrastructure;
• To assist in developing and update technical documentation;
• To perform other related duties as may be assigned by the Supervisor.

QUALIFICATION AND EXPERIENCE
• Holder of Bachelor’s Degree in one of the following fields: Computer Science, Electronic Science, Computer Engineering, Information Technology, Information Systems, Data Science or equivalent qualifications from recognized institution

REMUNERATION TCRAS 6

Job Experience: No Requirements

Work Hours: 8

Experience in Months:

Level of Education: Bachelor Degree

Job application procedure
Interested and Qualified? Click here to apply

All Jobs

QUICK ALERT SUBSCRIPTION