Candidate for the role is expected to be passionate about working with huge datasets and have the experience working with businesses to build data products and services to turn data into insights using advanced analytics. Candidate should have experience with curation of data for analytics/AI, and a strategic/long term view on architecting data eco systems. Candidate is experienced in building efficient and scalable data services and has the ability to integrate data systems with relevant tools and services to support a variety of customer use cases/applications.
- Translate business requirements into robust, scalable, operable solutions with a flexible and adaptable data architecture.
- Implement and adopt best practices in data system creation, data integrity, test design, analysis, validation, and documentation.
- Analyse and organize raw data – structured and unstructured data. Designing, implementing, and operating large-scale, high-volume, high-performance data structures for analytics and data science
- Build/Develop data systems and pipelines.
- Data sources - Liaise with Source system teams to identify and validate data to ensure that data are complete, reliable and clean for data ingestion.
- Ingestion components - Implementing data ingestion routines both real time and batch using best practices in data modeling, ETL/ELT processes by leveraging on relevant technologies and big data tools
- Transformation functions (e.g. filtering and aggregation)
- Destinations (a data warehouse or data lake)
- Design, develop, test and deploy frontend visualization (dashboards and reports) in collaboration with business end users. Helping continually to improve ongoing reporting and analysis processes, automating or simplifying self-service modeling and production support for users.
- Implement solutions to facilitate more effective data discovery by data users
- Working internally with business units and Data Analytic teams to:
- Identify opportunities for enhancements in data management capabilities, and work with relevant stakeholders to address operational or data issues in the data pipeline
- Ensure data management processes comply with established framework & policies
- Develop a good understanding of existing solutions and be able to support and enhance them
- Maintain good documentation of the solutions and effectively communicate stakeholders about the value-added
- Preparing data for prescriptive and predictive modelling
- With the implementation of the data lake, candidate will be part of the team to build up, pilot and gain knowledge and proficiency in the AWS cloud hosted infrastructure and data analytic tools, as well as managing and developing data insights from the new data sources.
- As the position has a key role in continuing to support and further enhance/extend the business analytics needs of the organization, searching for an appropriate candidate to fill the vacancy is highly essential and immediate.
- At least 5 years of relevant work experience as a Big Data Engineer. With demonstrated strength in ETL/ELT (SSIS, Talend, Informatica), data modelling, data warehouse technical architecture and reporting/analytic tools.
- Hands-on experience in AWS cloud services. E.g. S3, redshift
- Related working experience, specifically in the areas of data management and quality
- Strong pyspark, python, SQL, Tableau skills
- Ability to deal with ambiguity and prioritise/manage multiple tasks, with good problem-solving skills to ensure smooth data-to-insights conversion
- Willing to listen to multiple stakeholders and forge consensus on win-win solutions to meet sound data governance and management principles
- Exhibit values and principles of an Agile mindset