At Great Eastern, we are building a data platform to realize the full value of our data. We seek an experienced Principal Big Data Engineer to join our growing data engineering and analytics team. The right person for the job will have strong knowledge of Big Data Engineering and proven ability to strategize the implementation of data and analytics capabilities on a data platform (on premise and cloud) — from conception through release and production. The ideal candidate will be a strong data engineer; be willing to wrangle data, optimize data systems and products, and build them from the ground up. Experience with AWS, Snowflake, and migration to public could would be an advantage.The JobThe Principal Big Data Engineer will work with cross-functional teams like Data & Analytics (Data Engineer, Data Visual Analyst and Data Scientist), Data Management and Governance, IT and Digital
Platform to support data curation and analytics.
- Assess and implement robust and scalable Big Data Technologies/Architecture to enable optimal data extraction, ingestion, transformation and storage from a wide variety of data sources.
- Architect end-to-end solution for Business Analytics Product (Dashboards or Statistical Model) from the curation of data, contextualizing data for business analytics and integrating of Product with Business process.
- Build and maintain an efficient, scalable and future-proof deployment infrastructure to enable the development and deployment of production quality analytics and AI applications.
- Engineer, optimize, fine-tune and maintain efficient, secure and reliable data pipelines to ingest, clean and consolidate data sources into the analytics systems and solutions.
- Provide technical guidance to Junior Data Engineers/Analysts on complex data issues, handling of big data sets and use of advanced methodologies.
- Recommend and implement ways to improve data reliability, efficiency and quality through the use of programming languages and big data processing/manipulation tools.
- Conduct research on emerging Big Data Architecture, Technologies and Systems to ensure that they continue to support the requirements of the data scientists and the business stakeholders in the mid to long-term.
- Work with data scientists to operationalize and maintain analytics/ artificial intelligence solutions (e.g. applications, models) by integrating them into business processes. This includes converting proof-of-concepts developed by data scientists into production-grade products and converting machine learning models in Application Program Interfaces (APIs).
- Takes accountability in considering business and regulatory compliance risks and takes appropriate steps to mitigate the risks.
- Maintains awareness of industry trends on regulatory compliance, emerging threats and technologies in order to understand the risk and better safeguard the company.
- Highlights any potential concerns /risks and proactively shares best risk management practices.
- Degree in Computer Engineering, Computer Science, Mathematics, Software Engineering, equivalent fields or proven experience in data engineering
- At least 5-7 years of experience in data engineering, which includes developing and maintaining data, data science infrastructure, and pipelines that use big data and/or cloud platforms
- Technical Skillset
- Experience in delivering and maintaining Big Data Technologies/Architecture
- Experience modelling data for analytics and with DevOps, ModelOps, automation, containerization is preferred
- Hand-on experience working on both relational and non-relational databases
- Minimum 3 years experiences with Cloud Platforms with AWS/Azure
- Minimum 3 years of experience in Hadoop Cloudera and Scala
- Proficiency in SQL, Spark, Python is a must
- Specialized experience, not limited, with Nifi, Kafka, Spark, Hbase, Spark Streaming
- Minimum 5 years of programming experience in preferred languages – Python, R, Java
- Team Player
- Strong verbal and written communication skills and ability to explain technical solutions to the business users and partners
- Good coaching skills. Able to provide effective guidance to junior team members
- High level of integrity, takes accountability of work and good attitude over teamwork.
- Takes initiative to improve current state of things and adaptable to embrace new changes.
To all recruitment agencies: Great Eastern does not accept unsolicited agency resumes. Please do not forward resumes to our email or our employees. We will not be responsible for any fees related to unsolicited resumes.