The candidate will be working with business users and members of the technical team on the architecture, design, and development of highly scalable ETL processes. You will also assist in building interfaces from various upstream systems and ingest the data into Cloudera Hadoop data store. This is a great opportunity for someone who is interested in joining a lean, innovative group with the possibility of tremendous career development in data engineering, big data management and data analytics.
- Influencing role to product specifications, build and support a reliable enterprise data warehouse to meet the development and maintenance requirements of systems/platforms
- Develop and implement technical best practices for data movement, data quality, data cleansing, and other ETL-related activities. Develop and ensure adherence to locally defined standards for all developed components.
- Maintain “common code” libraries, so objects can be shared and folders for all Environments (Dev, System Test and Production), Code Check-In/Check-Out procedures and assists in managing SIT/DEV.
- Use the SSIS platform to extract, transform, and load data. Develop, test, integrate, and deploy ETL routines using ETL tools and external programming/scripting languages as necessary. Write and maintain unit test cases
- Design ETL processes and develop source-to-target data mappings, integration workflows, and load processes.
- Over 10 years of experience with the Data Warehouse architecture, design, development, and deployment of ETL processes using SSIS and development through Microsoft Visual C#
- Knowledge in SSIS, SQL, Kafka, Data Modeling, design & analytics
- Good knowledge of BI architecture, data warehousing concepts, and data vault model
- Strong knowledge of Python, SPARK, Java Scala, HIVE and Beeline with hands on experience
- Knowledge and experience of web service development using Java is an advantage
- Knowledge and experience with Control M Scheduler is a plus
- Knowledge and experience with DevOps practice and tools is a plus
- Bachelor’s degree in Business, Computer Science, Management Information Systems, Engineering, or a related field