>

Principal Data Engineer (Big Data)

ST Engineering

  View Insights

Apply

Job Summary


Job Type
Permanent

Seniority
Principal

Years of Experience
At least 6 years

Tech Stacks
ETL
Oracle
NewSQL
Oracle DB
Kubeflow
Analytics
HDFS
UNIX
Neo4j
Hive
Spark
NoSQL
Kubernetes
kafka
PostgreSQL
Scala
MongoDB
Cassandra
Hadoop
Linux
Python
Java

Job Description


  • Participate in the design, development, and testing of highly scalable, expandable and maintainable ‘Smart Enterprise Platform’ product 
  • Setup and maintain big data, machine learning and Kubernetes clusters 
  • Develop tools and software components for the platform that can be used by different projects. 
  • Work with Data Scientists, Data Analysts, and other internal stakeholders to assist with data-related technical issues and support their data pipeline infrastructure and data preparation needs. 
  • Convert models from Data Scientists/Analysts into optimized & scalable production grade ML code. 
  • Design, develop, and maintain optimal data pipeline for Big Data platform. 
  • Develop, evaluate, recommend and maintain Big Data reporting and visualization applications 
  • Troubleshoot software and system issues.
 
Job Requirements:
  • Bachelor or Master’s degree computer science, software engineering, information systems or related field. 
  • The candidate should have at least 6 years of technical experience in Information Technology with at least 3 years, preferably 5 years in Big Data, Data Warehousing or Business Intelligence technology with knowledge of analytics and AI technologies 
  • Broad knowledge of various aspects of Big Data with good understanding and hands-on experience in Hadoop based technologies such as HDFS, Hive, Spark, Kafka etc. 
  • Deep understanding of relational, NoSQL, NewSQL database technologies such as PostgreSQL, Oracle DB, CitusDB, MemSQL, Cassandra, MongoDB, Neo4J etc. 
  • Good knowledge in programming languages such as Java, Python or Scala on Linux/Windows platforms. 
  • Ability to write production grade ML code for edge devices will be a major plus point 
  • Experience in Kubernetes and Kubeflow is a plus point 
  • Experience in Big Data visualization and reporting software. 
  • Experience in designing ETL/BI solutions. 
  • Experience in DevOps and DataOps
  • Familiar with Linux/UNIX system administration
  • Experience in operational support in delivering Big Data solutions.
  • Effective oral and written communication with strong analytical, problem solving, multitasking and project management skills are essential on the job 

Salaries

There are no salaries from ST Engineering that are similar to this job

View more salaries from ST Engineering
Apply

NodeFlair Insights of ST Engineering