Ingesting data from multiple external vastly different sources at hundreds of rich data points per second, moving terabytes of data while processing it in real-time, running complex and complicated prediction and forecasting AI models while coupling their output into a hybrid human-machine data refinement process and presenting the result through a nimble low-latency SaaS solution used by customers around the globe is no small feat of science and engineering. This processing requires a highly reliable, stable, fault-tolerant infrastructure that can withstand multiple and varied uses and abuses by data analysts, data scientists, industry experts, and the end-users.
The Data Services Team is responsible for the fabric, the foundation, the impressive Amazon AWS estate Vortexa uses to achieve its purpose. We use a wide variety of technologies like Java, Python, Jupyter, Apache Kafka, and AWS services like AWS MSK, EKS, RDS, Neptune, Athena, Airflow, and others.
You will be a key member of the small, globally distributed team, instrumental in assessing current technologies, re-defining stacks and implementing the infrastructure roadmap, ensuring 100% uptime, availability, and fault-tolerance of every component of the platform, optimising AWS costs, as well as helping other teams implement best practices in using key technologies. You will look after multiple mature CI/CD pipelines and ensure they grow and adapt to the infrastructure. Your customers would be data analysts, data scientists, and other engineers and industry experts from across the company.
You Are:
● Striving to be more skilled than Google with Kubernetes
● An AWS power user and evangelist
● Familiar with Apache Kafka and Kafka Streams - deployment, monitoring, resiliency, fault- tolerance, cluster planning and operations, applications debugging
● Experienced with ElasticSearch operations and usage
● Fluent in Terraform, bash scripting and can read some Go
● Familiar with cloud-native application design and deployment
● Not afraid of challenges and infrastructure troubleshooting
● A "can-do" person - "I've done my bit" has no place at Vortexa
Awesome If You:
● Are familiar with Airflow or other workflow orchestration tools
● Have some relevant AWS or Kafka certifications
● Can code in Java or other JVM languages like Kotlin
● Can write some good Python
● Understand data lakes like Parquet, Orc, Athena
We Are:
● A vibrant company pushing ourselves and the technology to deliver beyond the cutting edge
● A team of motivated characters and top minds striving to be the best at what we do at all times
● Learning something new ourselves and help learn others every day
● Acting as company owners, which all of us are - in a business-savvy and responsible way
● Enjoying a friendly working environment
● Not only teammates but friends, often finishing the week enjoying a glass of a favourite drink together
● Offering generous salary and equity
