We are looking for a Senior Data Engineer who will be a key part of our team and will help bring structure to vast amounts of data, making it digestible and build scalable data platforms that enable data products, business analytics, and data science. This role requires technical expertise and willingness to learn a wide variety of technologies to develop real-time data pipelines, data product APIs, and modern & scalable data infrastructure. If you are interested in working in a fast-paced environment and like being challenged with fun data problems to solve, come join us.
- Simplify access to real-time data for internal stakeholders (Kafka).
- Design and implement our real-time data pipelines using Kafka.
- Improve and maintain the data lake setup (S3, EMR, Presto).
- Set up fully automated alerting systems to proactively monitor the infrastructure (Datadog, Pagerduty).
- Continuously improve the overall performance of analytical queries.
- Build scalable data infrastructure using Terraform.
- Build customer-facing data products with a focus on reliability and scalability.
- Ensure data quality through automated audits.
- Collaborate with analysts, engineers, and business users to design solutions.
- Research innovative technologies and make continuous improvements.
You may be a good fit if
- 5+ years of relevant industry experience.
- Excellent knowledge of Python and SQL.
- Demonstrated ability to build high-volume data ingestion and streaming pipelines (e.g. Kafka, AWS Kinesis, Spark Streaming)
- Working experience with big data technologies (e.g. Spark, Presto, Hive).
- Experience designing, building, and scaling a production-ready event streaming system.
- Experience in optimizing SQL queries (e.g. data partitioning, bucketing, indexing)
- Experience in building data lake/warehouse solutions consisting of structured and unstructured data.
- Experience with IaC is a plus (e.g. Terraform, CloudFormation)
- Bachelor's degree in a technical field or equivalent work experience.
- You have built data products that have scaled on AWS or another cloud.
- You thrive on nimble, lean, fast-paced startups, like autonomy, and have proven you can push towards a goal by yourself.
- Coachable. Able to own mistakes, reflect, and take feedback with maturity and a willingness to improve.
- You communicate with clarity and precision and you are able to effectively present results.
What We Care About
- Solve for the customer first: You build what customers want. You think about what is right for customers, not what is easiest for you
- Demonstrate mastery of honey badgery: You make ambitious goals. Then execute…no matter what stands in the way. When knocked down, you get up
- Take on challenges willingly and can be trusted to execute: You can be trusted to get things done right the first time quickly. You hit your deadlines
- You’re like us: You smile a lot, think work is fun and don’t take yourself too seriously. You measure yourself against the best and believe feedback is the breakfast of champions. You follow the golden rule.
- You’re remarkable: People naturally talk about how awesome you are. If we can’t find someone who raves about you then it’s unlikely we will too.