- Advanced SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases.
- Experience building and optimizing big data' data pipelines, architectures and data sets.
- Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
- Working knowledge of message queuing, stream processing, and highly scalable big data stores.
- We are looking for a candidate with 5+ years of experience in a Data Engineer role, who has attained a Graduate degree in Computer Science. They should also have experience using the following software/tools:
- Experience with big data tools: Hadoop, Spark / PySpark, Kafka.
- Experience with relational SQL and NoSQL databases.
- Experience with object-oriented/object function scripting languages: Python.
- Expertise in data modeling and building data-driven systems.
- Well-versed with Shell Scripting.
- Hands-on experience on various AWS services like EMR, EC2 Glue.
- Deploy existing data projects using CICD pipelines.
- Knowledge on Docker is a plus.