Data Engineer

Job Description

We are looking for an experienced developer responsible for the design, development and maintenance of applications. You will be working alongside other engineers and developers working on different layers of the infrastructure. Therefore, a commitment to collaborative problem solving, sophisticated design, and the creation of quality products is essential.

Responsibilities:

Collaborate with Solution Architects to design, prototype, implement, and optimize data ingestion pipelines so that data is shared effectively across various business systems

Build data ingestion pipelines and design optimal data storage and analytics solutions using cloud and on-perm technologies

Ensure the design, code and procedural aspects of the solution are production ready, in terms of operational, security and compliance standards

Participate in day to day project and production delivery status meetings, and provide technical support for faster resolution of issues

Skills

Proficient in Python or Java/Scala

Hands-on Experience With Some Of The Following Technologies:

YARN, HDFS, Hive, ELK/Splunk

Kubernetes

Airflow/Oozie

Spring or any other DI framework,

RESTful Web Services.

Proficient in querying and manipulating data from various DB (relational and big data)

Experience of writing effective and maintainable unit and integration tests for ingestion pipelines

Experience of using static analysis and code quality tools and building CI/CD pipelines

Exposure to microservices architecture would be preferable

A self-motivated learner and builder