Data Engineering Lead

Key Responsibilities

• lead a team of experienced team members of Data Engineers in designing, developing, and delivering scalable, reliable, and high performing software solutions
• lead the design, development, and maintenance of scalable data pipelines and ETL processes. Monitor and optimize data infrastructure performance, identifying and resolving bottlenecks and issues
• lead the team from a technical standpoint, and drive operational excellence, including code reviews, design reviews, testing, and deployment processes
• be an individual contributor (~60%) engineering the software products/solutions, jointly with the team
• ensure that the team adheres to coding standards, best practices, and architectural guidelines, oversee team spirit and team performance, guide and mentor team members
• oversee the implementation of the technical architecture, solve immediate technical challenges
• implement good practices, coding standards and modern architecture for DataOps; be a “go-to-person for technical decisions and problem-solving within the team
• ensure that the execution of DataOps is in place in the team’s daily work
• inspire, advise, and drive the selection of development approach
• coordinate software development and address technical debt in the team
• hire, onboard, mentor, and develop top engineering talents, fostering a culture of learning
• collaboration, and continuous improvement
• lead technical discussions with other teams/departments and oversees state-of-art quality of the stack
• may be involved in cross-functional discussions, representing the domain in broader technical discussions across domains
• responsible for designing and improving processes that enhance efficiency and quality
• communication with Engineering Manager, Product Owner, Business Analyst and Scrum Master to align on project. / sprint goals, timeline and resource allocation.

Qualification & Requirements

You are a good match if you have:

• 8+ years of working experience in in field of Data Engineering, Software Engineering
• 5+ years of experience in managerial position
• hands-on experience and in-depth knowledge of the technologies listed as mandatory in the Technology Stack section
• strong understanding and implementation of software development principles, coding standards, and modern architecture
• hands-on experience in implementing and managing End-to-End DataOps / Data Engineering projects in a team
• proven ability to lead software development teams of engineers with varying experience and adapt to team sizes from small to large
• experience in working in diverse projects with varying technologies, products, and systems
• strong problem-solving skills and ability to make critical technical decisions
• ability to guide / mentor other team members
• stakeholder engagement/influence skills
• pragmatic, and collaborative team player
• experience with Databricks, Data pipelines, CI/CD
• proficiency in Python, SQL and big data technologies in Cloud environment
• pragmatic, and collaborative team player.

You Are a Good Match If You Know

• strong experience in Azure cloud data services & technologies
• Azure Databricks, Unity Catalog, Delta live tables
• data Modelling & Architecture
• ETL pipeline design
• expert in Python, Pyspark and SQL
• Azure Data Factory
• Azure DevOps
• Logging and Monitoring using Azure / Databricks services
• Apache Kafka
• Nice to have: Synapse, Fabric, PowerBI, Azure Functions, Azure Logic apps, Azure API services, Microsoft Entra ID, GCP & AWS cloud services knowledge, Azure Networking, Databricks online table, Jira, Shell scripting / Azure CLI knowledge.

Benefits

• Insurance Benefits
• Retirement Benefits
• Vacation Policy
• Other Perks and Benefits…