Azure Data Engineer

CONTEXT

Imagine the pulsating heart of a major European capital, where millions rely daily on a vast public transport network. Every journey, every signal, every fleet movement generates a colossal stream of data. Our client, a major player in its sector, is on a mission to revolutionize urban mobility, making it smarter, more efficient, and more responsive than ever before. To achieve this vision, they need to harness the power of this data, transforming it from raw bytes into strategic insights. This role is a strategic pillar in their ambitious digital transformation journey, aimed at building a data-driven ecosystem that will redefine urban transport for the next decade. You will be joining a dynamic Data & Analytics team that is instrumental in shaping the future of public transportation, ensuring seamless operations, predictive maintenance, and an enhanced passenger experience for a vast urban population.

ACTIVITIES

• Design, develop, and optimize scalable and resilient data pipelines using Azure Databricks and Azure Data Factory, ensuring high data quality, availability, and performance for critical operational systems.
• Industrialize and operationalize existing data models and analytical calculations, focusing on advanced techniques like anomaly detection to anticipate and prevent operational disruptions in real-time.
• Collaborate intimately with domain experts and business stakeholders across various departments to identify key data use cases and translate complex business requirements into robust, data-driven technical solutions.
• Develop and maintain interactive Power BI dashboards and reports, providing critical, real-time insights for strategic decision-making, operational improvements, and performance monitoring.
• Implement data governance best practices, ensuring data security, privacy, and compliance across all data solutions, safeguarding sensitive information.
• Proactively monitor data pipeline performance, troubleshoot complex issues, and implement continuous improvements for efficiency, reliability, and cost-effectiveness.
• Contribute to the architectural evolution of the Azure data platform, exploring new technologies and methodologies to enhance data processing capabilities.

EXPECTED ACHIEVEMENTS

• By the end of 2026, you will have contributed significantly to delivering a suite of strategic data use cases, directly impacting operational efficiency, predictive maintenance schedules, and overall passenger satisfaction.
• Successfully industrialize several critical data models, enabling automated anomaly detection and proactive problem-solving across the transport network, leading to measurable reductions in downtime and service interruptions.
• Establish a robust, scalable, and high-performance Azure data platform capable of processing vast amounts of real-time and batch data, acting as the bedrock for future innovations.
• Empower business users with intuitive and insightful Power BI reports that drive data-driven decisions and foster a culture of continuous improvement within the organization.
• Play a key role in the digital transformation of urban mobility, leaving a tangible legacy on how millions commute daily and how public transport services are managed and optimized.

ENVIRONMENT

Azure Data Factory, Azure Databricks, Apache Spark, Python, SQL, Power BI, Azure DevOps, Data Lake Storage (Gen2), Azure Synapse Analytics, Data Governance Frameworks, Agile methodologies (Scrum/Kanban), Git.