Azure Data Engineer

Company Description Showtime Consulting is a leading provider of Shielded Cloud and Digital Solutions across Australia and New Zealand. We specialise in delivering secure, enterprise-scale technology solutions across cloud, infrastructure, cybersecurity, business applications, data, and digital transformation programs. We partner with government and enterprise organisations to build high-performing technology teams that deliver scalable, secure, and future-ready business solutions. The Role We are seeking an experienced Data Engineer to design, build, and support enterprise-scale data platforms and data integration solutions within Microsoft Azure environments. This role is suited to a hands-on data professional with strong experience across data engineering, data warehousing, cloud-based analytics platforms, and modern data architectures. You will work closely with business stakeholders, solution architects, analysts, and development teams to deliver high-quality, trusted, and scalable data solutions that support reporting, analytics, and business decision-making. You will play a key role in designing and optimising data pipelines, improving data quality, implementing governance controls, and driving best practices across the data ecosystem.

Key Responsibilities
• Design, develop, and maintain scalable and reliable data pipelines and ETL/ELT processes.
• Build and support Azure-based data platforms and enterprise data solutions.
• Develop data integration frameworks to ingest, transform, and deliver data from multiple source systems.
• Implement and support solutions using Azure Data Factory (ADF), Azure Databricks, Azure Data Lake Storage (ADLS), Azure Synapse Analytics, and Azure SQL.
• Design and optimise data warehouse and lakehouse architectures.
• Develop data transformation logic using Python, PySpark, and SQL.
• Create and maintain reusable data engineering frameworks, pipelines, and automation processes.
• Perform data modelling activities to support reporting, analytics, and operational requirements.
• Manage large-scale structured and unstructured datasets across cloud platforms.
• Monitor, troubleshoot, and optimise data pipelines to ensure performance, reliability, and scalability.
• Conduct root cause analysis and resolve complex data-related issues.
• Implement data validation, reconciliation, and quality assurance processes.
• Develop monitoring and alerting mechanisms to proactively identify data issues.
• Support data migration and modernisation initiatives.
• Collaborate with business and technical stakeholders to gather and analyse requirements.
• Translate business requirements into scalable technical solutions and data models.
• Implement and maintain data governance, security, and compliance controls.
• Ensure adherence to enterprise standards, privacy requirements, and data management policies.
• Produce technical documentation, data dictionaries, solution designs, and operational runbooks.
• Contribute to continuous improvement initiatives and cloud data platform best practices.
• Work within Agile delivery teams and participate in sprint planning, stand-ups, reviews, and retrospectives.

What You'll Need
• Minimum 5 years' experience in Data Engineering, Data Warehousing, or a related data-focused role.
• Demonstrated experience designing, developing, and maintaining scalable data pipelines and integration solutions.
• Strong hands-on experience with the Microsoft Azure Data Platform, including:
• Azure Data Factory (ADF)
• Azure Databricks
• Azure Data Lake Storage (ADLS)
• Azure Synapse Analytics
• Azure SQL
• Strong experience with SQL development, performance tuning, database optimisation, and query troubleshooting.
• Proven experience in data modelling for enterprise data platforms and analytics solutions.
• Strong understanding of ETL/ELT methodologies and modern cloud-based data architectures.
• Experience developing solutions using Python, PySpark, and/or Scala.
• Experience managing data quality, validation, monitoring, reconciliation, and issue resolution processes.
• Strong understanding of data governance, security, privacy, and compliance requirements.
• Experience working with large and complex datasets in enterprise environments.
• Ability to produce high-quality technical documentation and solution designs.
• Strong stakeholder engagement and communication skills.
• Experience working within Agile and cross-functional delivery teams. Preferred Skills
• Experience designing modern cloud-native data platforms and lakehouse architectures.
• Knowledge of data governance frameworks, master data management, and metadata management practices.
• Experience with DevOps, CI/CD pipelines, and infrastructure-as-code practices.
• Strong analytical and problem-solving skills.
• Ability to work collaboratively with technical and non-technical stakeholders.
• Experience supporting enterprise reporting, analytics, and business intelligence initiatives.
• Exposure to government, regulated, or highly secure environments is highly regarded. Why Join Us?
• Deliver large-scale Azure data and analytics transformation programs.
• Work with leading cloud technologies including Azure Databricks, Synapse Analytics, and Data Factory.
• Partner with architects, data specialists, and business leaders on strategic initiatives.
• Contribute to modern data platform, governance, and analytics uplift programs.
• Join a consulting culture focused on innovation, technical excellence, and continuous improvement.
• Access exciting projects across government and enterprise organisations throughout Australia and New Zealand.