Principal Data Engineer

About INGRITYINGRITY is a Microsoft Data and AI Solution Partner dedicated to delivering transformative data and AI-driven solutions.
We collaborate with some of the best ASX-listed companies and medium-sized businesses, working closely with Microsoft to create measurable value for our customers.
Our success is built on innovation, customer advocacy, and a strategic partnership with Microsoft.About The JobWe are seeking an experienced Principal Data Engineer to join our team in Sydney.
This is a hands-on role requiring deep expertise in data engineering and solution architecture, with a strong focus on the Azure data platform.
You will work directly with clients to understand their requirements, design robust data solutions, and lead delivery from inception to production.Your ResponsibilitiesClient Collaboration and Requirement Gathering - Engage closely with clients to thoroughly understand their business requirements, define comprehensive data strategies, and translate complex needs into scalable, practical solutions.Designing End-to-End Data Solutions - Deliver high-quality, end-to-end data solutions across data ingestion, processing, modelling, and analytics in the cloud to enable scalable and efficient business intelligence.Solution Estimation and Project Scoping - Take ownership of solution estimation and sizing, ensuring accurate scoping of projects and meticulous planning of delivery efforts to align with client expectations and timelines.Solution Architecture and Optimisation - Design and architect secure, high-performing, and cost-effective data solutions spanning all layers, including data ingestion, storage, compute, and visualisation.Azure Data Stack and Solutions Implementation - Leverage the full Azure data stack, including Azure Data Factory, Azure Synapse Analytics, Azure Databricks, Azure Data Lake Storage, and Azure SQL Database, to build scalable, integrated, and optimised data platforms tailored to client needs.Data Modelling and Optimisation - Develop and optimise robust data models that underpin analytics, reporting, and business intelligence, ensuring maintainability and optimal performance.Development of Data Pipelines and ETL/ELT Frameworks - Lead the creation and implementation of data pipelines and ETL/ELT frameworks utilising modern cloud-native technologies, guaranteeing data quality and reliability.Data Integration and Workflow Development - Construct integration workflows to ingest both structured and semi-structured data from a variety of data sources.CI/CD Pipeline Setup and Automation - Spearhead the establishment and automation of CI/CD pipelines using cloud-native DevOps tools, facilitating seamless continuous delivery and deployment.Stakeholder Collaboration and Architectural Alignment - Collaborate with internal teams and external stakeholders (including clients' stakeholders) to ensure all solutions adhere to recognised architectural standards such as the Well-Architected Framework or Cloud Adoption Framework.Technical Leadership and Project Lifecycle Management - Provide ongoing technical leadership throughout the project lifecycle, encompassing design, development, testing, deployment, and post-production support.Performance Monitoring and Cost Optimisation - Utilise monitoring and logging tools to oversee system performance, optimise resource utilisation, and manage costs efficiently.Data Governance and Security Compliance - Champion best practices in data governance, including data lineage, access control, security protocols, and compliance with relevant regulatory requirements.Client Relationship Management - Build and sustain strong client relationships by acting as a trusted technical advisor, offering expert guidance and strategic thought leadership.Your Technical Skills And ExperienceProven 7+ years in data engineering and solution architecture within enterprise-scale environments, specialising in Azure technologiesProven hands-on expertise with the Azure data platform, including Azure Data Factory, Azure Synapse Analytics, Azure Databricks, Azure Data Lake Storage and Azure SQL DatabaseDemonstrated client-facing experience, effectively engaging with stakeholders to understand and address their needs.Strong skills in solutioning and project estimation, accurately scoping and planning data engineering projects.Strong practical knowledge of data modelling principles, including star and snowflake schemas, dimensional modelling, and Slowly Changing Dimensions.Expert in SQL and Python, with production-ready code for both batch and real-time use cases; experience with R, Scala, or Java is a plus.Experience designing and implementing scalable ETL/ELT pipelines and data integration workflows.Demonstrated experience leading platform migrations, particularly from Azure Synapse or similar platforms.Experience in Microsoft Fabric is highly valued.Strong understanding of CI/CD practices, Git branching, automated testing, and infrastructure-as-code.Strong knowledge and hands-on implementation of cloud security best practices, including identity and access management, encryption, and compliance.Experience monitoring cloud services and optimising performance and costs using Azure Monitor, Log Analytics, or equivalent tools.Excellent communication skills with the ability to act as a trusted advisor to clients and stakeholders.Experience with GCP is a bonus, especially with BigQuery, Dataflow, Pub/Sub, and Cloud Composer.Please note to apply for this role you MUST be in Sydney with a valid work visa, PR or Citizenship.
Overseas applicants will not be considered.Seniority levelMid-Senior levelEmployment typeFull-timeJob functionOtherIndustriesSoftware DevelopmentReferrals increase your chances of interviewing at Ingrity by 2x
#J-18808-Ljbffr