Lead Data Engineer

Location: Sydney

Job Type: Contract

We are looking for an experienced Lead Data Engineer with deep hands-on expertise in Data and ETL to lead strategic initiatives and transform our data platform. You will leverage your technical leadership, strategic thinking, and problem-solving skills to guide a team of engineers, enhance platform efficiency through AI and automation, and manage business and technical stakeholders effectively.

Your Impact:

• Lead the design, development, and optimization of large-scale data pipelines and platforms.
• Take full ownership of projects, driving them from conception to delivery while managing business and technical stakeholders successfully.
• Make decisive technical and strategic decisions to ensure project success and platform efficiency.
• Mentor and guide a team of data engineers, setting best practices and technical standards.
• Ensure robust data architecture, governance, and quality across all data initiatives.

Your Expertise:

• Expert in Big Data technologies including Spark, Hive, Presto (or other query engines), and big data storage formats (Parquet, ORC, Avro).
• Strong Spark SQL expertise with proven ability in query tuning and optimization.
• Hands-on experience with data integration tools such as Talend, DataStage, or Informatica, and building/maintaining data warehouses.
• Strong experience with AWS (EC2, S3, RDS, EMR) and GCP.
• Proficient in Airflow for orchestrating data pipelines.
• Strong data architecture expertise, including data modeling techniques.
• Knowledge of data governance, including data lineage, technical metadata, data quality, and reconciliation.
• Ability to drive platform efficiency through automation and AI, with Python experience preferred.
• Exceptional stakeholder management skills, able to navigate complex business and technical landscapes effectively.

Why Join Us:

You'll be part of a forward-thinking team where innovation meets execution. You'll lead transformative projects, implement cutting-edge technologies, and shape the future of our data platforms while collaborating closely with business and technical stakeholders.

Location: Sydney

Job Type: Contract

We are looking for an experienced Lead Data Engineer with deep hands-on expertise in Data and ETL to lead strategic initiatives and transform our data platform. You will leverage your technical leadership, strategic thinking, and problem-solving skills to guide a team of engineers, enhance platform efficiency through AI and automation, and manage business and technical stakeholders effectively.

Your Impact:

• Lead the design, development, and optimization of large-scale data pipelines and platforms.
• Take full ownership of projects, driving them from conception to delivery while managing business and technical stakeholders successfully.
• Make decisive technical and strategic decisions to ensure project success and platform efficiency.
• Mentor and guide a team of data engineers, setting best practices and technical standards.
• Ensure robust data architecture, governance, and quality across all data initiatives.

Your Expertise:

• Expert in Big Data technologies including Spark, Hive, Presto (or other query engines), and big data storage formats (Parquet, ORC, Avro).
• Strong Spark SQL expertise with proven ability in query tuning and optimization.
• Hands-on experience with data integration tools such as Talend, DataStage, or Informatica, and building/maintaining data warehouses.
• Strong experience with AWS (EC2, S3, RDS, EMR) and GCP.
• Proficient in Airflow for orchestrating data pipelines.
• Strong data architecture expertise, including data modeling techniques.
• Knowledge of data governance, including data lineage, technical metadata, data quality, and reconciliation.
• Ability to drive platform efficiency through automation and AI, with Python experience preferred.
• Exceptional stakeholder management skills, able to navigate complex business and technical landscapes effectively.

Why Join Us:

You'll be part of a forward-thinking team where innovation meets execution. You'll lead transformative projects, implement cutting-edge technologies, and shape the future of our data platforms while collaborating closely with business and technical stakeholders.

Must hold full Australian working rights

About INGRITY:

We are a fast-growing, progressive Sydney-based data & analytics company. We service many large corporates and SMBs in the financial services sector, to derive value from data. At Ingrity, we are committed to building a diverse and inclusive workforce to deliver value to customers. As a value-driven organisation, we nurture and support our people by having a laser-sharp focus on skill and talent development, collaboration, and flexibility.