Lead Data Engineer Job in Sydney, Australia

2 weeks ago


Council of the City of Sydney, Australia Virtusa Full time

Qualifications

- Extensive experience in designing, building, and delivering enterprise-wide data ingestion, data integration and data pipeline solutions

- Strong Data Architecture expertise including different data modelling techniques and design patterns (conceptual, logical, physical, semantic is preferred)

- Strong knowledge of data governance such as data lineage, technical metadata, data quality and reconciliation

- Ability to drive platform efficiency through automation and AI capabilities

- AWS Data Stack: EMR, Glue, Redshift, Athena, S3, Lambda, ECS

- Data Orchestration & Pipelines: Airflow, Dataform

- Data Formats & Modelling: Iceberg, JSON, XML, CSV, Data Modelling Programming & DevOps: Python, SQL, Git, GitHub Actions, Team City, Jenkins, Octopus Unix shell scripting

- ETL & Ingestion: File ingress/egress solutions in AWS

- Security and Observability: DevSecOps, Artifactory, Observability tooling

- Testing & Automation: test automation frameworks, Jupyter Notebooks

- Familiarity with data warehousing and build experience in Teradata, Oracle

- Experience in visualisation tools such as PowerBI, Tableau

- Familiarity and experience with Agile processes

- AWS Data Engineer Associate certification

#J-18808-Ljbffr



  • Council of the City of Sydney, Australia Virtusa Full time

    Cross-Skills: - Hands-on experience with Python programming for data-related tasks and automation. - Knowledge of version control systems, particularly Git, and platforms like BitBucket. - Experience writing SQL for data profiling purposes, to enable evidence-based design decisions. Responsibilities and Qualifications: - Strong expertise in data modelling...


  • Council of the City of Sydney, Australia Sydney Metro Full time

    Update your details, view your application and progress. Project Engineer, Sydney (Royal Prince Alfred Hospital Redevelopment) At CPB Contractors, we know what it takes to construct a better future. Our track record speaks for itself – more than 90 years’ experience in transforming communities to help people live, work and move better. Today we are...

  • Data Engineer

    4 weeks ago


    Council of the City of Sydney, Australia GrainCorp Full time

    Overview GrainCorp is currently seeking a Data Engineer to join a motivated and collaborative team delivering data, infrastructure, and system activities to support advanced analytic use cases. You’ll have the opportunity to work from multiple exciting AI and IoT projects and be trained by experts who’ve been in this space for...


  • Council of the City of Sydney, Australia Nike Sydney Full time

    Nike - 3IC - Sydney WHO WE ARE AF1 Pty. Ltd. is an Authorised Nike Dealer that has been in business since 2006, having expanded to 7 locations across Sydney including Sydney City, Bondi Junction, Burwood, Castle Hill, Chatswood, Neutral Bay and Warringah Mall. Leading the way with our staff development and culture, we pride ourselves in providing an...


  • Council of the City of Sydney, Australia FinXL Full time

    Senior Data Engineer Leading government client (consulting through FinXL) FinXL Professional Services is a leading IT consultancy that partners with clients to bridge the gap between business needs and technology capabilities. Since 2003, we've been delivering high-value ICT services across government, finance, telecommunications, and enterprise...


  • Council of the City of Sydney, Australia black.ai Full time

    Location Melbourne HQ or Sydney Employment Type Full time Department Engineering Who is Heidi? Heidi is building an AI Care Partner that supports clinicians every step of the way, from documentation to delivery of care. We exist to double healthcare’s capacity while keeping care deeply human. In 18 months, Heidi has returned more than 18 million...

  • Data Engineer

    3 weeks ago


    Council of the City of Sydney, Australia DroneShield Group Pty Ltd Full time

    Work with cutting edge technology, making the world a safer and more secure place. DroneShield (ASX:DRO) offers an opportunity to solve some of world’s most challenging technical problems in the Electronic Warfare, Artificial Intelligence and Machine Learning, RF sensing, Sensor Fusion and distributed systems. Working with high profile customers across...

  • Data Engineer

    1 day ago


    Council of the City of Sydney, Australia DroneShield Group Pty Ltd Full time

    Work with cutting edge technology, making the world a safer and more secure place. DroneShield (ASX:DRO) offers an opportunity to solve some of world’s most challenging technical problems in the Electronic Warfare, Artificial Intelligence and Machine Learning, RF sensing, Sensor Fusion and distributed systems. Working with high profile customers across...

  • Senior Data Engineer

    3 weeks ago


    Council of the City of Sydney, Australia Michael Page Australia Full time

    Respected firm driving innovation & teamwork in digital transformation. A clear runway to build, create, and move with velocity. About Our Client This organisation operates within the business services industry and boasts a robust presence as a mid-sized player. They are known for their focus on digital innovation and delivering data-driven solutions to...


  • Council of the City of Sydney, Australia DroneShield Group Pty Ltd Full time

    Data Platform Engineering Team Lead (AU) Work with cutting edge technology, making the world a safer and more secure place. DroneShield (ASX:DRO) offers an opportunity to solve some of the world’s most challenging technical problems in Electronic Warfare, Artificial Intelligence and Machine Learning, RF sensing, Sensor Fusion and distributed systems....