Senior Data Engineer

13 hours ago

Canberra, ACT, Australia Ontrack Systems Aus Pty Full time $100,000 - $120,000 per year


Job details

The Department of Agriculture, Fisheries and Forestry (DAFF) is looking for a Data Engineer to join the Digital Transformation Program in theAustralian Bureau of Agricultural and Resource Economics and Sciences(ABARES) to work across several data and analytics platforms. The candidate will develop and optimise data pipelines in Azure Databricks, with a strong focus on Python and SQL. The candidate will have expertise in Azure Data Factory, Azure DevOps, CI/CD, and Git version control, as well as a deep understanding of Kimball dimensional modelling and Medallion architecture. This role requires strong collaboration skills to translate business requirements into effective technical solutions.

Key duties and responsibilities

Key Responsibilities:

  • Develop, optimise, and maintain data pipelines using Python and SQL within Azure Databricks Notebooks.
  • Design and implement ETL/ELT workflows in Azure Data Factory, ensuring efficient data transformation and loading.
  • Apply Kimball dimensional modelling and Medallion architecture best practices for scalable and structured data solutions.
  • Collaborate with team members and business stakeholders to understand data requirements and translate them into technical solutions.
  • Implement and maintain CI/CD pipelines using Azure DevOps, ensuring automated deployments and version control with Git.
  • Monitor, troubleshoot, and optimise Databricks jobs and queries for performance and efficiency.
  • Work closely with data analysts and business intelligence teams to provide well-structured, high-quality datasets for reporting and analytics.
  • Ensure compliance with data governance, security, and privacy best practices.
  • Contribute to code quality improvement through peer reviews, best practices, and knowledge sharing.

Preferred Skills & Experience:

  • Strong proficiency in Python for data transformation, automation, and pipeline development.
  • Advanced SQL skills for query optimisation and performance tuning in Databricks Notebooks.
  • Hands-on experience with Azure Databricks for large-scale data processing.
  • Expertise in Azure Data Factory for orchestrating and automating data workflows.
  • Experience with Azure DevOps, including setting up CI/CD pipelines and managing code repositories with Git.
  • Strong understanding of Kimball dimensional modelling (fact and dimension tables, star/snowflake schemas) for enterprise data warehousing.
  • Knowledge of Medallion architecture for structuring data lakes with bronze, silver, and gold layers.
  • Familiarity with data modelling best practices for analytics and business intelligence.
  • Strong analytical and problem-solving skills with a proactive approach to identifying and resolving issues.
  • Excellent collaboration and communication skills, with the ability to engage both technical and business stakeholders effectively.

  • Senior Data and Research Data Engineer

    14 hours ago

    Canberra, ACT, Australia Infinite Consulting Full time $90,000 - $120,000 per year

    Job DescriptionSenior Data and Research Engineer – TSPV Cleared12 months contract + 2x12-months extension optionsAustralian Citizens with current TSPV clearanceACT, VIC – OnsiteOur esteemed Federal Govt. client is seeking a Senior Data Engineer (Data and Research) to support the design, development, deployment, testing and documentation of solutions.In...

  • Senior IT Infrastructure Engineer

    1 week ago

    Canberra, ACT, Australia Eclectic Data Solutions Full time $120,000 - $180,000 per year

    Senior IT Infrastructure EngineerLocation: Canberra | Work Type: Full-time | Classification: Information & Communication Technology → Networks & Systems AdministrationAbout the RoleWe are seeking a highly skilled Senior IT Infrastructure Engineer to join our team. This role is responsible for managing and optimising a diverse range of infrastructure...

  • Senior Data Engineer

    19 hours ago

    Canberra, ACT, Australia Canberra Consulting Full time $120,000 - $180,000 per year

    About UsCanberra Consulting focuses on delivering highly skilled and qualified professionals to our clients. We have the experience and capability to understand your specific requirements and match them to the right environment, ensuring professional satisfaction and development. We focus on fit — for both our clients and our candidates — and give you...

  • Microsoft Fabric Administrator/Engineer

    2 weeks ago

    Canberra, ACT, Australia NTT DATA Full time $80,000 - $120,000 per year

    Role – Microsoft Fabric Administrator / EngineerThis role is open to Australian citizen (clearance - NV1 preferred or willingness to undergo clearance)Duration – 21 weeks of engagement with potential extension. Start Date: ASAP – Immediate start preferredLocations:- Canberra preferred (Remote work available)Your Role:NTT DATA is delivering an...

  • Data Engineers

    17 hours ago

    Canberra, ACT, Australia Softtest pays pty Full time $120,000 - $150,000 per year

    Job Description: Australian Citizens residing in Australia only respond.Must be able to obtain BaselineOverviewAt the Clean Energy Regulator, youre signing up for a challenging and rewarding workplace. All our work is connected to measuring, managing, reducing or offsetting carbon emissions and increasing the use of clean energy in Australia. Role...

  • Senior Data Engineer

    1 week ago

    Canberra, ACT, Australia Agylex Full time $120,000 - $150,000 per year

    NEEDED AN AUSTRALIAN CITIZEN WITH BASELINE SECURITY CLEARANCE.Job detailsSoftware Engineers have experience overseeing software development projects, writing optimised code, and provide technical leadership to a team of software engineers. They are responsible for the quality and functionality of the software produced by their team.Key duties and...

  • Data Engineer

    17 hours ago

    Canberra, ACT, Australia Nuvento Full time $80,000 - $120,000 per year

    Who we are and how are we different? Nuvento is a Canberra-based specialist technical data consultancy that provides consultancy services to both private and public organizations throughout Australia. Our team consists of passionate and highly skilled professionals who are dedicated to delivering the right solutions with the right tech in the right way, on...

  • Senior Infrastructure Engineer

    3 days ago

    Canberra, ACT, Australia Quality People Full time $120,000 - $180,000 per year

    Our client URGENTLY requires a Senior Infrastructure Engineer for a 6 months + Contract in Canberra.The Senior Infrastructure Engineer must possess deep product knowledge, exceptional troubleshooting abilities, and a proactive approach to problem-solving, ensuring timely resolution of any challenges that may arise during the project's implementation and...

  • Senior Storage Engineer

    13 hours ago

    Canberra, ACT, Australia Softtest pays pty Full time $80,000 - $120,000 per year

    Job Description: Australian Citizens residing in Australia only respond. eligible to get baseline cleranceWe are seeking a Senior Storage Engineer:who has experience working in large complex ICT environments. The successful candidate will be responsible for deploying, configuring, managing and tuning the departments storage systems at physical and virtual...

  • Cyber Security Engineer

    2 days ago

    Canberra, ACT, Australia Data#3 People Solutions Full time $120,000 - $140,000 per year

    Location: Canberra (Hybrid)Engagement: ASAP – 30 June 2026 (likely 12-month extension)Security Clearance: NV1 (mandatory)Join a high-performing Security Operations Centre and play a key role in advancing enterprise cyber capability.We're seeking a skilled Cyber Security Engineer to support a major centralised logging refresh and uplift project within a...