Data Engineer

3 weeks ago


Perth, Western Australia VitalTrace Full time

Summary

We are seeking an experienced Data Engineer with a proven track record in building and maintaining data pipelines, developing applications, and implementing data management solutions. The ideal candidate combines technical expertise with hands-on development skills to drive our data engineering initiatives.

Requirements

  1. 4-5 years of professional experience in data engineering
  2. Strong proficiency in Python for data processing, ETL/ELT development, and application building
  3. Advanced SQL skills for complex data querying, optimization, and database design
  4. Experience with Apache Airflow for workflow orchestration and scheduling
  5. Knowledge of low-code development platforms (such as Tooljet) for rapid application creation
  6. Ability to build and deploy data applications using Streamlit
  7. Extensive data management experience including data governance, quality control, and metadata management

Key Responsibilities

  1. Design, build, and maintain efficient data pipelines to process structured and unstructured data
  2. Create and optimize SQL queries for data extraction and transformation
  3. Develop and manage Airflow DAGs for reliable workflow orchestration
  4. Utilize low-code tools to accelerate application development where appropriate
  5. Build interactive dashboards and tools using Streamlit for data visualization and exploration
  6. Implement data management best practices throughout the data lifecycle
  7. Collaborate with data scientists, analysts, and other stakeholders to understand requirements

Technical Skills

  1. Programming Languages: Python, Javascript
  2. Database Technologies: SQL (PostgreSQL, MySQL, or similar)
  3. Orchestration: Apache Airflow
  4. Low-Code Development: Tooljet or similar platforms
  5. ETL/ELT Tools: Apache Nifi, KNIME, Apache Spark
  6. Lakehouse Solution: Snowflake
  7. Application Development: Streamlit
  8. Version Control: Git
  9. Data Processing: Experience with batch and streaming data processing

Preferred Qualifications

  1. Experience with cloud platforms (AWS, Azure, or GCP)
  2. Knowledge of containerization technologies (Docker, Kubernetes)
  3. Familiarity with big data technologies (Spark, Hadoop)
  4. Understanding of data modeling and dimensional modeling concepts

Success in this Role

  1. The successful candidate will bridge the gap between raw data and valuable insights by building reliable data infrastructure and intuitive applications, ensuring that data is accessible, trustworthy, and actionable across the organization.
#J-18808-Ljbffr
  • Data Engineer

    11 hours ago


    Perth, Western Australia Brunel GmbH Full time

    We are currently seeking a Data Engineer on a contract basis for an upcoming green energy project.Key accountabilities will include:Work on cutting-edge technology on mobile heavy mining equipment and railway equipment with the project team and a multidisciplinary engineering team.Responsible for test data management and data offloading pipeline for all...

  • Data Engineer

    1 week ago


    Perth, Western Australia ClearCompany Full time

    Hiring Now: Data Engineer (6-Month Contract)A well-established organization seeks an experienced Data Engineer to design, develop, and optimize Azure-based data solutions. This hybrid role (minimum 3 days on-site) is ideal for professionals skilled in Azure Data Services, ETL processes, Python, SQL, Power BI, and data warehousing concepts.Key...

  • Data Engineer

    1 week ago


    Perth, Western Australia Perth Airport Full time

    Perth Airport is Australia's Western Hub connecting the people, businesses and communities of Western Australia with the rest of Australia and the world. Operating 24 hours a day, seven days a week throughout the year, Perth Airport plays a significant role in providing economic, social and cultural benefits to West Australians. Perth Airport is investing...

  • Data Engineer

    2 days ago


    Perth, Western Australia Perth Airport Pty Full time

    Perth Airport is Australia's Western Hub connecting the people, businesses and communities of Western Australia with the rest of Australia and the world. Operating 24 hours a day, seven days a week throughout the year, Perth Airport plays a significant role in providing economic, social and cultural benefits to West Australians.Perth Airport is investing...

  • Data Engineer

    2 weeks ago


    Perth, Western Australia ROBERT WALTERS AUSTRALIA Full time

    Our client is seeking a dedicated and enthusiastic Data Engineer to join their dynamic team. This role offers the opportunity to design and implement efficient data pipelines using Azure Databricks and Synapse Analytics, develop and maintain SQL queries for data transformation and cleansing, and work with cloud storage solutions like Azure Blob Storage for...

  • Data Engineer

    3 weeks ago


    Perth, Western Australia ROBERT WALTERS AUSTRALIA Full time

    Our client is seeking a dedicated and enthusiastic Data Engineer to join their dynamic team. This role offers the opportunity to design and implement efficient data pipelines using Azure Databricks and Synapse Analytics, develop and maintain SQL queries for data transformation and cleansing, and work with cloud storage solutions like Azure Blob Storage for...

  • Data Engineer

    11 hours ago


    Perth, Western Australia Pyramid Global Technologies Full time

    Job Requirements:Strong Data Engineering skills using Azure and PysparkGood Knowledge on SQLPreferred experience in Big data/Hadoop technologies like Spark, Hive, Hbase, and KafkaPreferred experience in ETL processGood Communication SkillsDesired Experience Range: 5 – 10 yearsLocation of Requirement: Perth/AustraliaDesired Competencies...

  • Data Engineer

    2 days ago


    Perth, Western Australia NES Fircroft Full time

    We have partnered with our Tier One Mining & Energy client to find the next Data Engineer to join the team as a key player in their journey developing heavy industry products.Role Details:1. Location: Perth Based (near the airport)2. Hours: 40 hours per week / 8 hour days3. Duration: 01/05/2025 - 30/04/2026Key accountabilities will include:1. Work on cutting...

  • Data Engineer

    3 weeks ago


    Perth, Western Australia Robert Walters Full time

    Our client, a leading player in the financial services sector, is seeking a Data Engineer to play a pivotal role to leverage technology.This is an opportunity to revolutionise financial wellness by merging innovative technology with human expertise. As a key contributor, you will build and maintain their data infrastructure, ensuring data is reliable,...


  • Perth, Western Australia Justice Full time

    The Department of Justice is seeking a highly skilled Data Engineer Specialist to join its team. The successful candidate will be responsible for the provision of support and maintenance of the corporate data warehouse, as well as undertaking all activities associated with the analysis, design, development, and implementation of enhancements and new business...


  • Perth, Western Australia Paxus - Technology + Digital Talent Full time

    6 month initial contractPerth based; hybrid - 3 days onsiteMultiple roles *3AWS ToolsetTasks to be performed on a daily basis:The Principal Data Engineer shall be responsible for the building, testing, and operation of an enterprise data platform.The engineer shall ensure that the platform is developed and deployed with a secure-by-design approach,...

  • Data Engineer

    3 weeks ago


    Perth, Western Australia Whizdom Full time

    Whizdom Perth, Western Australia, AustraliaState Manager - taking the Whizdom success story further into NSW, QLD and VICAbout Our ClientOur client is a globally recognized consulting house with a strong reputation for delivering innovative solutions across various sectors, including technology, energy, and finance. With a presence in multiple countries,...


  • Perth, Western Australia Verse Full time

    As a Data Scientist and Engineer at Verse, you'll have the opportunity to work on cutting-edge projects that leverage machine learning, data engineering, and cloud computing to drive business growth and innovation.Key Responsibilities and ChallengesDesign and develop machine learning models that predict business outcomes and drive growth.Collaborate with...


  • Perth, Western Australia Perth Airport Pty Full time

    Perth Airport Pty is Australia's Western Hub, connecting the people, businesses and communities of Western Australia with the rest of Australia and the world. As a Data Engineer at Perth Airport Pty, you will be part of our evolving data and analytics ecosystem.About the RoleYou will design, implement, and optimize scalable data pipelines to ensure high...

  • Data Engineer

    2 days ago


    Perth, Western Australia Verse Full time

    Verse Perth, Western Australia, AustraliaTech Recruiter | Helpdesk | Infrastructure | Cloud Technologies | Verse GroupAre you a creative problem-solver with a passion for numbers, engineering, and data? We're looking for a skilled Data Engineer to help revolutionise our operational processes, deliver impactful insights, and drive measurable improvements in...

  • Data Engineer

    2 weeks ago


    Perth, Western Australia Tbwa ChiatDay Inc Full time

    VGW is an interactive entertainment company, harnessing technology and creativity to deliver world-class, free-to-play games.We have an exciting opportunity to join our Technology team in Perth and are currently looking for a Data Engineer to join the team.The Data Engineer will deliver and maintain data products that delight our customers and stakeholders....

  • Big Data Engineer

    8 hours ago


    Perth, Western Australia Pyramid Global Technologies Full time

    At Pyramid Global Technologies, we are looking for an exceptional Big Data Engineer to join our team.The ideal candidate will have strong data engineering skills using Azure and Pyspark, with a good knowledge of SQL. Preferred experience in big data/Hadoop technologies like Spark, Hive, HBase, and Kafka is also desired.We are seeking someone with excellent...


  • Perth, Western Australia Tbwa ChiatDay Inc Full time

    VGW is an interactive entertainment company, harnessing technology and creativity to deliver world-class, free-to-play games. We have an exciting opportunity to join our Technology team in Australia and are currently looking for a Head of Data Engineering to join the team. As the Head of Data Engineering at VGW, you will lead the strategy to deliver...


  • Perth, Western Australia BHP Group Full time

    About the RoleWe are seeking an experienced Data Engineer to join our team at BHP Group. As a key member of our data engineering team, you will play a crucial role in designing and developing innovative solutions that drive business growth and improvement.The successful candidate will have expertise in data warehousing, ETL tools, and cloud-based services....


  • Perth, Western Australia NES Fircroft Full time

    We are seeking a skilled Data Engineer to join our team as a key player in developing heavy industry products. As a member of our multidisciplinary engineering team, you will work on cutting-edge technology for mobile heavy mining equipment and railway equipment.Key Responsibilities:Responsible for test data management and data offloading pipeline for all...