Data Engineer
3 weeks ago
Summary
We are seeking an experienced Data Engineer with a proven track record in building and maintaining data pipelines, developing applications, and implementing data management solutions. The ideal candidate combines technical expertise with hands-on development skills to drive our data engineering initiatives.
Requirements
- 4-5 years of professional experience in data engineering
- Strong proficiency in Python for data processing, ETL/ELT development, and application building
- Advanced SQL skills for complex data querying, optimization, and database design
- Experience with Apache Airflow for workflow orchestration and scheduling
- Knowledge of low-code development platforms (such as Tooljet) for rapid application creation
- Ability to build and deploy data applications using Streamlit
- Extensive data management experience including data governance, quality control, and metadata management
Key Responsibilities
- Design, build, and maintain efficient data pipelines to process structured and unstructured data
- Create and optimize SQL queries for data extraction and transformation
- Develop and manage Airflow DAGs for reliable workflow orchestration
- Utilize low-code tools to accelerate application development where appropriate
- Build interactive dashboards and tools using Streamlit for data visualization and exploration
- Implement data management best practices throughout the data lifecycle
- Collaborate with data scientists, analysts, and other stakeholders to understand requirements
Technical Skills
- Programming Languages: Python, Javascript
- Database Technologies: SQL (PostgreSQL, MySQL, or similar)
- Orchestration: Apache Airflow
- Low-Code Development: Tooljet or similar platforms
- ETL/ELT Tools: Apache Nifi, KNIME, Apache Spark
- Lakehouse Solution: Snowflake
- Application Development: Streamlit
- Version Control: Git
- Data Processing: Experience with batch and streaming data processing
Preferred Qualifications
- Experience with cloud platforms (AWS, Azure, or GCP)
- Knowledge of containerization technologies (Docker, Kubernetes)
- Familiarity with big data technologies (Spark, Hadoop)
- Understanding of data modeling and dimensional modeling concepts
Success in this Role
- The successful candidate will bridge the gap between raw data and valuable insights by building reliable data infrastructure and intuitive applications, ensuring that data is accessible, trustworthy, and actionable across the organization.
-
Data Engineer
11 hours ago
Perth, Western Australia Brunel GmbH Full timeWe are currently seeking a Data Engineer on a contract basis for an upcoming green energy project.Key accountabilities will include:Work on cutting-edge technology on mobile heavy mining equipment and railway equipment with the project team and a multidisciplinary engineering team.Responsible for test data management and data offloading pipeline for all...
-
Data Engineer
1 week ago
Perth, Western Australia ClearCompany Full timeHiring Now: Data Engineer (6-Month Contract)A well-established organization seeks an experienced Data Engineer to design, develop, and optimize Azure-based data solutions. This hybrid role (minimum 3 days on-site) is ideal for professionals skilled in Azure Data Services, ETL processes, Python, SQL, Power BI, and data warehousing concepts.Key...
-
Data Engineer
1 week ago
Perth, Western Australia Perth Airport Full timePerth Airport is Australia's Western Hub connecting the people, businesses and communities of Western Australia with the rest of Australia and the world. Operating 24 hours a day, seven days a week throughout the year, Perth Airport plays a significant role in providing economic, social and cultural benefits to West Australians. Perth Airport is investing...
-
Data Engineer
2 days ago
Perth, Western Australia Perth Airport Pty Full timePerth Airport is Australia's Western Hub connecting the people, businesses and communities of Western Australia with the rest of Australia and the world. Operating 24 hours a day, seven days a week throughout the year, Perth Airport plays a significant role in providing economic, social and cultural benefits to West Australians.Perth Airport is investing...
-
Data Engineer
2 weeks ago
Perth, Western Australia ROBERT WALTERS AUSTRALIA Full timeOur client is seeking a dedicated and enthusiastic Data Engineer to join their dynamic team. This role offers the opportunity to design and implement efficient data pipelines using Azure Databricks and Synapse Analytics, develop and maintain SQL queries for data transformation and cleansing, and work with cloud storage solutions like Azure Blob Storage for...
-
Data Engineer
3 weeks ago
Perth, Western Australia ROBERT WALTERS AUSTRALIA Full timeOur client is seeking a dedicated and enthusiastic Data Engineer to join their dynamic team. This role offers the opportunity to design and implement efficient data pipelines using Azure Databricks and Synapse Analytics, develop and maintain SQL queries for data transformation and cleansing, and work with cloud storage solutions like Azure Blob Storage for...
-
Data Engineer
11 hours ago
Perth, Western Australia Pyramid Global Technologies Full timeJob Requirements:Strong Data Engineering skills using Azure and PysparkGood Knowledge on SQLPreferred experience in Big data/Hadoop technologies like Spark, Hive, Hbase, and KafkaPreferred experience in ETL processGood Communication SkillsDesired Experience Range: 5 – 10 yearsLocation of Requirement: Perth/AustraliaDesired Competencies...
-
Data Engineer
2 days ago
Perth, Western Australia NES Fircroft Full timeWe have partnered with our Tier One Mining & Energy client to find the next Data Engineer to join the team as a key player in their journey developing heavy industry products.Role Details:1. Location: Perth Based (near the airport)2. Hours: 40 hours per week / 8 hour days3. Duration: 01/05/2025 - 30/04/2026Key accountabilities will include:1. Work on cutting...
-
Data Engineer
3 weeks ago
Perth, Western Australia Robert Walters Full timeOur client, a leading player in the financial services sector, is seeking a Data Engineer to play a pivotal role to leverage technology.This is an opportunity to revolutionise financial wellness by merging innovative technology with human expertise. As a key contributor, you will build and maintain their data infrastructure, ensuring data is reliable,...
-
Data Engineer Specialist
5 days ago
Perth, Western Australia Justice Full timeThe Department of Justice is seeking a highly skilled Data Engineer Specialist to join its team. The successful candidate will be responsible for the provision of support and maintenance of the corporate data warehouse, as well as undertaking all activities associated with the analysis, design, development, and implementation of enhancements and new business...
-
Principle Data Engineer
3 weeks ago
Perth, Western Australia Paxus - Technology + Digital Talent Full time6 month initial contractPerth based; hybrid - 3 days onsiteMultiple roles *3AWS ToolsetTasks to be performed on a daily basis:The Principal Data Engineer shall be responsible for the building, testing, and operation of an enterprise data platform.The engineer shall ensure that the platform is developed and deployed with a secure-by-design approach,...
-
Data Engineer
3 weeks ago
Perth, Western Australia Whizdom Full timeWhizdom Perth, Western Australia, AustraliaState Manager - taking the Whizdom success story further into NSW, QLD and VICAbout Our ClientOur client is a globally recognized consulting house with a strong reputation for delivering innovative solutions across various sectors, including technology, energy, and finance. With a presence in multiple countries,...
-
Data Scientist and Engineer
1 day ago
Perth, Western Australia Verse Full timeAs a Data Scientist and Engineer at Verse, you'll have the opportunity to work on cutting-edge projects that leverage machine learning, data engineering, and cloud computing to drive business growth and innovation.Key Responsibilities and ChallengesDesign and develop machine learning models that predict business outcomes and drive growth.Collaborate with...
-
Data Engineering Specialist
2 days ago
Perth, Western Australia Perth Airport Pty Full timePerth Airport Pty is Australia's Western Hub, connecting the people, businesses and communities of Western Australia with the rest of Australia and the world. As a Data Engineer at Perth Airport Pty, you will be part of our evolving data and analytics ecosystem.About the RoleYou will design, implement, and optimize scalable data pipelines to ensure high...
-
Data Engineer
2 days ago
Perth, Western Australia Verse Full timeVerse Perth, Western Australia, AustraliaTech Recruiter | Helpdesk | Infrastructure | Cloud Technologies | Verse GroupAre you a creative problem-solver with a passion for numbers, engineering, and data? We're looking for a skilled Data Engineer to help revolutionise our operational processes, deliver impactful insights, and drive measurable improvements in...
-
Data Engineer
2 weeks ago
Perth, Western Australia Tbwa ChiatDay Inc Full timeVGW is an interactive entertainment company, harnessing technology and creativity to deliver world-class, free-to-play games.We have an exciting opportunity to join our Technology team in Perth and are currently looking for a Data Engineer to join the team.The Data Engineer will deliver and maintain data products that delight our customers and stakeholders....
-
Big Data Engineer
8 hours ago
Perth, Western Australia Pyramid Global Technologies Full timeAt Pyramid Global Technologies, we are looking for an exceptional Big Data Engineer to join our team.The ideal candidate will have strong data engineering skills using Azure and Pyspark, with a good knowledge of SQL. Preferred experience in big data/Hadoop technologies like Spark, Hive, HBase, and Kafka is also desired.We are seeking someone with excellent...
-
Head Of Data Engineering
3 weeks ago
Perth, Western Australia Tbwa ChiatDay Inc Full timeVGW is an interactive entertainment company, harnessing technology and creativity to deliver world-class, free-to-play games. We have an exciting opportunity to join our Technology team in Australia and are currently looking for a Head of Data Engineering to join the team. As the Head of Data Engineering at VGW, you will lead the strategy to deliver...
-
Data Engineering Specialist
7 days ago
Perth, Western Australia BHP Group Full timeAbout the RoleWe are seeking an experienced Data Engineer to join our team at BHP Group. As a key member of our data engineering team, you will play a crucial role in designing and developing innovative solutions that drive business growth and improvement.The successful candidate will have expertise in data warehousing, ETL tools, and cloud-based services....
-
Data Engineering Expert
2 days ago
Perth, Western Australia NES Fircroft Full timeWe are seeking a skilled Data Engineer to join our team as a key player in developing heavy industry products. As a member of our multidisciplinary engineering team, you will work on cutting-edge technology for mobile heavy mining equipment and railway equipment.Key Responsibilities:Responsible for test data management and data offloading pipeline for all...