Data Engineer
1 day ago
Whizdom Perth, Western Australia, Australia
State Manager - taking the Whizdom success story further into NSW, QLD and VICAbout Our Client
Our client is a globally recognized consulting house with a strong reputation for delivering innovative solutions across various sectors, including technology, energy, and finance. With a presence in multiple countries, they offer a diverse range of services and have a proven track record of success. Their unique selling points include a commitment to excellence, a collaborative work environment, and access to cutting-edge technology.
About the Role
We are seeking a Principal Data Engineer to join our client's dynamic team. In this role, you will be responsible for building, testing, and operating an enterprise data platform.
Key criteria for this position include:
- Experience integrating with timeseries source systems such as Honeywell Plant Historian Database and OSI Pi.
- Extensive experience in authentication mechanisms including OAuth 2.0, OIDC, Microsoft Entra, Key Pair Authentication, Certificate-based authentication, and SAML-based SSO.
- Proficiency in using AWS services for cloud-based platform tooling infrastructure, including Lambda, ECS, MSK, RDS, EC2, Secrets Manager, ALB, CloudWatch, and EventBridge.
- Development of standard patterns for data pipelines and workflows utilizing Streamsets, Kestra, dbt, and Git.
- APIs/Integration for data access and manipulation.
- Development of APIs and Web Applications to support the management of the platform (Security, Sandbox creation, etc.).
- Experience in Terraform for AWS and Azure deployments.
- Python scripting for data processing and automation tasks.
- CLI development to automate common platform tasks.
- Data storage design and processing solutions employing Snowflake.
- Quality assurance and testing.
The Successful Candidate
The ideal candidate will have the following credentials, qualifications, and experience:
- Proven experience as a Data Engineer with expertise in AWS toolset.
- Ability to design, build, and operate batch and real-time data pipelines using AWS services.
- Proficiency in developing data layers on AWS RedShift, Aurora, and DynamoDB.
- Experience in migrating data using AWS DMS.
- Familiarity with open-source technologies like Apache Airflow and dbt, Spark/Python or Spark/Scala on AWS Platform.
- Strong technical support skills for delivery data engineers working on the platform.
- Conducting security reviews for data engineering projects to ensure compliance with standards.
- Offering technical reviews and feedback to enhance the quality of deliverables.
- Passionate, diligent, hardworking, with exceptional communication skills.
What is on Offer
This opportunity offers the chance to work in a prestigious consulting house with amazing end clients. You will benefit from competitive contracting rates, access to world-class technology, and learning and development opportunities. You will also have the chance to work with talented managers and colleagues in a collaborative and innovative environment.
Seniority levelMid-Senior level
Employment typeContract
Job functionInformation Technology
IndustriesIT Services and IT Consulting, Wholesale Petroleum and Petroleum Products, and Mining
#J-18808-Ljbffr-
Data Engineer
1 day ago
Perth, Western Australia Robert Walters Full timeOur client, a leading player in the financial services sector, is seeking a Data Engineer to play a pivotal role to leverage technology.This is an opportunity to revolutionise financial wellness by merging innovative technology with human expertise. As a key contributor, you will build and maintain their data infrastructure, ensuring data is reliable,...
-
Principle Data Engineer
1 day ago
Perth, Western Australia Paxus - Technology + Digital Talent Full time6 month initial contractPerth based; hybrid - 3 days onsiteMultiple roles *3AWS ToolsetTasks to be performed on a daily basis:The Principal Data Engineer shall be responsible for the building, testing, and operation of an enterprise data platform.The engineer shall ensure that the platform is developed and deployed with a secure-by-design approach,...
-
Data Engineering Specialist
3 days ago
Perth, Western Australia Fortescue Full timeAt Fortescue, we are committed to leading the world in decarbonisation and sustainability. As a key player in our Heavy Industry projects, you will play a crucial role in developing and testing technologies to power our mining fleet and rail locomotives.We are seeking an experienced Data Engineer to join our team at our new Redcliffe and Hazelmere R&D...
-
Cloud Data Engineering Manager
7 days ago
Perth, Western Australia workable - ATS Full timeAt DataDivers, we empower organizations to unlock the full potential of their data through expert guidance and innovative solutions. As a Cloud Data Engineering Manager, you will lead our team in designing and delivering world-class cloud data engineering capabilities.About the RoleAs a Cloud Data Engineering Manager, you will be responsible for:Leading a...
-
Data Engineer
2 weeks ago
Perth, Western Australia Tideri Jobbörse Full timeDo you want a job that looks different every day? Have you got an unquenchable passion for data and problem solving? Are you aching to jump into a work environment where you will work with some of the keenest minds in the industry? We've got a career you will enjoy At InterWorks, we like our work with a hefty side of play. Our team of high achievers is...
-
Cloud Data Engineer
3 days ago
Perth, Western Australia Fortescue Full timeFortescue is driving innovation in heavy industry through our Heavy Industry projects, focusing on green energy and carbon neutrality. We are seeking a talented Data Engineer to join our team at our Redcliffe and Hazelmere R&D facilities.In this role, you will work collaboratively with a multidisciplinary engineering team to develop and test technologies...
-
Senior Data Development Engineer
5 days ago
Perth, Western Australia VitalTrace Full timeAbout the Role:VitalTrace is seeking an experienced Data Engineer to lead our data engineering initiatives. As a key member of our team, you will be responsible for designing, building, and maintaining data pipelines, developing applications, and implementing data management solutions.Main Responsibilities:Develop and maintain efficient data pipelines to...
-
Data Engineer
3 hours ago
Perth, Western Australia InterWorks Full timeDo you want a job that looks different every day? Have you got an unquenchable passion for data and problem solving? Are you aching to jump into a work environment where you will work with some of the keenest minds in the industry? We've got a career you will enjoyAt InterWorks, we like our work with a hefty side of play. Our team of high achievers is...
-
AWS Data Platform Engineer
1 day ago
Perth, Western Australia Paxus - Technology + Digital Talent Full timeAbout the PositionThe AWS Data Platform Engineer will be responsible for designing, building, and operating data engineering solutions on the AWS Cloud ecosystem. The successful candidate will have expertise in AWS toolset and advise on, develops, and maintains data engineering solutions.Key ResponsibilitiesDesign, build, and operate batch and real-time data...
-
Data Driven AI Engineer
1 day ago
Perth, Western Australia NXTGIG Full timeNXTGIG is a leading technology company based in Perth, Australia, that delivers innovative AI solutions.AI Engineer PositionWe are seeking a highly skilled AI Engineer with expertise in Python to join our team. As an AI Engineer, you will design, develop, and deploy AI-driven solutions that leverage advanced machine learning techniques and algorithms. You...
-
Cloud Data Engineer Leader
1 day ago
Perth, Western Australia Robert Walters Full timeThis is an exciting opportunity for a Cloud Data Engineer Leader to join Robert Walters and play a pivotal role in revolutionising financial wellness. As a key contributor, you will design, develop, and maintain our data infrastructure, ensuring data is reliable, secure, and accessible for driving insights and delivering better wealth experiences.Key...
-
Mechanical Engineer for Data Centers
7 days ago
Perth, Western Australia Capital Executive Search Full timeAbout the RoleThis is a mid-senior level, full-time position within the Engineering department. The ideal candidate will have a Bachelor's degree in Mechanical Engineering or Petroleum Engineering and a minimum of 5 years of experience in Data Centers and/or mission critical facilities design, construction, or operations.Solid knowledge of large-scale water...
-
Senior Data Engineering Specialist
1 day ago
Perth, Western Australia Paxus - Technology + Digital Talent Full timeAbout the RoleThe Senior Data Engineering Specialist will be responsible for building, testing, and operating an enterprise data platform on AWS. This role requires a secure-by-design approach to develop robust, thoroughly tested, and maintainable solutions.Key ResponsibilitiesDesign, build, and operate batch and real-time data pipelines using AWS services...
-
Cloud and Data Engineering Specialist
3 days ago
Perth, Western Australia Business in Heels International Pty Full timeBecoming part of Business in Heels International Pty means being part of a pioneering organization that pushes boundaries through innovative science and technology. As a leading research organization, we deliver solutions with real impact, transforming lives and shaping the future.We are seeking a highly skilled software engineer to join our team in Perth,...
-
Assistant Data Engineer
3 hours ago
Perth, Western Australia VenuesWest Full timePosition number: 019604Work type: Permanent - Full TimeLocation: Perth CBDThis is a permanent, full-time vacancy.About JusticeWhen you join the Department of Justice, you become part of a people-focused organisation, working hard to make our community a fairer and safer place to live. We offer many different employment opportunities in locations around...
-
OT Engineering Team Manager
5 days ago
Perth, Western Australia C015 APT MANAGEMENT SERVICES PTY LIMITED Full timeAbout the JobWe're looking for an experienced Data and Modelling Solutions Architect to join our OT Engineering Team Lead – Data and Modelling team.Key Responsibilities:Design and develop data management solutions to support operational technology and data science initiatives.Collaborate with cross-functional teams to identify and prioritize data science...
-
Senior Engineer
5 days ago
Perth, Western Australia C015 APT MANAGEMENT SERVICES PTY LIMITED Full timeOT Engineering Team Manager PositionWe're seeking a skilled OT Engineering Team Manager to lead our Data and Modelling team.Key Responsibilities:Lead the development of team capabilities to manage Pi Historian and Online Sim modelling applications.Collaborate with cross-functional teams to identify and prioritize operational technology and data science...
-
Data Solutions Specialist
5 days ago
Perth, Western Australia ROBERT WALTERS AUSTRALIA Full timeThe ideal candidate will possess a strong background in data engineering, with expertise in designing and implementing efficient data pipelines using Azure Databricks and Synapse Analytics.Key responsibilities include developing and maintaining SQL queries for data transformation and cleansing, as well as designing integration solutions using...
-
Lead Data Architect
6 days ago
Perth, Western Australia Fortescue Full timeData Engineer Leadership OpportunityFortescue is seeking a Lead Data Architect to join our team. As a technical leader, you will define the data architecture strategy, establish engineering standards, and design scalable ETL/ELT frameworks.Key ResponsibilitiesDefine and implement enterprise data architecture standards, including ETL/ELT processes, data...
-
Senior Data Science Software Engineer
7 days ago
Perth, Western Australia Commonwealth Scientific and Industrial Research Organisation Full timeThe Commonwealth Scientific and Industrial Research Organisation (CSIRO) is seeking a skilled software engineer to join our team in Perth, Western Australia. Our organisation is building one of the world's largest science facilities, with next-generation telescopes driving astrophysics research, innovation, and skills development.Job DescriptionWe are...