Principle Data Engineer
2 days ago
- 6 month initial contract
- Perth based; hybrid - 3 days onsite
- Multiple roles *3
- AWS Toolset
Tasks to be performed on a daily basis:
The Principal Data Engineer shall be responsible for the building, testing, and operation of an enterprise data platform.
The engineer shall ensure that the platform is developed and deployed with a secure-by-design approach, delivering robust, thoroughly tested, and maintainable solutions. Moreover, the Platform Data Engineer shall provide support, security reviews, and technical reviews for delivery data engineers working on the platform.
A data engineer with expertise in AWS toolset advises on, develops, and maintains data engineering solutions on the AWS Cloud ecosystem. They design, build, and operate batch and real-time data pipelines using AWS services such as AWS EMR, AWS Glue, Glue Catalog, and Kinesis. Additionally, they create data layers on AWS RedShift, Aurora, and DynamoDB.
The data engineer also migrates data using AWS DMS and is proficient with various AWS Data Platform components, including S3, RedShift, RedShift Spectrum, AWS Glue with Spark, AWS Glue with Python, Lambda functions with Python, AWS Glue Catalog, and AWS Glue Databrew. They are experienced in developing batch and real-time data pipelines for Data Warehouse and Datalake, utilizing AWS Kinesis and Managed Streaming for Apache Kafka. They are also proficient in using open source technologies like Apache Airflow and dbt, Spark / Python or Spark / Scala on AWS Platform.
The data engineer schedules and manages data services on the AWS Platform, ensuring seamless integration and operation of data engineering solutions.
Mandatory Requirements:
- Development experience with Go, SQL, C#, .net, JavaScript, shell scripts & container platforms like Docker.
- The engineer shall have experience integrating with timeseries source systems: Honeywell Plant Historian Database, OSI Pi.
- The engineer shall have extensive experience in Authentication mechanisms including but not limited to (OAuth 2.0, OIDC, Microsoft Entra, Key Pair Authentication, Certificate based authentication, SAML based SSO).
- Use of AWS services for cloud-based platform tooling infrastructure including but not limited to: Lambda, ECS, MSK, RDS, EC2, Secrets Manager, ALB, Cloud Watch, Event Bridge.
Required Professional and Technical Expertise:
- Development of standard patterns for data pipelines and workflows utilizing Streamsets, Kestra, dbt, Git.
- APIs / Integration for data access and manipulation.
- Development of APIs and Web Applications to support the management of the Platform (Security, Sandbox creation etc.).
- Experience in Terraform for AWS and Azure deployments.
- Python scripting for data common processing and automation tasks.
- CLI development to automate common platform tasks.
- Data storage design and processing solutions employing Snowflake.
- Quality assurance and testing.
Desired Attributes and Skills:
- Provide technical support to delivery data engineers working on the platform.
- Conduct security reviews for data engineering projects to ensure compliance with standards.
- Offer technical reviews and feedback to enhance the quality of deliverables.
- Testing and Quality Assurance experience.
To be considered for the role, click the 'apply' button or for more information about this and other opportunities please contact Gizaille Shaheen on 03 86804305 or email: Gshaheen@paxus.com.au and quote the above job reference number.
#J-18808-Ljbffr-
Performance Data Engineer
11 hours ago
Perth, Western Australia VenuesWest Full timeAbout the RoleVenuesWest is seeking a Performance Data Engineer to join our team. In this role, you'll be responsible for designing and implementing performance data solutions. This includes developing and maintaining data pipelines, creating reports and dashboards, and ensuring data quality and security.Key Responsibilities:Design and implement performance...
-
Director of Data Architecture and Engineering
7 hours ago
Perth, Western Australia Tbwa ChiatDay Inc Full timeAbout VGWVGW is a leading interactive entertainment company that harnesses technology and creativity to deliver world-class, free-to-play games.We have an exciting opportunity to join our Technology team in Australia and are currently seeking a highly skilled Director of Data Architecture and Engineering to lead our data engineering efforts.As the Director...
-
Data Engineer
2 weeks ago
Perth, Western Australia Tideri Jobbörse Full timeDo you want a job that looks different every day? Have you got an unquenchable passion for data and problem solving? Are you aching to jump into a work environment where you will work with some of the keenest minds in the industry? We've got a career you will enjoy At InterWorks, we like our work with a hefty side of play. Our team of high achievers is...
-
Head Of Data Engineering
11 hours ago
Perth, Western Australia Tbwa ChiatDay Inc Full timeVGW is an interactive entertainment company, harnessing technology and creativity to deliver world-class, free-to-play games. We have an exciting opportunity to join our Technology team in Australia and are currently looking for a Head of Data Engineering to join the team. As the Head of Data Engineering at VGW, you will lead the strategy to deliver...
-
Data Engineer
15 hours ago
Perth, Western Australia InterWorks Full timeDo you want a job that looks different every day? Have you got an unquenchable passion for data and problem solving? Are you aching to jump into a work environment where you will work with some of the keenest minds in the industry? We've got a career you will enjoyAt InterWorks, we like our work with a hefty side of play. Our team of high achievers is...
-
Data Engineer
2 days ago
Perth, Western Australia Robert Walters Full timeOur client, a leading player in the financial services sector, is seeking a Data Engineer to play a pivotal role to leverage technology.This is an opportunity to revolutionise financial wellness by merging innovative technology with human expertise. As a key contributor, you will build and maintain their data infrastructure, ensuring data is reliable,...
-
Data Engineer
2 days ago
Perth, Western Australia Whizdom Full timeWhizdom Perth, Western Australia, AustraliaState Manager - taking the Whizdom success story further into NSW, QLD and VICAbout Our ClientOur client is a globally recognized consulting house with a strong reputation for delivering innovative solutions across various sectors, including technology, energy, and finance. With a presence in multiple countries,...
-
Data Engineering Specialist
3 days ago
Perth, Western Australia Fortescue Full timeAt Fortescue, we are committed to leading the world in decarbonisation and sustainability. As a key player in our Heavy Industry projects, you will play a crucial role in developing and testing technologies to power our mining fleet and rail locomotives.We are seeking an experienced Data Engineer to join our team at our new Redcliffe and Hazelmere R&D...
-
Cloud Data Engineer
3 days ago
Perth, Western Australia Fortescue Full timeFortescue is driving innovation in heavy industry through our Heavy Industry projects, focusing on green energy and carbon neutrality. We are seeking a talented Data Engineer to join our team at our Redcliffe and Hazelmere R&D facilities.In this role, you will work collaboratively with a multidisciplinary engineering team to develop and test technologies...
-
Assistant Data Engineer
15 hours ago
Perth, Western Australia VenuesWest Full timePosition number: 019604Work type: Permanent - Full TimeLocation: Perth CBDThis is a permanent, full-time vacancy.About JusticeWhen you join the Department of Justice, you become part of a people-focused organisation, working hard to make our community a fairer and safer place to live. We offer many different employment opportunities in locations around...
-
Senior Data Development Engineer
5 days ago
Perth, Western Australia VitalTrace Full timeAbout the Role:VitalTrace is seeking an experienced Data Engineer to lead our data engineering initiatives. As a key member of our team, you will be responsible for designing, building, and maintaining data pipelines, developing applications, and implementing data management solutions.Main Responsibilities:Develop and maintain efficient data pipelines to...
-
AWS Data Platform Engineer
2 days ago
Perth, Western Australia Paxus - Technology + Digital Talent Full timeAbout the PositionThe AWS Data Platform Engineer will be responsible for designing, building, and operating data engineering solutions on the AWS Cloud ecosystem. The successful candidate will have expertise in AWS toolset and advise on, develops, and maintains data engineering solutions.Key ResponsibilitiesDesign, build, and operate batch and real-time data...
-
Data Driven AI Engineer
2 days ago
Perth, Western Australia NXTGIG Full timeNXTGIG is a leading technology company based in Perth, Australia, that delivers innovative AI solutions.AI Engineer PositionWe are seeking a highly skilled AI Engineer with expertise in Python to join our team. As an AI Engineer, you will design, develop, and deploy AI-driven solutions that leverage advanced machine learning techniques and algorithms. You...
-
Cloud Data Engineer Leader
2 days ago
Perth, Western Australia Robert Walters Full timeThis is an exciting opportunity for a Cloud Data Engineer Leader to join Robert Walters and play a pivotal role in revolutionising financial wellness. As a key contributor, you will design, develop, and maintain our data infrastructure, ensuring data is reliable, secure, and accessible for driving insights and delivering better wealth experiences.Key...
-
Engineering Cost Controller
2 days ago
Perth, Western Australia Zenith Search Full timeEngineering Cost Controller WantedSeeking a skilled Engineering Cost Controller to join our team. Salary: AUD130000 - AUD160000 per annum + SuperAbout the OpportunityThis is an excellent opportunity for a seasoned professional to take on a challenging role in cost management. As an Engineering Cost Controller, you will be responsible for overseeing the...
-
Senior Data Engineering Specialist
2 days ago
Perth, Western Australia Paxus - Technology + Digital Talent Full timeAbout the RoleThe Senior Data Engineering Specialist will be responsible for building, testing, and operating an enterprise data platform on AWS. This role requires a secure-by-design approach to develop robust, thoroughly tested, and maintainable solutions.Key ResponsibilitiesDesign, build, and operate batch and real-time data pipelines using AWS services...
-
Cloud and Data Engineering Specialist
3 days ago
Perth, Western Australia Business in Heels International Pty Full timeBecoming part of Business in Heels International Pty means being part of a pioneering organization that pushes boundaries through innovative science and technology. As a leading research organization, we deliver solutions with real impact, transforming lives and shaping the future.We are seeking a highly skilled software engineer to join our team in Perth,...
-
AI Engineer with Python
2 days ago
Perth, Western Australia NXTGIG Full timeNXTGIG Perth, Western Australia, AustraliaAI Engineer with PythonNXT GIG is looking for an AI Engineer with expertise in Python to join our dynamic tech team. In this role, you will be tasked with designing, developing, and deploying AI-driven solutions that leverage advanced machine learning techniques and algorithms. Collaborating closely with data...
-
Senior Telecommunications Engineer
3 weeks ago
Perth, Western Australia Airswift Full time1x Senior Project Telecommunications Engineer Location: Perth The company: Major EPC Player Benefits: Competitive rate with long-term prospect Indicative assignment length: Initial 12-month contract (subject to agreement) The Company Our client is a well-known consultancy with multiple offices across the globe.As a result of ongoing business growth, they...
-
Engineering Project Manager
7 days ago
Perth, Western Australia Jacobs Full timeJacobs is a global company dedicated to delivering technical solutions across diverse clients. We're seeking a seasoned Principal Systems Engineer to join our Perth office and support the ANZ Rail & Transit team in delivering technical solutions.The role requires strong client relationships, clear technical direction, and exceptional project delivery...