Data Engineer

1 month ago

Sydney, Australia Experis Full time

We are seeking an experienced Data Engineer with a strong background in AWS toolsets to advise, develop, and maintain data engineering solutions within the AWS Cloud ecosystem. This role involves designing, building, and managing both batch and real-time data pipelines using various AWS services.

Key Responsibilities:

  • Design, build, and operate batch and real-time data pipelines using AWS services like AWS EMR, AWS Glue, Glue Catalog, and Kinesis.
  • Develop data layers on AWS RedShift, Aurora, and DynamoDB.
  • Migrate data using AWS DMS and manage data integration using AWS Data Platform components such as S3, RedShift, RedShift Spectrum, AWS Glue with Spark, AWS Glue with Python, Lambda functions with Python, AWS Glue Catalog, and AWS Glue Databrew.
  • Build data pipelines for Data Warehouses and Data Lakes using AWS Kinesis and Managed Streaming for Apache Kafka.
  • Utilize open-source technologies, including Apache Airflow and dbt, as well as Spark/Python or Spark/Scala on the AWS Platform.
  • Schedule and manage data services on AWS, ensuring efficient operation and integration.

Mandatory Requirements:

  • Experience working on traditional Data Warehousing with any database (Oracle, DB2, or SQL Server; Redshift experience is a plus).
  • Strong SQL skills with the ability to write complex queries using analytical functions.
  • Prior experience working on the AWS platform.
  • Proficient in Python programming for data engineering.
  • Experience with PySpark/Spark.
  • Working knowledge of data pipeline tools like Apache Airflow.

Preferred Skills and Expertise

  • Experience with DBT.
  • Familiarity with Agile working environments.
  • Proven ability to troubleshoot and resolve production issues under a DevOps model.
  • A consistent track record of identifying opportunities to improve performance and quality within data ecosystems.
  • Experience monitoring system performance and ensuring high-quality data operations.

if the above looks of intererest, please make an applicatoin and Experis will be in contact with you.

Rowan Mollison
Senior Consultant, Experis

ManpowerGroup is committed to being a Diversity Confident Recruiter and encourages applications from people from a diverse range of backgrounds, including people with a disability. Please indicate your preferred method of communication in your resume and please let us know if you require any reasonable adjustments should you be contacted for an interview.

Aboriginal and Torres Strait Islander people are encouraged to apply.

By submitting your resume and other personal information with this application you are consenting to this information being collected in line with our privacy policy. Follow the link to learn more - www.manpowergroup.com.au/privacy-policy

Experis Pty Ltd is a wholly owned subsidiary of ManpowerGroup

State: QLD, licensee/s Manpower Services (Australia) Pty Ltd, LHL-02026-D5L4Q. State: QLD, licensee/s Experis Pty Ltd, LHL-02014-Y5F6D. State: SA, licensee/s Manpower Services (Australia) Pty Ltd, LHS 288856

  • Data Engineer

    4 weeks ago

    Sydney, New South Wales, Australia Data - AI & Data Science Full time

    Drive Business Value with Data Engineering at Domain GroupWe are seeking a skilled Data Engineer to join our Data Decision Science team in Sydney. As a key member of our data department, you will play a critical role in building scalable data pipelines that power Australia's property market.About the RoleThis is an exciting opportunity to work with...

  • Data Property Solutions Engineer

    4 weeks ago

    Sydney, New South Wales, Australia Data-Property Data Solns Shared Full time

    We are seeking a highly skilled Solutions Data Engineer to join our team at Data-Property Data Solns Shared. As a Senior member of our organization, you will play a critical role in shaping the future of property data solutions.About UsWe shine a light on all things property, aiming to simplify the property journey for all involved. Our business is motivated...

  • Senior Data Engineer- Data Decision Science Group

    4 weeks ago

    Sydney, Australia Data - AI & Data Science Full time

    Who are weWe shine a light on all things property. Our business aims to simplify the property journey for all involved; motivated by expertise and our exclusive data. Changing the way people engage with property requires a team of diverse thinkers.At Domain Group, our data department is at the forefront of driving value across the organisation. We create...

  • Senior Data Engineer

    3 weeks ago

    Sydney, New South Wales, Australia Data-Property Data Solns Shared Full time

    We are seeking a highly skilled Senior Data Engineer to join our team at Data-Property Data Solutions Shared. This is an exciting opportunity for a seasoned professional to lead the development of cutting-edge data solutions and drive business growth.About UsOur company specializes in simplifying the property journey for all stakeholders, leveraging our...

  • Data Engineering Expert

    4 weeks ago

    Sydney, New South Wales, Australia Data - AI & Data Science Full time

    About UsWe shine a light on all things property, simplifying the journey for everyone involved. Our expertise and exclusive data drive value across the organisation.Role OverviewThis is an exciting opportunity to make your mark in data engineering at Domain Group's Data Decision Science team. You'll play a key role in building data pipelines that power...

  • Senior Azure Data Engineer

    8 months ago

    North Sydney, Australia NTT DATA Full time

    NTT DATA is heavily investing globally in providing state of the art analytical solutions to its clients. Our capability has been focused locally, regionally in APAC and globally by forming strategic partnerships including Microsoft Azure, SAP Analytics, Snowflake, Matillion and many more, leveraging the strength of our cloud and network assets via the NTT...

  • Senior Data Engineer- SVP

    1 month ago

    Sydney, Australia Data-Property Data Solns Shared Full time

    Who are weWe shine a light on all things property. Our business aims to simplify the property journey for all involved; motivated by expertise and our exclusive data. Changing the way people engage with property requires a team of diverse thinkers.Our data department is at the forefront of driving value across the organisation. We create reusable data assets...

  • Senior Data Architect for Property Insights

    4 weeks ago

    Sydney, New South Wales, Australia Data - AI & Data Science Full time

    Company Overview">We shine a light on all things property at Domain Group, a business that aims to simplify the property journey for all involved. Our expertise and exclusive data drive value across the organisation.About the Role">This Senior Data Architect will play a key role in building scalable ETL pipelines using AWS and Snowflake, and designing data...

  • Data Analyst

    6 months ago

    Sydney, Australia The Data Foundry Full time

    Company Description We are a customer-obsessed, “one stop shop” for all things data-related. We are deliberately narrow in our focus areas but we go deep in each of them. We provide solutions, services and training in the **data strategy, data governance, data security, data management, data classification, data wrangling, data lake, data analytics,...

  • Data Engineer

    4 weeks ago

    Sydney, New South Wales, Australia Data-Property Data Solns Shared Full time

    About UsData-Property Data Solns Shared is a leading company that simplifies the property journey for all involved. Our mission is to empower every department with reusable data assets, driving value across the organization.

  • Technical Solutions Engineer

    2 months ago

    Sydney, Australia Cdc Data Centres Full time

    About us: Established in 2007, CDC Data Centres is Australia’s leading operator of sovereign, secure, world-class data centre facilities. Headquartered in Canberra with multiple facilities in the ACT, NSW and New Zealand, CDC’s diversified operations provide secure data centre configurations that support co-location, containerised and hyperscale...

  • Technical Solutions Engineer

    2 months ago

    Sydney, Australia CDC Data Centres Full time

    About us: Established in 2007, CDC Data Centres is Australia’s leading operator of sovereign, secure, world-class data centre facilities. Headquartered in Canberra with multiple facilities in the ACT, NSW and New Zealand, CDC’s diversified operations provide secure data centre configurations that support co-location, containerised and hyperscale...

  • Data Engineer

    2 months ago

    Sydney, Australia Lifebyte Systems Full time

    At LifeByte, we are a dynamic and innovative collective of tech visionaries driven by a relentless pursuit of excellence. Each of us brings a unique set of skills to the table, collaborating on projects that shape the future.Founded in 2017, we are dedicated to fostering an ecosystem of seamless resource exchange, where efficiency and precision are...

  • Data Engineer

    1 month ago

    Sydney, Australia Atlassian Full time

    Working at AtlassianAtlassians can choose where they work – whether in an office, from home, or a combination of the two. That way, Atlassians have more control over supporting their family, personal goals, and other priorities. We can hire people in any country where we have a legal entity. Interviews and onboarding are conducted virtually, a part of...

  • Cybersecurity Expert for NTT DATA

    4 weeks ago

    Sydney, New South Wales, Australia NTT DATA Full time

    About the RoleNTT DATA is seeking an experienced Senior Security Architect to join our team in Australia and New Zealand. This role offers a unique opportunity to work with leading-edge security solutions and make a significant impact on our clients' success.Job DescriptionWe are looking for a skilled Security Architect to provide technical leadership and...

  • Data Engineer

    5 months ago

    Sydney, Australia Attribute Group Full time

    Data EngineerPermanent $130k-$140k + Super Our client is seeking a seasoned Data Engineer with expertise in the Cloudera Data Platform (CDP) and Spark-based ingestion pipelines. This role is ideal for a professional with a strong background in building and optimising data pipelines and a track record of working with large-scale data infrastructures.Key...

  • Data Engineer

    4 months ago

    Sydney, Australia Mane Consulting Full time

    Are you a customer-obsessed data engineer who can focus on the delivery and innovation of new services? Can you use your Snowflake experience and knowledge to advise, design, and implement cost-effective solutions to our customers, whilst maintaining an excellent delivery posture? Do you want to shape your world and change the way the world works? Do you...

  • Principal Data Engineer

    1 month ago

    Sydney, Australia Talent Insights Full time

    Microsoft Azure, Databricks, Data Factory and Gen AI applications Hybrid, flexible work Opportunity to lead and shape a DE function, greenfield project Principal / Lead Data Engineer ( Microsoft Azure /...

  • Cyber Security Engineer with Expertise in Advanced Threat Detection

    3 weeks ago

    Sydney, New South Wales, Australia NTT DATA Full time

    NTT DATA is a leading provider of digital and AI infrastructure, serving 75% of the Fortune Global 100. We are committed to helping clients innovate, optimize, and transform for long-term success.As a Cyber Security Engineer with NTT DATA, you will play a vital role in securing and resolving cyber threats for our key clients. You will be responsible for...

  • Data Engineer Leader

    4 weeks ago

    Sydney, New South Wales, Australia The Drive Group Full time

    Unlock Your Data Engineering PotentialWe are seeking a talented Data Engineer Leader to join our team at The Drive Group, a fast-growing Australian consultancy. This exciting opportunity will allow you to utilize your expertise in data engineering concepts and techniques to drive business growth.As a Data Engineer Leader, you will be responsible for...