Current jobs related to Big Data Engineer for Large-Scale Data Solutions - Sydney, New South Wales - Luxoft

  • Senior Data Engineer

    4 weeks ago

    Sydney, New South Wales, Australia Data-Property Data Solns Shared Full time

    We are seeking a highly skilled Senior Data Engineer to join our team at Data-Property Data Solutions Shared. This is an exciting opportunity for a seasoned professional to lead the development of cutting-edge data solutions and drive business growth.About UsOur company specializes in simplifying the property journey for all stakeholders, leveraging our...

  • Big Data Solutions Architect

    4 weeks ago

    Sydney, New South Wales, Australia Zone IT Solutions Full time

    About the RoleWe are seeking a highly skilled Big Data Solution Engineer to join our team. As a key member of our technical staff, you will be responsible for designing and implementing scalable big data solutions to support our clients' analytical needs.This role involves leveraging your expertise in big data technologies and architectures to develop...

  • Data Architect and Big Data Engineer

    4 weeks ago

    Sydney, New South Wales, Australia Commonwealth Bank of Australia Full time

    We're looking for a highly skilled Senior Data Engineer to join our team. With a focus on delivering high-quality data solutions, you'll be at the forefront of Big Data and Data warehouse technologies.As a key member of our engineering team, you'll be working on state-of-the-art data platforms that power seamless experiences for millions of customers. Your...

  • Senior Data Engineer for Large-Scale Guidewire Migration

    4 weeks ago

    Sydney, New South Wales, Australia Cognizant Full time

    About the Role:Cognizant is seeking a highly skilled Senior Data Engineer to lead our large-scale Guidewire migration project. This exciting opportunity will challenge your technical expertise and allow you to contribute to the design, development, and deployment of complex data solutions.Key Responsibilities:- Collaborate with cross-functional teams to...

  • Data Solutions Architect

    1 month ago

    Sydney, New South Wales, Australia Experis Full time

    We are seeking an experienced Data Solutions Architect to lead our team in designing and delivering future-ready data solutions for a prestigious financial services organisation. As a technical lead, you will work closely with cross-functional teams to provide cutting-edge data solutions while mentoring junior team members and ensuring excellence in project...

  • Data Operations Engineer

    4 weeks ago

    Sydney, New South Wales, Australia Tabcorp Full time

    About the RoleWe are seeking a highly skilled Data Operations Engineer to join our team, focusing on developing and delivering scalable data solutions using cutting-edge technologies such as Databricks, Teradata, Redshift, Talend, and AWS. As part of this role, you will be responsible for ensuring the successful implementation of data engineering projects,...

  • Senior Data Engineer

    4 weeks ago

    Sydney, New South Wales, Australia Zone IT Solutions Full time

    About the RoleWe are seeking a Senior Data Engineer with expertise in cloud and analytics to join our team. As a key member of our organization, you will be responsible for designing and implementing scalable big data solutions to support our clients' analytical needs.Key Responsibilities:Design and implement robust big data architectures on cloud platforms...

  • Senior Big Data Solutions Architect

    4 weeks ago

    Sydney, New South Wales, Australia Whizdom Recruitment Full time

    Drive business transformation with a challenging role at Whizdom Recruitment. We are seeking an experienced Business/Systems Analyst to lead the development and support of a big data platform in Sydney.About the Role:The Senior Big Data Solutions Architect will act as a bridge between business stakeholders, architects, designers, and the delivery team....

  • Enterprise Solution Strategist

    4 weeks ago

    Sydney, New South Wales, Australia NTT DATA Full time

    Job DescriptionWe are seeking a seasoned Enterprise Solution Strategist to join our team at NTT DATA. This role is ideal for a highly skilled professional with expertise in designing and implementing complex solutions that address clients' needs.About the Role:You will work closely with clients and internal teams to develop strategic plans, transformational...

  • Data Architect for Big Data Solutions

    1 month ago

    Sydney, New South Wales, Australia Pyramid Global Technologies Full time

    About Pyramid Global TechnologiesWe are a cutting-edge technology company seeking an experienced Data Architect for Big Data Solutions to lead our team in developing end-to-end big data pipelines.Estimated Salary: $120,000 - $180,000 per yearAs a Data Architect for Big Data Solutions, you will be responsible for designing and implementing scalable big data...

  • Data Engineering Expert

    1 month ago

    Sydney, New South Wales, Australia Data - AI & Data Science Full time

    About UsWe shine a light on all things property, simplifying the journey for everyone involved. Our expertise and exclusive data drive value across the organisation.Role OverviewThis is an exciting opportunity to make your mark in data engineering at Domain Group's Data Decision Science team. You'll play a key role in building data pipelines that power...

  • Big Data Transformation Expert

    4 weeks ago

    Sydney, New South Wales, Australia Experis Full time

    We are seeking a highly skilled Business Transformation Consultant to join our dynamic team in Experis, Australia. This is a crucial role in building and supporting a big data platform, collaborating with stakeholders, and driving significant enterprise projects.Job SummaryThis role involves working closely with business stakeholders, process owners, and...

  • Big Data Solutions Expert

    2 weeks ago

    Sydney, New South Wales, Australia Databricks Full time

    Big Data Solutions ExpertWe are seeking a highly skilled Big Data Solutions Expert to join our team at Databricks in Australia. As a key member of our Consulting and System Integrator (C&SI) partner ecosystem, you will work closely with our partners, teammates, and customers to develop technical champions within their top C&SI Partners.Your expertise in...

  • Big Data Engineer Position in Australia

    4 weeks ago

    Sydney, New South Wales, Australia Zone IT Solutions Full time

    About Zone IT Solutions">We are a recruitment company based in Australia, specializing in digital and IT services.">Salary Range: AU$120,000 - AU$180,000 per annum">Job Description:">We are seeking an experienced AWS Hadoop Data Engineer to design, build, and optimize data pipelines and processing frameworks on the AWS cloud platform. You will be responsible...

  • Big Data Solutions Architect Lead

    4 weeks ago

    Sydney, New South Wales, Australia Databricks Full time

    This is an exceptional opportunity to join Databricks, a leading company in the AU region, as a Big Data Solutions Architect Lead. The role offers a chance to be part of an amazing team that is growing rapidly.The Big Data Solutions Architect Lead will work closely with Databricks' Consulting and System Integrator (C&SI) partners, teammates, and technical...

  • Data Engineering Leader

    4 weeks ago

    Sydney, New South Wales, Australia Simple Machines Full time

    Company OverviewSimple Machines is a pioneering technology firm with a global presence, offering bespoke data solutions at the intersection of data, AI, machine learning, and software engineering.We help enterprises, technology companies, and governments unlock the potential of data through innovative technologies and tailored services. Our mission is to...

  • Lead Data Engineer

    2 months ago

    Sydney, New South Wales, Australia Cognizant Full time

    About This RoleCognizant is a leading provider of information technology, consulting, and business process services, dedicated to helping the world's leading companies build stronger businesses. As a Lead Data Engineer - Big Data Architect, you will play a key role in designing and developing scalable data infrastructure to support the needs of internal and...

  • Data Transformation Consultant

    4 weeks ago

    Sydney, New South Wales, Australia Whizdom Recruitment Full time

    Job DescriptionOur client, a well-established multinational consultancy, is seeking an experienced Data Transformation Consultant to join their team in Sydney. This 2-3 year contract offers a unique opportunity to work with a big data platform, driving its development and support.About the Role:The successful candidate will act as a critical bridge between...

  • Senior Data Architect for Cloud and Big Data Solutions

    4 weeks ago

    Sydney, New South Wales, Australia Experis Full time

    Job DescriptionWe are seeking a highly experienced Senior Data Architect to lead the design, development, and delivery of future-ready data solutions. As a technical leader, you will work closely with cross-functional teams to provide cutting-edge data solutions while mentoring junior team members and ensuring excellence in project delivery.Key...

  • Cloud Solution Strategist

    4 weeks ago

    Sydney, New South Wales, Australia NTT DATA Full time

    About the RoleWe are seeking a seasoned Cloud Solution Strategist to join our team at NTT DATA.As a key member of our cloud solutions group, you will work closely with clients and internal teams to develop strategies, transformational designs, and architectural visions for complex solutions that meet their needs.The ideal candidate will have extensive...

Big Data Engineer for Large-Scale Data Solutions

1 month ago

Sydney, New South Wales, Australia Luxoft Full time
Job Summary

Luxoft is seeking a highly skilled Big Data Engineer to design, develop, and maintain large-scale data solutions using PySpark. As a key member of our team, you will work closely with data scientists, data analysts, and software engineers to drive robust, data-driven insights for business decisions.

About the Role

We are looking for an experienced engineer who can optimize ETL pipelines for performance and scalability. You will work with large volumes of structured and unstructured data, transforming it to meet business needs. Your expertise in integrating data from multiple sources into the data platform will ensure data integrity and quality.

In this role, you will collaborate with cross-functional teams to understand data requirements and translate them into efficient data workflows. You will implement best practices for data governance, monitoring, and data security. Additionally, you will debug and troubleshoot issues across ETL pipelines and data workflows.

Key Responsibilities
  • Design, develop, and maintain ETL pipelines using PySpark, optimizing for performance and scalability.
  • Work with large volumes of structured and unstructured data, transforming it to meet business needs.
  • Integrate data from multiple sources into the data platform, ensuring data integrity and quality.
  • Collaborate with cross-functional teams to understand data requirements and translate them into efficient data workflows.
  • Implement best practices for data governance, monitoring, and data security.
Required Skills and Qualifications

To be successful in this role, you will need:

  • Proficiency in PySpark: In-depth experience with PySpark for data processing and transformation tasks.
  • SQL Knowledge: Strong command of SQL for querying and processing data.
  • Data Warehousing Concepts: Familiarity with data warehousing, data lakes, and data integration principles.
  • Cloud Platforms: Experience with cloud environments like AWS, GCP, or Azure for data storage and processing.
  • Big Data Technologies: Hands-on experience with Hadoop and Spark ecosystem (Spark SQL, Spark Streaming).
  • Data Modeling: Experience in designing and implementing efficient data models.
  • Python Programming: Strong Python skills, particularly in data manipulation and analysis.
Estimate Salary: $120,000 - $180,000 per year