Data Engineering Lead

1 day ago

Sydney, New South Wales, Australia Luxoft Full time
Data Engineer Role at Luxoft

We are seeking a skilled Data Engineer to join our team at Luxoft, where you will be working on high-impact data engineering projects. As a Data Engineer, you will play a crucial role in designing, developing, and maintaining large-scale ETL pipelines using PySpark.

Responsibilities:

  • Design and develop efficient ETL pipelines using PySpark for data processing and transformation tasks.
  • Collaborate with cross-functional teams to understand data requirements and translate them into efficient data workflows.
  • Implement best practices for data governance, monitoring, and security.
  • Work closely with data scientists and analysts to drive robust, data-driven insights for business decisions.
  • Optimize performance, scalability, and reliability of existing data pipelines.
  • Provide documentation and training for data workflows and processes.

Requirements:

  • Proficiency in PySpark: In-depth experience with PySpark for data processing and transformation tasks.
  • Strong SQL skills for querying and processing data.
  • Familiarity with data warehousing concepts, including data lakes and integration principles.
  • Experience with cloud environments like AWS, GCP, or Azure for data storage and processing.
  • Hands-on experience with Hadoop and Spark ecosystem (Spark SQL, Spark Streaming).
  • Strong Python programming skills, particularly in data manipulation and analysis.

Benefits:

  • Average salary: $150,000 per year.
  • Opportunity to work on high-impact data engineering projects.
  • Collaborative and dynamic work environment.
  • Ongoing training and professional development opportunities.

About Us:

Luxoft is a global technology company that provides innovative software solutions to clients worldwide. We are committed to delivering exceptional results and fostering a culture of innovation and excellence.

  • Data Engineer

    6 days ago

    Sydney, New South Wales, Australia Data - AI & Data Science Full time

    Drive Business Value with Data Engineering at Domain GroupWe are seeking a skilled Data Engineer to join our Data Decision Science team in Sydney. As a key member of our data department, you will play a critical role in building scalable data pipelines that power Australia's property market.About the RoleThis is an exciting opportunity to work with...

  • Data Engineer

    1 week ago

    Sydney, New South Wales, Australia Data - AI & Data Science Full time

    About UsWe shine a light on the property market, simplifying the journey for all involved. Our expertise and exclusive data drive change. At Domain Group, our data department leads the way in driving value across the organisation.The RoleAs a Senior Data Engineer, you'll play a key role in building data pipelines that power Australia's property market....

  • Data Centre Engineering Specialist

    1 month ago

    Sydney, New South Wales, Australia Cdc Data Centres Full time

    About UsCDC Data Centres is a leading operator of sovereign, secure, world-class data centre facilities in Australia.Our mission is to attract, retain, and grow the world's best data centre team.About the RoleWe are seeking a Technical Solutions Engineer to provide thought leadership, planning, management, and technical design development for our...

  • Data Centre Engineering Specialist

    1 month ago

    Sydney, New South Wales, Australia CDC Data Centres Full time

    About us: CDC Data Centres is a leading operator of sovereign, secure, world-class data centre facilities in Australia and New Zealand.Our mission is to attract, retain and grow the world's best data centre team, promoting an inclusive workplace that values diversity.We believe that everyone's unique experiences enable us to deliver meaningful benefits for...

  • Senior Data Engineer

    1 month ago

    Sydney, New South Wales, Australia Cdc Data Centres Full time

    About UsCDC Data Centres is a leading operator of sovereign, secure, world-class data centre facilities in Australia. Our mission is to attract, retain, and grow the world's best data centre team.About the RoleWe are seeking a Senior Data Engineer to design, implement, and support telemetry systems and reporting solutions within our IT environments. The...

  • Data Property Solutions Engineer

    6 days ago

    Sydney, New South Wales, Australia Data-Property Data Solns Shared Full time

    We are seeking a highly skilled Solutions Data Engineer to join our team at Data-Property Data Solns Shared. As a Senior member of our organization, you will play a critical role in shaping the future of property data solutions.About UsWe shine a light on all things property, aiming to simplify the property journey for all involved. Our business is motivated...

  • Data Engineer for Property Insights

    1 week ago

    Sydney, New South Wales, Australia Data-Property Data Solns Shared Full time

    We are seeking a highly skilled Senior Data Engineer to join our Single View Property (SVP) team at Domain. This role is perfect for someone who thrives in a dynamic environment and has a passion for working with data to drive business outcomes.

  • Data Centre Design Engineer Position

    1 week ago

    Sydney, New South Wales, Australia Cdc Data Centres Full time

    About UsCDC Data Centres is a leading operator of sovereign, secure, world-class data centre facilities in Australia.Established in 2007, our company has grown to become the largest provider of co-location, containerised and hyperscale compute environments with high-speed carrier-grade networks connecting all campuses.We are committed to creating an...

  • Data Architect Lead

    4 weeks ago

    Sydney, New South Wales, Australia Data Privacy Full time

    Data Architect LeadThis role involves providing technical leadership to the project delivery teams through design reviews, advising on implementation best practices, and defining release processes. Key Responsibilities:Provide technical leadership to the project delivery teamsReview and refine the data delivery framework to enhance integration of new sources...

  • Data Engineering Lead

    4 weeks ago

    Sydney, New South Wales, Australia Canva Full time

    At Canva, we're looking for a highly skilled Data Engineering Lead to join our Business Data Platform team. Role Overview As a Data Engineering Lead, you will be responsible for designing and developing end-to-end cloud data warehouse solutions, ensuring data is reliable, accurate, and consistent. You will work collaboratively with the People Analytics...

  • Data Engineering Lead

    4 weeks ago

    Sydney, New South Wales, Australia Correlate Resources Full time

    Correlate Resources is seeking a seasoned Data Engineering Lead to join their team in Australia. This is a high-profile opportunity to work with a reputable brand and contribute to building a 'best in class' data-driven operation.Key Responsibilities:Lead the design, development, and optimisation of back-end NoSQL and Python-related data platforms for...

  • Senior Software Engineer

    4 weeks ago

    Sydney, New South Wales, Australia Dicker Data Full time

    Job Description:As a senior software engineer in data analysis, you will be responsible for developing and maintaining large-scale data processing systems. Your primary focus will be on designing and implementing data pipelines using Python, ensuring data quality and integrity. You will work closely with cross-functional teams to deliver high-quality...

  • Senior Data Engineer

    1 month ago

    Sydney, New South Wales, Australia CDC Data Centres Full time

    About UsCDC Data Centres is a leading operator of sovereign, secure, world-class data centre facilities in Australia.Established in 2007, we have a strong presence in Canberra, NSW, and New Zealand, with a diverse range of operations that provide secure data centre configurations for co-location, containerised, and hyperscale compute environments.We are...

  • Senior Data Engineer

    4 weeks ago

    Sydney, New South Wales, Australia CDC Data Centres Full time

    About CDC Data CentresWe are a leading operator of sovereign, secure, world-class data centre facilities in Australia and New Zealand.Our mission is to attract, retain and grow the world's best data centre team by creating an inclusive workplace that promotes and values diversity.About the RoleWe are seeking a highly skilled Senior Data Engineer to design,...

  • Data Engineering Leader

    2 weeks ago

    Sydney, New South Wales, Australia Cdc Data Centres Full time

    About UsCDC Data Centres is a leading provider of sovereign and secure data centre facilities in Australia.We are passionate about creating an inclusive workplace that promotes diversity and values individual experiences. Our mission is to attract, retain, and grow the world's best data centre team.Our team is dedicated to delivering meaningful benefits for...

  • Data Engineer Lead

    4 weeks ago

    Sydney, New South Wales, Australia BaptistCare Full time

    BaptistCare Data Engineer Lead Job DescriptionAre you a skilled data engineer looking for a challenging and rewarding role? We are seeking an experienced Data Engineer Lead to join our team at BaptistCare, a leading for-purpose Christian-based care organisation.Key Responsibilities:Design, implement, and optimise data pipelines on Azure Data Platform for...

  • Data Engineering Expert

    2 days ago

    Sydney, New South Wales, Australia Data - AI & Data Science Full time

    About UsWe shine a light on all things property, simplifying the journey for everyone involved. Our expertise and exclusive data drive value across the organisation.Role OverviewThis is an exciting opportunity to make your mark in data engineering at Domain Group's Data Decision Science team. You'll play a key role in building data pipelines that power...

  • Data Engineering Lead

    19 hours ago

    Sydney, New South Wales, Australia Commonwealth Bank of Australia Full time

    Transformative Opportunity at Commonwealth Bank of AustraliaWe are seeking an exceptional Data Engineering Lead to join our Financial Services Technology – Finance domain, driving the delivery of digital era-experiences that are simple, smart and secure. As part of our 3 Lines of Assurance (3LOA) Crew, you will play a pivotal role in expanding and...

  • Data Engineering Lead

    1 month ago

    Sydney, New South Wales, Australia TMGM Full time

    Design and Implement Scalable Data SolutionsAs a Data Engineer Lead at TMGM, you will be responsible for designing, constructing, installing, testing, and maintaining highly scalable data management systems. You will ensure that our systems meet business requirements and adhere to industry practices, while also integrating cutting-edge data management and...

  • Lead Data Engineer

    1 month ago

    Sydney, New South Wales, Australia REA Group Full time

    About the RoleWe are seeking a highly skilled and experienced Lead Data Engineer to join our PropTrack Engineering team. As a key member of our technical leadership team, you will be responsible for providing technical direction and guidance to your team, ensuring the delivery of high-quality technical outcomes.Key ResponsibilitiesProvide technical...