Data Pipeline Architect

7 days ago


Sydney, New South Wales, Australia Commonwealth Bank Full time

We are seeking an experienced Senior Data Engineer to join our team. As a key member of our big data operations team, you will play a critical role in driving our transition from existing technology stacks into cloud-based, future-ready technologies and agile, DevSecOps ways of working.

As a Senior Data Engineer, you will be responsible for developing and implementing state-of-the-art data pipelines using Big Data Hadoop, leveraging your scripting knowledge in Unix, Java or Python to automate tasks and improve efficiency.

We value creativity, innovation, and collaboration, and offer flexible work arrangements, professional development opportunities, and a comprehensive benefits package.

  • Have Risk Mindset – All CommBank employees are expected to proactively identify and understand current and future risks.
  • Have experience in building and supporting data pipelines using Bigdata Hadoop.
  • Have good experience with Spark and Hive.


  • Sydney, New South Wales, Australia desda pty Full time

    Enterprise Data Pipeline ArchitectAbout the RoleWe are seeking an experienced Enterprise Data Pipeline Architect to join our team. In this role, you will be responsible for designing, developing, and optimizing ETL pipelines for data migration and transformation across multiple enterprise systems. You will also work closely with our technical teams to...

  • Data Architect

    1 day ago


    Sydney, New South Wales, Australia Zone IT Solutions Full time

    We are looking for a talented Data Architect to join our team at Zone IT Solutions. As a global consulting firm, we're seeking an expert in Big Data platforms, specifically Cloudera Hadoop.Key responsibilities:Design and implement efficient data pipelines using MapReduce, PySpark, Kafka, Sqoop, Impala, Hive, and other tools in the Hadoop ecosystem.Develop...

  • Data Architect

    3 weeks ago


    Sydney, New South Wales, Australia XPERT CONSULTING PTY LTD Full time

    Xpert Consulting Pty Ltd (trading as Novon) is a large Sydney based IT Services business. We operate a number of Technology centric brands including Novon , our IT Services business focused on delivering Data Transformation Services for clients large and small.Do you like to design strategic data solutions for complex enterprise environments? We have several...


  • Sydney, New South Wales, Australia Optiver US LLC Full time

    About OptiverOptiver is a global market maker founded in Amsterdam, with offices around the world. Established in 1986, we are a leading liquidity provider, with close to 2,000 employees united in our commitment to improve the market through competitive pricing, execution and risk management.We provide liquidity to financial markets using our own capital, at...


  • Sydney, New South Wales, Australia Virtusa Full time

    Job SummaryThe Enterprise Data Architect Lead is responsible for designing, developing, testing, deploying, and maintaining a data platform (storage, API, pipelines, catalogues) with various types of data - JSON, XML, Parquet, Avro, Delimited files.Key Responsibilities:Design and develop scalable data pipelines from multiple sources that includes batch data...


  • Sydney, New South Wales, Australia CreditorWatch Full time

    Job SummaryWe are seeking a highly skilled Senior Software Engineer to join our Data Engineering team. As a Senior Software Engineer, you will design and implement high-performance data products and services that power customer-facing applications.You will architect and build end-to-end solutions spanning data pipelines, storage layers, and APIs, with a...


  • Sydney, New South Wales, Australia Tideri Jobbörse Full time

    Job OverviewWe are looking for an experienced Azure Data Solutions Architect to lead our cloud engineering efforts. The ideal candidate will have expertise in Azure Data Lake, Databricks, Azure Data Factory (ADF), and Azure Synapse Analytics.Key Responsibilities:Design and implement data migration strategies from AWS to AzureDevelop and optimize data...


  • Sydney, New South Wales, Australia Macquarie Bank Limited Full time

    Company OverviewMacquarie is a global team that is passionate about accelerating the digital enterprise, connecting people and data, building platforms and applications, and designing tomorrow's technology solutions.We enable every aspect of Macquarie, for our people, our customers, and our communities. Our advantage is bringing together diverse people and...


  • Sydney, New South Wales, Australia FinXL Full time

    We are seeking a skilled Cloud-Based Data Specialist to join our team at FinXL IT. In this role, you will be responsible for designing and implementing data pipelines to support business growth and innovation.Main ResponsibilitiesDevelop and maintain ETL processes to ensure accurate and timely data integration.Collaborate with cross-functional teams to...

  • Data Architect

    2 days ago


    Sydney, New South Wales, Australia Luxoft Full time

    Luxoft is seeking a skilled Data Modeler to join their team and contribute to the modernization of their customer entitlements platform.The ideal candidate will have hands-on experience with SQL DB, NoSQL DB, and GraphDB, as well as strong stakeholder management skills and problem-solving abilities.This role involves designing, architecting, and implementing...


  • Sydney, New South Wales, Australia CareCone Group Full time

    CareCone Group is seeking a Senior Data Architect to lead our data architecture initiatives. As a Senior Data Architect, you will be responsible for designing and implementing scalable data architectures using AWS services such as Amazon S3, Amazon Redshift, and Amazon DynamoDB.You will work closely with our engineering team to ensure seamless integration of...


  • Sydney, New South Wales, Australia Optiver US LLC Full time

    The RoleAs a senior member of our data engineering team, you will be responsible for architecting and implementing large-scale data pipelines to support our research teams' needs. This includes collaborating with researchers to design and develop production-ready data pipelines, as well as working with cross-functional teams to align on data strategy and...


  • Sydney, New South Wales, Australia Bank of America Full time

    Role Overview:The Enterprise Cloud Platforms Team at Bank of America is seeking a skilled Site Reliability Engineer to join our global team. This role involves designing, building, and maintaining our next-generation cloud platforms. As an SRE, you will collaborate with engineers, architects, and teams to develop secure, robust, highly available, and...


  • Sydney, New South Wales, Australia NXTGIG Full time

    Data Warehouse Architect Role at NXTGIGNXTGIG is seeking a skilled professional to lead the design and implementation of our data warehousing solutions. In this role, you will be responsible for creating scalable and efficient data warehouse architectures that enable high-quality data analysis and reporting.Key Responsibilities:Design and implement robust...


  • Sydney, New South Wales, Australia PayTech Nexus Ltd Full time

    About the CompanyWe are PayTech Nexus Ltd, a leading provider of innovative payment solutions.About the Role:This is a 12-month contract opportunity with an ASAP start date.The role is based in Sydney, with a hybrid work arrangement.You will have the opportunity to work with one of the largest and most advanced Data Engineering teams in the...


  • Sydney, New South Wales, Australia Capgemini Full time

    **Job Overview**We are seeking an experienced Data Governance Architect to lead the design and delivery of master data management solutions using industry-leading tools such as Informatica MDM.The ideal candidate has deep expertise in data governance, data quality, and master/reference data management, with a proven track record of delivering at least two...


  • Sydney, New South Wales, Australia Big Red Group Full time

    Key Responsibilities- Architect and build scalable data pipelines: Design and implement efficient data workflows, ensuring data is readily available for analysis.- Integrate diverse data sources: Work with a variety of systems and datasets, bringing together information from across the business.- Ensure data quality and reliability: Implement data governance...

  • Senior Data Architect

    3 hours ago


    Sydney, New South Wales, Australia INGRITY Full time

    About the RoleAs a Senior Data Engineer at Ingrity, you will play a key role in designing, developing, and optimising data solutions on Google Cloud Platform (GCP). You will work closely with clients to implement scalable and reliable data pipelines, ensuring efficient data processing and analytics.Key ResponsibilitiesDesign, develop, and maintain scalable...

  • Cloud Data Architect

    3 hours ago


    Sydney, New South Wales, Australia Culture Amp Full time

    Job DescriptionWe're seeking a talented Data Engineer to join our Data Platform team. As a key member of this team, you'll be responsible for designing and building data pipelines that ingest data from different sources.You will ensure those pipelines are tested, performant, and observable. You'll also collaborate with stakeholders to ensure we're...


  • Sydney, New South Wales, Australia Pyramid Global Technologies Full time

    At Pyramid Global Technologies, we are seeking a highly skilled Senior Data Architect to lead our data engineering and warehousing initiatives. With 5+ years of experience in Data Engineering and Warehousing, you will be responsible for designing, implementing, and maintaining large-scale data systems.We are looking for an expert who has 3-5 years of...