Platform Data Engineer

1 month ago

Melbourne, Australia V2 Digital Full time

About the Role:

We are looking for a talented Platform & Data Engineer - Contractor to join our greenfield project, where you'll play a critical role in building and deploying a cutting-edge data platform in Azure Databricks. As part of a highly technical team, you’ll be responsible for designing, implementing, and managing the entire infrastructure pipeline, from provisioning cloud resources in Azure to deploying scalable data solutions on Databricks.

This role is ideal for an engineer who thrives at the intersection of platform engineering, data engineering, and cloud infrastructure. You’ll be working with modern Infrastructure as Code (IAC) practices, automating deployment workflows, and ensuring the robustness and scalability of the entire platform. Your expertise will be pivotal in ensuring the seamless operation of the platform, from cloud infrastructure provisioning to continuous deployment and monitoring on Databricks.

What Your Day Will Look Like:

  • Designing, deploying, and managing cloud infrastructure on Azure, ensuring it supports the data platform and applications at scale.
  • Implementing and managing Infrastructure as Code (IAC) using Terraform to automate provisioning and maintain cloud resources.
  • Supporting Databricks deployment, including the integration of data pipelines, notebooks, and cluster management.
  • Working with the broader data engineering team to optimise the data platform for scalability, performance, and security.
  • Collaborating with other engineering teams to ensure a smooth deployment process and end-to-end delivery of the solution.
  • Automating continuous integration and continuous deployment (CI/CD) pipelines for the data platform, ensuring efficient and reliable delivery of updates.
  • Ensuring platform security, monitoring, and alerting are in place to maintain a high level of uptime and performance.
  • Assisting with troubleshooting and optimising infrastructure and data pipelines to meet operational and performance goals.

Requirements:

  • 5+ years of experience in platform and data engineering, with hands-on experience building and managing cloud-based platforms.
  • Strong experience with Azure cloud infrastructure, including provisioning resources, managing services, and configuring network security.
  • Expertise in Infrastructure as Code (IaC) using Terraform or Bicep for automating cloud resource provisioning and infrastructure management.
  • Experience deploying, managing, and optimising Databricks environments, including clusters, notebooks, jobs, and integrations with data pipelines.
  • A solid understanding of CI/CD pipelines, automation, and deployment processes for cloud-based data platforms.
  • Strong technical background in data engineering, including building data pipelines, managing large-scale data systems, and working with cloud-native tools.
  • Familiarity with dbt and data modelling principles to work closely with the data engineering team in deploying solutions.
  • Excellent communication and problem-solving skills, with the ability to collaborate effectively across multiple teams.
  • Ability to quickly adapt to new technologies and solve complex infrastructure and deployment challenges.

Bonus Points:

  • Experience with machine learning pipelines or MLflow on Databricks.
  • Familiarity with containerisation technologies like Docker or Kubernetes for managing data workflows.
  • Experience with monitoring tools, FinOps and cost management for cloud infrastructure and Databricks environments.

If you are passionate about building robust platforms and data solutions at scale, automating infrastructure, and ensuring smooth deployment workflows in Azure Databricks, we’d love to hear from you

  • Platform Data Engineer

    1 month ago

    Melbourne, Australia V2 Digital Full time

    About the Role:We are looking for a talented Platform & Data Engineer to join our greenfield project, where you'll play a critical role in building and deploying a cutting-edge data platform in Azure Databricks. As part of a highly technical team, you’ll be responsible for designing, implementing, and managing the entire infrastructure pipeline, from...

  • Data Platform Engineer

    4 weeks ago

    Melbourne, Victoria, Australia Binance Full time

    Are you passionate about working with big data platforms?We are looking for a skilled Data Platform Engineer to join our team at Binance in Singapore.About the Role:The ideal candidate will have experience with developing big data platform tools, researching and applying big data platform component technology, and troubleshooting performance optimization,...

  • Data Insights Engineer

    4 weeks ago

    Melbourne, Victoria, Australia NTT DATA Full time

    About the RoleAt NTT DATA, we're seeking a seasoned Business Intelligence engineer to join our team. This role is responsible for transforming data into critical information and knowledge that can be used to make sound business decisions.Key Responsibilities:Work as part of a team to deliver modern data solutionsImplementation of modern data platformsElicit...

  • Senior Data Engineer Position

    1 month ago

    Melbourne, Victoria, Australia NTT DATA Full time

    Job Summary:We are seeking an experienced Senior Data Engineer to join our team at NTT DATA. The successful candidate will be responsible for designing, developing, and implementing cloud-based data platforms and associated technologies. This is an excellent opportunity to work with a leading provider of IT services and solutions.Key Responsibilities:Design...

  • Snowflake Data Platform Engineer

    1 month ago

    Melbourne, Victoria, Australia Zone IT Solutions Full time

    We are seeking a skilled Snowflake Data Platform Engineer to join our team at Zone IT Solutions. In this role, you will be responsible for managing and optimizing our Snowflake data platform, ensuring seamless integration and deployment across our data pipelines.

  • Data Engineering Lead for Cloud Data Platforms

    3 weeks ago

    Melbourne, Victoria, Australia Charterhouse Full time

    A challenging opportunity has arisen for a skilled Data Architect Manager to join our client, a leading telecom brand with strong year-over-year growth.As part of their team, you will play a key role in designing and implementing cutting-edge enterprise data platforms on Microsoft Azure, fostering a culture of excellence in data engineering.Data Architecture...

  • Platform Data Engineer

    1 month ago

    Melbourne, Victoria, Australia V2 Digital Full time

    About the Role:We are seeking an experienced Platform and Data Engineer - Contractor to join our greenfield project. This is a unique opportunity to build and deploy a cutting-edge data platform in Azure Databricks. As part of a highly technical team, you will be responsible for designing, implementing, and managing the entire infrastructure pipeline, from...

  • Senior Azure Data Engineer

    1 month ago

    Melbourne, Victoria, Australia NTT DATA Full time

    Job DescriptionWe are seeking a highly skilled Senior Azure Data Engineer to join our team at NTT DATA. As a key member of our analytics practice, you will play a crucial role in developing innovative solutions leveraging cloud-based data platforms and associated technologies.The ideal candidate will have extensive experience in designing and implementing...

  • Data Engineering Leader

    4 weeks ago

    Melbourne, Victoria, Australia Charterhouse Full time

    Charterhouse is seeking an experienced Data Architect Manager to lead the development of a cutting-edge enterprise data platform on the Microsoft Azure platform. About the Role:We are looking for a highly skilled professional with expertise in modern data architecture, lambda architecture, and data lake implementation. The successful candidate will have...

  • Data Engineering Leader for Cloud Data Platforms

    2 weeks ago

    Melbourne, Victoria, Australia Energy Australia Full time

    About this roleThe Senior Data Engineer will lead the development of data solutions to enable effective and efficient value realisation from EnergyAustralia's data. This role is responsible for designing, building, and deploying data pipelines in the AWS analytics ecosystem.This position requires strong experience in building data engineering pipelines and...

  • Data Platform Architect

    3 weeks ago

    Melbourne, Victoria, Australia John Lewis Partnership Full time

    OverviewThe John Lewis Partnership is a leading UK retailer that operates two of the busiest websites in the country. Our Engineering practice includes over 60 teams working on various projects, including cloud platforms, mobile apps, and e-commerce functionality.We are seeking a skilled Senior Data Engineer to join our Data Platform team at either our...

  • Senior Azure Data Engineer

    1 month ago

    Melbourne, Australia NTT DATA Full time

    Job Description NTT DATA is heavily investing globally in providing state of the art analytical solutions to its clients. Our capability has been focused locally, regionally in APAC and globally by forming strategic partnerships including Microsoft Azure, SAP Analytics, Snowflake, Matillion and many more, leveraging the strength of our cloud and network...

  • Senior Azure Data Engineer

    3 weeks ago

    Melbourne, Australia NTT DATA Full time

    Job Description NTT DATA is heavily investing globally in providing state of the art analytical solutions to its clients. Our capability has been focused locally, regionally in APAC and globally by forming strategic partnerships including Microsoft Azure, SAP Analytics, Snowflake, Matillion and many more, leveraging the strength of our cloud and network...

  • Data Platform Strategist

    3 weeks ago

    Melbourne, Victoria, Australia Opus Recruitment Solutions Full time

    Data Platform StrategistLocation: Melbourne, AustraliaJob Type: PermanentSalary: $130,000 - $150,000 per annumAbout the RoleWe are seeking a seasoned Data Platform Lead to spearhead the development and implementation of a robust data platform with our client in the wellness and fitness services space.This is a unique opportunity to shape the future of our...

  • Data Platform Strategist

    3 weeks ago

    Melbourne, Victoria, Australia Charterhouse Full time

    Seeking a Data Architect Manager to lead the development of a cutting-edge enterprise data platform on Microsoft Azure.About Our ClientNationally recognised telecom brand with amazing year-on-year growth, offering a great place to work and build your career.About the RoleFoster a culture of excellence in data engineering by implementing industry-leading...

  • Data Science and Analytics Strategist

    2 weeks ago

    Melbourne, Victoria, Australia NTT DATA Full time

    About the RoleNTT DATA is a global leader in business and technology services, with a commitment to innovation, optimization, and transformation. As a seasoned Business Intelligence Engineer, you will play a key role in delivering modern data solutions that drive business value.Job SummaryThis is a hybrid working opportunity where you will work closely with...

  • Senior Data Engineer

    1 month ago

    Melbourne, Victoria, Australia John Lewis Partnership Full time

    Key ResponsibilitiesThe Senior Data Engineer will design, implement, and support secure cloud-based data platforms, ensuring high-quality delivery outcomes across our data landscape.This role will participate in shaping and delivering data solutions to meet the needs of a wide variety of stakeholders from across the Partnership.As a Senior Data Engineer, you...

  • Cloud Engineering Leader

    3 weeks ago

    Melbourne, Victoria, Australia Capgemini Full time

    Capgemini is a global leader in consulting, digital transformation, technology, and engineering services. As a Cloud Engineering Leader on our Google Cloud Data Platform team, you will play a key role in shaping the future of our clients' technology infrastructure.We are seeking an experienced engineer with a strong background in cloud technologies to lead...

  • Azure Platform Engineer

    2 months ago

    Melbourne, Australia Softtest pays pty ltd Full time

    Australian Citizens with baseline clearance. able to obtain Negative Vetting Level 1. Essential criteria: 1. Experience designing, building and maintaining cloud hosted platforms and services (Infrastructure, Platform, and Software as a Service)- 25% 2. Experience developing, implementing, and supporting cloud infrastructure using an infrastructure as code...

  • Data Transformation Specialist

    2 weeks ago

    Melbourne, Victoria, Australia NTT DATA Full time

    Make an impact at NTT DATAWe are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society.Our workplace embraces diversity and inclusion – it's a place where you can grow, belong and thrive.About the Role:The Business Intelligence engineer is a seasoned subject matter expert, tasked with...