Data Engineer

1 day ago

Melbourne, Victoria, Australia beBeeDataEngineer Full time $90,000 - $120,000
About This Role

We're seeking a skilled professional to drive our data pipeline efficiency and scalability.

Job Summary

This role focuses on optimizing data workflows, implementing cloud infrastructure, and ensuring seamless data delivery across the organization.

Key Responsibilities
  • Data Pipeline Automation: Design, implement, and maintain CI/CD pipelines for automated data workflow deployment.
  • Cloud Infrastructure Management: Develop infrastructure-as-code (IaC) templates for provisioning cloud-based data environments/resources.
  • ETL Workflow Automation: Automate ETL/ELT workflow management using orchestration tools like Apache Airflow, Prefect, or AWS Step Functions.
  • Pipeline Optimization: Optimize data ingestion, transformation, and processing pipelines for scalability and efficiency.
  • Data Security: Manage data infrastructure in AWS, Azure, or GCP, ensuring high availability and security.
  • Access Control: Implement role-based access control (RBAC), encryption, and data security best practices.
  • Compliance: Collaborate with Security teams to ensure compliance with industry regulations.
  • Cost Optimization: Monitor cloud costs and optimize resource usage for storage, compute, and networking.
  • Monitoring and Incident Response: Develop real-time monitoring dashboards and alerts for data pipeline performance and failures.
  • Observability Tools: Use observability tools such as Datadog, Prometheus, Grafana, CloudWatch etc.to track latency, errors, and throughput.
  • Incident Response Workflows: Establish incident response workflows and automated rollback strategies for failures.
  • Collaboration: Work with Site Reliability Engineers (SRE) and DevOps teams to maintain high system uptime and resilience.

  • Full Stack Engineer

    3 weeks ago

    Melbourne, Victoria, Australia Champion Data Full time

    Get AI-powered advice on this job and more exclusive features.At Champion Data, our dedicated team are united by sport and driven by data. We strive to revolutionise and transform sport data globally by leveraging our cutting-edge technology and expert team, ensuring we deliver the best possible outcomes for our partners and clients.About the jobChampion...

  • Full Stack Engineer

    3 weeks ago

    Melbourne, Victoria, Australia Champion Data Full time

    Get AI-powered advice on this job and more exclusive features.At Champion Data, our dedicated team are united by sport and driven by data. We strive to revolutionise and transform sport data globally by leveraging our cutting-edge technology and expert team, ensuring we deliver the best possible outcomes for our partners and clients.About the jobChampion...

  • Software Engineer

    3 days ago

    Melbourne, Victoria, Australia Champion Data Full time

    At Champion Data, our dedicated team are united by sport and driven by data. We strive to revolutionise and transform sport data globally by leveraging our cutting-edge technology and expert team, ensuring we deliver the best possible outcomes for our partners and clients. About the job Champion Data has an exciting opportunity for an experienced and...

  • Software Engineer

    3 days ago

    Melbourne, Victoria, Australia Champion Data Full time

    At Champion Data, our dedicated team are united by sport and driven by data. We strive to revolutionise and transform sport data globally by leveraging our cutting-edge technology and expert team, ensuring we deliver the best possible outcomes for our partners and clients.About the jobChampion Data has an exciting opportunity for an experienced and dedicated...

  • Data Engineer

    1 day ago

    Melbourne, Victoria, Australia Career Success Australia Full time

    Data Engineers are in high demand, so an Internship in this particular stream of data science is a terrific way to start your career. Through this Data Engineer Internship, you will learn a wide range of skills in data extraction, manipulation, artificial intelligence, and database development to help you pursue a career in Data Engineering.As a Data...

  • MS Engineer

    3 weeks ago

    Melbourne, Victoria, Australia NTT DATA, Inc. Full time

    Join to apply for the MS Engineer (L1): Network role at NTT DATA, Inc.Join to apply for the MS Engineer (L1): Network role at NTT DATA, Inc.Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society....

  • MS Engineer

    3 weeks ago

    Melbourne, Victoria, Australia NTT DATA, Inc. Full time

    Join to apply for the MS Engineer (L1): Network role at NTT DATA, Inc.Join to apply for the MS Engineer (L1): Network role at NTT DATA, Inc.Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society....

  • Data Engineer

    3 weeks ago

    Melbourne, Victoria, Australia Vertali APAC Pty Ltd Full time

    Our Client is looking foran experienced Data Engineer for an initial 6-month contract, with likely extensions based on your fit for the role and the project timeline.Reporting to the Head of Data and Analytics, the Data Engineer is responsible for the development of data models, data pipelines, test automation solutions and mapping the business requirements...

  • Data Engineer

    3 weeks ago

    Melbourne, Victoria, Australia triniti Full time

    This range is provided by triniti. Your actual pay will be based on your skills and experience — talk with your recruiter to learn more.Base pay rangeA$120,000.00/yr - A$125,000.00/yrFlexible hours | 2 days in CBD office | Supportive, friendly team | $120-125k + Super (per annum)About Our ClientOur client is a well-established, innovative business with a...

  • Data Engineer

    3 weeks ago

    Melbourne, Victoria, Australia triniti Full time

    This range is provided by triniti. Your actual pay will be based on your skills and experience — talk with your recruiter to learn more.Base pay rangeA$120,000.00/yr - A$125,000.00/yrFlexible hours | 2 days in CBD office | Supportive, friendly team | $120-125k + Super (per annum)About Our ClientOur client is a well-established, innovative business with a...