Senior Kafka DataOps Engineer

2 days ago

Greater Sydney Area, Australia Excolo. Full time $120,000 - $180,000 per year

We're working with a globally recognised financial services organisation that's building a distributed, scalable and reliable data streaming platform. They're looking for a Senior Kafka DataOps Engineer to design, build and manage high-performance streaming infrastructure that underpins mission-critical systems.

You will provision, configure and maintain messaging, integration and data distribution environments, primarily around Kafka, across hybrid infrastructures that combine on-premises servers and AWS cloud services. This is a greenfield enterprise Kafka deployment, offering the opportunity to shape the platform's foundations and automation practices.

Key Responsibilities

  • Configure, deploy and manage Kafka clusters across hybrid environments (on-prem + AWS)
  • Build and maintain CI/CD pipelines for Kafka components and infrastructure automation
  • Implement observability and monitoring solutions (Grafana, ELK, OpenTelemetry) to ensure operational health and visibility
  • Use Python or PySpark for automation, monitoring and operational tooling (not for data engineering pipelines)
  • Manage authentication, DR and cluster configuration to ensure resilience and security
  • Create and maintain technical documentation and Standard Operating Procedures for Kafka operations
  • Collaborate with platform, data and cloud teams to support, enhance and maintain Kafka ecosystems
  • Troubleshoot and resolve incidents, conduct root-cause analysis and participate in on-call rotations

Required Experience

  • Proven experience deploying and managing Kafka clusters in production environments
  • Deep knowledge of AWS and infrastructure-as-code tools (Terraform, CloudFormation etc.)
  • Strong background in platform engineering and DevOps practices
  • Skilled with UNIX/Linux systems and shell scripting
  • Experience implementing monitoring and observability frameworks (Grafana, ELK, OpenTelemetry)
  • Proficiency in Python (or PySpark) for automation, monitoring and operational tooling
  • Strong troubleshooting, performance tuning and analytical skills

Preferred Skills

  • Exposure to multi-cloud or hybrid architectures
  • Knowledge of ServiceNow Observability, Telegraf or InfluxDB
  • Familiarity with API Gateway tools (e.g., Apigee)
  • Experience in financial services or trading environments
  • Understanding of Agile delivery and automated incident/change management processes

Why Apply

This is a chance to join a greenfield Kafka platform build within a global institution where you will define streaming architecture, automation and observability standards from the ground up. If you thrive in high-availability environments and enjoy bridging infrastructure and data operations, this role offers immediate and visible technical impact.

  • Senior Kafka Streaming Engineer

    2 days ago

    Greater Sydney Area, Australia Excolo. Full time $120,000 - $180,000 per year

    We're partnering with a leading financial services organisation seeking an experienced Senior Kafka Streaming Engineer to design, implement and support real-time data streaming solutions across a hybrid environment. You'll play a key role in building scalable pipelines using Kafka, SQL Server CDC, Terraform and Databricks, integrating on-premises and AWS...

  • Senior Data Engineer

    2 days ago

    Council of the City of Sydney, Australia SONITEC Full time

    About the role The company is seeking a Senior Data Engineer with focus on AI/ML DataOps pipelines to join the Sensor Fusion AI Team in Sydney CBD. Responsibilities, Duties and Expectations - Design, develop, maintain and/or refine internal software tools used for processing data, workflow automation, benchmarking and testing the a Sensor AI engine -...

  • Kafka Engineer

    2 weeks ago

    Sydney, New South Wales, Australia Optiver Full time $120,000 - $180,000 per year

    Who We AreOptiver is a global market maker founded in Amsterdam, with offices in London, Chicago, Austin, New York, Sydney, Shanghai, Hong Kong, Singapore, Taipei and Mumbai. Established in 1986, today we are a leading liquidity provider, with close to 2,000 employees in offices around the world, united in our commitment to improve the market through...

  • Senior Integration Engineer

    6 days ago

    Greater Sydney Area, Australia Excolo. Full time $120,000 - $180,000 per year

    We're looking for a Senior Integration Engineer to design and deliver secure scalable integrations across our fintech platforms. You'll work primarily with Azure integration services and a leading iPaaS platform (MuleSoft, Boomi or Workato) to build APIs and workflows that keep data flowing across core systems.What you'll doBuild and maintain APIs and...

  • Senior Python Backend Engineer

    2 days ago

    Greater Sydney Area, Australia Themelia Full time $120,000 - $180,000 per year

    Senior Python Backend Engineer (Performance & Parallel Execution)Company DescriptionThemelia is a crypto index and analytics platform entering a performance and scaling phase. We're seeking a senior Python engineer with expertise in computational optimisation and concurrent systems to improve index calculation performance and parallelise exchange trade...

  • Senior Software Engineer

    2 days ago

    Greater Perth Area, Australia Synechron Full time $120,000 - $180,000 per year

    We are seeking two experienced Senior Software Engineers to join the client's Tech team. In this role, you will work on developing and maintaining enterprise-grade solutions, utilizing a broad spectrum of tools and technologies. Your expertise will contribute to the design, development, and deployment of scalable and secure applications, supporting their...

  • Power Query and Kafka Consultant

    1 week ago

    Sydney, Australia Delivery Centric Technologies Full time

    We are currently looking for **Power Query AND Kafka Consultant **who are based out of **Sydney**, to help our clients deliver solutions. **Role Title: Power Query AND Kafka Consultant** **Location: Sydney** **Employment Type**:Full Time** **Responsibilities and Duties** We are seeking a skilled **Power BI Developer with expertise in integrating Power...

  • Senior Platform Engineer

    2 weeks ago

    Greater Sydney Area, Australia Cognizant Full time $100,000 per year

    What makes Cognizant a unique place to work? The combination of rapid growth and an international and innovative environment This is creating many opportunities for people like YOU — people with an entrepreneurial spirit who want to make a difference in this world.At Cognizant, together with your colleagues from all around the world, you will collaborate...

  • Senior Data Engineer

    2 days ago

    Greater Sydney Area, Australia Viva Leisure Full time $120,000 - $180,000 per year

    Company DescriptionAustralian-owned, Award-winning, and ASX listed, Viva Leisure are Australia's second-largest and fastest-growing operator of health and fitness experiences and facilities. Viva Leisure operates in all segments of health and fitness and our range of offerings includes big-box health clubs, express gyms, high-intensity group training...

  • Senior Data Engineer

    1 week ago

    Sydney, New South Wales, Australia SONITEC Full time $120,000 - $180,000 per year

    About the roleOur Client is seeking a Senior Data Engineer to join our team in Sydney, Australia. Reporting to the SensorFusionAI (SFAI) Team Lead, you will develop the core data pipelines and tools that turn real-world, multi-sensor data into actionable AI intelligence — powering systems that operate in the world's most complex and high-stakes...