Data Pipeline Automation Specialist

12 hours ago

Brisbane, Queensland, Australia beBeeDataOps Full time $180,000 - $200,000
Job Summary

The Data Engineer's role involves developing and implementing data pipeline automation solutions to ensure efficient, scalable, and reliable data workflows.

Key Responsibilities:
  • Design and implement CI/CD pipelines for automated data workflow deployment using DevOps methodologies.
  • Develop infrastructure-as-code (IaC) templates for provisioning cloud-based data environments/resources.
  • Automate Extract, Transform, Load/Extract, Load, Transform (ETL/ELT) workflow management using orchestration tools like Apache Airflow, Prefect, or AWS Step Functions.
  • Optimize data ingestion, transformation, and processing pipelines for scalability and efficiency.
Required Skills and Qualifications:
  • Proficiency in programming languages such as Python, Java, or C++.
  • Experience with containerization using Docker and Orchestration using Kubernetes.
  • Familiarity with cloud platforms including AWS, Azure, or GCP.
  • Knowledge of data warehousing and business intelligence tools.
Benefits:
  • Competitive salary and benefits package.
  • Opportunities for career growth and professional development.
  • Collaborative and dynamic work environment.
Other Information:
  • Ability to work collaboratively with cross-functional teams.
  • Strong problem-solving and analytical skills.
  • Excellent communication and interpersonal skills.
Cloud Infrastructure & Security:
  • Manage data infrastructure in cloud platforms ensuring high availability and security.
  • Implement role-based access control (RBAC), encryption, and data security best practices.
  • Collaborate with Security teams to ensure compliance with industry regulations.
  • Monitor cloud costs and optimize resource usage for storage, compute, and networking.
Monitoring, Observability & Incident Response:
  • Develop real-time monitoring dashboards and alerts for data pipeline performance and failures.
  • Use observability tools such as Datadog, Prometheus, Grafana, CloudWatch etc. to track latency, errors, and throughput.
  • Establish incident response workflows and automated rollback strategies for failures.
  • Work with Site Reliability Engineers (SRE) and DevOps teams to maintain high system uptime and resilience.
Keyword: Dataops Specialist

  • Infrastructure Automation Specialist

    2 days ago

    Brisbane, Queensland, Australia beBeeAutomation Full time $105,388 - $142,949

    Job DescriptionWe are seeking a highly skilled Infrastructure Automation Specialist to join our team. In this role, you will play a key part in designing, maintaining, and optimizing the infrastructure, automation pipelines, and tooling that support our engineering teams.Key ResponsibilitiesYou will work closely with software engineers and IT to improve...

  • Senior Data Pipeline Architect

    2 weeks ago

    Brisbane, Queensland, Australia beBeeDataPipeline Full time $120,000 - $180,000

    Data Pipeline ExpertWe are seeking a highly skilled Data Pipeline Expert to join our team. As a key member of our data engineering group, you will be responsible for designing, implementing, and maintaining scalable and efficient data pipelines that support our business operations.

  • Automation Quality Assurance Specialist

    2 days ago

    Brisbane, Queensland, Australia beBeeQuality Full time $90,000 - $125,000

    Job OpportunityWe are seeking a skilled Automation Quality Assurance Specialist to play a pivotal role in delivering high-quality technology solutions.This specialist will lead automation testing activities across complex systems, working closely with developers, product managers, and stakeholders to ensure applications meet business requirements.Key...

  • Data Science Manager, Digital

    2 weeks ago

    Brisbane, Queensland, Australia Rockwell Automation Full time

    Join to apply for the Data Science Manager, Digital role at Rockwell Automation5 days ago Be among the first 25 applicantsJoin to apply for the Data Science Manager, Digital role at Rockwell AutomationRockwell Automation is a global technology leader focused on helping the world's manufacturers be more productive, sustainable, and agile. With more than...

  • Pipeline Engineering Specialist

    12 hours ago

    Brisbane, Queensland, Australia beBeePiping Full time $120,000 - $170,000

    Job Title: Pipeline Engineering SpecialistThe role of a pipeline engineering specialist is to provide technical expertise in piping design and engineering.This position involves working on diverse projects across the Energy, Chemicals, and Mineral Processing industries.

  • Senior Test Automation Expert

    2 weeks ago

    Brisbane, Queensland, Australia beBeeAutomation Full time $100,000 - $150,000

    Automation SpecialistThis role entails maintaining the consistency and usability of online systems through ongoing automated testing. It also involves ensuring that application changes meet both specification and customer journey requirements.The Automation Specialist will be responsible for developing and maintaining automation frameworks, executing...

  • Senior Automation Tester

    10 hours ago

    Brisbane, Queensland, Australia beBeeAutomationtestanalyst Full time $120,000 - $140,000

    Automation Test Analyst RoleTalent International is seeking an experienced Automation Test Analyst (QA Engineer) to join a global leader in manufacturing and supply of construction and mining equipment. This role is based in the Brisbane CBD on a 6-month contract, with likely extensions and possibility to go permanent.ResponsibilitiesAct as quality assurance...

  • Senior Automation Specialist

    4 days ago

    Brisbane, Queensland, Australia beBeeAutomation Full time

    Automation Tester PositionWe are seeking a skilled Automation Tester to join our team. As an Automation Tester, you will be responsible for designing, implementing, and maintaining automated test suites for both frontend and backend components.Key Responsibilities:Design and Implement Automated Test Suites: Ensure alignment with business requirements and...

  • Senior Automation Specialist

    2 weeks ago

    Brisbane, Queensland, Australia beBeeDevOps Full time $120,000 - $165,000

    In today’s fast-paced digital landscape, businesses require innovative solutions to stay ahead of the competition. As a DevOps Engineer at CrowdStrike, you will play a vital role in building and deploying automation and configuration to drive massive scale.We are looking for a highly-technical, hands-on engineer with experience using configuration...

  • OT & Cybersecurity Branch Manager

    3 weeks ago

    Brisbane, Queensland, Australia Alliance Automation Full time

    Excellent salary and employee benefits availableWe are seeking a skilled and strategic OT & Cybersecurity Branch Manager to lead our operations across key industries including mining, energy, water and wastewater, infrastructure, manufacturing, and defence. In this leadership role, you will oversee a team of specialists delivering secure, resilient, and...