Data Pipeline Architect

2 days ago

Brisbane, Queensland, Australia beBeeDataAutomation Full time $100,000 - $150,000
Senior Data Automation Specialist

About the Role:

We are seeking a highly skilled Senior Data Automation Specialist to lead our data operations team. This role is responsible for designing, implementing, and maintaining automated data workflows and infrastructure to ensure efficient, scalable, and reliable data delivery.

Key Responsibilities:

  • Design and implement CI/CD pipelines for automated data workflow deployment using industry-leading tools like Apache Airflow and Prefect.
  • Develop infrastructure-as-code (IaC) templates using Terraform for provisioning cloud-based data environments/resources.
  • Automate ETL/ELT workflow management using orchestration tools and optimize data ingestion, transformation, and processing pipelines for scalability and efficiency.
  • Manage data infrastructure in AWS, Azure, or GCP, ensuring high availability and security through best practices and compliance with industry regulations.
  • Implement role-based access control (RBAC), encryption, and data security measures to protect sensitive data.
  • Collaborate with Security teams to ensure compliance with industry regulations and maintain secure data handling practices.
  • Develop real-time monitoring dashboards and alerts for data pipeline performance and failures, utilizing observability tools like Datadog and Prometheus.
  • Establish incident response workflows and automated rollback strategies for failures to minimize downtime and ensure business continuity.

Requirements:

  • Bachelor's or Master's degree in Computer Science, Data Engineering, Cloud Computing, or a related field/relevant experience.
  • 5+ years of experience in DevOps, Data Engineering, or Cloud Infrastructure roles, with expertise in CI/CD tools and orchestration technologies.
  • Proficiency in Python, Bash, or PowerShell for automation and scripting, as well as SQL and NoSQL databases like PostgreSQL and Snowflake.
  • Strong understanding of ETL/ELT pipelines, workflow scheduling, and automation, with hands-on experience with cloud data platforms and observability tools.

  • Enterprise Architect

    5 days ago

    Brisbane, Queensland, Australia Data#3 Full time

    Data#3 Brisbane City, Queensland, AustraliaJoin or sign in to find your next jobJoin to apply for the Enterprise Architect role at Data#3Data#3 Brisbane City, Queensland, Australia2 days ago Be among the first 25 applicantsJoin to apply for the Enterprise Architect role at Data#3Direct message the job poster from Data#34-month contract | Brisbane CBD | Qld...

  • Enterprise Architect

    4 days ago

    Brisbane, Queensland, Australia Data#3 Full time

    Data#3 Brisbane City, Queensland, AustraliaJoin or sign in to find your next jobJoin to apply for the Enterprise Architect role at Data#3Data#3 Brisbane City, Queensland, Australia2 days ago Be among the first 25 applicantsJoin to apply for the Enterprise Architect role at Data#3Direct message the job poster from Data#34-month contract | Brisbane CBD | Qld...

  • Data Architect

    2 weeks ago

    Brisbane, Queensland, Australia Allens Full time

    Join to apply for the Data Architect role at Allens2 days ago Be among the first 25 applicants Join to apply for the Data Architect role at Allens Get AI-powered advice on this job and more exclusive features.Your role At Allens, our business teams are also specialists in their field, using their unique knowledge and expertise to solve critical client and...

  • Data Engineer

    3 days ago

    Brisbane, Queensland, Australia beBeeSolution Full time $120,000 - $137,500

    Key Role: Solution ArchitectJob Summary:We are seeking an experienced and innovative Solution Architect to join our team. The successful candidate will have demonstrated experience in designing, developing and implementing complex ICT solutions and efficient data pipelines.The ideal candidate will be responsible for providing technical advice, support and...

  • Solution Architect – Application Platform

    2 days ago

    Brisbane, Queensland, Australia Data#3 Full time

    Solution Architect – Application PlatformData#3 Brisbane City, Queensland, AustraliaJoin or sign in to find your next jobJoin to apply for the Solution Architect – Application Platform role at Data#3Solution Architect – Application PlatformData#3 Brisbane City, Queensland, Australia1 day ago Be among the first 25 applicantsJoin to apply for the...

  • Leading Data Architect

    2 days ago

    Brisbane, Queensland, Australia beBeeDataEngineering Full time $150,000 - $200,000

    Role SummaryWe are seeking an experienced Senior Data Engineer to deliver an evolving central data ecosystem, playing a significant part in the delivery of a shared cloud data platform.Key ResponsibilitiesDevelop and deploy technical functionality against the shared cloud data platform and existing warehouse platforms.Create and maintain optimal data...

  • Visionary Data Architect

    1 day ago

    Brisbane, Queensland, Australia beBeeDataEngineer Full time $140,000 - $160,000

    Job DescriptionWe are seeking an accomplished Senior Data Engineer to spearhead the development of a world-class data platform. This high-profile role offers an exceptional opportunity for growth, innovation and autonomy in a fully remote environment.This is a critical position where you will play a pivotal role in shaping the technical direction of our core...

  • Data Architect

    4 days ago

    Brisbane, Queensland, Australia Healthy Land & Water Full time

    Join to apply for the Data Architect role at Healthy Land & Water3 days ago Be among the first 25 applicantsJoin to apply for the Data Architect role at Healthy Land & WaterThis range is provided by Healthy Land & Water. Your actual pay will be based on your skills and experience — talk with your recruiter to learn more.Base pay rangeA$120,000.00/yr -...

  • Chief Data Architect

    5 days ago

    Brisbane, Queensland, Australia beBeeData Full time

    Job OpportunityWe are seeking a highly skilled Senior Data Engineer to join our dynamic team in Brisbane. This is a unique opportunity to deliver an evolving central data ecosystem and play a significant part in the delivery of a shared cloud data platform.Key Responsibilities:Deliver technical functionality developed against the shared cloud data platform...

  • Data Architect

    3 weeks ago

    Brisbane, Queensland, Australia Latitude IT Full time

    A leading Australian company is undergoing a large-scale digital transformation and is looking for a Data & Analytics Architect to play a key role in shaping its data strategy. This is a fantastic opportunity to work on complex projects, drive innovation, and make a real impact.About the RoleAs a Data & Analytics Architect, you will be responsible for...