Chief Data Infrastructure Architect

As a Lead Data DevOps professional, you will play a pivotal role in designing and implementing high-availability, scalable, and secure data pipelines and platforms.

Key responsibilities include ensuring efficient Continuous Integration/Continuous Deployment (CI/CD) processes, infrastructure automation, and monitoring to support the organization's data ecosystem.

• Data Platform & Pipeline Automation:
  • Design, implement, and maintain CI/CD pipelines for data engineering workflows.
  • Automate data pipeline deployments, monitoring, and scaling to ensure reliability and performance.
  • Develop and enforce Infrastructure-as-Code (IaC) practices using tools like Terraform, AWS CloudFormation, or Azure Bicep.
  • Collaborate with Data Engineers and IT teams to optimize Extract, Transform, Load/Extract, Load Transform (ETL/ELT) processes and cloud data platforms.

• Cloud Infrastructure & Security:
  • Oversee data infrastructure in cloud environments (AWS, Azure, GCP), ensuring high availability and security.
  • Implement Role-Based Access Control (RBAC), encryption, and network security best practices within data environments.
  • Collaborate with Security and Compliance teams to ensure regulatory compliance.
  • Optimize cloud storage, compute, and networking costs while maintaining performance.

• Monitoring, Observability & Incident Response:
  • Implement monitoring solutions for real-time alerting, anomaly detection, and performance optimization.
  • Utilize logging and observability tools such as Prometheus, Grafana, Datadog, Splunk, or CloudWatch.
  • Establish incident response processes, including automated rollback strategies and disaster recovery plans.
  • Work with Site Reliability Engineering (SRE) teams to ensure high system availability and minimal downtime.

• Data Ops & DevOps Best Practices:
  • Advocate for DataOps methodologies to improve collaboration between Data Engineering, IT, and Analytics teams.
  • Implement version control, automated testing, and release management for data pipelines.
  • Continuously improve CI/CD workflows to enhance deployment frequency and reduce failures.
  • Enforce best practices for data governance, lineage tracking, and auditability.

• Collaboration & Stakeholder Engagement:
  • Work closely with Data Engineers, Analytics, Security, and IT Operations to streamline data infrastructure.
  • Act as the subject matter expert (SME) on Data DevOps, providing training and mentorship to team members.
  • Engage with business leaders and data users to align infrastructure improvements with business goals.

Required Skills and Qualifications:

• 3-5+ years of experience in DevOps, Data Engineering, or Cloud Infrastructure roles.
• Hands-on experience with CI/CD tools (e.g., GitHub Actions, Jenkins, Azure DevOps).
• Proficiency in containerization and orchestration (Docker, Kubernetes, ECS, EKS, AKS) as plus.
• Experience implementing IaC using Terraform.
• Expertise in cloud data platforms (AWS, Azure, GCP) with services like AWS Lambda, S3, RDS, Redshift, Glue, EMR, Databricks, BigQuery, etc.
• Strong understanding of cloud networking, security policies, and identity management (IAM, VPC, KMS, Secrets Manager, etc.).
• Experience setting up monitoring and alerting using tools like Prometheus, Grafana, CloudWatch, Datadog, etc.
• Ability to troubleshoot performance issues and optimize system reliability.
• Strong experience in Python, Bash, or PowerShell for automation and scripting.
• Ability to write automation scripts for infrastructure provisioning, deployments, and data pipeline management.
• Ability to lead a small team and mentor junior engineers and promote DevOps best practices across teams.
• Strong communication skills to work with both technical and non-technical stakeholders.

Benefits:

• Career Opportunities
• A competitive salary plus generous uncapped commission scheme
• The opportunity to work with a well-known brand
• Work with a team of highly energized, motivated, and driven professionals
• 3 'recharge' days per year
• A free coffee each day
• An impressive Employee Benefits Program

What You Need To Do:

• Bachelor's or Master's degree in Computer Science, Data Engineering, Cloud Computing, or a related field/relevant experience.