Big Data Infrastructure Manager

2 weeks ago

Sydney, New South Wales, Australia beBeeInfrastructure Full time $150,000 - $200,000
Job Title: Big Data Infrastructure Manager

We are seeking an experienced Big Data Infrastructure Manager to join our team. In this role, you will be responsible for managing and maintaining our Big Data infrastructure, including Hive, Spark, Ranger, Oozie, HBase, NiFi.

Key Responsibilities:

  • Deploy, upgrade, configure, and manage Cloudera platforms;
  • Build and upgrade CDH distributions;
  • Manage services, backup, and disaster recovery;
  • Identify ineffective processes proactively and propose industry best practices;
  • Plan capacity, schedule, and evaluate clusters for business requirements;
  • Configure High Availability for all Hadoop components;
  • Troubleshoot and perform root cause analysis;
  • Support batch processing and real-time data pipelines such as Kafka;
  • Automate complex tasks using Hadoop ecosystem and document day-to-day activities;
  • Perform frequent platform health checks;
  • Work with product vendors to attain maximum performance;
  • Implement security around Hadoop clusters using Kerberos and Ranger;
  • Handle data between Hadoop ecosystems;
  • Experience with Linux shell scripting, Oozie, Crontab jobs, and DevOps tools like Jenkins, Bitbucket;
  • Configure Oozie jobs for control flow between different ecosystems;
  • Experience with databases like MySQL and Oracle;
  • Tableau Server administration and support activities;
  • Excellent communication and written skills;
  • Willingness to work in a support role with flexible hours.

Requirements:

  • 6-8 years of hands-on experience in managing Big Data technologies and Hadoop ecosystem;
  • Strong understanding of Hadoop architecture and ecosystem;
  • Experience with Cloudera platforms and CDH distributions;
  • Knowledge of Linux operating system and scripting languages;
  • Ability to work in a fast-paced environment and prioritize multiple tasks;
  • Excellent problem-solving and analytical skills;
  • Good communication and interpersonal skills;
  • Willingness to learn and adapt to new technologies and processes.

  • Big Data Expert

    2 weeks ago

    Sydney, New South Wales, Australia beBeeDataSpecialist Full time $75,000 - $85,000

    Job Opportunity:We are looking for a highly skilled and experienced big data specialist to join our team.The ideal candidate will have strong expertise in designing, developing, and maintaining robust, scalable, and high-performance big data solutions using modern big data tools and technologies.You will be responsible for developing data pipelines,...

  • Big Data Developer

    7 days ago

    Sydney, New South Wales, Australia AI Hustler Full time $104,000 - $130,878 per year

    We are seeking a skilled and experiencedBig Data Engineerto join our growing data team. You will be instrumental in building, maintaining, and optimizing the data pipelines and infrastructure that power our analytics and machine learning initiatives. This is an exciting opportunity to work with complex datasets and cutting-edge technologies in a dynamic and...

  • Big Data Specialist

    2 weeks ago

    Sydney, New South Wales, Australia beBeeData Full time $110,000 - $140,000

    Job Overview">We are seeking a Big Data Specialist to manage and maintain our company's Hadoop infrastructure. The successful candidate will be responsible for deploying, upgrading, configuring, and managing Cloudera platforms.">Key responsibilities include identifying inefficient processes, proposing industry best practices, and implementing High...

  • Big Data Expert

    2 weeks ago

    Sydney, New South Wales, Australia beBeeData Full time

    Job Description:We are seeking a highly skilled and experienced data professional to join our team. The ideal candidate will have strong expertise in building and maintaining robust, scalable, and high-performance data solutions using modern tools and technologies.You will be responsible for developing data pipelines, transforming large volumes of data, and...

  • Big Data Developer

    5 days ago

    Sydney, New South Wales, Australia Cognizant Full time

    OverviewPosition Summary: We are seeking a highly skilled and experienced Big Data Developer to join our engineering team. The ideal candidate will have strong expertise in building and maintaining robust, scalable, and high-performance big data solutions using modern big data tools and technologies. You will be responsible for developing data pipelines,...

  • Big Data Developer

    5 days ago

    Sydney, New South Wales, Australia Cognizant Full time

    OverviewPosition Summary: We are seeking a highly skilled and experienced Big Data Developer to join our engineering team. The ideal candidate will have strong expertise in building and maintaining robust, scalable, and high-performance big data solutions using modern big data tools and technologies. You will be responsible for developing data pipelines,...

  • Big Data Developer

    3 weeks ago

    Sydney, New South Wales, Australia Cognizant Full time

    Join to apply for the Big Data Developer role at Cognizant1 day ago Be among the first 25 applicantsJoin to apply for the Big Data Developer role at CognizantPosition SummaryWe are seeking a highly skilled and experienced Big Data Developer to join our engineering team. The ideal candidate will have strong expertise in building and maintaining robust,...

  • Big Data Developer

    2 weeks ago

    Sydney, New South Wales, Australia Cognizant Technology Solutions Full time US$75,000 - US$85,000 per year

    Position Summary:We are seeking a highly skilled and experienced Big Data Developer to join our engineering team. The ideal candidate will have strong expertise in building and maintaining robust, scalable, and high-performance big data solutions using modern big data tools and technologies. You will be responsible for developing data pipelines, transform...

  • Big Data Developer

    2 weeks ago

    Sydney, New South Wales, Australia Cognizant Full time

    Position SummaryWe are seeking a highly skilled and experienced Big Data Developer to join our engineering team. The ideal candidate will have strong expertise in building and maintaining robust, scalable, and high-performance big data solutions using modern big data tools and technologies. You will be responsible for developing data pipelines, transform...

  • Big Data Developer

    2 weeks ago

    Sydney, New South Wales, Australia Cognizant Full time

    Position SummaryWe are seeking a highly skilled and experienced Big Data Developer to join our engineering team. The ideal candidate will have strong expertise in building and maintaining robust, scalable, and high-performance big data solutions using modern big data tools and technologies. You will be responsible for developing data pipelines, transform...