Data Engineer- Hadoop Big Data Platform

4 weeks ago


Sydney, Australia Virtusa Full time

Data Engineer- Hadoop Big Data Platform - 104329 Description Implementation experience in BigData Platform ; preferably in Cloudera Hadoop platform Minimum 2 years of Development experience using Hadoop eco system tools & utilities: MapReduce, Spark, Kafka, Sqoop, Impala, Hive etc Ability to work independently and also contribute to overall architecture and design Experience in writing Shell scripts in Linux Platform Knowledge on API management concepts and design Developed Apache Spark applications and comfortable developing in Python. (Preferred) Performed debugging and performance tuning of Spark applications. Primary Location Sydney, New South Wales, Australia Job Type Experienced Years of Experience 5 Qualification Implementation experience in BigData Platform ; preferably in Cloudera Hadoop platform Minimum 2 years of Development experience using Hadoop eco system tools & utilities: MapReduce, Spark, Kafka, Sqoop, Impala, Hive etc Ability to work independently and also contribute to overall architecture and design Experience in writing Shell scripts in Linux Platform Knowledge on API management concepts and design Developed Apache Spark applications and comfortable developing in Python. (Preferred) Performed debugging and performance tuning of Spark applications. Travel No


  • Data Engineer

    4 weeks ago


    Sydney, Australia Virtusa Full time

    Data Engineer - Hadoop Bigdata Platform - 99291 Description Implementation experience in BigData Platform ; preferably in Cloudera Hadoop platform Minimum 2 years of Development experience using Hadoop eco system tools & utilities: MapReduce, Spark, Kafka, Sqoop, Impala, Hive etc Ability to work independently and also contribute to overall...


  • Sydney, Australia TikTok Full time

    About the team Be part of TikTok's groundbreaking Data Application Team within the Video Cloud Infrastructure division, where we empower our Live CDN and ultra-edge CDN service teams to revolutionize video on demand and video streaming through data-driven insights and cutting-edge algorithms. By harnessing the vast log data generated by our CDNs, we...

  • Senior Data Engineer

    4 weeks ago


    Sydney, Australia Virtusa Full time

    Senior Data Engineer - CREQ183850 Description ETL Dev and Testing Teradata Scala Python SQL Selenium Test automation Datawarehousing / BI experience Spark Hadoop / Big Data / HDFS Cloud migration experience AWS Cloud experience Primary Location Sydney, New South Wales, Australia Job Type Experienced Primary Skills Snowflake...


  • North Sydney, Australia NTT DATA Full time

    NTT DATA is heavily investing globally in providing state of the art analytical solutions to its clients. Our capability has been focused locally, regionally in APAC and globally by forming strategic partnerships including Microsoft Azure, SAP Analytics, Snowflake, Matillion and many more, leveraging the strength of our cloud and network assets via the NTT...

  • Executive Manager

    4 weeks ago


    Sydney, Australia Commonwealth Bank Full time

    **Executive Manager - Data Engineering (Crew Lead -Controls and Ingestion)** - You are an Executive Leader who’s passionate about managing large engineering teams - We are a team of big thinkers who are engineering data-driven solutions in the world of Chief Data and Analytics Office - Together we can rewrite the experience for millions of customers **Do...

  • Data Engineer

    4 weeks ago


    Sydney, Australia Practiv Full time

    As a Practiv Data Engineer you will join a team of passionate data-cloud technologists who help deliver end to end data engineering solutions for our customers from building data pipelines, warehousing the data and visualisation/presentation. Practiv Data Engineers understand both data pipeline development, operations and management as well as delivery of...


  • Sydney, Australia DATA MACHINE Full time

    **Responsibilities**: - Collect and organize social media data from various sources, including social media platforms, analytics tools, and client reports. - Enter and manage data in spreadsheets or databases, ensuring accuracy and completeness. - Conduct quality checks to identify and correct errors or inconsistencies in the data. - Prepare and present...

  • Big Data Lead

    4 weeks ago


    Sydney, Australia XPT Software Australia PTY Ltd Full time

    **About us**: **About XPT** - XPT Software Australia PTY Ltd, incorporated in 2016, is a Software Services company - XPT works with top clients across Australia in Banking, Insurance, Telecom, Retail, Energy, Mining and Manufacturing domains. - We have 120+ technocrats in Australia working at our client locations. - XPT Software Australia is part of group...

  • Big Data Lead

    4 weeks ago


    Sydney, Australia XPT Software Australia PTY Ltd Full time

    **About us**: **About XPT** - XPT Software Australia PTY Ltd, incorporated in 2016, is a Software Services company - XPT works with top clients across Australia in Banking, Insurance, Telecom, Retail, Energy, Mining and Manufacturing domains. - We have 120+ technocrats in Australia working at our client locations. - XPT Software Australia is part of group...

  • Data Engineer

    1 week ago


    Sydney, Australia Cognizant Full time

    Job Description What makes Cognizant a unique place to work? The combination of rapid growth and an international and innovative environment! This is creating many opportunities for people like YOU — people with an entrepreneurial spirit who want to make a difference in this world. At Cognizant, together with your colleagues from all around the world,...

  • Data Engineer

    1 week ago


    Sydney, Australia Hydrogen Group Full time

    The Client My client is a leading financial organisation who are continuing their journey with multiple projects where they are looking for Data Engineers. This client is working on multiple projects which would give you an exciting opportunity to cutting edge Technology tools. The Role This role offers a great chance to join a team of Engineering...

  • Data Engineer

    7 days ago


    Sydney, New South Wales, Australia Cognizant Full time

    Job Description What makes Cognizant a unique place to work? The combination of rapid growth and an international and innovative environment This is creating many opportunities for people like YOU - people with an entrepreneurial spirit who want to make a difference in this world. At Cognizant, together with your colleagues from all around the world, you...

  • Data Engineer

    2 weeks ago


    Sydney, Australia Virtusa Full time

    Data Engineer - CREQ186610 Description Role Name: Data Engineer Must have 5+ years of experience in below mentioned skills. Must Have: Big Data Concepts , Python, SQL, Shell Scripting, AWS S3 Good to Have: Event-driven/AWA SQS, Microservices, API Development, Kafka, Kubernetes, Argo, Amazon Redshift, Amazon Aurora Primary Location Sydney, New South Wales,...

  • Data Engineer

    9 hours ago


    Sydney, Australia INGRITY Remote Work Freelance Full time

    Seeking a skilled Data Engineer with data and platform engineering experience to design, build, and maintain data platforms for a top banking client. Collaborate with teams to deliver high-quality data solutions. At INGRITY:Imagine the Impossible: It's not just an aspiration for our clients; we expect the same from our team.INGRITY isn't your typical...

  • Senior Data Engineer

    4 weeks ago


    Sydney, Australia Hydrogen Group Full time

    The Client My client is a leading financial organisation who are continuing their journey with multiple projects where they are looking for Data Engineers. This client is working on multiple projects which would give you an exciting opportunity to cutting edge Technology tools. The Role This role offers a great chance to join a team of Engineering...


  • Sydney, Australia Commonwealth Bank Full time

    **_You are determined give your Crew and Squads purpose to drive our Enterprise Data Platform agenda leading Product Owners and partnering with Engineering Leaders and Centre of Excellence Delivery _**: - **_Together we can bring to life state-of-the-art data solutions that power seamless experiences for millions of customers. _** **Do work that...


  • Sydney, Australia Latitude IT Full time

    Collaborative, supportive, Values driven environmentGreat coffee (and a variety of tea) and fresh fruitAvoid the traffic: we are outside the CBD and close to public transportOptimize Azure-based data infrastructure.Drive decision-making with insightful analytics.Foster innovation through technology integration.Are you passionate about leveraging data to...

  • Data Engineer

    4 weeks ago


    Sydney, Australia ED - Group Support Functions Full time

    About You You’re customer focused and passionate about being part of a tribe of great people and you actively contribute to your workplace community. With a growth mindset, you orient around doing what’s meaningful, purposeful and delivers value. You’re excited about being part of a data-led transformation at one of Australia’s biggest...

  • Data Engineer

    4 weeks ago


    Sydney, Australia PayTech Nexus Ltd Full time

    Term: 12 monthsExperience Requirement: 3-5 yearsJob location: Sydney, AustraliaProject scope: Data Engineering in Credit Risk EnvironmentBackgroundWe are working with one of the major card schemes and a Tier 1 Australian Bank, based in Sydney. This role is a 12-month contract (likely extending to 2 years) and is for an experienced Data Engineer / Analyst...

  • Cloud Admin

    4 weeks ago


    Sydney, Australia CareCone Australia Full time

    **Profile**: - 5-8 years of hands-on experience in managing Big Data technologies and Hadoop eco system including Hive, Spark, ranger, Oozie, HBase, CDSW, NiFi. - Responsible for deploying, upgrading, configuring, and managing of Cloudera platforms. - Responsible to building CDH, upgrade CDH distributions, managing services, backup and disaster recovery. -...