Pyspark Consultant

1 month ago


Melbourne, Australia luvo Full time

Melbourne based Permanent OR Contract position.
- Hybrid working - max 3 days from office.
- Great organization to work with.
- Extract Transform and Load data from sources Systems to Azure Data Storage services using a combination of Azure Data factory, T-SQL, Spark SQL, and U-SQL Azure Data Lake Analytics.
- Data ingestion to one or more Azure services (Azure Data Lake, Azure Storage, Azure SQL, Azure DW) and processing the data in Azure Databricks.
- Hands on experience on developing SQL Scripts for automation.
- Responsible for estimating the cluster size, monitoring, and troubleshooting of the Spark databricks cluster
- Good understanding of Spark Architecture including spark core, spark SQL, DataFrame, Spark streaming, Driver Node, Worker Node, Stages, Executors and Tasks, Deployment modes, the Execution hierarchy, fault tolerance, and collection.

**Please note you must have full Aussie work rights and be willing to work in the Melbourne office.


  • Pyspark Consultant

    2 months ago


    Melbourne, Australia Aurec Human Capital Group Full time

    Our client is a Global leader in IT Consulting services recognised for their extensive portfolio of services & commitment to sustainability. We celebrate diversity and are committed to create an inclusive environment for all. Don't worry if you can't tick every single one of these boxes, we would still like to hear from you. The Role: Required Skills and...

  • Pyspark Consultant

    2 months ago


    Melbourne, Australia Aurec Full time

    Our client is a Global leader in IT Consulting services recognised for their extensive portfolio of services & commitment to sustainability. We celebrate diversity and are committed to create an inclusive environment for all. Don't worry if you can't tick every single one of these boxes, we would still like to hear from you. The Role: Required Skills and...


  • Melbourne City Centre, Australia Ayan Infotech Full time

    **AYAN InfoTech** is looking for **Azure Data Factory Consultant** to join an exciting project based in **Melbourne**. The role offers you the opportunity to contribute towards an extremely well structured and mature environment, working on sophisticated enhancement projects. **Role**: Azure Data Factory Consultant **Location**: Melbourne **Contract...


  • Melbourne, Australia Arinco Full time

    At Arinco, we focus on outcomes - not where or when do your work. - We are innovators in cloud technology, recently awarded Microsoft's Partner of the Year, 2023 - We are innovators in cloud technology, and have a uniquely wonderful company culture! **We are Arinco.** We are innovators in cloud technology, awarded as Microsoft's Azure Partner of the Year...


  • Melbourne, Australia Mastercard, Inc. Full time

    **Our Purpose** We work to connect and power an inclusive, digital economy that benefits everyone, everywhere by making transactions safe, simple, smart and accessible. Using secure data and networks, partnerships and passion, our innovations and solutions help individuals, financial institutions, governments and businesses realize their greatest potential....


  • Melbourne, Australia Mastercard Full time

    Our Purpose We work to connect and power an inclusive, digital economy that benefits everyone, everywhere by making transactions safe, simple, smart and accessible. Using secure data and networks, partnerships and passion, our innovations and solutions help individuals, financial institutions, governments and businesses realize their greatest potential. Our...

  • Azure Data Engineer

    3 weeks ago


    Melbourne, Australia Capgemini Full time

    About Capgemini: Capgemini is a diverse collective of more than 350,000 strategic and technological experts based across more than 50 countries, partnering with world-renowned clients to transform and manage their businesses. We are dedicated to leveraging cloud, data, AI, connectivity, software, digital engineering, and platforms to address the entire...

  • Azure Data Engineer

    2 weeks ago


    Melbourne, Australia Capgemini Full time

    About Capgemini: Capgemini is a diverse collective of more than 350,000 strategic and technological experts based across more than 50 countries, partnering with world-renowned clients to transform and manage their businesses. We are dedicated to leveraging cloud, data, AI, connectivity, software, digital engineering, and platforms to address the entire...

  • Data Solution Designer

    2 months ago


    Melbourne, Australia Resource Corner IT Consulting service Full time

    Solid experience, knowledge and skills in Data Engineering(Global data warehousing), BI/software development such as ELT/ETL, data extraction and manipulation in Data Lake/Data Warehouse environment. - Proven experience in data modelling, analysis, and profiling - Experience of working in diverse data domains. - Continuous improvement and performance...

  • Tech Lead

    2 months ago


    Melbourne, Australia Resource Corner Pty Ltd Full time

    **Description** - Solid experience, knowledge and skills in Data Engineering(Global data warehousing), BI/software development such as ELT/ETL, data extraction and manipulation in Data Lake/Data Warehouse environment. - Proven experience in data modelling, analysis, and profiling - Experience of working in diverse data domains. - Continuous improvement and...

  • Data Professionals

    2 weeks ago


    Melbourne, Australia Capgemini Full time

    Come and join a thriving company and become part of a diverse global collective of free-thinkers, entrepreneurs and industry experts who are all driven to use technology to reimagine what’s possible. Capgemini. Get the future you want. This is a Melbourne based Job only. Let’s talk about the role and responsibilities: As a Data Professional, your...


  • Melbourne City Centre, Australia HAYS Full time

    Want to take the next step in your career as a senior manager? **Your new company** Your new company is a Big 4 Bank that is all about serving their customers well and helping communities prosper. We are looking for a Data and Analytics Stream Lead to join the newly created Customer Identification Remediation Analytics (CIR) team on a 12 month FTC. **Your...


  • Melbourne, Australia Sourced Group, an Amdocs Company Full time

    Sourced is a global cloud consultancy that helps enterprises make the most of cloud services with a focus on security, governance and compliance. With offices around the globe, we provide professional services for securing, migrating and managing the cloud infrastructure of large enterprise customers. We specialize in configuration management, automation,...

  • Solution Designers

    2 months ago


    Melbourne, Australia Paxus Australia Pty Ltd Full time

    Posted 07 March 2023 - SalaryAU$900.00 - AU$980.00 per day - LocationMelbourne - Job type Contract - DisciplineSoftware Development + Testing - ReferenceBBBH260576_1678152576 **Job description**: We are looking for experienced Solution Designers for a global consulting firm. There are two roles; one focussed on UI/ UX design and the second focussed on...


  • Melbourne, Australia Salt Recruitment Full time

    **About us!** Salt is partnering with one of Australia’s leading banks in search of a Analytics Consultant in a newly created Customer Identification Remediation (CIR) Analytics team. You are a data technology professional who enjoys building relationships with key stakeholders and supporting data analysts under squad delivery model. **About The...


  • Melbourne, Australia Sourced Group Full time

    Welcome to the Amdocs Cloud Business Unit!We are the result of a strategic fusion of three prominent Cloud Consulting companies: Sourced, Kenzan, and DevOps Group. Today, we proudly operate under the unified brand of Sourced.As a global team of cloud specialists, we are laser-focused on driving innovation across three critical domains: Platform, Workloads,...

  • Azure Data Engineer

    3 weeks ago


    Melbourne, Australia Experis Full time

    Experis have the pleasure to announce a partnership with a global consultancy in their search for an experienced Data Engineer. This consultancy focuses on accelerating their customers migration to cloud and aims to digitalize and automate their operations. You will have the opportunity to work within a dynamic team focused on developing large-scale data...

  • Azure Data Engineer

    2 weeks ago


    Melbourne, Australia Experis Full time

    Experis have the pleasure to announce a partnership with a global consultancy in their search for an experienced Data Engineer. This consultancy focuses on accelerating their customers migration to cloud and aims to digitalize and automate their operations. You will have the opportunity to work within a dynamic team focused on developing large-scale data...


  • Melbourne, Australia VISA Full time

    Job Description And Responsibilities About the role: Do you want to be a part of Visa’s fastest growing team in Australia and New Zealand? Do you enjoy delivering and getting things done? Do you want to make positive impacts for customers and accelerate the pace of innovation in the payments industry? If so, Visa Managed Services (VMS) is the team...