Senior Bigdata Engineer-AWS/Azure

4 days ago

Melbourne, Victoria, Australia Cognizant Full time
Job Description


What makes Cognizant a unique place to work? The combination of rapid growth and an international and innovative environment This is creating many opportunities for people like YOU - people with an entrepreneurial spirit who want to make a difference in this world.


At Cognizant, together with your colleagues from all around the world, you will collaborate on creating solutions for the world's leading companies and help them become more flexible, more innovative, and successful.

Moreover, this is your chance to be part of the success story.

Position Summary:

  • We are looking for a Sr. Big Data Engineer that will work on the collecting, storing, processing, and analyzing of huge sets of data. The primary focus will be on choosing optimal solutions to use for these purposes, then maintaining, implementing, and monitoring them. You will also be responsible for integrating them with the architecture used across the company.
  • Exposure to, or an understanding of a broad cross-section of on prem, hybrid and cloud data platforms, including Azure/AWS.
  • Scope, architect, design and engineer innovative cloud-based solutions in multi-cloud scenarios across the Microsoft Azure/AWS.
  • 7+ years demonstrated coding experience in Spark Streaming, batch framework & hive/impala. (Language such as Scala/Java/Python)
  • Hands on designing a micro service architecture solution using AWS Lambda, Kinesis, and Azure Event Hub, which process streaming data in real-time.
  • 2+ expertise with writing applications or APIs using Azure Function, Logic Apps, Azure Service Bus, Azure Databricks, AWS lambda.
  • A demonstrated experience in design and build real-time streaming pipeline using AWS kinesis, lambda, DynamoDB, Azure EventHub & Azure synapse.
  • Inhouse Proprietary Tool - 3+ years of working experience with JOF Job, Merlin and JOF Ingestion framework to develop data ingestions pipelines and deploy the code artifact.
  • Hands on implementation of end-to-end data pipelines using Azure Data Lake, Azure Data factory, Azure Databricks.
  • Expertise with database services & data lake such as Azure Synapse, Cosmos DB, Azure SQL, Azure Blob Storage, and Azure Data Lake Storage Gen2.
  • Solid experience in setting up advanced deployment techniques like CI/CD using Azure DevOps, GITHUB integration, ARM templates, Terraform, CloudFormation, CodePipeline, CodeCommit, CodeBuild, Jenkins, Bamboo & etc.
  • End to End project execution, Requirement gathering, transforming legacy design to Bigdata eco system. Development, Testing, UAT Support and GO Live support.
  • Understanding and creating Tableau/PowerBI reports would be an additional advantage.

Mandatory Skills:

Big Data Stack:

  • Hadoop, Spark Streaming, Kafka, Hive, Impala, Hue, Kudu, Oozie, AirFlow, Sqoop, HBase, Pig, Flume, Scala, Java, Python, Shell, Google Apigee, Tableau, PowerBI, Cloudera Manager, Ambari, Control-M.

Azure Stack:

  • Azure Function, Azure EventHub, Azure HDInsight, Azure Data Bricks, Azure VM, Azure Data Factory, Azure storage, Azure Data Lake, Azure Cosmos DB, Azure KeyVault, Azure SQLDW, Synapse, Azure Policy, Azure Blueprint, Logic App.

AWS Stack:

  • Kinesis, Lambda, S3, EC2, ASG, DynamoDB, API Gateway, VPC, CloudTrail, IAM roles & Policies
  • Dev-ops Tools: - Terraform, CloudFormation, Azure DevOps, JIRA, GIT, Bitbucket, Bamboo, Jenkins, Confluence.

Client Specific:

  • JOF Job, Merlin, JOF Ingestion framework and Paul tool

Duties and Responsibilities:

  • Associate would be responsible for design, code, test, document and maintain high-quality and scalable Big Data solutions on-prem or on-cloud.
  • Research, evaluate and deploy new tools frameworks and design patterns to build sustainable Big Data platform.
  • Proactively communicate and collaborate with stakeholders, product specialists and data architects to capture functional and non-functional requirements.
  • Define and develop APIs for integration of various data sources.
  • Make accurate development effort estimates to assist management in project planning.
  • Responsible for migrating on-premises workloads to the cloud and debug cloud stacks.
  • Responsible for the design and build of the CI/CD and/or DevOps pipelines and looks to improve existing workflows through deployment automation.

Salary Range:
>$100,000

Date of Posting : 10/6/2024

Next Steps:

If you feel this opportunity suits you, or Cognizant is the type of organization you would like to join, we want to have a conversation with you Please apply directly with us.

For a complete list of open opportunities with Cognizant, visit is committed to providing Equal Employment Opportunities. Successful candidates will be required to undergo a background check.

About Cognizant:


Cognizant (Nasdaq-100: CTSH) is one of the world's leading professional services companies, transforming clients' business, operating and technology models for the digital era.

Our unique industry-based, consultative approach helps clients envision, build, and run more innovative and efficient businesses. Learn how Cognizant helps clients lead with digital at or follow us @Cognizant.

  • Senior Bigdata Engineer

    2 weeks ago

    Melbourne, Victoria, Australia Cognizant Full time

    Job Description What makes Cognizant a unique place to work? The combination of rapid growth and an international and innovative environment This is creating many opportunities for people like YOU — people with an entrepreneurial spirit who want to make a difference in this world.At Cognizant, together with your colleagues from all around the world, you...

  • Mid - Senior Aws Devops Engineer

    2 weeks ago

    Melbourne, Victoria, Australia Opus Recruitment Solutions Ltd Full time

    Apply Mid - Senior AWS Dev Ops Engineer Melbourne, Australia Permanent$ $150000 per annum Mid-Senior AWS Dev Ops Engineer Our client is a start-up poised for exponential growth.To catalyse their journey, they are seeking a Senior AWS Dev Ops Engineer to join their team.As the sole Dev Ops engineer, you will play a pivotal role in shaping their infrastructure...

  • APAC Senior Azure Engineer

    2 weeks ago

    Melbourne, Victoria, Australia Bluefin Resources Full time

    APAC Senior Azure Engineer - Greenfield project Azure Cloud migration from Legacy systems Global co. - APAC Senior Azure Engineer - Greenfield project - APPLY NOW Salary - $150 - $170K plus super, plus benefits Location - East Melbourne - hybrid role - 2 days a week office based Reporting to APAC Head of Cloud and Strategy his role will be managing...

  • Azure Cloud Engineer

    2 weeks ago

    Melbourne, Victoria, Australia Alliancesrcare Full time

    KPMG - Melbourne, VICIT Source: uWorkin JOB DESCRIPTION Job DescriptionOur Enterprise Technology & System Integration team brings together skills in intelligent automation, data engineering, software integration and cloud platforms to help our clients align their technology capabilities with their strategic and operational objectives. We also draw on KPMG's...

  • Azure Cloud Engineer

    2 weeks ago

    Melbourne, Victoria, Australia Alliancesrcare Full time

    KPMG - Melbourne, VICIT Source: u Workin JOB DESCRIPTION Job Description Our Enterprise Technology & System Integration team brings together skills in intelligent automation, data engineering, software integration and cloud platforms to help our clients align their technology capabilities with their strategic and operational objectives.We also draw on KPMG's...

  • Senior Identity Integration Engineer

    2 months ago

    Melbourne, Victoria, Australia Randstad Australia Full time

    *Australian Citizens only*Our Government client is looking for a talented IAM/IDAM Integration Engineer for a long term 12 month contract in Melbourne.This position involves designing, implementing, and enhancing identity integration solutions to ensure smooth and secure access for external users within an Azure and AWS Cloud environment.What you'll do:Lead...

  • Azure Cloud Engineer

    2 weeks ago

    Melbourne, Victoria, Australia InfoCentric Full time

    Info Centric is growing, and with this growth, many exciting programs are starting that require data architecture and data modelling expertise.In this role, you will work closely with our Head of Architecture in designing and implementing cloud platforms for high-profile Australian brands across various industries.Your involvement will help organisations...

  • Azure Cloud Engineer

    2 weeks ago

    Melbourne, Victoria, Australia KPMGau Full time

    Job DescriptionOur Enterprise Technology & System Integration team brings together skills in intelligent automation, data engineering, software integration and cloud platforms to help our clients align their technology capabilities with their strategic and operational objectives. We also draw on KPMG's extensive national and global resources, adopting a...

  • Azure Cloud Engineer

    2 weeks ago

    Melbourne, Victoria, Australia Infocentric Full time

    InfoCentric is growing, and with this growth, many exciting programs are starting that require data architecture and data modelling expertise. In this role, you will work closely with our Head of Architecture in designing and implementing cloud platforms for high-profile Australian brands across various industries. Your involvement will help organisations...

  • Azure Cloud Engineer

    2 weeks ago

    Melbourne, Victoria, Australia InfoCentric Full time

    InfoCentric is growing, and with this growth, many exciting programs are starting that require data architecture and data modelling expertise. In this role, you will work closely with our Head of Architecture in designing and implementing cloud platforms for high-profile Australian brands across various industries. Your involvement will help organisations...

  • Senior Azure Data Engineer

    2 weeks ago

    Melbourne, Victoria, Australia NTT DATA Business Solutions Full time

    Press Tab to Move to Skip to Content Link Select how often (in days) to receive an alert: NTT DATA is heavily investing globally in providing state of the art analytical solutions to its clients. Our capability has been focused locally, regionally in APAC and globally by forming strategic partnerships including Microsoft Azure, SAP Analytics, Snowflake,...

  • Senior AWS Cloud Engineer

    2 weeks ago

    Melbourne, Victoria, Australia Luxoft Full time

    Project description A leading client in Australia is seeking to migrate the existing Murex solution to Amazon Web Services (AWS). The migration of Murex from on-premises to AWS will enable bank to automate environment provisioning, improve application performance, modernize Murex platform, and potentially reduce their total cost of ownership (TCO).DXC has...

  • Azure DevOps Engineer

    2 weeks ago

    Melbourne, Victoria, Australia Firesoft People Full time

    DevOps & Infrastructure Engineer - Azure - NV1/NV2 clearance mandatory - 6 months rolling contract - Remote working anywhere in AustraliaAs a Platforms & DevOps Consultant, you'll assist customers on their cloud enablement and optimization journey, applying automated delivery and operational processes. You must have a deep infrastructure background and...

  • Devops Engineer

    2 weeks ago

    Melbourne, Victoria, Australia Intrinzic Full time

    $100k-$165k base + super Junior, Mid and Senior Dev Ops Engineering roles - Brand new teams Security clearance required (Baseline, NV1 or NV2) Intrinzic has partnered with a rapidly growing industry leader in the search for multiple Dev Ops Engineers, ranging from Junior to Senior in levelling, to work across public cloud and automation projects.Due to the...

  • Azure Devops Engineer

    2 weeks ago

    Melbourne, Victoria, Australia Green Light Full time

    Azure Devops Engineer (IRAP Accessor)1-3 Months Contract ( Remote)Contract BasedGreen Light Worldwide are looking for a Senior Azure DevOps Engineer to assist in delivering projects for a leading Industry provider. In this role candidate will be responsible for containerize a software solution suitable run on a government AZURE environment and network. They...

  • Senior Devops Engineer

    2 weeks ago

    Melbourne, Victoria, Australia ClearCompany Full time

    A global fintech, my client is looking to bring in a Senior Dev Ops Engineer to help in the growth and expansion of their presence in Dev Ops and AWS.Partnering closely and reporting to the Tech Lead, with the Chief Technology Officer above, this role is a great opportunity for a specialist that is passionate within Dev Ops looking to grow into a leadership...

  • Senior Devops Engineer

    2 weeks ago

    Melbourne, Victoria, Australia Sirius People Full time

    Sirius Technology Melbourne part of Sirius People Pty Ltd Engineering - Software (Information & Communication Technology) A global fintech, my client is looking to bring in a Senior Dev Ops Engineer to help in the growth and expansion of their presence in Dev Ops and AWS.Partnering closely and reporting to the Tech Lead, with the Chief Technology Officer...

  • Senior DevOps Engineer

    2 weeks ago

    Melbourne, Victoria, Australia ClearCompany Full time

    A global fintech, my client is looking to bring in a Senior DevOps Engineer to help in the growth and expansion of their presence in DevOps and AWS.Partnering closely and reporting to the Tech Lead, with the Chief Technology Officer above, this role is a great opportunity for a specialist that is passionate within DevOps looking to grow into a leadership...

  • Senior Aws Cloud Engineer

    2 weeks ago

    Melbourne, Victoria, Australia Alliancesrcare Full time

    A leading client in Australia is seeking to migrate the existing Murex solution to Amazon Web Services (AWS). The migration of Murex from on-premises to AWS will enable bank to automate environment provisioning, improve application performance, modernize Murex platform, and potentially reduce their total cost of ownership (TCO). DXC has been engaged to...

  • Senior Aws Cloud Engineer

    2 weeks ago

    Melbourne, Victoria, Australia Alliancesrcare Full time

    A leading client in Australia is seeking to migrate the existing Murex solution to Amazon Web Services (AWS).The migration of Murex from on-premises to AWS will enable bank to automate environment provisioning, improve application performance, modernize Murex platform, and potentially reduce their total cost of ownership (TCO).DXC has been engaged to execute...