Agentic AI Senior Engineer

1 day ago

Sydney, New South Wales, Australia N2S Full time $150,000 - $200,000 per year

Exp. Level
: 3-5 years

Location:
Sydney/Melbourne/AU offices

Job Description
:

We are looking for a hands-on and technically strong Senior software engineer to drive the development, and deployment of evaluation frameworks for Agentic AI and Retrieval-Augmented Generation (RAG) solutions. This role is pivotal in enabling the deployment of scalable multi-agent AI architecture (20–30+ agents) in production, with a strong emphasis on Agent-Level and System-Level evaluation in compliance-heavy domains such as finance and banking.

Responsibilities:

  • Develop and implement framework leveraging advanced AI Agents using models such as OpenAI GPT, Anthropic Clude, Meta Llama2, Google Gemini etc. focusing on enhancing developer and business productivity.
  • Knowledge of mapping suitable LLMs according to agent roles (e.g., DevSecOps Agent with OpenAI).
  • Must have experience with Agentic AI frameworks like
    LangGraph, CrewAI, AutoGen etc.
  • Must have experience in framework evaluation tools and techniques such as AgentBench, RAGA and OpenAI Evals
    .
  • Develop the solution incorporating advanced techniques like Retrieval Augmented Generation (RAG), Prompt Engineering ensuring optimal performance and scalability.
  • Must know how to measure and benchmark the performance of the Agents.
  • Drive end-to-end implementation and deployment with extensive knowledge of Azure and AWS services.
  • Stay abreast of emerging trends, complex patterns, data dependencies, and advancements in AI architecture, contributing to the refinement and innovation of application development processes.

Qualifications:

  • Bachelor's degree or master's degree (or equivalent experience) or PhD in Computer Science, Electrical Engineering, Artificial Intelligence, or Applied Math with 3+ years of experience, with at least 1 year of hand-on in GenAI.
  • Excellent programming skills in Python with strong fundamentals in programming, optimizations and software design.
  • Proven expertise in developing scalable evaluation systems for GenAI, RAG, or multi-agent architectures.
  • Experience with agentic AI frameworks (e.g., LangChain, CrewAI, AutoGPT), RAG pipelines and orchestration tools.
  • Strong collaborative and interpersonal skills, specifically a proven ability to effectively guide and influence within a dynamic matrix environment.

  • Senior AI Engineer

    2 weeks ago

    Sydney, New South Wales, Australia Relevance AI Full time

    OverviewSenior AI Engineer role at Relevance AI, Sydney, Australia (Hybrid - 3 Days In Office).About UsAt Relevance AI, our mission is to empower anyone to delegate work to the AI workforce. We're building a new type of automation platform for anyone to create and use AI agents that can replicate human quality work, decision making and collaboration. We're...

  • Senior AI Engineer

    2 weeks ago

    Sydney, New South Wales, Australia Relevance AI Full time

    OverviewSenior AI Engineer role at Relevance AI, Sydney, Australia (Hybrid - 3 Days In Office).About UsAt Relevance AI, our mission is to empower anyone to delegate work to the AI workforce. We're building a new type of automation platform for anyone to create and use AI agents that can replicate human quality work, decision making and collaboration. We're...

  • AI Agent Solutions Engineer

    2 weeks ago

    Sydney, New South Wales, Australia Relevance AI Full time US$120,000 - US$150,000 per year

    Location :Sydney, Australia (Hybrid - 3 Days In Office)About Us At Relevance AI, our mission is to empower anyone to delegate work to the AI workforce.We're building a new type of automation platform, for anyone to create and use AI agents that can replicate human quality work, decision making and collaboration.We are scaling fast to meet unprecedented...

  • Agentic AI Engineer

    1 day ago

    Sydney, New South Wales, Australia Resource Corner Full time $120,000 - $180,000 per year

    Location:Sydney/Melbourne/AU offices Job Description:We are looking for a hands-on and technically strong Senior software engineer to drive the development, anddeployment of evaluation frameworks for Agentic AI and Retrieval-Augmented Generation (RAG) solutions. Thisrole is pivotal in enabling the deployment of scalable...

  • Staff AI Engineer

    1 day ago

    Sydney, New South Wales, Australia Relevance AI Full time $120,000 - $180,000 per year

    Location :Sydney, Australia (Hybrid - 3 Days In Office)About Us At Relevance AI, our mission is to empower anyone to delegate work to the AI workforce.We're building a new type of automation platform, for anyone to create and use AI agents that can replicate human quality work, decision making and collaboration.We are scaling fast to meet unprecedented...

  • Ai Principal Consultant

    1 week ago

    Sydney, New South Wales, Australia V2 AI Full time

    2 days ago Be among the first 25 applicants V2 AI is a leading Data & AI consultancy backed by $30m in VC funding, allowing us to meet our customers' needs.We harness the power of Data & AI to accelerate business outcomes for some of the world's largest brands.We bring decades of experience and a unique delivery model to partner with our customers on the...

  • Senior Software Engineer

    1 week ago

    Sydney, New South Wales, Australia Ninjatech Ai Full time

    OverviewWe invite you to join NinjaTech AI as a Senior Software Engineer (React, , Python) to build autonomous AI agents. For this role we will prioritize candidates who are leaning frontend development (in React, ). This is an onsite position designed to build web applications and agentic AI systems with our strong ex-AWS founding engineering team.About the...

  • Senior Software Engineer

    2 weeks ago

    Sydney, New South Wales, Australia NinjaTech AI Full time

    OverviewWe invite you to join NinjaTech AI as a Senior Software Engineer (React, , Python) to build autonomous AI agents. For this role we will prioritize candidates who are leaning frontend development (in React, ). This is an onsite position designed to build web applications and agentic AI systems with our strong ex-AWS founding engineering team.About the...

  • Senior Software Engineer

    2 weeks ago

    Sydney, New South Wales, Australia NinjaTech AI Full time

    Overview We invite you to join NinjaTech AI as a Senior Software Engineer (React, Next.js, Python) to build autonomous AI agents. For this role we will prioritize candidates who are leaning frontend development (in React, Next.js). This is an onsite position designed to build web applications and agentic AI systems with our strong ex-AWS founding...

  • Senior Software Engineer

    2 weeks ago

    Sydney, New South Wales, Australia NinjaTech AI Full time $150,000 - $200,000 per year

    We invite you to join NinjaTech AI as a Senior Software Engineer as we innovate together to build autonomous AI agents. For this role we will prioritize candidates who are leaning Frontend development (in React, ). This is an onsite position designed to build web applications and agentic AI systems with our strong ex-AWS founding engineering team. About the...