Agentic AI Senior Engineer

2 days ago

Sydney, New South Wales, Australia N2S Full time $150,000 - $200,000 per year

Exp. Level
: 3-5 years

Location:
Sydney/Melbourne/AU offices

Job Description
:

We are looking for a hands-on and technically strong Senior software engineer to drive the development, and deployment of evaluation frameworks for Agentic AI and Retrieval-Augmented Generation (RAG) solutions. This role is pivotal in enabling the deployment of scalable multi-agent AI architecture (20–30+ agents) in production, with a strong emphasis on Agent-Level and System-Level evaluation in compliance-heavy domains such as finance and banking.

Responsibilities:

  • Develop and implement framework leveraging advanced AI Agents using models such as OpenAI GPT, Anthropic Clude, Meta Llama2, Google Gemini etc. focusing on enhancing developer and business productivity.
  • Knowledge of mapping suitable LLMs according to agent roles (e.g., DevSecOps Agent with OpenAI).
  • Must have experience with Agentic AI frameworks like
    LangGraph, CrewAI, AutoGen etc.
  • Must have experience in framework evaluation tools and techniques such as AgentBench, RAGA and OpenAI Evals
    .
  • Develop the solution incorporating advanced techniques like Retrieval Augmented Generation (RAG), Prompt Engineering ensuring optimal performance and scalability.
  • Must know how to measure and benchmark the performance of the Agents.
  • Drive end-to-end implementation and deployment with extensive knowledge of Azure and AWS services.
  • Stay abreast of emerging trends, complex patterns, data dependencies, and advancements in AI architecture, contributing to the refinement and innovation of application development processes.

Qualifications:

  • Bachelor's degree or master's degree (or equivalent experience) or PhD in Computer Science, Electrical Engineering, Artificial Intelligence, or Applied Math with 3+ years of experience, with at least 1 year of hand-on in GenAI.
  • Excellent programming skills in Python with strong fundamentals in programming, optimizations and software design.
  • Proven expertise in developing scalable evaluation systems for GenAI, RAG, or multi-agent architectures.
  • Experience with agentic AI frameworks (e.g., LangChain, CrewAI, AutoGPT), RAG pipelines and orchestration tools.
  • Strong collaborative and interpersonal skills, specifically a proven ability to effectively guide and influence within a dynamic matrix environment.

  • AI Agent Solutions Engineer

    2 weeks ago

    Sydney, New South Wales, Australia Relevance AI Full time US$120,000 - US$150,000 per year

    Location :Sydney, Australia (Hybrid - 3 Days In Office)About Us At Relevance AI, our mission is to empower anyone to delegate work to the AI workforce.We're building a new type of automation platform, for anyone to create and use AI agents that can replicate human quality work, decision making and collaboration.We are scaling fast to meet unprecedented...

  • Senior AI Engineer

    4 days ago

    Sydney, New South Wales, Australia Relevance AI Full time $120,000 - $180,000 per year

    Location : Sydney, Australia (Hybrid - 3 Days In Office)About UsAt Relevance AI, our mission is to empower anyone to delegate work to the AI workforce.We're building a new type of automation platform, for anyone to create and use AI agents that can replicate human quality work, decision making and collaboration.We are scaling fast to meet unprecedented...

  • Senior AI Engineer

    4 days ago

    Sydney, New South Wales, Australia Relevance AI Full time $150,000 - $200,000 per year

    Location : Sydney, Australia (Hybrid - 3 Days In Office)About Us At Relevance AI, our mission is to empower anyone to delegate work to the AI workforce.We're building a new type of automation platform, for anyone to create and use AI agents that can replicate human quality work, decision making and collaboration.We are scaling fast to meet unprecedented...

  • Senior AI Engineer

    4 days ago

    Sydney, New South Wales, Australia Relevance AI Full time $120,000 - $180,000 per year

    Location : Sydney, Australia (Hybrid - 3 Days In Office) About Us At Relevance AI, our mission is to empower anyone to delegate work to the AI workforce. We're building a new type of automation platform, for anyone to create and use AI agents that can replicate human quality work, decision making and collaboration. We are scaling fast to meet...

  • Staff AI Engineer

    2 days ago

    Sydney, New South Wales, Australia Relevance AI Full time $120,000 - $180,000 per year

    Location :Sydney, Australia (Hybrid - 3 Days In Office)About Us At Relevance AI, our mission is to empower anyone to delegate work to the AI workforce.We're building a new type of automation platform, for anyone to create and use AI agents that can replicate human quality work, decision making and collaboration.We are scaling fast to meet unprecedented...

  • Staff AI Engineer

    4 days ago

    Sydney, New South Wales, Australia Relevance AI Full time $120,000 - $180,000 per year

    Location : Sydney, Australia (Hybrid - 3 Days In Office)About Us At Relevance AI, our mission is to empower anyone to delegate work to the AI workforce.We're building a new type of automation platform, for anyone to create and use AI agents that can replicate human quality work, decision making and collaboration.We are scaling fast to meet unprecedented...

  • Staff AI Engineer

    4 days ago

    Sydney, New South Wales, Australia Relevance AI Full time $120,000 - $180,000 per year

    Location : Sydney, Australia (Hybrid - 3 Days In Office) About Us At Relevance AI, our mission is to empower anyone to delegate work to the AI workforce. We're building a new type of automation platform, for anyone to create and use AI agents that can replicate human quality work, decision making and collaboration. We are scaling fast to meet...

  • Agentic AI Engineer

    2 days ago

    Sydney, New South Wales, Australia Resource Corner Full time $120,000 - $180,000 per year

    Location:Sydney/Melbourne/AU offices Job Description:We are looking for a hands-on and technically strong Senior software engineer to drive the development, anddeployment of evaluation frameworks for Agentic AI and Retrieval-Augmented Generation (RAG) solutions. Thisrole is pivotal in enabling the deployment of scalable...

  • Agentic AI

    10 hours ago

    Sydney, New South Wales, Australia PhoenixDX Full time $180,000 - $250,000 per year

    Agentic AI Practice Lead – PhoenixDXPhoenixDX is one of Australia's fastest-growing digital innovation companies. We specialise in rapid application development using the latest low-code enterprise platforms, legacy modernisation, leading technologies, and mission critical software delivery with top talent.We're also a certified Great Place to Work, home...

  • Senior Software Engineer

    2 weeks ago

    Sydney, New South Wales, Australia NinjaTech AI Full time $150,000 - $200,000 per year

    We invite you to join NinjaTech AI as a Senior Software Engineer as we innovate together to build autonomous AI agents. For this role we will prioritize candidates who are leaning Frontend development (in React, ). This is an onsite position designed to build web applications and agentic AI systems with our strong ex-AWS founding engineering team. About the...