Agentic AI Senior Engineer

2 days ago

Sydney, New South Wales, Australia N2S Full time $150,000 - $200,000 per year

Exp. Level
: 3-5 years

Location:
Sydney/Melbourne/AU offices

Job Description
:

We are looking for a hands-on and technically strong Senior software engineer to drive the development, and deployment of evaluation frameworks for Agentic AI and Retrieval-Augmented Generation (RAG) solutions. This role is pivotal in enabling the deployment of scalable multi-agent AI architecture (20–30+ agents) in production, with a strong emphasis on Agent-Level and System-Level evaluation in compliance-heavy domains such as finance and banking.

Responsibilities:

  • Develop and implement framework leveraging advanced AI Agents using models such as OpenAI GPT, Anthropic Clude, Meta Llama2, Google Gemini etc. focusing on enhancing developer and business productivity.
  • Knowledge of mapping suitable LLMs according to agent roles (e.g., DevSecOps Agent with OpenAI).
  • Must have experience with Agentic AI frameworks like
    LangGraph, CrewAI, AutoGen etc.
  • Must have experience in framework evaluation tools and techniques such as AgentBench, RAGA and OpenAI Evals
    .
  • Develop the solution incorporating advanced techniques like Retrieval Augmented Generation (RAG), Prompt Engineering ensuring optimal performance and scalability.
  • Must know how to measure and benchmark the performance of the Agents.
  • Drive end-to-end implementation and deployment with extensive knowledge of Azure and AWS services.
  • Stay abreast of emerging trends, complex patterns, data dependencies, and advancements in AI architecture, contributing to the refinement and innovation of application development processes.

Qualifications:

  • Bachelor's degree or master's degree (or equivalent experience) or PhD in Computer Science, Electrical Engineering, Artificial Intelligence, or Applied Math with 3+ years of experience, with at least 1 year of hand-on in GenAI.
  • Excellent programming skills in Python with strong fundamentals in programming, optimizations and software design.
  • Proven expertise in developing scalable evaluation systems for GenAI, RAG, or multi-agent architectures.
  • Experience with agentic AI frameworks (e.g., LangChain, CrewAI, AutoGPT), RAG pipelines and orchestration tools.
  • Strong collaborative and interpersonal skills, specifically a proven ability to effectively guide and influence within a dynamic matrix environment.

  • Senior AI Engineer

    3 days ago

    Sydney, New South Wales, Australia Relevance AI Full time $120,000 - $180,000 per year

    Location : Sydney, Australia (Hybrid - 3 Days In Office)About UsAt Relevance AI, our mission is to empower anyone to delegate work to the AI workforce.We're building a new type of automation platform, for anyone to create and use AI agents that can replicate human quality work, decision making and collaboration.We are scaling fast to meet unprecedented...

  • Senior AI Engineer

    3 days ago

    Sydney, New South Wales, Australia Relevance AI Full time $120,000 - $180,000 per year

    Location : Sydney, Australia (Hybrid - 3 Days In Office) About Us At Relevance AI, our mission is to empower anyone to delegate work to the AI workforce. We're building a new type of automation platform, for anyone to create and use AI agents that can replicate human quality work, decision making and collaboration. We are scaling fast to meet...

  • Senior AI Engineer

    3 days ago

    Sydney, New South Wales, Australia Relevance AI Full time $150,000 - $200,000 per year

    Location : Sydney, Australia (Hybrid - 3 Days In Office)About Us At Relevance AI, our mission is to empower anyone to delegate work to the AI workforce.We're building a new type of automation platform, for anyone to create and use AI agents that can replicate human quality work, decision making and collaboration.We are scaling fast to meet unprecedented...

  • Staff AI Engineer

    2 days ago

    Sydney, New South Wales, Australia Relevance AI Full time $120,000 - $180,000 per year

    Location :Sydney, Australia (Hybrid - 3 Days In Office)About Us At Relevance AI, our mission is to empower anyone to delegate work to the AI workforce.We're building a new type of automation platform, for anyone to create and use AI agents that can replicate human quality work, decision making and collaboration.We are scaling fast to meet unprecedented...

  • Staff AI Engineer

    3 days ago

    Sydney, New South Wales, Australia Relevance AI Full time $120,000 - $180,000 per year

    Location : Sydney, Australia (Hybrid - 3 Days In Office) About Us At Relevance AI, our mission is to empower anyone to delegate work to the AI workforce. We're building a new type of automation platform, for anyone to create and use AI agents that can replicate human quality work, decision making and collaboration. We are scaling fast to meet...

  • Staff AI Engineer

    3 days ago

    Sydney, New South Wales, Australia Relevance AI Full time $120,000 - $180,000 per year

    Location : Sydney, Australia (Hybrid - 3 Days In Office)About Us At Relevance AI, our mission is to empower anyone to delegate work to the AI workforce.We're building a new type of automation platform, for anyone to create and use AI agents that can replicate human quality work, decision making and collaboration.We are scaling fast to meet unprecedented...

  • Agentic AI Engineer

    2 days ago

    Sydney, New South Wales, Australia Resource Corner Full time $120,000 - $180,000 per year

    Location:Sydney/Melbourne/AU offices Job Description:We are looking for a hands-on and technically strong Senior software engineer to drive the development, anddeployment of evaluation frameworks for Agentic AI and Retrieval-Augmented Generation (RAG) solutions. Thisrole is pivotal in enabling the deployment of scalable...

  • Staff AI Engineer

    3 days ago

    Sydney, New South Wales, Australia Relevance AI Full time $120,000 - $180,000 per year

    Location :Sydney, Australia (Hybrid - 3 Days In Office)About Us At Relevance AI, we're building the home of the AI workforce, a new generation of automation built for real work.Our mission is simple: empower every team to delegate meaningful work to AI agents that think, act, and collaborate like experts. With Relevance AI, anyone can create and manage...

  • Agentic AI

    34 minutes ago

    Sydney, New South Wales, Australia PhoenixDX Full time $180,000 - $250,000 per year

    Agentic AI Practice Lead – PhoenixDXPhoenixDX is one of Australia's fastest-growing digital innovation companies. We specialise in rapid application development using the latest low-code enterprise platforms, legacy modernisation, leading technologies, and mission critical software delivery with top talent.We're also a certified Great Place to Work, home...

  • Agentic AI Lead

    2 days ago

    Sydney, New South Wales, Australia PhoenixDX Full time $120,000 - $250,000 per year

    Agentic AI Lead at PhoenixDX — a senior role blending thought leadership, client evangelism, and hands-on technical delivery to scale next-gen AI solutions.PhoenixDX is one of Australia's fastest-growing digital innovation companies. We specialise in rapid application development using the latest low-code enterprise platforms, legacy modernisation, leading...