Agentic AI Lead

7 hours ago

Sydney, New South Wales, Australia N2S Full time $180,000 - $250,000 per year

Exp. Level
: 8-10 years

Location:
Sydney/Melbourne/AU offices

Job Description
:

We are looking for a hands-on and technically strong Technical Lead to drive the design, development, and deployment of evaluation frameworks for Agentic AI and Retrieval-Augmented Generation (RAG) solutions. This role is pivotal in enabling the deployment of scalable multi-agent AI architecture (20–30+ agents) in production, with a strong emphasis on Agent-Level and System-Level evaluation in compliance-heavy domains such as finance and banking.

Responsibilities:

  • Develop and implement framework leveraging advanced AI Agents using models such as OpenAI GPT, Anthropic Clude, Meta Llama2, Google Gemini etc. focusing on enhancing developer and business productivity.
  • Knowledge of mapping suitable LLMs according to agent roles (e.g., DevSecOps Agent with OpenAI).
  • Must have experience with Agentic AI frameworks like
    LangGraph, CrewAI, AutoGen etc.
  • Must have experience in framework evaluation tools and techniques such as AgentBench, RAGA and OpenAI Evals
    .
  • Develop the solution incorporating advanced techniques like Retrieval Augmented Generation (RAG), Prompt Engineering ensuring optimal performance and scalability.
  • Must know how to measure and benchmark the performance of the Agents.
  • Drive end-to-end implementation and deployment with extensive knowledge of Azure and AWS services.
  • Understanding and experience in LLMOps and AgentOps with multi-agent coordination and debugging.
  • Stay abreast of emerging trends, complex patterns, data dependencies, and advancements in AI architecture, contributing to the refinement and innovation of application development processes.

Qualifications:

  • Bachelor's degree or master's degree (or equivalent experience) or PhD in Computer Science, Electrical Engineering, Artificial Intelligence, or Applied Math with 8+ years of experience, with at least 1.5 years of hand-on in GenAI.
  • Excellent programming skills in Python with strong fundamentals in programming, optimizations and software design.
  • Proven expertise in developing scalable evaluation systems for GenAI, RAG, or multi-agent architectures.
  • Experience with agentic AI frameworks (e.g., LangChain, CrewAI, AutoGPT), RAG pipelines and orchestration tools.
  • Strong collaborative and interpersonal skills, specifically a proven ability to effectively guide and influence within a dynamic matrix environment.

  • AI Agent Solutions Engineer

    2 weeks ago

    Sydney, New South Wales, Australia Relevance AI Full time US$120,000 - US$150,000 per year

    Location :Sydney, Australia (Hybrid - 3 Days In Office)About Us At Relevance AI, our mission is to empower anyone to delegate work to the AI workforce.We're building a new type of automation platform, for anyone to create and use AI agents that can replicate human quality work, decision making and collaboration.We are scaling fast to meet unprecedented...

  • Agentic AI Lead

    5 hours ago

    Sydney, New South Wales, Australia PhoenixDX Full time $120,000 - $250,000 per year

    Agentic AI Lead at PhoenixDX — a senior role blending thought leadership, client evangelism, and hands-on technical delivery to scale next-gen AI solutions.PhoenixDX is one of Australia's fastest-growing digital innovation companies. We specialise in rapid application development using the latest low-code enterprise platforms, legacy modernisation, leading...

  • Senior AI Engineer

    2 weeks ago

    Sydney, New South Wales, Australia Relevance AI Full time

    OverviewSenior AI Engineer role at Relevance AI, Sydney, Australia (Hybrid - 3 Days In Office).About UsAt Relevance AI, our mission is to empower anyone to delegate work to the AI workforce. We're building a new type of automation platform for anyone to create and use AI agents that can replicate human quality work, decision making and collaboration. We're...

  • Senior AI Engineer

    2 weeks ago

    Sydney, New South Wales, Australia Relevance AI Full time

    OverviewSenior AI Engineer role at Relevance AI, Sydney, Australia (Hybrid - 3 Days In Office).About UsAt Relevance AI, our mission is to empower anyone to delegate work to the AI workforce. We're building a new type of automation platform for anyone to create and use AI agents that can replicate human quality work, decision making and collaboration. We're...

  • Ai Principal Consultant

    1 week ago

    Sydney, New South Wales, Australia V2 AI Full time

    2 days ago Be among the first 25 applicants V2 AI is a leading Data & AI consultancy backed by $30m in VC funding, allowing us to meet our customers' needs.We harness the power of Data & AI to accelerate business outcomes for some of the world's largest brands.We bring decades of experience and a unique delivery model to partner with our customers on the...

  • Staff AI Engineer

    5 hours ago

    Sydney, New South Wales, Australia Relevance AI Full time $120,000 - $180,000 per year

    Location :Sydney, Australia (Hybrid - 3 Days In Office)About Us At Relevance AI, our mission is to empower anyone to delegate work to the AI workforce.We're building a new type of automation platform, for anyone to create and use AI agents that can replicate human quality work, decision making and collaboration.We are scaling fast to meet unprecedented...

  • AI Agentic Lead – Voice Recognition

    6 hours ago

    Sydney, New South Wales, Australia tekFinder Full time $200,000 - $300,000 per year

    AI Agentic Lead – Voice Recognition – Conversational AIWe're seeking a highly skilled Agentic AI Lead with a specialization in voice to lead the design, development, and deployment of voice‑based autonomous agents.As a lead role, you'll be responsible for pushing the boundaries of voice interaction: ASR (automatic speech recognition), TTS...

  • AI Agent

    2 weeks ago

    Sydney, New South Wales, Australia Peak Advisory Australia Pty Ltd Full time $80,000 - $90,000 per year

    About UsWe are a growing automation and AI solutions provider, specialising in AI Agent development, business process automation, and data-driven insights for Australian small-to-medium businesses. Our projects combine self-built AI tools with integrations into existing platforms like Xero, Snowflake, and Alteryx — helping clients transform their...

  • Advanced Agentic AI Innovation Lead

    6 days ago

    Sydney, New South Wales, Australia beBeeDeveloper Full time $120,000 - $200,000

    Agentic AI Solutions LeaderWe are seeking a highly skilled Staff Software Engineer to lead the development of innovative agentic AI solutions that drive business growth.Key Responsibilities:

  • Agentic AI Engineer

    6 hours ago

    Sydney, New South Wales, Australia Resource Corner Full time $120,000 - $180,000 per year

    Location:Sydney/Melbourne/AU offices Job Description:We are looking for a hands-on and technically strong Senior software engineer to drive the development, anddeployment of evaluation frameworks for Agentic AI and Retrieval-Augmented Generation (RAG) solutions. Thisrole is pivotal in enabling the deployment of scalable...