Senior AI Engineer/Lead Engineer

1 day ago

Sydney, New South Wales, Australia ALOIS Solutions Full time $120,000 - $180,000 per year

Job role – Senior AI Engineer/Lead Engineer

Job Location – Sydney/Melbourne

Job type – Contract

Job Description
:

We are looking for a hands-on and technically strong Senior software engineer to drive the development, and deployment of evaluation frameworks for Agentic AI and Retrieval-Augmented Generation (RAG) solutions. This role is pivotal in enabling the deployment of scalable multi-agent AI architecture (20–30+ agents) in production, with a strong emphasis on Agent-Level and System-Level evaluation in compliance-heavy domains such as finance and banking.

Responsibilities:

  • Develop and implement framework leveraging advanced AI Agents using models such as OpenAI GPT, Anthropic Clude, Meta Llama2, Google Gemini etc. focusing on enhancing developer and business productivity.
  • Knowledge of mapping suitable LLMs according to agent roles (e.g., DevSecOps Agent with OpenAI).
  • Must have experience with Agentic AI frameworks like
    LangGraph, CrewAI, AutoGen etc.
  • Must have experience in framework evaluation tools and techniques such as AgentBench, RAGA and OpenAI Evals
    .
  • Develop the solution incorporating advanced techniques like Retrieval Augmented Generation (RAG), Prompt Engineering ensuring optimal performance and scalability.
  • Must know how to measure and benchmark the performance of the Agents.
  • Drive end-to-end implementation and deployment with extensive knowledge of Azure and AWS services.
  • Stay abreast of emerging trends, complex patterns, data dependencies, and advancements in AI architecture, contributing to the refinement and innovation of application development processes.

Qualifications:

  • Bachelor's degree or master's degree (or equivalent experience) or PhD in Computer Science, Electrical Engineering, Artificial Intelligence, or Applied Math with 3+ years of experience, with at least 1 year of hand-on in GenAI.
  • Excellent programming skills in Python with strong fundamentals in programming, optimizations and software design.
  • Proven expertise in developing scalable evaluation systems for GenAI, RAG, or multi-agent architectures.
  • Experience with agentic AI frameworks (e.g., LangChain, CrewAI, AutoGPT), RAG pipelines and orchestration tools.
  • Strong collaborative and interpersonal skills, specifically a proven ability to effectively guide and influence within a dynamic matrix environment.

  • Senior AI Engineer

    2 weeks ago

    Sydney, New South Wales, Australia Relevance AI Full time $120,000 - $180,000 per year

    Location : Sydney, Australia (Hybrid - 3 Days In Office)About UsAt Relevance AI, our mission is to empower anyone to delegate work to the AI workforce.We're building a new type of automation platform, for anyone to create and use AI agents that can replicate human quality work, decision making and collaboration.We are scaling fast to meet unprecedented...

  • Senior AI Engineer

    3 days ago

    Sydney, New South Wales, Australia Relevance AI Full time $120,000 - $180,000 per year

    Location : Sydney, Australia (Hybrid - 3 Days In Office)About Us At Relevance AI, we're building the home of the AI workforce.Our mission is simple: empower every team to delegate meaningful work to AI agents that think, act, and collaborate like experts.With Relevance AI, anyone can create and manage intelligent agents that handle workflows, decisions, and...

  • Senior AI Engineer

    2 weeks ago

    Sydney, New South Wales, Australia Relevance AI Full time $120,000 - $180,000 per year

    Location : Sydney, Australia (Hybrid - 3 Days In Office) About Us At Relevance AI, our mission is to empower anyone to delegate work to the AI workforce. We're building a new type of automation platform, for anyone to create and use AI agents that can replicate human quality work, decision making and collaboration. We are scaling fast to meet...

  • Senior AI Engineer

    2 weeks ago

    Sydney, New South Wales, Australia Relevance AI Full time $150,000 - $200,000 per year

    Location : Sydney, Australia (Hybrid - 3 Days In Office)About Us At Relevance AI, our mission is to empower anyone to delegate work to the AI workforce.We're building a new type of automation platform, for anyone to create and use AI agents that can replicate human quality work, decision making and collaboration.We are scaling fast to meet unprecedented...

  • Senior AI Engineer

    2 weeks ago

    Sydney, New South Wales, Australia Nuage Technology Group Full time $180,000 - $250,000 per year

    Senior AI EngineerNuage is proud to partner exclusively with an innovative, tech-driven organisation as they scale their next-generation AI capabilities. We're looking for multiple Senior AI Engineers to join a high-impact team developing cutting-edge Gen AI proof-of-concepts and delivering full-scale enterprise implementations.You'll work within a mature,...

  • Staff AI Engineer

    2 weeks ago

    Sydney, New South Wales, Australia Relevance AI Full time $120,000 - $180,000 per year

    Location :Sydney, Australia (Hybrid - 3 Days In Office)About Us At Relevance AI, our mission is to empower anyone to delegate work to the AI workforce.We're building a new type of automation platform, for anyone to create and use AI agents that can replicate human quality work, decision making and collaboration.We are scaling fast to meet unprecedented...

  • Staff AI Engineer

    2 weeks ago

    Sydney, New South Wales, Australia Relevance AI Full time $120,000 - $180,000 per year

    Location :Sydney, Australia (Hybrid - 3 Days In Office)About Us At Relevance AI, we're building the home of the AI workforce, a new generation of automation built for real work.Our mission is simple: empower every team to delegate meaningful work to AI agents that think, act, and collaborate like experts. With Relevance AI, anyone can create and manage...

  • Staff AI Engineer

    3 days ago

    Sydney, New South Wales, Australia Relevance AI Full time $120,000 - $180,000 per year

    Location : Sydney, Australia (Hybrid - 3 Days In Office)About Us At Relevance AI, we're building the home of the AI workforce.Our mission is simple: empower every team to delegate meaningful work to AI agents that think, act, and collaborate like experts.With Relevance AI, anyone can create and manage intelligent agents that handle workflows, decisions, and...

  • Staff AI Engineer

    2 weeks ago

    Sydney, New South Wales, Australia Relevance AI Full time $120,000 - $180,000 per year

    Location : Sydney, Australia (Hybrid - 3 Days In Office) About Us At Relevance AI, our mission is to empower anyone to delegate work to the AI workforce. We're building a new type of automation platform, for anyone to create and use AI agents that can replicate human quality work, decision making and collaboration. We are scaling fast to meet...

  • Staff AI Engineer

    2 weeks ago

    Sydney, New South Wales, Australia Relevance AI Full time $120,000 - $180,000 per year

    Location : Sydney, Australia (Hybrid - 3 Days In Office)About Us At Relevance AI, our mission is to empower anyone to delegate work to the AI workforce.We're building a new type of automation platform, for anyone to create and use AI agents that can replicate human quality work, decision making and collaboration.We are scaling fast to meet unprecedented...