Senior AI Engineer/Lead Engineer

2 weeks ago

Sydney, New South Wales, Australia ALOIS Solutions Full time $120,000 - $180,000 per year

Job role – Senior AI Engineer/Lead Engineer

Job Location – Sydney/Melbourne

Job type – Contract

Job Description
:

We are looking for a hands-on and technically strong Senior software engineer to drive the development, and deployment of evaluation frameworks for Agentic AI and Retrieval-Augmented Generation (RAG) solutions. This role is pivotal in enabling the deployment of scalable multi-agent AI architecture (20–30+ agents) in production, with a strong emphasis on Agent-Level and System-Level evaluation in compliance-heavy domains such as finance and banking.

Responsibilities:

  • Develop and implement framework leveraging advanced AI Agents using models such as OpenAI GPT, Anthropic Clude, Meta Llama2, Google Gemini etc. focusing on enhancing developer and business productivity.
  • Knowledge of mapping suitable LLMs according to agent roles (e.g., DevSecOps Agent with OpenAI).
  • Must have experience with Agentic AI frameworks like
    LangGraph, CrewAI, AutoGen etc.
  • Must have experience in framework evaluation tools and techniques such as AgentBench, RAGA and OpenAI Evals
    .
  • Develop the solution incorporating advanced techniques like Retrieval Augmented Generation (RAG), Prompt Engineering ensuring optimal performance and scalability.
  • Must know how to measure and benchmark the performance of the Agents.
  • Drive end-to-end implementation and deployment with extensive knowledge of Azure and AWS services.
  • Stay abreast of emerging trends, complex patterns, data dependencies, and advancements in AI architecture, contributing to the refinement and innovation of application development processes.

Qualifications:

  • Bachelor's degree or master's degree (or equivalent experience) or PhD in Computer Science, Electrical Engineering, Artificial Intelligence, or Applied Math with 3+ years of experience, with at least 1 year of hand-on in GenAI.
  • Excellent programming skills in Python with strong fundamentals in programming, optimizations and software design.
  • Proven expertise in developing scalable evaluation systems for GenAI, RAG, or multi-agent architectures.
  • Experience with agentic AI frameworks (e.g., LangChain, CrewAI, AutoGPT), RAG pipelines and orchestration tools.
  • Strong collaborative and interpersonal skills, specifically a proven ability to effectively guide and influence within a dynamic matrix environment.

  • Senior AI Engineer

    2 weeks ago

    Sydney, New South Wales, Australia Relevance AI Full time $120,000 - $180,000 per year

    Location : Sydney, Australia (Hybrid - 3 Days In Office)About Us At Relevance AI, we're building the home of the AI workforce.Our mission is simple: empower every team to delegate meaningful work to AI agents that think, act, and collaborate like experts.With Relevance AI, anyone can create and manage intelligent agents that handle workflows, decisions, and...

  • Senior AI Engineer

    3 days ago

    Sydney, New South Wales, Australia Relevance AI Full time

    Location :Sydney, Australia (Hybrid - 3 Days In Office)About Us At Relevance AI, we're building the home of the AI workforce.Our mission is simple: empower every team to delegate meaningful work to AI agents that think, act, and collaborate like experts.With Relevance AI, anyone can create and manage intelligent agents that handle workflows, decisions, and...

  • Senior AI Engineer

    2 weeks ago

    Sydney, New South Wales, Australia Future Secure AI Pty Ltd Full time $120,000 - $180,000 per year

    About UsFuture Secure AI (FSAI) is a leading enterprise AI company that was created in deep partnership with one of the world's largest financial institutions.Operating as an enterprise company with deep relationships at C-suite with large global enterprises, FSAI believes humans and AI Workers will collaborate seamlessly to raise organisational performance....

  • Staff AI Engineer

    1 day ago

    Sydney, New South Wales, Australia Relevance AI Full time

    Location :Sydney, Australia (Hybrid - 3 Days In Office)About Us At Relevance AI, we're building the home of the AI workforce, a new generation of automation built for real work.Our mission is simple: empower every team to delegate meaningful work to AI agents that think, act, and collaborate like experts. With Relevance AI, anyone can create and manage...

  • Staff AI Engineer

    2 weeks ago

    Sydney, New South Wales, Australia Relevance AI Full time $120,000 - $180,000 per year

    Location : Sydney, Australia (Hybrid - 3 Days In Office)About Us At Relevance AI, we're building the home of the AI workforce.Our mission is simple: empower every team to delegate meaningful work to AI agents that think, act, and collaborate like experts.With Relevance AI, anyone can create and manage intelligent agents that handle workflows, decisions, and...

  • Senior AI Engineer

    2 weeks ago

    Sydney, New South Wales, Australia Helia Full time $120,000 - $200,000 per year

    Helia, as Australia's first Lenders Mortgage Insurance (LMI) provider, with over 50 years expertise in the Australian housing market are in the process of re-inventing ourselves.We exist to accelerate financial wellbeing through home ownership, now and for the future. Our mission  is to create innovative and tailored solutions in partnership with our...

  • Senior AI Solution Engineer

    2 weeks ago

    Sydney, New South Wales, Australia Fyndr Group Full time $180,000 - $275,000 per year

    Senior Solution Engineer – Applied AI Systems Salary: $180,000 to $275,000 + share options About the Role This senior position suits an accomplished engineer who can lead the end-to-end delivery of AI solutions. You will design system architecture, guide technical direction and mentor others while staying hands-on across build and integration work. The...

  • Senior Solutions Engineer

    2 weeks ago

    Sydney, New South Wales, Australia Relevance AI Full time $1,200,000 - $2,000,000 per year

    Location : Sydney, Australia (Hybrid - Travel to customer office)About Us At Relevance AI, we're building the home of the AI workforce.Our mission is simple: empower every team to delegate meaningful work to AI agents that think, act, and collaborate like experts.With Relevance AI, anyone can create and manage intelligent agents that handle workflows,...

  • Senior AI/ML Engineer

    5 days ago

    Sydney, New South Wales, Australia Preacta Recruitment Full time

    Senior AI/ML Engineer - (GenAI) 200k + Super + Bonus Join a leading Australian financial services innovator transforming the way people access credit and manage finance. The company is known for using data and technology to deliver flexible, fair, and customer-first financial solutions. About the Role As a Senior AI/ML Engineer, you will own the...

  • Senior AI Engineer

    5 days ago

    Sydney, New South Wales, Australia Motion Recruitment Full time

    The CompanyI'm working with one of the most exciting new companies in the Australian startup ecosystem. They doing a lot more than just riding the AI hype wave; they are building a product with deep utility that has already generated massive demand.They are looking for a Senior Software / AI Engineer who is working day-to-day with Generative AI. You will be...