Member of Technical Staff

Location:

US (on-site in San Francisco), Australia (Hybrid)

About Artificial Analysis

Artificial Analysis is an independent AI benchmarking and insights company. We support engineers and companies to understand AI capabilities and make critical decisions about their AI strategies. 

Our public benchmarks are trusted by tens of thousands of users every week and have been cited by publications from
TechCrunch
to the
Wall Street Journal
to the All-In Podcast (E165:
52:00 to 1:12:00
; E167:
28:20 to 31:30
). Our work is regularly cited by leading AI companies, with recent examples including
NVIDIA
,
Meta
,
Amazon
and
Bytedance
. 

We are backed by Nat Friedman (Github, Meta), Daniel Gross (SSI), Andrew Ng (Google Brain, , Amazon), Adam D'Angelo (Quora, Poe, OpenAI), Clem Delangue (Hugging Face) and other industry leaders.

The Opportunity

We're looking for someone to join our team and lead projects in AI benchmarking and analysis. You'll work closely with our team and founders on product development, including developing new evaluation techniques, and analysis of AI to support companies in leveraging AI.

This will be a technical role where the bar for success will be becoming a world expert in modern AI technologies - but with more of a generalist scope than our
Software Engineer positions
. The key responsibilities won't be about shipping code or developing our main benchmarking stack - but you might be writing code to run experiments or build out a new technique for testing a particular AI capability. 

The coming wave of AI scaling is going to change the world in ways we don't yet understand - and we're offering a front row seat. 

What You'll Do

• AI Benchmarking Product Development:

Structure, design and execute projects to evaluate AI systems and technologies, including developing new AI evaluation methodologies and datasets that advance our benchmarking capabilities
- Strategic Analysis of AI:

Drive developing reports and data visualizations to communicate complex AI concepts to enterprises looking to shape their AI strategy
- Partner with Leading AI Companies:

Collaborate with leading AI companies to support them in benchmarking their technologies, from agentic AI applications to models to hardware. This also requires identifying opportunities to enhance our leading AI benchmarking platform and working with our developers to make it happen
- Become AI-Native:

Embrace an AI-native workflow, using cutting-edge AI tools to generate leverage in a fast-changing industry and maintain our competitive edge in AI benchmarking
- Company strategy:

We are a startup and are looking for the future leaders of Artificial Analysis. Team members will be expected to contribute to company strategy and should expect to drive large initiatives that will support us in being the leading AI benchmarking company

What We're Looking For

Required:

• Strong analytical and critical thinking skills
• Excitement about AI and eagerness to work at the forefront of technological innovation
• Demonstrated excellence, both academically and professionally

Beyond that, this opportunity could potentially suit a wide range of backgrounds. We believe that the following skills and experience would likely be helpful for success in this role. We're not expecting any single candidate to tick off the entire list, but these are some things that we think would be valuable:

• Experience in strategy consulting
• Academic success in a technical degree programme, especially related to AI (e.g. Computer Science, Engineering), or in related technical field (e.g. Physics, Maths)
• A strong perspective on AI scaling laws
• Experience with AI/ML research, especially any academic research directly relating to modern language models
• Publication record in AI/ML conferences or journals (or more broadly any publication record, including non-technical or non-traditional)
• Familiarity with modern AI language models (e.g. developing systems leveraging LLMs as a component)
• Experience working in software engineering or data science
• Proficiency in Python and data analysis libraries (e.g., pandas, numpy, matplotlib); experience with AI/ML frameworks (e.g. PyTorch, Tensor Flow)
• Strong writing skills, especially in relation to explaining technical topics to both technical and non-technical audiences
• Visual communication skills (e.g. making slides, visualizing data)

Why Artificial Analysis?

• Work with frontier AI technologies and collaborate with companies building the future of AI
• Make a significant impact in a fast-growing startup
• Competitive compensation package (including potential for equity)
• We offer flexible work arrangements, including remote and hybrid options. Our team is currently split between San Francisco, Sydney, Melbourne.

If you're excited about the challenge of building the future of AI benchmarking and analysis,

we'd love to chat. 

• To apply for the role, please email
  
  with (1) your CV (and LinkedIn URL), (2) a brief note on why you're excited about joining Artificial Analysis, and (3) a two sentence description of your favorite AI paper or industry development from the last 6 months.