About Artificial Analysis
Artificial Analysis is the leading independent benchmarking company for AI models, inference providers and hardware. Our benchmarks are the go-to reference for AI labs including OpenAI, Google, Meta, NVIDIA and Anthropic, and are cited by the Wall Street Journal, Bloomberg, the Financial Times and The Economist. We’re a growing team across San Francisco and Melbourne, on track to double, backed by Nat Friedman, Daniel Gross, Andrew Ng, Adam D’Angelo and Clem Delangue.
About the role
You’ll build core parts of the benchmarking stack, working closely with our founders and taking projects from concept to shipped.
What you’ll do
Ship production backend systems, data pipelines and APIs that benchmark AI models
Work with LLM APIs (OpenAI, Anthropic and others): rate limits, batching, failure handling and structured outputs
Build a simple working version first, then scale it with cloud infrastructure and monitoring
Run custom analyses for enterprise customers and present the results clearly
About you
3+ years as a professional software engineer shipping production backend systems, data pipelines and APIs
Strong Python, including data libraries (pandas or similar), with clean, well-structured code
Hands-on production experience with LLM APIs
Pragmatic: you get a v0 working, then scale; comfortable with cloud infrastructure and monitoring
Use AI coding tools daily, and know how to steer them and critically review their output
Can explain clearly what you personally built and the trade-offs you made
Nice to have: PyTorch or other ML frameworks; model training, inference or evaluation work; analytical report writing
Why join
Shape how AI gets built: leading labs track our benchmarks to guide their development priorities
Become a world expert: you’ll evaluate every major model across every major capability as it’s released
Join early: the people who join now will shape the product, the team and the strategy
Competitive salary plus equity
Location and work rights: Melbourne office, at least 3 days a week. You must have full working rights in Australia.