United States
Staff Technical Product Manager - Evals
Hercules · United States
- Salary
- $200,000–$350,000 per year
- Contract
- Full time
- Occupation
- Product Manager
Advertised in 2 locations:
- San Francisco, California, United States
- United States
Posted: 10/21/2026 · Last checked: 11/17/2026
Job description
Quoted in full from Hercules's job posting. Gavroche does not edit it.
What are we looking for?
Hercules is looking for a senior or staff level product manager to own evals for our AI coding agent
What will you do?
Your #1 and only goal is to make Hercules' AI coding agent cheaper, faster, and more accurate (our pareto frontier)
You will own Hercules Bench, our offline eval set
You will own Hercules online evals
You will leverage our 500k+ users, 1M+ code repos, and millions of chat threads to guide your eval creation
You will anonymize data to protect our customer's privacy
What skills do we expect you to have?
We expect candidates to be incredibly strong across all of the following areas:
Software fundamentals: You are building an AI coding agent. We expect you to be technical and able to code. We expect you to have a decent understanding of system design, architecture, data modeling, and API design. In saying this, you will work with an engineering counterpart to work on complex implementation
Evals. You have explicitly created and/or managed technical evals before. You should be able to work across a variety of different eval methodologies and deeply understand optimal reward modeling,
Product sense: strong intuition for the goals customers are trying to achieve and novel solutions to help them achieve them
Design sense: Track record of building simple, delightful UX/UI that makes it easy for customers to achieve their goals
AI context and harness engineering: strong experience with the most up to date context and harness engineering best practices like summarization/compression, prompting, prompt caching, subagents, memory, context management, tools, async tools, deferred tools, skills, etc
Yes, we know this industry is evolving rapidly and the "best practices" change every week, but yes, we are still actively looking for someone who has wrestled with these challenges at scale and has strong opinions about what works and what doesn't
Our core stack: TypeScript, React, (some sort of agent framework like AI SDK), (some sort of eval platform like LangSmith)
AI: able to truly 10x your workflow by being on frontier of AI coding
What’s Hercules’ stack?
Technologies: Typescript, React, AI SDK, Postgres, Redis, Clickhouse, Tailwind, IaC (CDK, Terraform), Docker, Otel, Hono, Drizzle ORM, Temporal
Clouds: Cloudflare, AWS
AI Models: OpenAI, Anthropic, Google
What are Hercules’ operating principles?
Hercules’ operating principles guide how we make decisions every day. We take them very seriously. Most of the principles can be learned on the job, however, we do specifically filter for the following principles in our hiring process:
We ship. We ship fast, iterate, and correct mistakes quickly. Our bias for action speeds up our learning and delights our customers.
We strive for excellence. Doing things well is in our DNA. We have good taste. We get the details right. Our outputs are exceptionally simple and self-serve. We’re proud of our work.
We are hungry. We are willing to work hard and break through walls to achieve the outcomes we want. We are willing to operate at any level to get things done.
Is Hercules in-office or remote?
Hercules founding team works in-office in San Francisco (Kearny and Bush). We have a very strong preference for in office. We are open to remote for exceptional candidates who are willing to 1) work in US time zones and 2) travel to the office regularly.
What are Hercules’ benefits?
Unlimited AI token budget
Take what you need vacation policy
Healthcare, vision, and dental
Daily lunches, dinners and snacks in the office
We guarantee you will do the best work of your career here
What’s the compensation?
$200k-$350k + equity (variable based on seniority, equity, and location)
From Gavroche data
Figures computed from live adverts on Gavroche; the job text above is the employer's.
Similar roles in San Francisco
- Product Manager · Convex, San Francisco
- Product Manager - Connections · Notion, San Francisco
- Inference Engineering and Product Lead · Modal, San Francisco
- Product Manager · Exa, San Francisco
- Product Manager - Forward Deployed · Exa, San Francisco
- Head of Product Marketing · Exa, San Francisco
200 open roles like this one; at most three per employer shown.
What product manager adverts in San Francisco state
Middle half: 185,925 – 257,000 USD a year; median 215,375 USD.
From 64 adverts by 36 employers that state a salary, annualised.
How product manager roles in the United States are worked
1% fully remote · 0% hybrid · 99% do not say
Of 226 adverts. An advert that does not state a work mode is counted as not saying, not as on-site.
New openings in San Francisco, by week
- 8/24/2026 323
- 8/31/2026 415
- 9/7/2026 365
- 9/14/2026 446
- 9/21/2026 515
- 9/28/2026 623
- 10/5/2026 574
- 10/12/2026 683
- 10/19/2026 562
- 10/26/2026 562
- 11/2/2026 602
- 11/9/2026 554
- 11/16/2026 44
6,268 openings over 12 weeks, by the week they were published.
Other roles at Hercules
- Head of Sales - Technical · San Francisco
- Senior Technical Account Executive - SMB · San Francisco
- Senior Technical Account Executive - Mid-market · San Francisco
- Social Media Manager · San Francisco
- Head of Technical Recruiting · San Francisco
- Creator Partnerships Lead · San Francisco
10 open.