Founding Engineer: Reinforcement Learning Environments, LLM Evals & Benchmarks | San Francisco On-site | Up to $260K + Equity

"In God we trust. All others must bring data." W. Edwards Deming

Here is exactly what this role is, exactly who it is for, and exactly what it pays. No guessing required.

THE COMPANY

An early-stage AI agent company in San Francisco that is growing fast. Their agent platform runs inside real companies, handling ops, go-to-market and engineering work. They also build reinforcement learning environments, evals and benchmarks for frontier AI labs, using the real agent failures they see in production. They are hiring two Founding Engineers to own that work.

THE JOB, IN SEVEN LINES

• Build reinforcement learning environments from scratch, including full simulated companies with generated email, records and files that an agent works inside for weeks
• Build mock software (inboxes, CRMs, internal tools) that survives real agent tool calls
• Mine production agent traces, separate real failures from noise, and turn them into benchmarks a lab will accept
• Write graders that hold up when the model knows more than the reviewer, and harden them against reward hacking
• Validate that benchmark rankings match how agents actually perform in production
• Own each lab contract end to end: scope it, build it, ship it
• Work directly with the founders on what makes an environment great or useless

MUST HAVE: THE DEALBREAKER

You have personally built evals, benchmarks, reinforcement learning environments or training datasets for large language models or AI agents. Industry, a research lab or an internship all count. Running someone else's benchmark does not.

MUST HAVE: THE BASELINE

• 1 to 5 years of engineering or research experience, including real work at an established organization (not internships only). Exceptional new grads with serious lab or research experience will be considered.
• Experience at an AI lab, an AI data or reinforcement learning environment company, an AI agent startup, or a top research group
• A computer science, math or related degree from a top-30 university
• Strong Python and hands-on large language model experience
• AI-forward: you use AI coding tools every day and have real opinions on agent products

NICE TO HAVE

• You found a real flaw in a widely used benchmark
• You caught a model reward hacking your grader, and fixed it
• You have been a founder, or gone through YC, South Park Commons, the Thiel Fellowship or similar
• A competitive spike: math, programming, debate or games competitions, or high-level athletics

NOT A FIT

• Resistant to AI-assisted coding or new tools
• Hostile toward specific AI labs (they are the customer)
• Several short stints in a row without clear reasons
• Background mainly in bio or life sciences
• Would need to relocate from overseas

COMPENSATION

• Base salary: $150K to $260K, depending on level
• Profit share: a large share of the profit, after costs, on every lab deliverable you ship
• Meaningful founding equity
• Lunch and dinner covered

WORK SETUP

• On-site in San Francisco, Monday to Friday, arriving between 9 and 11am
• Expect some work on one weekend day; the other is yours
• Bay Area candidates preferred, no relocation support
• Visa transfers welcome (H-1B transfer, OPT)

HOW TO APPLY

Apply with your resume and answer the screening questions. Strong profiles are reviewed fast. Be ready to walk through an eval, benchmark or reinforcement learning environment you built, and explain how you knew it measured the right thing.

Big Wave Digital is recruiting this role on behalf of our client.

Plata: $150k - $260k per year

Izvor: https://www.linkedin.com/jobs/view/4485197066/

Nivo: mid level