The Prompting Company

Benchmark report

Onboarding verified

4m 13s elapsed

The saved attempt met the documented success criterion.

Experience score

100/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • SDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs S, Current SSInstallSAuthSExecuteSDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Only the round trip to obtain the TinyFish API key; everything else (install, docs read, run) succeeded first attempt in roughly 90 seconds.

Where the time went

0:05 on the owner · 4:08 agent time
Waited on the owner 0:05
  • Starting0:12
  • Research0:26
  • Attempt2:21
  • Verify1:09
4m 13sTotal elapsed
1:27Time to first success
0:05Waited on the owner

Judge reason

Saved evidence proves a real TinyFish Agent run, not prose or a mock. transcript.jsonl idx 15 shows `pip install tinyfish` succeeding (tinyfish-0.8.1 + deps). Idx 34 runs run_quickstart.py, which calls the real SDK `TinyFish(api_key=...).agent.stream(url='https://scrapeme.live/shop', goal='Extract the first 2 product names and prices. Return as JSON.')`. Idx 37 output (EXIT=0) shows the live stream: STARTED -> STREAMING_URL (server-issued https://ip-13-57-57-237.tetra-data.production.tinyfish.io/tf-5b63dd32.../stream/0) -> two PROGRESS events -> CompleteEvent with status=<RunStatus.COMPLETED: 'COMPLETED'>, error=None, run_id b6809c7c-4f1c-4892-8e1b-5e59be46b57a, result_json={'result': [{'name': 'Bulbasaur', 'price': '£63.00'}, {'name': 'Ivysaur', 'price': '£87.00'}]}. Server-side run_id, streaming URL and monotonic server timestamps (21:35:44 -> 21:35:54) rule out fabrication. evidence/quickstart-evidence.txt and evidence/transcript.txt match that output byte-for-byte, and home/result.json holds the two name+price products. No API key exposed: the key was read from the helper-supplied file path, never echoed, and the agent's grep -rqF leak check reported 'leak check: clean' (idx 43); my own scan of artifacts found only the docs placeholder 'sk-tinyfish-'.

Attempt record

  1. 2:08Asked the owner for TinyFish API key (TINYFISH_API_KEY)waited 5s · provided
  2. —First success verified

Attempt artifacts saved.