Free baseline, read-only
Get a free credit-farming baseline.
For self-serve AI products with a meaningful free grant and enough signup volume to observe abuse within weeks. Two small scripts run on your machine against your own exports. Nothing is enforced during the analysis.
- Who it is forYou offer self-serve signup with a free grant that has real variable cost (credits, inference, API calls).
- What you send
signups.csv(user_id, signed_up_at, email_domain, ip, credits_granted) andusage.csv(user_id, used_at, credits_used), optionalconversions.csv. Timestamps ISO orYYYY-MM-DD HH:MM:SS. - What you get backA measured dollar figure on flagged farming cohorts (share of total burn, 24-72h burn), cluster breakdown (shared email domain / IP prefix / burst windows), and conversion contrast. Rounded aggregates only in
summary.json.
The guarantees
Nothing leaves your system.
Trust before you run anything. The scripts are offline by construction, print aggregates only, and ship with public checksums you can verify.
Offline by construction
No HTTP, no sockets, no DNS. Standard library only. Run it in a sandbox with the network unplugged: it works, and your security team can prove it.
Aggregates only
The scripts print group counts and rounded dollar values. No emails, no IPs, no user IDs are written to the output: verify by reading the ~200 lines, or pre-hash anything before export.
Checksummed and public
Identical copies live on GitHub and this page, with published sha256 hashes. Diff the emailed copy against the public one before running.
What the comparison looks like
Same abuse stopped. Fewer false blocks.
Numbers below are from the published example/ fixture, the same files you can download and re-run. Not a customer cohort. Your baseline returns this shape on your data.
Your current rules
funnel-blind decisions
- False blocks
- 3
- Abuse dollars missed
- $450
- Saved per false block
- $440
Outcome-tuned decisions
joined to burn and conversion
- False blocks
- 1
- Abuse dollars missed
- $150
- Saved per false block
- $1,320
- Abuse stopped5 of 7 events, both sides
- False blocks removed2
- $/false-block lift+$880
Source: example/summary.json · events.csv. Reproduce with python3 outcome-backtest.py --events example/events.csv.
Run it
An hour, one export, one number.
Baseline first. If the figure is material, compare your current block decisions against ones joined to credit burn and conversion. If it is not, we stop.
Signups and usage CSV
user_id, signed_up_at, and metered usage, from your own warehouse, in whatever shape you have. Conversions optional but make the contrast stronger.
The baseline
python3 farming-baseline.py --signups signups.csv --usage usage.csv --credit-cost-usd 0.002 --out summary.json
The backtest
python3 outcome-backtest.py --events events.csv --out summary.json: same event IDs, decisions joined to outcomes.
30 minutes, if material
Walk the cohort breakdown together. An honest "not enough pain" result is useful. A material number starts a short design-partner conversation, still in monitor mode.
The two tools
Baseline first, then the comparison.
The baseline puts a dollar figure on automated activity consuming your free tier. The backtest answers the follow-up: would decisions joined to your outcomes beat your current rules?
farming-baseline.py / .mts
How much automated activity costs you
Cluster analysis on your signups, usage, and conversion exports, looking for patterns such as accounts that look alike, arrive together, and burn credits fast. Output: a dollar figure priced at your cost basis.
outcome-backtest.py / .mts
Outcomes vs. funnel-blind rules
Compare your current block decisions against ones joined to credit burn and conversion on the metric that matters: dollars saved per false block. Same abuse stopped, fewer paying customers blocked.
example/ · README.md
Read every line first
Synthetic datasets with reference output, full CSV schemas, and per-file sha256 checksums. Your engineers can audit the scripts in ten minutes, or skip them and reproduce the math themselves.
Verify the files
Checksums, published and current.
Run shasum -a 256 <file> (macOS) or sha256sum (Linux) and compare against these values and the GitHub mirror.
| File | sha256 | What it is |
|---|---|---|
| farming-baseline.py | 0deedbbef0c94009fab291a94f4cf74e4239b48d073ac9a3b6219f523a0e66f4 | Baseline: Python (3.9+) |
| farming-baseline.mts | f6e26a91df84638bbf37a3ecce5c30c9c8aaaf3edb6d969a34caf53f4e290263 | Baseline: TypeScript (Node ≥ 24) |
| outcome-backtest.py | 0b0bb67d090064b9e45a509825c8ea42cd150d3eb73a4e52280b823fe9ddc9d4 | Backtest: Python (3.9+) |
| outcome-backtest.mts | 652b309c682f836f12d360d0da50bd147728199544183efa79c8a60c1a15a19d | Backtest: TypeScript (Node ≥ 24) |
The example datasets and reference outputs are in the example/ folder shipped with the scripts: run the scripts on them before touching real data.
Next step
Run it on your data. Get your own comparison.
The fixture above is the shape of the result. Your exports fill in the numbers. When the comparison is on the table, book a 30-minute review. If it is not material, we stop. No obligation either way.