# Manual benchmark commands — runs 5, 6, 7

Runs 1-4 used `benchmark-runner.py` (preserved in this folder). Runs 5-7 were executed manually to work around problems the runner had with multi-day SSH connections. This file documents the exact commands so anyone can reproduce.

## Prerequisites

- Test server reachable via `ssh benchmarking` (alias for `root@<server>`)
- WP-CLI as user `tests46427` with the site at `/var/www/tests.superspeedyplugins.com/htdocs/`
- Hetzner snapshot `benchmark-ready-empty` (image_id used in tests: 390805640) — clean state with both import configs (SSI id=2, WPAI id=2) pointing at `shirt-test-50k-with-images.csv`
- CSV file `shirt-test-50k-update.csv` already uploaded to `/var/www/.../uploads/super-speedy-imports/`
- Helper PHP files on the server in `/tmp/`:

```bash
# Upload helper scripts to the server
cat > /tmp/count-products.php << 'EOF'
<?php
require_once("/var/www/tests.superspeedyplugins.com/htdocs/wp-load.php");
echo wp_count_posts("product")->publish;
EOF

cat > /tmp/count-products-all.php << 'EOF'
<?php
require_once("/var/www/tests.superspeedyplugins.com/htdocs/wp-load.php");
$c = wp_count_posts("product");
foreach ($c as $s => $n) { if ($n > 0) echo "$s\t$n\n"; }
EOF

scp /tmp/count-products.php /tmp/count-products-all.php benchmarking:/tmp/
```

## Snapshot operations

Replace `<TOKEN>` with the Hetzner project API token; `<IMAGE_ID>` with the snapshot ID.

```bash
# Restore snapshot
curl -s -X POST -H "Authorization: Bearer <TOKEN>" -H "Content-Type: application/json" \
  -d '{"image": <IMAGE_ID>}' \
  "https://api.hetzner.cloud/v1/servers/<SERVER_ID>/actions/rebuild"

# Wait for SSH to come back
until ssh -o ConnectTimeout=5 benchmarking "echo OK" 2>/dev/null; do sleep 10; done

# Create snapshot
curl -s -X POST -H "Authorization: Bearer <TOKEN>" -H "Content-Type: application/json" \
  -d '{"description":"<SNAPSHOT_NAME>","type":"snapshot"}' \
  "https://api.hetzner.cloud/v1/servers/<SERVER_ID>/actions/create_image"

# Wait for snapshot to become available
until curl -s -H "Authorization: Bearer <TOKEN>" "https://api.hetzner.cloud/v1/images/<NEW_IMAGE_ID>" \
  | python3 -c "import sys,json; s=json.load(sys.stdin)['image']['status']; print(s); exit(0 if s=='available' else 1)"; do
    sleep 30
done
```

---

## Run 5: WPAI 50k fresh with images

Started 2026-05-27 02:02:35 UTC, finished 2026-05-29 20:01:44 UTC = **66 hours wall clock**.
50,000 products created. Disk delta: +50.5 GB.

```bash
# 1. Restore clean baseline (image_id=390805640)
curl -s -X POST -H "Authorization: Bearer <TOKEN>" -H "Content-Type: application/json" \
  -d '{"image": 390805640}' \
  "https://api.hetzner.cloud/v1/servers/<SERVER_ID>/actions/rebuild"

# 2. Wait for SSH and verify clean state
until ssh -o ConnectTimeout=5 benchmarking "echo OK" 2>/dev/null; do sleep 10; done
ssh benchmarking "sudo -u tests46427 php /tmp/count-products.php"  # expects 0

# 3. Start sar telemetry in background on the server
ssh benchmarking "nohup sar -u -r -d 5 > /tmp/sar-wpai-50k-fresh.log 2>&1 & disown"

# 4. Run WPAI import id=2 in background on the server (avoids SSH timeout)
ssh benchmarking "nohup sudo -u tests46427 wp all-import run 2 \
  --path=/var/www/tests.superspeedyplugins.com/htdocs/ \
  > /tmp/wpai-50k-fresh.log 2>&1 < /dev/null &"

# 5. Poll progress (in a separate window or loop)
while true; do
  ssh benchmarking "sudo -u tests46427 php /tmp/count-products.php; \
    ps aux | grep 'all-import' | grep -v grep | wc -l"
  sleep 3600  # check hourly
done

# 6. When complete (process gone), collect data
ssh benchmarking "pkill -f 'sar -u' 2>/dev/null"
scp benchmarking:/tmp/sar-wpai-50k-fresh.log .benchmark-data/sar-wpai-50k-fresh-images.log
scp benchmarking:/tmp/wpai-50k-fresh.log .benchmark-data/

# 7. Export
ssh benchmarking "sudo -u tests46427 wp ssi export /tmp/export-wpai-50k-fresh.csv product \
  --exclude-dates --path=/var/www/tests.superspeedyplugins.com/htdocs/"
scp benchmarking:/tmp/export-wpai-50k-fresh.csv .benchmark-data/export-wpai-50k-fresh-images.csv

# 8. Snapshot the loaded state for later comparison/update runs
curl -s -X POST -H "Authorization: Bearer <TOKEN>" -H "Content-Type: application/json" \
  -d '{"description":"wpai-50k-loaded","type":"snapshot"}' \
  "https://api.hetzner.cloud/v1/servers/<SERVER_ID>/actions/create_image"
# Got image_id 391877612
```

**Data collected**: sar (19MB CPU/RAM/disk over 66h), wall-clock from server start/end timestamps, final product count via PHP helper, full product export CSV. **No HTTP probes** for this run — the runner-based probe collector would have died after the 2h SSH timeout anyway.

---

## Run 6: SSI 50k fresh with images (re-run to disprove environmental cause)

Started 2026-05-29 20:26:44 UTC, finished ~20:43 UTC = **985 seconds (~16.4 minutes)**.
50,000 products created. Disk delta: +1.6 GB.

Purpose: prove that WPAI's 66h wasn't caused by the test server, image-host CDN, or Cloudflare. Same server, same images, same network — SSI completes in 16 min, WPAI takes 66 h. The 240× factor is plugin behaviour, not infrastructure.

```bash
# 1. Restore clean baseline
curl -s -X POST -H "Authorization: Bearer <TOKEN>" -H "Content-Type: application/json" \
  -d '{"image": 390805640}' \
  "https://api.hetzner.cloud/v1/servers/<SERVER_ID>/actions/rebuild"

# 2. Wait and verify
until ssh -o ConnectTimeout=5 benchmarking "echo OK" 2>/dev/null; do sleep 10; done
scp /tmp/count-products.php /tmp/count-products-all.php benchmarking:/tmp/
ssh benchmarking "sudo -u tests46427 php /tmp/count-products.php"  # expects 0

# 3. Start sar
ssh benchmarking "nohup sar -u -r -d 5 > /tmp/sar-ssi-50k-rerun.log 2>&1 & disown"

# 4. Run SSI import id=2 in background on the server
ssh benchmarking "nohup sudo -u tests46427 wp ssi 2 --workers=8 \
  --path=/var/www/tests.superspeedyplugins.com/htdocs/ \
  > /tmp/ssi-50k-rerun.log 2>&1 < /dev/null &"

# 5. Poll until process gone
while ssh benchmarking "ps aux | grep 'wp ssi 2' | grep -v grep" 2>/dev/null | grep -q wp; do
  sleep 60
done

# 6. Collect data (do this BEFORE any subsequent snapshot restore — /tmp gets wiped)
ssh benchmarking "pkill -f 'sar -u' 2>/dev/null"
scp benchmarking:/tmp/sar-ssi-50k-rerun.log .benchmark-data/sar-ssi-50k-fresh-images-rerun.log
scp benchmarking:/tmp/ssi-50k-rerun.log .benchmark-data/

# 7. Export
ssh benchmarking "sudo -u tests46427 wp ssi export /tmp/export-ssi-50k-rerun.csv product \
  --exclude-dates --path=/var/www/tests.superspeedyplugins.com/htdocs/"
scp benchmarking:/tmp/export-ssi-50k-rerun.csv .benchmark-data/export-ssi-50k-fresh-images-rerun.csv

# 8. Snapshot the SSI-loaded state
curl -s -X POST -H "Authorization: Bearer <TOKEN>" -H "Content-Type: application/json" \
  -d '{"description":"ssi-50k-loaded","type":"snapshot"}' \
  "https://api.hetzner.cloud/v1/servers/<SERVER_ID>/actions/create_image"
# Got image_id 391877796
```

**Data collected**: SSI stage-level timings (parsed from `/tmp/ssi-50k-rerun.log`), final product count, full product export CSV. **sar log was LOST** — I restored the next snapshot before downloading it. **No HTTP probes.**

---

## Run 7: WPAI 50k update (currently running)

Started 2026-05-29 21:06:56 UTC. In progress at time of writing.
Runs against the wpai-50k-loaded snapshot (Run 5's final state).
Update file changes: 2,500 removed + 2,500 added + 5,000 changed + 40,000 unchanged.

```bash
# 1. Restore wpai-50k-loaded (image_id=391877612)
curl -s -X POST -H "Authorization: Bearer <TOKEN>" -H "Content-Type: application/json" \
  -d '{"image": 391877612}' \
  "https://api.hetzner.cloud/v1/servers/<SERVER_ID>/actions/rebuild"

# 2. Wait and verify (should show 50000 products from WPAI run)
until ssh -o ConnectTimeout=5 benchmarking "echo OK" 2>/dev/null; do sleep 10; done
scp /tmp/count-products.php /tmp/count-products-all.php benchmarking:/tmp/
ssh benchmarking "sudo -u tests46427 php /tmp/count-products.php"  # expects 50000

# 3. Overwrite the WPAI CSV with the update file
# (WPAI has no --file= flag — it reads the path stored at import-create time;
# we keep the original filename and replace its contents)
ssh benchmarking "cp /var/www/tests.superspeedyplugins.com/htdocs/wp-content/uploads/super-speedy-imports/shirt-test-50k-update.csv \
  /var/www/tests.superspeedyplugins.com/htdocs/wp-content/uploads/wpallimport/files/shirt-test-50k-with-images.csv && \
  chown tests46427:tests46427 /var/www/tests.superspeedyplugins.com/htdocs/wp-content/uploads/wpallimport/files/shirt-test-50k-with-images.csv"

# 4. Start sar
ssh benchmarking "nohup sar -u -r -d 5 > /tmp/sar-wpai-50k-update.log 2>&1 & disown"

# 5. Launch WPAI update
ssh benchmarking "nohup sudo -u tests46427 wp all-import run 2 \
  --path=/var/www/tests.superspeedyplugins.com/htdocs/ \
  > /tmp/wpai-50k-update.log 2>&1 < /dev/null &"

# 6. Poll progress until process gone
while ssh benchmarking "ps aux | grep 'all-import' | grep -v grep" 2>/dev/null | grep -q all-import; do
  ssh benchmarking "sudo -u tests46427 wp all-import list \
    --path=/var/www/tests.superspeedyplugins.com/htdocs/ 2>&1 | grep -v '^PHP' | tail -1"
  sleep 3600
done

# 7. When done, collect (DO THIS BEFORE ANY SUBSEQUENT RESTORE)
ssh benchmarking "pkill -f 'sar -u' 2>/dev/null"
scp benchmarking:/tmp/sar-wpai-50k-update.log .benchmark-data/sar-wpai-50k-update.log
scp benchmarking:/tmp/wpai-50k-update.log .benchmark-data/

# 8. Export
ssh benchmarking "sudo -u tests46427 wp ssi export /tmp/export-wpai-50k-update.csv product \
  --exclude-dates --path=/var/www/tests.superspeedyplugins.com/htdocs/"
scp benchmarking:/tmp/export-wpai-50k-update.csv .benchmark-data/export-wpai-50k-update.csv
```

**Data being collected**: sar (preserved every check), final WPAI metrics from `wp all-import list` (Created / Updated / Skipped / Deleted), full product export CSV after completion.

---

## Reproducing on your own server

To run these benchmarks yourself:

1. Get a Hetzner test server (or any test box you can snapshot)
2. Install WP + WooCommerce + Super Speedy Imports + WP All Import Pro (with WooCommerce Add-on)
3. Generate the CSV files using `super-speedy-imports/.docs/2026-05-21-benchmark-system-v1-spec.md` (Rust generator)
4. Set up the SSI import config from `super-speedy-imports/.benchmark-data/ssi-benchmark-1k-config.json` (adjust for 50k)
5. Manually create the WPAI import via admin UI (no CLI to create) — map the columns per the doc
6. Take the empty snapshot
7. Follow the run-5 commands above for each scenario

**Important**: WPAI doesn't have a CLI flag for the CSV file path. To run an update against a different CSV, you MUST overwrite the file at the path WPAI was configured with. The file path lives in `wp_pmxi_imports.path` if you need to inspect it.

## Notes on data we lost / didn't collect

- **HTTP frontend response-time probes for the 50k runs**: lost (Run 5) or never collected (Runs 6 + 7). This was the wp-admin / category-page response-time-during-import data I had planned. We have it for the 1k runs.
- **SSI 50k re-run sar log**: lost (snapshot restored before download). We have the SSI CLI output with stage timings instead.
- **SSI 50k first run probe data**: lost when runner crashed at 2h SSH timeout. sar from that run survived.

What we DO have, for ALL runs: wall-clock times, final product counts, disk deltas, full product exports, and sar telemetry (CPU / RAM / disk I/O) — minus the SSI re-run's sar.
