# 10 · APK harvest — reproduce from scratch

> A standalone, end-to-end guide so an **independent third party** can reproduce the
> APK harvest ([`07`](07-apk-harvest-and-identification.md)), the integration matrix
> ([`08`](08-integration-matrix.md)) and the live km-range + kill-switch audit
> ([`09`](09-active-config-km-killswitch.md)) from nothing. It lists prerequisites
> with the **exact versions used**, the precise commands (one-shot **or**
> step-by-step), the **expected outputs**, and how to **verify** the result against
> the published checksums. Where output can legitimately vary, it says so.

This complements the deeper techniques in
[`../methodology-and-reproduction.md`](../methodology-and-reproduction.md) (decompile,
GPS spoofing, HTTPS MITM); here we focus only on harvest → identify → scan → read
config → verify.

---

## 1. Prerequisites

| Need | Why | Version used here |
|---|---|---|
| **Rooted Android device**, USB debugging on | Root is required **only** to read each app's private Crowd Connected DB/prefs (step 5). APK pulling (steps 2–4) needs **no** root. | Pixel 10a, Android 16, Magisk root |
| The Appmiral apps **installed** on it | You can only harvest what's installed; only **launched** apps have a live config. | 140 apps |
| **Android platform-tools** (`adb`) | Enumerate packages, pull APKs, pull DBs. | adb 1.0.41 |
| **Android build-tools** (`aapt2`) — optional | Cross-check badging/labels. | build-tools 36.1.0 |
| **Python 3.11+** | Runs the scan + config reader. | Python 3.11.9 |
| **androguard** | Robust APK manifest/DEX analysis. | androguard 4.1.4 (`pip install androguard==4.1.4`) |
| **sqlite3** | Decode the Crowd Connected Room DB. | Python stdlib, SQLite 3.45.1 |
| **PowerShell 7** (`pwsh`) | Runs the `.ps1` harvester/orchestrator. | Windows 11 |

The exact versions for this capture are recorded **programmatically** in
[`../../evidence/appmiral-apks_2026-06-28_154246/logs/tool-versions.txt`](../../evidence/appmiral-apks_2026-06-28_154246/logs/tool-versions.txt)
and [`../../tools/apk-harvest/logs/tool-versions.txt`](../../tools/apk-harvest/logs/).
Confirm the device is connected:

```powershell
adb devices            # must list exactly one "<serial>   device"
adb shell su -c id     # must print uid=0(root) — needed for step 5 only
```

The reusable, documented scripts live in
[`../../tools/apk-harvest/`](../../tools/apk-harvest/): `harvest-apks.ps1`,
`scan-integrations.py`, `read-cc-config.py`, and the one-command `run-all.ps1`.
None hard-codes a device serial — it is resolved from `adb devices` (override with
`-Serial` / `--serial`).

---

## 2. One command (does everything)

```powershell
pwsh tools/apk-harvest/run-all.ps1 `
  -PrivateDir <big/private location for APK + DB bytes> `
  -DataDir    docs/appmiral-comparison/data
```

This runs harvest → scan → config-read, then generates the private
`SHA256SUMS.txt` byte manifests. Heavy bytes go to `-PrivateDir`
(`apks_<stamp>/`, `cc_dbs_<stamp>/`); the lightweight CSV/JSON tables go to
`-DataDir`. To run the phases individually (or re-run just one), use the steps
below.

---

## 3. Step by step

### Step 1 — Enumerate & reconcile

```powershell
adb shell pm list packages -3        # third-party (user-installed) packages
```

Filter to the Appmiral namespace (`appmiral`, `appstrakt`, `mysticfestival`,
`triple.verstappen`). The harvester does this for you and writes the raw list to
`logs/appmiral_packages_<date>.txt`. **Expected:** ~140 packages.

### Step 2 — Harvest the APKs (no root)

```powershell
pwsh tools/apk-harvest/harvest-apks.ps1 `
  -OutDir      <private>/apks_<stamp> `
  -ManifestCsv docs/appmiral-comparison/data/apk_files_manifest_<date>.csv
# optional: -Serial <id>  -Filter <regex>  -Force (re-pull)
```

For each package it resolves every split with `adb shell pm path` and `adb pull`s
the exact installed bytes into `apks_<stamp>/<pkg>/`. Idempotent: already-pulled
packages are skipped (use `-Force` to re-pull).

**Expected output** — `apk_files_manifest_<date>.csv`, one row per file:

```
package,versionName,versionCode,file,size_bytes,sha256,on_device_path,captured_ts
com.appmiral.ttfestival,"2.x.x",N,base.apk,<bytes>,<sha256>,/data/app/~~…/base.apk,2026-…
```

~**375 files / ~7.08 GB** for 140 apps (varies with your app set/versions).

### Step 3 — Identify + integration scan

```powershell
python tools/apk-harvest/scan-integrations.py `
  --apks <private>/apks_<stamp> `
  --out  docs/appmiral-comparison/data/integration_matrix_<date>
```

Writes three files: `…csv` (app × integration matrix), `….identify.csv` (label,
versions, minSdk/targetSdk, ABIs, location perms), `….json` (every hit **with the
matched byte-pattern**, for audit).

**Expected:** baseline integrations (Instagram, Firebase, Fused Location) at ~140/140;
the egress stack (Cognito, Kinesis Firehose) at ~139/140; variable-adoption ones
lower (Permutive ~36, Mapbox ~16); ~19 advertised partners at 0. All 140 request
fine+coarse location, ~30 add background.

> **Silence androguard's logging** if you adapt the script: it floods loguru DEBUG.
> The scanner already calls `logger.remove()` +
> `logging.getLogger("androguard").setLevel(ERROR)` before importing any APK.

### Step 4 — Read the live km-range + kill-switch (root)

```powershell
python tools/apk-harvest/read-cc-config.py `
  --serial  <id> `
  --workdir <private>/cc_dbs_<stamp> `
  --out     docs/appmiral-comparison/data/cc_live_config_<date>
```

For every installed app it checks (root) for
`databases/CrowdConnected-RoomDatabase`, copies it (+ `-wal`/`-shm`) and
`shared_prefs/CrowdConnected.xml` off the device, then decodes — tolerant of both
SDK schemas:

- **new SDK** — `Settings.active` column (`1`=ON), `GeoZone.radius2` (outer km),
  `radius1` (inner ring), centre lat/lng;
- **legacy SDK** — the `activeAPIResponse=<poll>,<active>,…` heartbeat in the prefs
  XML (`…,false,…` = OFF).

Identifiers are **redacted** (deviceId → salted prefix; alias values dropped). The
raw DBs/prefs stay in `-workdir` (**private** — they contain identifiers).

**Expected:** a row per app with a populated config. In this capture, 6 apps —
**`com.appmiral.ttfestival` ON @ 60 km**, the rest OFF but armed at 40–80 km. See
[`09`](09-active-config-km-killswitch.md).

---

## 4. Verify

The public snapshot folder
[`../../evidence/appmiral-apks_2026-06-28_154246/`](../../evidence/appmiral-apks_2026-06-28_154246/)
carries the checksums:

```bash
# 1. the public artifacts (tables, logs, the APK manifest itself)
cd evidence/appmiral-apks_2026-06-28_154246 && sha256sum -c SHA256SUMS.txt

# 2. the actual APK bytes (in the private repo) against the public manifest
cd <private>/apks_2026-06-28_154246 && sha256sum -c /path/to/apk-manifest.sha256
```

`apk-manifest.sha256` (public) is **byte-for-byte identical** to
`apks_2026-06-28_154246/SHA256SUMS.txt` (private), so the public repo attests
exactly which APK bytes exist without re-hosting them. Regenerate any manifest with
[`../../tools/gen-sha256sums.ps1`](../../tools/gen-sha256sums.ps1).

---

## 5. What should — and shouldn't — match

| Output | Should match byte-for-byte? | Why |
|---|---|---|
| **APK SHA-256s** | ✅ yes, **if** the same app versions are installed | APKs are the exact Play-delivered bytes; a Play update changes versionCode → new hash. |
| **Integration matrix** | ✅ largely | Same APKs → same static signatures. Differences mean a different app/version set. |
| **Live km range / kill-switch** | ❌ **not necessarily** | Config is **server-pushed** and changes over time; the operator can flip the switch or change radii at any moment. |
| **Which apps have a config** | ❌ no | Only **launched** apps have a cached DB; your launch history differs. |

A different live switch state is **not** a failed reproduction — it confirms the
key finding that activation is a **remote, server-side** decision. Capture your own
evidence header (device, build, wall-clock) so your run is independently dated, the
same way this one is.

---

## 6. Determinism caveats (summary)

- **Server config drift** — re-running later may show different radii / ON-OFF.
- **Obfuscation** — R8 can strip a literal SDK package name → static **false
  negatives**; corroborate location apps at runtime (step 4).
- **Launch state** — installed-but-never-opened apps have no Crowd Connected DB yet.
- **Locale/encoding** (Windows) — write files UTF-8 no-BOM and format coordinates
  with InvariantCulture; see the gotchas in
  [`../methodology-and-reproduction.md`](../methodology-and-reproduction.md) §3.3.