# AI SEO Manager — integration guide

How the pieces connect, and the order to connect them in. Written for the
developer deploying this to `dghanalytics.com/aiseo/`.

**Requirements:** PHP 8.1+ with `pdo_mysql`, `curl`, `mbstring`, `dom`, `libxml`,
`openssl` · MySQL 8.0+ or MariaDB 10.6+ · cron access · Node 20+ *only* for the
render service (optional). No Composer required.

---

## 0. The one-page version

```bash
# 1. get the code onto the server
cd ~/public_html && git clone <your-repo> aiseo && cd aiseo

# 2. configure
cp .env.example .env && chmod 600 .env
nano .env                      # DB_* and APP_KEY are the only required values

# 3. verify the environment BEFORE anything else
php bin/doctor.php             # fix every FAIL line, top to bottom

# 4. create the schema
php bin/migrate.php

# 5. point the sites at real domains (the seed uses placeholders)
mysql -u USER -p DBNAME -e "UPDATE site SET primary_domain='dolphinhvac.ae' WHERE division='hvac';"

# 6. smoke test end to end — under ten HTTP requests
php bin/audit.php --list
php bin/audit.php --site=1 --type=canary

# 7. the worker (this is what actually crawls)
crontab -e
* * * * * cd /home/dolrad/public_html/aiseo && /usr/local/bin/php bin/worker.php >> var/logs/worker.log 2>&1
```

`bin/doctor.php` is the important one. It checks PHP version and extensions,
whether `.env` is being read, whether MySQL connects **and why not if it
doesn't**, whether the user can create tables, whether migrations are applied,
whether `var/` is writable, **whether `.env` is readable over HTTP**, which
integrations have credentials, and whether outbound HTTPS works. Run it after
every deploy.

---

## 1. Directory layout and what goes where

```
Every line ending in / is a FOLDER. Every other line is a FILE.
See MANIFEST.md for the complete listing with sizes and purposes.

aiseo/
├── bootstrap.php             the anchor: autoloader + env + error handling
├── .env                      your credentials. chmod 600. never committed.
├── .env.example              template to copy
├── .htaccess                 use when the front controller is at the app root
├── MANIFEST.md               every path, marked folder or file
├── INTEGRATION-GUIDE.md      this document
├── README.md
│
├── public/                   THE ONLY WEB-ACCESSIBLE FOLDER
│   ├── index.php             front controller: all pages + the JSON API
│   ├── styleguide.html       component reference, opens without PHP
│   ├── .htaccess             use when the doc root points at public/
│   └── assets/
│       ├── css/
│       │   ├── tokens.css    design tokens
│       │   └── app.css       components
│       └── js/
│           └── app.js
│
├── config/
│   ├── paths.php             set your folder paths explicitly (optional)
│   └── path-resolver.php     finds your folders; runs pre-autoloader
│
├── src/                      = 3-source-code/src on your server
│   ├── Core/
│   │   ├── Env.php
│   │   ├── Logger.php
│   │   └── Queue.php
│   ├── Db/
│   │   ├── Database.php
│   │   └── Migrator.php
│   ├── Http/
│   │   ├── HttpClient.php
│   │   ├── HttpResponse.php
│   │   ├── Router.php
│   │   └── View.php
│   ├── Providers/            one file per external API
│   │   ├── DataForSeo.php
│   │   ├── GooglePerformance.php
│   │   ├── GoogleOAuth.php
│   │   └── SearchConsole.php     <-- a FILE, not a folder
│   └── Audit/                the engine — 8 files, no sub-folders
│       ├── RobotsTxt.php
│       ├── UrlNormalizer.php
│       ├── PageParser.php
│       ├── Crawler.php
│       ├── AiAccessProbe.php
│       ├── CheckRunner.php
│       ├── Scorer.php
│       └── AuditService.php
│
├── templates/                = 3-source-code/templates on your server
│   ├── layout/
│   │   └── app.php
│   ├── pages/
│   │   ├── dashboard.php
│   │   ├── site.php
│   │   ├── integrations.php
│   │   └── error.php
│   └── partials/
│       └── sidebar.php
│
├── db/
│   └── migrations/           = 3-source-code/db/migrations on your server
│       ├── 001_core.sql      schema, 34 tables
│       └── 002_seed.sql      checks, crawlers, countries, blocklist
│
├── bin/                      CLI only — never reached by a browser
│   ├── doctor.php            run this first
│   ├── migrate.php
│   ├── worker.php            the cron process that crawls
│   └── audit.php
│
├── tests/
│   ├── robots_test.php       33 assertions
│   └── env_test.php          17 assertions
│
├── render-service/           optional Node service, separate process
│   ├── server.mjs
│   └── package.json
│
└── var/                      writable. MUST NOT be web-readable.
    ├── logs/
    └── cache/
```

**20 folders, 51 files.** Names like `SearchConsole`, `RobotsTxt` and `Crawler`
are **files** (`SearchConsole.php` and so on), not directories — `src/Audit/` and
`src/Providers/` have no sub-folders at all. You do not need to create any of
these folders by hand; extracting the zip creates them.

### Your layout — source in `3-source-code/src`

**Paths are not hardcoded.** The package default is `<root>/src`, but your
hosting keeps source at `/aiseo/3-source-code/src`, so `config/path-resolver.php`
resolves every directory in three passes:

1. an explicit value in `.env` (`AISEO_SRC_DIR`, `AISEO_TEMPLATES_DIR`, …)
2. an explicit value in `config/paths.php`
3. auto-discovery — it scans `src`, `3-source-code/src`, `source-code/src`,
   `app/src`, `../3-source-code/src`

Discovery requires a **marker file** (`Core/Env.php` for `src`,
`layout/app.php` for templates, `001_core.sql` for migrations) so it cannot latch
onto an unrelated directory that happens to share a name. Tested against your
exact layout: source, templates, migrations, assets and `var/` all resolve with
zero configuration.

For your box the layout resolves as:

| What | Where |
|---|---|
| root (holds `bootstrap.php`) | `/aiseo` |
| src | `/aiseo/3-source-code/src` |
| templates | `/aiseo/3-source-code/templates` |
| migrations | `/aiseo/3-source-code/db/migrations` |
| public (holds `index.php`, `assets/`) | `/aiseo` |
| var (logs, cache, locks) | `/aiseo/var` — **move this above the web root if you can** |

**On production, set the paths explicitly** rather than relying on discovery.
Discovery is a filesystem scan; an explicit path is faster and unambiguous, and a
wrong one **fails loudly** instead of silently falling back to a directory you
are not editing:

```
AISEO_SRC_DIR=/home/dolrad/public_html/aiseo/3-source-code/src
AISEO_TEMPLATES_DIR=/home/dolrad/public_html/aiseo/3-source-code/templates
AISEO_MIGRATIONS_DIR=/home/dolrad/public_html/aiseo/3-source-code/db/migrations
AISEO_PUBLIC_DIR=/home/dolrad/public_html/aiseo
AISEO_VAR_DIR=/home/dolrad/aiseo-var
```

`bootstrap.php` is the one anchor: `AISEO_ROOT` is the directory containing it.
Put it at `/aiseo/bootstrap.php` and everything else is found from there. The
front controller and every CLI script locate `bootstrap.php` by walking up a few
levels, so `index.php` works whether it sits in `public/` or at the app root.

**Document root.** Two options, in order of preference:

- **Point the document root at a dedicated `public/`.** Source is then not under
  the web root at all, and `public/.htaccess` is the file to use.
- **Keep the front controller at `/aiseo/`** (what you have now). Use the root
  `.htaccess`, which hard-blocks `3-source-code/`, `config/`, `bin/`, `db/`,
  `tests/`, `var/`, plus **every numbered directory** except asset paths — so a
  new `4-something/` folder is private by default rather than public by accident.

Two things I checked against your live site:

- `/aiseo/3-source-code/src/` returns **403** — directory listing is off. Good.
- `/aiseo/3-source-code/` returns **your app page**, which means a catch-all
  rewrite is absorbing unknown paths into the front controller.

That second one is a trap for verification: a source path can return **200 with
your app's HTML**, which looks fine but tells you nothing about whether the file
itself is readable. That is why the block rules in `.htaccess` run *before* the
front-controller rewrite, and why `doctor.php` inspects the response **body** —
it grades 200 + PHP source as a failure, 200 + app HTML as "the rewrite absorbed
it", and 403/404 as properly blocked.

**Verify after deploying.** All four must return 403 or 404:

```
/aiseo/.env
/aiseo/3-source-code/src/Core/Env.php
/aiseo/config/paths.php
/aiseo/var/logs/app.log
```

`php bin/doctor.php` probes all of them and fails loudly if any is readable.

---

## 2. Connection 1 — MySQL (this is what is currently broken)

The banner on your Integrations page says the API is reachable but MySQL
refused the connection. `src/Db/Database.php` turns that into an actionable
message instead of a generic one. The five real causes, in order of likelihood
on cPanel:

| Symptom in `doctor.php` | Cause | Fix |
|---|---|---|
| `Access denied for user` | The MySQL user exists but was never **added to the database** | cPanel → MySQL Databases → *Add User To Database* → grant ALL PRIVILEGES |
| `Unknown database` | Missing the account prefix | The real name is `dolrad_aiseo`, not `aiseo`. Same for the user. |
| `No such file or directory` | PHP is trying a unix socket that doesn't exist | Set `DB_HOST=127.0.0.1` to force TCP, or set `DB_SOCKET` to the real path (`SHOW VARIABLES LIKE 'socket'`) |
| `Connection refused` | Nothing listening | MySQL is down, or `DB_PORT` is wrong, or the DB is on a separate host |
| Credentials look right but still denied | The password contains `#` or a quote and was truncated by the `.env` parser | Wrap it in double quotes: `DB_PASS="p@ss#word"` |

Three things `Database.php` does that a typical `config.php` does not:

1. **`ERRMODE_EXCEPTION`.** PDO's default silent mode is why a broken query
   surfaces three functions later as "call to a member function on null".
2. **`EMULATE_PREPARES = false` with the charset in the DSN.** Emulated prepares
   plus `SET NAMES` is the classic injection hole in otherwise-parameterised code.
3. **`upsertMany()`** — bulk insert with `ON DUPLICATE KEY UPDATE`. The crawler
   writes thousands of rows per run; one round trip per row is the difference
   between a 40-second crawl and a six-minute one.

Everything stores UTC (`SET SESSION time_zone='+00:00'`) and renders in
`Asia/Dubai`. Don't mix that up — audit deltas across a DST-free timezone are
still wrong if half the timestamps are local.

### Migrations

`php bin/migrate.php` applies every `db/migrations/*.sql` once, in filename
order, recording each in the `migration` table. Every statement is idempotent
(`CREATE TABLE IF NOT EXISTS`, `ON DUPLICATE KEY UPDATE`) because **MySQL DDL is
not transactional** — a half-applied migration cannot be rolled back, so it has
to be safe to re-run instead.

`002_seed.sql` is *configuration*, not data: check definitions with their
severities, weights and confidence values; the AI crawler registry; country
language weights; and the directory blocklist. Editing a weight is a re-run of
this file, not a code change.

Useful: `php bin/migrate.php --status` and `--dry-run`.

---

## 3. Connection 2 — external APIs

All outbound HTTP goes through `src/Http/HttpClient.php`, which gives you five
things a bare `curl_exec` does not:

- **Retry with full jitter**, honouring `Retry-After` exactly when present.
  Full jitter, not "backoff plus a little" — that's what stops synchronised
  retry storms when several workers hit the same 429.
- **A shared response cache in MySQL** (`api_cache`), keyed on method + URL +
  body. SERP TTL defaults to 24 h. You are buying this data; don't buy it twice.
- **Cost metering** (`api_usage`) with a **hard daily budget stop**. Set
  `DATAFORSEO_DAILY_BUDGET_USD` and the client refuses calls past it rather than
  discovering the overspend on the invoice.
- **Per-host politeness** shared through `host_state`, so concurrent workers
  respect one combined rate.
- **Redirects are not followed by default**, because the chain is itself a
  finding (T012).

### DataForSEO — the backbone

Basic auth; **the login is your account email**, not a username.

```
DATAFORSEO_LOGIN=you@dolrad.ae
DATAFORSEO_PASSWORD=...
DATAFORSEO_DAILY_BUDGET_USD=5.00
```

Pricing baked into the cost estimates (verified Aug 2026): SERP standard queue
**$0.60/1,000**, priority $1.20, live $2.00. Labs **$0.012/task + $0.00012/item**
— *except* Historical Rank and Historical Bulk Traffic Estimation, which are 10×,
and `include_clickstream_data`, which **doubles** the cost. `depth` doubles the
SERP price per extra 100 results, so depth 10 and depth 20 cost the same unit.
**Location targeting is free**, which is what makes multi-country cheap.

Two behaviours to know:

- The API returns **HTTP 200 with a non-`20000` `status_code`** on logical
  errors. `DataForSeo::call()` checks both; don't check HTTP status alone.
- `load_async_ai_overview: true` raises the per-request cost from $0.0006 to
  $0.0012 but **DataForSEO refunds the difference when the SERP has no async AI
  Overview**. Leave it on: AI Overview presence changes the value of ranking by
  roughly half, so guessing is worse than paying.

Verify `location_code` values against
`/v3/dataforseo_labs/locations_and_languages` rather than trusting the seed file.
Note that Labs is **country-level only** — no city granularity, unlike the SERP
API.

### PageSpeed Insights + CrUX — free, one API key

```
GOOGLE_API_KEY=...
```

Two rules `GooglePerformance.php` enforces:

1. **Lab and field data are never merged.** Lighthouse LCP and CrUX p75 LCP
   disagree routinely and legitimately. They're returned in separate shapes and
   stored with a `source` column.
2. **`strategy` is always set explicitly.** The PSI API defaults to *desktop*,
   which under mobile-first indexing is the wrong plane to audit.

CrUX allows **150 queries/minute per Google Cloud project**. A **404 means "not
enough traffic for field data"** — that's a fact about the site, not an error, and
the code falls back to origin level. Poll CrUX **weekly, not daily**: it's a
28-day rolling average updated once a day and about two days stale. Use
`cruxHistory()` to get 25 trailing periods in one call.

Google publishes no numeric PSI quota. The community figures (25k/day, 240/100s)
are unverified, so the client backs off on 429 rather than trusting a constant.

### Google Search Console — OAuth

This is the fiddly one. In Google Cloud Console:

1. Create a project; enable **Google Search Console API** (and the Analytics
   Data API if you want GA4).
2. Credentials → Create credentials → **OAuth client ID** → *Web application*.
3. Add the redirect URI **exactly** as it appears in
   `GOOGLE_OAUTH_REDIRECT_URI` — scheme, port and trailing path all count. One
   character of difference gives you `redirect_uri_mismatch`.
4. **OAuth consent screen:** while the app is in *Testing*, Google expires the
   refresh token after **7 days**. For a tool with scheduled runs you must
   publish the app or use an Internal app inside a Workspace org.

Two flags are easy to miss and both are set in `GoogleOAuth::authUrl()`:
`access_type=offline` (without it you get no refresh token) and
`prompt=consent` (without it Google only returns a refresh token on the *first*
ever authorisation, so a re-connect after losing the token silently yields none).

Then click **Connect Google account** on the Integrations page.

What GSC gives you that no crawler can: **clicks per URL** (which is what turns
"1,200 issues" into "3,410 visits at risk"), **query cannibalisation**, and
**Google's own selected canonical**.

What it does **not** give you, so don't plan features around it:

- **No API for the Links report or the Crawl Stats report.** At all.
- **URL Inspection is capped at 2,000/day per site.** On a 5,000-URL site you
  cannot enumerate — `inspectSample()` takes a sample and you feed it your
  highest-click, highest-severity URLs.
- **FAQ rich-result support ended August 2026.** Drop any dependency on it.
- **AI Overview clicks are counted as ordinary organic clicks.** There is no way,
  in GSC or GA4, to separate "clicked from an AI Overview" from "clicked a blue
  link". State this in reports rather than implying attribution you can't do.

---

## 4. Connection 3 — the crawler and the queue

### Why the button only enqueues

A 5,000-URL crawl takes minutes. Shared hosting kills it at
`max_execution_time`, leaving a half-written `audit_run` indistinguishable from a
real one. So:

```
[Run audit] → POST /api/sites/{id}/audits → INSERT audit_run + INSERT job → 202
                                                       ↓
cron every minute → bin/worker.php → reserve job → AuditService::execute()
                                                       ↓
browser polls GET /api/audits/{id} → progress → reload on completion
```

`bin/worker.php` exits cleanly after `WORKER_MAX_JOBS` (200) or
`WORKER_MAX_SECONDS` (280), whichever comes first, so a stuck job can never wedge
the queue and PHP memory never accumulates across a day. An **flock guard** means
scheduling it every minute is safe even when a run takes twenty — overlapping
invocations exit immediately instead of double-crawling.

If the UI reports "queued" for more than 45 seconds it toasts *"nothing is
consuming the queue"* with the cron line, rather than spinning silently. That is
the single most common deployment mistake.

Job claiming is race-safe without `SELECT FOR UPDATE`: the worker writes its own
id with a conditional `UPDATE`, then reads back only what it won. Jobs reserved
for over 15 minutes are reclaimed automatically, which is how a worker killed
mid-crawl recovers.

### Politeness — not optional

Per-host state lives in the `host_state` table, so several workers share one
combined rate:

- 2–4 concurrent connections per host, 400 ms default delay
- **`Crawl-delay` is honoured** even though Google ignores it. You are not
  Google, and a site owner's `crawl-delay: 10` is a legitimate instruction.
- After 3 consecutive 5xx the host is paused 15 minutes and the delay doubles
- The crawl **hard-aborts if a host's 5xx rate exceeds 20%** — you may be the cause
- Trap defence: max depth 10, max 100 distinct query-parameter combinations per
  path, max 5,000 URLs per path prefix
- `robots.txt` is cached 24 h, matching Googlebot's own behaviour

### robots.txt — read `tests/robots_test.php`

`src/Audit/RobotsTxt.php` exists because the naive implementation gets three
things wrong, and each produces a **confidently wrong finding**:

1. **Group selection.** The `*` group applies only when no group matches the
   agent's product token. Given `User-agent: * / Disallow: /` plus
   `User-agent: GPTBot / Allow: /`, **GPTBot is allowed**. Tools report this
   backwards constantly. Where more than one group matches the same agent,
   RFC 9309 §2.2.1 requires their rules to be **combined**, not "first wins".
2. **Rule precedence.** Longest match wins by path length in octets, not source
   order. On equal length, Allow wins.
3. **Token matching.** Case-insensitive, on the product token, not the full UA
   string.

`php tests/robots_test.php` asserts all 33 cases. Run it after any change.

### The render plane (optional but high value)

```bash
cd render-service && npm install && node server.mjs
# then in .env:
RENDER_SERVICE_URL=http://127.0.0.1:8790
```

PHP can't drive headless Chromium sensibly, so this is a small Node service the
PHP app calls over HTTP. Keeping it separate also means a render backlog can
never stall the raw crawl.

`POST /diff {"url":"…"}` is the endpoint that matters. It fetches the URL twice
— once with a real browser, once as a plain HTTP GET with an AI-crawler UA — and
returns the delta. **Most AI crawlers do not execute JavaScript**, so that delta
*is* the retrievability gap, and a gap above 30% is a P0 finding. Almost no
competing tool ships this check.

Rendering costs 10–20× a raw fetch, which is why the PHP side renders a
**stratified sample** — every template × 3–5 URLs plus the high-JS-dependency
ones, typically 1–3% of URLs catching ~99% of rendering issues. Without the
service you still get roughly 85% of the check catalogue.

---

## 5. Connection 4 — frontend to backend

`public/index.php` is the only web entry point. Routes:

| Method | Path | Purpose |
|---|---|---|
| GET | `/` | Portfolio dashboard |
| GET | `/sites/{id}` | Scores, priority queue, AI crawler matrix |
| GET | `/integrations` | Credential status, OAuth connect |
| GET | `/oauth/google/callback` | OAuth return |
| POST | `/api/sites/{id}/audits` | Enqueue a run → 202 |
| POST | `/api/audits/bulk` | **"Audit all sites"** — one run *per site* |
| GET | `/api/audits/{id}` | Status + progress + score (the poller) |
| GET | `/api/audits/{id}/findings` | Paginated, filterable |
| GET | `/api/audits/{id}/queue` | The developer work queue |
| POST | `/api/findings/{id}/status` | fixed / ignored / won't fix |
| GET | `/api/sites/{id}/ai-access-matrix` | Crawler × path grid |
| GET | `/api/health` | 200/503 for uptime monitoring |

**"Audit all sites" creates one run per site, not one run covering four.** Scores,
deltas and findings all stay per-site; the dashboard rolls them up for display.
It also means the per-host concurrency cap still applies — four simultaneous
audits of four sites on the same shared hosting would otherwise look like an
attack.

`APP_URL` drives `View::base()`, so the app works at `/aiseo/` as well as at a
domain root. CSRF: the token is in a `<meta>` tag and `app.js` sends it as
`X-CSRF-Token` on every mutating request.

**Every page renders even when MySQL is down.** `db_status()` catches the failure
and the layout shows the banner with the actual diagnosis. That's deliberate: a
tool whose error page is a white screen can't tell you what's wrong with it.

---

## 6. The CSS layer

Two files, both under `public/assets/css/`:

- **`tokens.css`** — the only file where a raw colour value may appear. Every
  component reads roles (`--fg-primary`, `--sev-critical-ink`, `--series-1`).
- **`app.css`** — base + components. No hex values below the first line.

Theme resolution: `<html data-theme>` (user choice, persisted to
`localStorage`) beats `prefers-color-scheme` beats light. The dark values are
declared under **both** scopes so the toggle wins either way. `app.js` stamps the
attribute before first paint, so there's no flash.

Dark mode is a **selected** set, not an inverted one — every categorical hue is
re-stepped for the dark surface and re-validated against it. Both modes were
validated against this app's actual surfaces (light `#ffffff`, dark `#14171c`):
adjacent-pair CVD ΔE 9.1 light / 8.4 dark, normal-vision ΔE 19.6 / 19.3. Three
light-mode series colours sit below 3:1 contrast, which is why every chart ships
**visible direct labels and a table view** — that's the relief rule, not an
optional nicety. Don't add a ninth series hue; fold to "Other" or facet.

Severity badges never rely on colour alone: each carries an icon and a text
label, and every badge ink clears 6.4:1 against its fill.

`public/styleguide.html` is a standalone reference page with every component in
both modes. Open it directly; it needs no PHP.

---

## 7. Scoring — why the number is defensible

Five properties, all in `src/Audit/Scorer.php`:

1. **Scored per check, not per URL.** Ahrefs divides by URL count, so one bad
   template dominates and the error *type* is invisible. Here the denominator is
   the sum of severity weights across *applicable* checks — which also makes the
   score invariant to crawl size.
2. **Sublinear prevalence:** `penalty = weight × sqrt(prevalence) × confidence`.
   One critical instance in ten thousand still costs ~1% of that check's weight,
   so a single catastrophic finding can't be averaged away.
3. **Confidence factor.** Heuristics (soft 404, intent mismatch, E-E-A-T proxies)
   carry 0.5–0.8, so a fuzzy check can never dominate a deterministic one. This
   is also your answer when a finding is disputed.
4. **Hard gates.** A blanket `Disallow: /` caps the score at 20. Mass `noindex`
   caps at 25. No HTTPS caps at 35. Failing mobile CWV at p75 caps at 69.

5. **No evidence is not a pass.** If the crawl fetched nothing usable — the origin
   403s a WAF, DNS is broken, the site is behind a login — every check is "not
   applicable", the denominator is zero, and naive arithmetic returns a perfect
   100. This was real: auditing a live domain that answered 403 produced
   `score=100.00, 0 findings, strength 100`. Such a run is now **inconclusive**:
   no overall, risk, strength or category row is written at all, `audit_run.status`
   becomes `inconclusive`, and `error_message` carries the reason. `score.value` is
   `NOT NULL` on purpose — a row that exists is a number someone can act on, so
   "inconclusive" is the *absence* of the row, not a null inside it. Locked down by
   `tests/scorer_test.php`.

Checks with `eligible_count = 0` are excluded from **both** sums, so a site with
no images is neither punished nor rewarded on image checks.

### Coverage — telling "we looked" from "nobody looked"

`002_seed.sql` loads **134** check definitions. **55** have working evaluators.
The other 79 write no `check_result` row, so they are excluded from both sums —
correct arithmetically, invisible editorially. `check_definition.implemented`
records which is which, and the site page prints the count beside the category
scores. To list the gaps:

```sql
SELECT code, category, severity, title FROM check_definition WHERE implemented = 0 ORDER BY code;
```

**Info-severity checks carry weight 0** and are reported only. Blocking a training
crawler, or charging AI crawlers through Cloudflare pay-per-crawl, is a policy
decision — it appears in the findings and costs nothing. (The seed originally gave
them weight 0.5, so sites were penalised for it; `005_info_weight.sql` fixes
existing databases.)

**`check_result.penalty` is authoritative.** The scorer writes
`weight × sqrt(prevalence) × confidence` back to every row, so a report built in
SQL agrees with the UI. It read 0.0000 for every check until this build.

Three scores are produced and never blended: **Risk**, **Strength**,
**Opportunity**. A site can be simultaneously strong and risky, and one number
hides exactly that.

The **priority queue** — not the score — is what the Findings page leads with:

```
priority = weight × log(1 + affected_clicks) × confidence ÷ fix_effort
```

with effort 1 for a template-level fix and `affected_count` for a per-URL content
fix. That ordering is why "robots.txt blocks Googlebot" outranks "96 pages have
short titles" even though both are real.

`scoring_model_version` is stored on every run. Without it, your trend charts
become lies the first time you tune a weight.

---

## 8. Adding a check

`check_definition` is config, so this is a two-step change:

1. Append a row to a new migration — `db/migrations/006_your_checks.sql`
   (001–005 are taken):

```sql
INSERT INTO check_definition
 (code, category, subcategory, title, severity, weight, confidence, plane,
  eligible_unit, evidence_tier, remediation) VALUES
('T130','sitemaps','content','Sitemap URL host does not match the sitemap host',
 'medium', 2, 1.00, 'crawl', 'sitemap_url', 'primary',
 'Cross-host sitemap entries require verified ownership of both hosts in Search Console.')
ON DUPLICATE KEY UPDATE title=VALUES(title), severity=VALUES(severity);
```

2. Add a method in `CheckRunner`, register it in `run()`, and call
   `$this->flagFromQuery('T130', $sql, $eligibleCount)`.

3. Flip the flag in the same commit as the code:
   `UPDATE check_definition SET implemented = 1 WHERE code = 'T130';`
   A definition without an evaluator is a silent gap, and the coverage line on the
   site page is only honest if this stays in step with the code.

The helper writes both the findings and the `check_result` row the scorer needs.
**Always pass a real eligible count**, and derive it *without looking at the
verdict* — a check that reports 5 affected out of an unknown denominator can't be
scored, and one whose eligible count equals its affected count always reads as
100% prevalence, which defeats the sublinear curve. That exact mistake kept the AI
access checks out of the score entirely: they wrote findings but no
`check_result`, so disallowing `OAI-SearchBot` showed a critical finding worth
zero points.

The remaining catalogue entries from the specification document are all additive
in exactly this way; nothing in the engine needs to change to accept them.

---

## 9. Cadence — what to schedule

```cron
# canary: robots.txt + homepage + one template. <10 requests. THE highest-ROI monitor.
0 * * * *   cd /path/aiseo && php bin/audit.php --all --type=canary --queue

# worker: consumes the queue. flock makes every-minute safe.
* * * * *   cd /path/aiseo && php bin/worker.php >> var/logs/worker.log 2>&1

# full crawl: weekly per site (<10k URLs)
0 2 * * 1   cd /path/aiseo && php bin/audit.php --all --type=full --queue

# AI SEO audit: weekly, separate so the matrix refreshes without a full crawl
0 3 * * 2   cd /path/aiseo && php bin/audit.php --all --type=ai_seo --queue

# log rotation
0 4 * * *   cd /path/aiseo && find var/logs -name '*.log' -size +50M -exec truncate -s 0 {} \;
```

Alert on **deltas and gate breaches**, never on absolute issue counts. A site
sitting at 1,200 low-severity issues generates no alert; the same site gaining
one critical, or its robots.txt hash changing, pages someone immediately.

---

## 10. Security checklist before go-live

- [ ] `.env` is `chmod 600` and returns 403/404 over HTTP (`doctor.php` probes it)
- [ ] `src/`, `db/`, `var/`, `templates/`, `tests/` are not web-readable
- [ ] `APP_DEBUG=false` and `APP_ENV=production`
- [ ] `APP_KEY` is set to 32+ random bytes
- [ ] The MySQL user has privileges on **one** database, not `*.*`
- [ ] The app itself is behind authentication — **there is none built in.** Put it
      behind cPanel Directory Privacy, an IP allowlist, or add a login before
      exposing it publicly. It exposes API cost data and full crawl output.
- [ ] `X-Robots-Tag: noindex, nofollow, noarchive` is set (it is, in
      `public/.htaccess`) — ironic for an SEO tool, and correct
- [ ] Credentials are never written to `integration.config_json`; secrets live in
      `.env` and `oauth_token` only
- [ ] `var/logs/app.log` redaction is working: `grep -i pass var/logs/app.log`
      should return only `***`

---

## 10a. Blank 500 — start here

A 500 with **nothing** in the browser means PHP died before it could print
anything. Static files still serve normally, which is how you tell it apart from
an Apache or `.htaccess` problem:

| Test | Result | Meaning |
|---|---|---|
| `/aiseo/assets/css/tokens.css` | 200 | Apache and `.htaccess` are fine |
| `/aiseo/index.php` | 500 | PHP is fataling — read on |
| both 500 | | it IS the `.htaccess`; rename it to `.htaccess.bak` and reload |

**Upload `preflight.php` to `/aiseo/` and open it in a browser.** It is written
in PHP 5.4-compatible syntax on purpose: a diagnostic that needs a modern PHP
cannot tell you your PHP is too old, because it hits a parse error first and
produces the very blank 500 it was meant to explain.

It reports the PHP version, the extensions, which folders it found and where,
**which PHP files fail to parse** (validated with `token_get_all(TOKEN_PARSE)`,
so nothing is executed), which `.env` keys are set — never their values — and the
tail of any error log it can find. Delete it afterwards.

The four causes, in order of how often they are the answer:

1. **PHP version too old.** The app needs **8.0+**. On 7.x a file using newer
   syntax fails to *parse*, and PHP returns an empty 500. Fix in cPanel →
   MultiPHP Manager → select `/aiseo` → PHP 8.1 or 8.2 → Apply. **Check the CLI
   version separately** (`php -v`): on cPanel it is often different from the web
   version, so the site can work while cron does not, or the reverse.
2. **A file truncated during upload.** FTP in text mode, or an interrupted
   transfer, leaves a half-written `.php`. The parse check names the file.
3. **A folder missing.** If `templates/` or `config/` did not make it across,
   path resolution fails. `preflight.php` lists exactly which folder it could not
   find and where it looked.
4. **A runtime fatal.** If preflight is clean, set `APP_DEBUG=true` in `.env`,
   reload, read the real message and stack trace, then **set it back to false**.

Where the real message lives when the browser shows nothing: cPanel → Metrics →
Errors, or a file literally named `error_log` next to `index.php`. That file
holds the fatal that the browser swallowed.

---

## 11. Troubleshooting

| Symptom | Cause | Fix |
|---|---|---|
| **Blank 500, no message** | PHP version too old, a truncated upload, or a missing folder | **upload and open `preflight.php`** — see §10a |
| 500 on `index.php` but assets serve fine | PHP fatal, not Apache | `preflight.php`, then `APP_DEBUG=true` |
| 500 on *everything* including CSS | an `.htaccess` directive the host rejects | rename `.htaccess` to `.htaccess.bak`, reload, then re-add rules one block at a time |
| Site works but cron does not | the CLI PHP is a different version from the web PHP | `php -v` vs the MultiPHP setting; use the full binary path in cron |
| Banner: "Database connection failed" | see §2 | `php bin/doctor.php` |
| "Queued" forever, nothing happens | worker cron not installed or not executing | check `var/logs/worker.log`; verify the PHP binary path with `which php` |
| Crawl returns 1 URL | placeholder domain, or robots.txt blocks everything, or the start URL redirects off-host | `php bin/audit.php --list`; check `host_state.robots_body` |
| `Class "AiSeo\..." not found` | filename doesn't match the class name (case-sensitive on Linux) | the autoloader maps `AiSeo\Foo\Bar` → `src/Foo/Bar.php` exactly |
| DataForSEO returns 40100 | login is a username, not the account email | use the email address |
| `redirect_uri_mismatch` | the registered URI differs by ≥1 character | copy `GOOGLE_OAUTH_REDIRECT_URI` verbatim into Cloud Console |
| GSC connects, then stops working after a week | OAuth consent screen still in *Testing* | publish the app, or use an Internal Workspace app |
| Worker dies mid-crawl on shared hosting | memory or a dropped MySQL socket | `Database::reconnect()` handles the socket; raise `memory_limit` to 512M |
| CrUX returns 404 for every URL | not enough real traffic for URL-level field data | expected — the code falls back to origin level |
| Findings reappear after being marked fixed | the underlying issue is still present | `status='fixed'` is per finding identity (check + URL); it returns if the check still matches |

---

## 12. What is deliberately not built

Stated so nobody spends a sprint discovering it:

- **No authentication layer.** Add one before exposing this publicly.
- **No off-page module yet.** The schema and check definitions exist (F001–F030);
  the provider adapters for Ahrefs/Majestic/DataForSEO Backlinks are not written.
- **No Site Analyze pipeline yet.** `DataForSeo::serpCompetitors()`,
  `competitorsDomain()`, `keywordOverview()` and `searchIntent()` are written and
  costed; the orchestration that turns them into competitors, keyword scores and
  content briefs is the next phase.
- **hreflang checks are stubbed.** `PageParser` extracts the annotations;
  persisting them to a `hreflang_edge` table activates T075–T083. This matters
  for the GCC plan, so it should be early in the next phase.
- **No FAQ rich-result anything.** Deprecated May–June 2026, API support ended
  August 2026.