1. specific group beats the wildcard group
  ok    GPTBot is ALLOWED despite Disallow: / under *
  ok    an unlisted agent is blocked by the * group

2. full UA string matches the product token
  ok    long UA string matches Googlebot
  ok    case insensitive

3. longest match wins; Allow wins ties
  ok    deeper Allow beats shallower Disallow
  ok    shallower Disallow still applies elsewhere
  ok    equal length: Allow wins

4. wildcards and end-anchors
  ok    *.pdf$ blocks a pdf
  ok    *.pdf$ does not block a pdf with a query
  ok    query wildcard blocks a sorted URL
  ok    unsorted URL is fine

5. empty Disallow means allow everything
  ok    empty Disallow is not a rule

6. multiple matching groups are COMBINED (RFC 9309 2.2.1)
  ok    /b/ blocked for mybot
  ok    /a/ from the "bot" group also applies to mybot (substring match, combined)

7. consecutive user-agent lines share one group
  ok    alpha blocked
  ok    beta blocked

8. a rule after a new user-agent starts a new group
  ok    a is blocked from /one/
  ok    a is NOT blocked from /two/

9. blanket disallow detection (T139)
  ok    blanket disallow detected
  ok    Googlebot also affected
  ok    narrow disallow is not blanket

10. sitemaps, unsupported directives, syntax issues
  ok    sitemap captured
  ok    crawl-delay read
  ok    noindex flagged as unsupported
  ok    noindex raised a syntax issue

11. rules before any user-agent apply to nobody
  ok    orphan rule ignored
  ok    orphan rule flagged

12. Cloudflare content-signal is parsed but never enforced
  ok    search signal
  ok    ai-train signal
  ok    access still allowed — signals express preference only

13. size limit
  ok    truncation flagged past 500 KiB

14. BOM does not eat the first group
  ok    BOM stripped, rule still applies

15. percent-encoding normalisation
  ok    encoded path matches

33 passed, 0 failed