Fix #19: unblock all public marketing/docs/SEO pages in the middleware
Details
The Clerk middleware 307'd anonymous visitors + crawlers to /sign-in for every page NOT in isPublicRoute — and the allowlist was missing almost all public pages: /docs (and /docs/interop, the interop inbound magnet just shipped), /conformance, /compare, /catena-x, /components, /changelog, /dpp, /battery-passport, /eu-machinery-2027, /dpa, plus /robots.txt and /sitemap.xml. So the magnet was dead in prod (bounce to sign-in) and the whole marketing surface was invisible to search engines. - Extract the allowlist to lib/auth/public-routes.ts (PUBLIC_ROUTE_PATTERNS) so it is unit-testable; proxy.ts imports it. - Add every public marketing/docs/SEO page + /robots.txt + /sitemap.xml. Authed surfaces (/studio/* sub-routes, /org, /suppliers) stay protected. - Add app/sitemap.ts (a real sitemap; robots.ts already advertised /sitemap.xml which previously 404'd). - Regression test __tests__/api/public-routes.test.ts: builds the same matcher and asserts every public page/API is public and every authed surface is protected (the matcher-regex unit cases the #19 playbook wants). Gate: tsc 0, public-routes 3/3, hygiene green. (The connectors-register-ssrf test fails on this machine with OR without this change — Avast DNS interception on the egress guard's re-resolve; green in CI.) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>