[]previous and next page

Metadata routes

sitemap.xml, robots.txt and manifest.webmanifest are API routes.

For agentsUpdated

sitemap.xml, robots.txt and manifest.webmanifest are API routes. A route file exports GET built by one of the helpers in @shadow-js/dusk/server. The helper receives the request event, so the answer can depend on the host:

src/routes/sitemap.xml.tsts
import { sitemapRoute } from "@shadow-js/dusk/server";

export const GET = sitemapRoute((event) => [
  { url: `${event.url.origin}/`, changeFrequency: "daily", priority: 1 },
  { url: "/docs", lastModified: "2026-10-09", alternates: { languages: { en: "/docs", ar: "/ar/docs" } } },
  { url: "/gallery", images: ["/a.png", { loc: "/b.png", caption: "B" }] },
]);

The helpers are sitemapRoute, sitemapIndexRoute (several sitemaps, one per file, the generateSitemaps shape), robotsRoute and manifestRoute. Their pure forms are sitemapXml, sitemapIndexXml, robotsTxt and manifestJson, which API routes can call directly. Each answer is application/xml, text/plain or application/manifest+json, with public, max-age=3600, stale-while-revalidate=86400 caching unless a cacheControl option replaces it, and a contentType option replaces the type (Next writes robots.txt as text/plain with no charset; set it to match). A HEAD request gets the headers and no body.

The bytes are Next's: User-Agent lines (the agent is written with that spelling), the rules, then a blank line per group, then the Host line and one Sitemap line per file. A sitemap has one element per line, a bare origin keeps its form (<loc>https://example.com</loc>, no added slash), and the manifest is compact JSON with no trailing newline. A sitemap's children follow the sitemaps.org order (loc, lastmod, changefreq, priority); Next writes the extensions first, which a site without images or alternates does not show. A sitemap loc is XML-escaped; Next writes it raw.

The limits are checked before a byte is sent: at most 50 000 URLs per sitemap file (split with sitemapIndexRoute; an index lists at most 50 000 files too), 50 MiB of UTF-8 per file, dates in W3C form, priorities between 0 and 1, and absolute http(s) URLs. The size is counted as the file is written, so a list a request made huge stops at the limit instead of after the whole document is in memory (maxBytes lowers it for a smaller budget).

What a value may hold, and what happens when it does not pass (each is a TypeError or a RangeError at the call, so the route answers with an error, never with a broken file):

Value Rule
Every URL (url, an image loc, a license, an alternate, a video thumbnailLoc, contentLoc, playerLoc, a robots sitemap) http(s) only (javascript:, data:, file: and the rest are refused); written with // after the scheme; no control character or line break and no space at either end; at most 2048 characters.
A relative URL Resolved against the request's origin and the app's base path. One that resolves anywhere else (//host/x, ///host/x, /\host/x) is refused. An absolute URL to another origin is yours to write: image CDNs and alternates on other domains are normal.
Text (title, caption, description, tag, language code) At most 8192 characters. XML escaping covers &, <, >, " and ', so ]]> and a CDATA opener are plain text. The characters XML 1.0 cannot hold at all (NUL and most C0 controls, U+FFFE, U+FFFF, a lone surrogate) are dropped, because a strict parser rejects a whole sitemap at the first one.
Lists per URL At most 1000 images, 1000 language alternates and 32 tags per video.
Video duration A finite number from 0 to 28 800 seconds, written as a whole number.
robots.txt values (agents, paths, host, sitemap) A string with no control character other than a tab, and none of U+0085, U+2028, U+2029: a value cannot begin a directive of its own. A user agent is not empty, and a rule has at least one.
manifest start_url, scope, id, and every url, src, action or href below it: a relative path or an http(s) URL; an image src may also be data:image/.... javascript:, vbscript:, file:, other data: URLs, //host and a backslash start are refused. The text writes <, >, &, U+2028 and U+2029 as \u escapes (the same JSON), so it is safe wherever it is embedded.

A route file can also serve these URLs (a sitemap.xml.ts that exports GET). The file conventions below are the other way: a file in src/ with the conventional name is served at the conventional URL, with no route file.