robotsTxt
The text of robots.txt, written as Next writes it: User-Agent lines, the rules, a blank line after each group, then the Host line and one Sitemap line per file.
The text of robots.txt, written as Next writes it: User-Agent lines, the rules, a blank line after each group,
then the Host line and one Sitemap line per file. Every line ends with a newline.
Import#
ts
import { robotsTxt } from "@shadow-js/dusk/server";
Signature#
ts
function robotsTxt(config: RobotsConfig): string
Parameters#
| Name | Description |
|---|---|
config |
The groups, the sitemaps and the host. |
Returns#
The file text.
Throws#
- TypeError for a value with a line break, a sitemap that is not an absolute http(s) URL, or no rules.
Examples#
ts
robotsTxt({ rules: { userAgent: "*", allow: "/", disallow: ["/api/"] }, sitemap: "https://example.com/sitemap.xml" });