Sitemap generation

Walk public board resources and serve a bounded sitemap index and XML buckets.

Generate the same eight-bucket sitemap model as Cavuno’s hosted board. The walker handles feature gating, pagination, canonical paths, and the minimum-content rule for job taxonomy pages.

Prerequisites

A stateless public board client.
A canonical public origin.
Server routes for /sitemap.xml and /sitemap/:filename.
A cache independent from candidate or employer sessions.

Build the sitemap service

1// src/lib/sitemaps.ts
2import {
3  SITEMAP_CHUNK_SIZE,
4  bucketFilename,
5  buildBucketUrls,
6  chunk,
7  listedBuckets,
8  parseBucketFilename,
9  renderSitemapIndex,
10  renderUrlset,
11} from '@cavuno/board/sitemap';
12
13import { board } from './board';
14
15const origin = 'https://jobs.example.com';
16
17export async function buildSitemapIndex() {
18  const entries: string[] = [];
19
20  for (const bucket of await listedBuckets(board)) {
21    const urls = await buildBucketUrls(board, origin, bucket);
22    const chunkCount = Math.max(
23      1,
24      chunk(urls, SITEMAP_CHUNK_SIZE).length,
25    );
26
27    for (let index = 0; index < chunkCount; index += 1) {
28      entries.push(`${origin}/sitemap/${bucketFilename(bucket, index)}`);
29    }
30  }
31
32  return renderSitemapIndex(entries);
33}
34
35export async function buildSitemapFile(filename: string) {
36  const parsed = parseBucketFilename(filename);
37  if (!parsed) return null;
38
39  const available = await listedBuckets(board);
40  if (!available.includes(parsed.bucket)) return null;
41
42  const urls = await buildBucketUrls(board, origin, parsed.bucket);
43  const chunks = chunk(urls, SITEMAP_CHUNK_SIZE);
44  const selected =
45    chunks[parsed.chunkIndex] ??
46    (parsed.chunkIndex === 0 && chunks.length === 0 ? [] : null);
47
48  return selected ? renderUrlset(selected) : null;
49}

Return these strings with Content-Type: application/xml; charset=utf-8. Return your framework’s 404 for an unknown filename or chunk.

Understand what is emitted

The buckets cover marketing routes, job categories, skills, locations and details, companies, salaries, and blog content. The blog bucket is removed when the feature is disabled. Other empty buckets produce valid empty URL sets.

Job taxonomy pages need at least five matching jobs before they are emitted. This avoids indexing thin category, skill, and location pages. Location counts include only jobs in the place itself, never jobs within a search distance, and distance views (?within=) are never emitted.

Expected result

/sitemap.xml is a sitemap index whose entries point to valid bucket files. Each bucket route returns a standards-compliant URL set, including a valid empty set when an enabled bucket has no URLs.

Verify the result

bash

1curl -fsS https://jobs.example.com/sitemap.xml
2curl -fsS https://jobs.example.com/sitemap/jobs-details.xml

Confirm that:

Every <loc> uses the production HTTPS origin.
Disabled blog routes are absent.
Job detail URLs include both company and job slugs.
A non-existent bucket or chunk returns 404.
XML reserved characters are escaped by the renderer.

Submit /sitemap.xml to your search-console tooling only after it works at the final public domain.

Errors and edge cases

A catalog above the offset ceiling logs a truncation warning until bulk support is available. Monitor sitemap generation logs.
Cross-axis salary pages and combined location/taxonomy pages are not currently emitted because the public API cannot enumerate them without an N+1 crawl.
The walker can make many public reads. Do not run it during every ordinary page render.
An origin with a path or the wrong environment creates invalid canonical URLs.

Production cautions

Cache generated XML and revalidate often enough to remove expired jobs.
Generate with a stateless client; never attach candidate authorization.
Alert on truncation warnings and generation failures.
Keep robots.txt’s Sitemap: line aligned with board.seo().canonicalBase.

Next, format board-owned data with Styling and formatting.