TudorandClaude Opus 5
c2c76c5817
X-Robots-Tag rather than the robots.txt Disallow alone, for the same reason the staging rule uses one: a Disallow blocks crawling, not indexing, so a URL found from an external link can be indexed without ever being fetched — and blocking the crawl means the noindex is never seen. Both mechanisms are applied to /admin and /cms-api. The existing CSP is frame-ancestors only, which restricts who may embed the site rather than what a page may load, so it cannot break the panel. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017YmbBhr8s7GusjDE12hrZM
12 lines
345 B
TypeScript
12 lines
345 B
TypeScript
import robots from '@/app/robots';
|
|
|
|
describe('robots.txt', () => {
|
|
it('disallows the admin panel and the CMS API', () => {
|
|
const rules = robots().rules;
|
|
const rule = Array.isArray(rules) ? rules[0] : rules;
|
|
expect(rule.disallow).toEqual(
|
|
expect.arrayContaining(['/api/', '/_next/', '/admin/', '/cms-api/']),
|
|
);
|
|
});
|
|
});
|