Domain Name: DAILY.DEV
Registrar: GoDaddy.com, LLC
Domain Status: client delete prohibited
Domain Status: client renew prohibited
Domain Status: client transfer prohibited
Domain Status: client update prohibited
Creation Date: 2019-08-19T12:41:12.282Z
Registry Expiry Date: 2026-08-19T12:41:12.282Z
Updated Date: 2026-05-31T07:42:51.429Z
Name Server: JAY.NS.CLOUDFLARE.COM
Name Server: KIMORA.NS.CLOUDFLARE.COM
REGISTRAR Contact: GoDaddy.com, LLC
>>> Last update of RDAP database: 2026-06-08T14:35:30Z
# Robots.txt for business.daily.dev
# This file is specific to the Business/Ads site
# AI Crawlers - explicitly allow for LLM visibility
# OpenAI crawlers
User-agent: GPTBot
Allow: /
User-agent: OAI-SearchBot
Allow: /
User-agent: ChatGPT-User
Allow: /
# Anthropic crawlers
User-agent: Anthropic-ai
Allow: /
User-agent: ClaudeBot
Allow: /
User-agent: Claude-Web
Allow: /
# Other AI crawlers
User-agent: CCBot
Allow: /
User-agent: PerplexityBot
Allow: /
User-agent: Google-Extended
Allow: /
User-agent: Cohere-ai
Allow: /
User-agent: Meta-ExternalAgent
Allow: /
User-agent: Meta-ExternalFetcher
Allow: /
User-agent: Bytespider
Allow: /
User-agent: AppleBot-Extended
Allow: /
User-agent: Amazonbot
Allow: /
User-agent: YouBot
Allow: /
User-agent: AI2Bot
Allow: /
User-agent: Diffbot
Allow: /
# All other crawlers
User-agent: *
Allow: /
# Content Signals: declare AI usage preferences (contentsignals.org)
# We actively want LLM visibility for this marketing site.
Content-Signal: ai-train=yes, search=yes, ai-input=yes
# Exclude form thank-you and scheduling pages from search
Disallow: /thank-you/
Disallow: /schedule/
Sitemap: https://business.daily.dev/sitemap.xml
# LLM/AI discoverability reference
# See https://llmstxt.org/
#
# `LLMs:` is intentionally NOT a directive here. Lighthouse's
# robots.txt validator (and Google's robots parser) flags any non-
# standard directive as "Unknown directive", failing the SEO audit
# (PR #189 fixed the same issue on main-site, 2026-05-07). The
# llms.txt advertisement happens through three already-supported
# mechanisms:
# 1. `` in the HTML head (Layout.astro)
# 2. `Link: ; rel="llms-txt"` HTTP header on every HTML
# route (vercel.json + middleware.ts)
# 3. The agent-readiness scanner reads /llms.txt directly as a
# well-known file
# So removing the `LLMs:` line costs nothing for discoverability and
# fixes the SEO audit.