Domain Name: BCCRESEARCH.COM
Registrar: GoDaddy.com, LLC
Domain Status: client delete prohibited
Domain Status: client renew prohibited
Domain Status: client transfer prohibited
Domain Status: client update prohibited
Creation Date: 2000-01-13T17:00:21Z
Registry Expiry Date: 2026-01-13T17:00:20Z
Updated Date: 2025-01-14T11:46:20Z
Name Server: NS-106.AWSDNS-13.COM
Name Server: NS-1400.AWSDNS-47.ORG
Name Server: NS-1680.AWSDNS-18.CO.UK
Name Server: NS-729.AWSDNS-27.NET
REGISTRAR Contact: GoDaddy.com, LLC
>>> Last update of RDAP database: 2026-01-13T16:31:15Z
# BCC Research robots.txt — AI-friendly with guardrails
# --- Global defaults (keep existing hygiene) ---
User-agent: *
Crawl-Delay: 2
Disallow: /index/advancedresult/
Disallow: /index/advancedsearch/
Disallow: /*?*
Disallow: /*?
Disallow: /aboutus/hubspotlead
Disallow: /statsana/piwik/
Allow: /public/
Allow: /public/images2017/
# --- AI crawlers: explicit allow (faster crawl) ---
User-agent: GPTBot
Allow: /
Crawl-Delay: 2
User-agent: OAI-SearchBot
Allow: /
Crawl-Delay: 2
User-agent: ClaudeBot
Allow: /
Crawl-Delay: 2
User-agent: Claude-SearchBot
Allow: /
Crawl-Delay: 2
User-agent: PerplexityBot
Allow: /
Crawl-Delay: 2
User-agent: CCBot
Allow: /
Crawl-Delay: 2
# Broader “infra” user-agents used for AI indexing/pipeline work
User-agent: GoogleOther
Allow: /
Crawl-Delay: 2
User-agent: Amazonbot
Allow: /
Crawl-Delay: 2
# Traditional search bots (often feed LLM features)
User-agent: Googlebot
Allow: /
User-agent: bingbot
Allow: /
# Training/usage policy toggles (not crawlers, but respected by some vendors)
User-agent: Google-Extended
Allow: /
User-agent: Applebot-Extended
Allow: /
# --- Machine Content Policy for AI & LLMs (already live) ---
MCP: https://www.bccresearch.com/mcp.yaml
# --- Sitemaps (already live) ---
Sitemap: https://www.bccresearch.com/public/sitemap/indexsitemap/sitemapindex.xml
| Позиция | фраза | Страница | фрагмент |
|---|---|---|---|
| 6 | /?srsltid=afmbooolsurzufssz3ybzcgcltflkc4c8-0ys9kmmtxbwso9bn8d_rgd |