Domain Name: GORETAILGROUP.COM
Registrar: GoDaddy.com, LLC
Domain Status: client delete prohibited
Domain Status: client renew prohibited
Domain Status: client transfer prohibited
Domain Status: client update prohibited
Creation Date: 2011-06-23T02:43:59Z
Registry Expiry Date: 2026-06-23T02:43:59Z
Updated Date: 2025-06-23T16:46:51Z
Name Server: NS03.DOMAINCONTROL.COM
Name Server: NS04.DOMAINCONTROL.COM
REGISTRAR Contact: GoDaddy.com, LLC
>>> Last update of RDAP database: 2026-03-21T01:20:18Z
# robots.txt for https://www.goretailgroup.com
# Consolidated and updated 11/04/2025
# Includes AI bot allowlist + SEO crawler rules + unwanted bot blocks
# --- Global rules ---
User-agent: *
Allow: /
Disallow: /admin*
Disallow: /cart*
Disallow: /checkout*
Disallow: /search*
Disallow: /my-account*
Disallow: /user*
Disallow: /util*
Disallow: /storefront/email/render/*
Disallow: /print-return*
Disallow: /guest-checkout*
Disallow: /print-order*
Disallow: /nomore*
Disallow: /registrations-sign-in*
Disallow: /home-temp*
Disallow: /test-pricelist*
Crawl-delay: 5
# --- AI and external bot settings ---
User-agent: GPTBot
Allow: /
Crawl-delay: 0.2
User-agent: OAI-SearchBot
Allow: /
User-agent: ChatGPT-User
Allow: /
User-agent: ChatGPT-User/2.0
Allow: /
User-agent: PerplexityBot
Allow: /
User-agent: ClaudeBot
Allow: /
User-agent: facebookexternalhit
Allow: /
User-agent: meta-externalagent
Allow: /
User-agent: meta-externalfetcher
Allow: /
# --- Major SEO tools and crawlers ---
User-agent: AhrefsSiteAudit
Allow: /
Crawl-delay: 10
User-agent: AhrefsBot
Crawl-delay: 10
User-agent: SemrushBot
Allow: /
Crawl-delay: 1
User-agent: Pinterestbot
Allow: /
Crawl-delay: 1
User-agent: AmazonBot
Crawl-delay: 3
User-agent: MJ12bot
Crawl-delay: 10
# --- Block unwanted bots ---
User-agent: Baiduspider
Disallow: /
# --- Host and Sitemap ---
Host: https://www.goretailgroup.com
Sitemap: https://www.goretailgroup.com/sitemap.xml