Domain Name: RIGIDINDUSTRIES.COM
Registrar: GoDaddy.com, LLC
Domain Status: client delete prohibited
Domain Status: client renew prohibited
Domain Status: client transfer prohibited
Domain Status: client update prohibited
Creation Date: 2004-02-18T02:23:48Z
Registry Expiry Date: 2028-02-18T02:23:48Z
Updated Date: 2026-02-18T16:03:07Z
Name Server: INGRID.NS.CLOUDFLARE.COM
Name Server: RUBEN.NS.CLOUDFLARE.COM
REGISTRAR Contact: GoDaddy.com, LLC
>>> Last update of RDAP database: 2026-02-19T05:27:05Z
# ============================================================
# AI TRAINING CRAWLERS — Blocked
# These crawlers index content for commercial AI model training.
# No SEO or traffic benefit to the site.
# ============================================================
User-agent: GPTBot
Disallow: /
User-agent: ChatGPT-User
Disallow: /
User-agent: OAI-SearchBot
Disallow: /
User-agent: ClaudeBot
Disallow: /
User-agent: anthropic-ai
Disallow: /
User-agent: Google-Extended
Disallow: /
User-agent: CCBot
Disallow: /
User-agent: cohere-ai
Disallow: /
User-agent: PerplexityBot
Disallow: /
User-agent: Bytespider
Disallow: /
User-agent: Amazonbot
Disallow: /
User-agent: Applebot-Extended
Disallow: /
User-agent: YouBot
Disallow: /
User-agent: Diffbot
Disallow: /
# ============================================================
# SEO & BACKLINK AUDIT TOOLS — Blocked
# These crawlers primarily benefit competitors performing
# competitive analysis. SemrushBot is intentionally omitted
# as it is used internally by the team.
# ============================================================
User-agent: AhrefsBot
Disallow: /
User-agent: MJ12bot
Disallow: /
User-agent: DotBot
Disallow: /
User-agent: BLEXBot
Disallow: /
User-agent: Serpstatbot
Disallow: /
User-agent: DataForSeoBot
Disallow: /
User-agent: PetalBot
Disallow: /
User-agent: SEOkicks-Robot
Disallow: /
User-agent: SiteAuditBot
Disallow: /
User-agent: SISTRIX
Disallow: /
User-agent: MauiBot
Disallow: /
# ============================================================
# META / FACEBOOK CRAWLERS — Selective
# facebookexternalhit is kept for social sharing link previews
# but blocked from static/media assets it does not need.
# meta-externalfetcher bypasses robots.txt by design (Meta policy)
# and cannot be controlled here — use Fastly VCL if needed.
# ============================================================
User-agent: meta-externalagent
Disallow: /
User-agent: meta-webindexer
Disallow: /
User-agent: meta-externalads
Disallow: /
User-agent: facebookexternalhit
Disallow: /static/
Disallow: /pub/static/
Disallow: /pub/
Disallow: /media/
# ============================================================
# ALL OTHER CRAWLERS
# Legitimate search engines (Google, Bing, DuckDuckGo, Yahoo,
# Apple, etc.) are allowed by omission above.
# The rules below apply to all unspecified crawlers.
# ============================================================
User-agent: *
Disallow: /index.php/
Disallow: /*?q
Disallow: /checkout/
Disallow: /app/
Disallow: /lib/
Disallow: /*.php$
Disallow: /pkginfo/
Disallow: /report/
Disallow: /var/
Disallow: /catalog/
Disallow: /customer/
Disallow: /sendfriend/
Disallow: /review/
Disallow: /*SID=
Disallow: /onestepcheckout/
Disallow: /customer/account/
Disallow: /customer/account/login/
Disallow: /catalogsearch/
Disallow: /catalog/product_compare/
Disallow: /catalog/category/view/
Disallow: /catalog/product/view/
Disallow: /*?dir*
Disallow: /*?dir=desc
Disallow: /*?dir=asc
Disallow: /*?limit=all
Disallow: /*?mode*
Disallow: /bin/
Disallow: /dev/
Disallow: /phpserver/
Disallow: /tag/
Disallow: /*catalogsearch/*
Disallow: /*?
Disallow: /blog/tag/*
Disallow: /blog/author/*
Disallow: /amlocator/*
Disallow: /graphql/*
| Položaj | Fraza | Stran | Izrezek |
|---|---|---|---|
| 6 | / |