Domain Name: SCAMMINDER.COM
Registrar: IONOS SE
Domain Status: client transfer prohibited
Creation Date: 2023-08-22T04:13:38Z
Registry Expiry Date: 2026-08-22T04:13:38Z
Updated Date: 2025-08-23T07:01:50Z
Name Server: GABE.NS.CLOUDFLARE.COM
Name Server: OLGA.NS.CLOUDFLARE.COM
REGISTRAR Contact: IONOS SE
>>> Last update of RDAP database: 2026-08-02T03:46:22Z
# ScamMinder Robots.txt
# Updated: 2026-05-29T08:30:13.800Z
User-agent: *
# Allow all public pages
Allow: /
Allow: /analyze
Allow: /websites
Allow: /website/
Allow: /blog
Allow: /blog/*
Allow: /verify
Allow: /gethelp
Allow: /about
Allow: /contact
Allow: /faq
Allow: /privacy
Allow: /terms
Allow: /disclaimer
# Disallow admin pages
Disallow: /admin
Disallow: /admin/*
# Disallow authentication pages
Disallow: /login
Disallow: /register
Disallow: /reset-password
Disallow: /auth/
# Disallow user dashboard and account pages
Disallow: /dashboard
Disallow: /dashboard/*
Disallow: /account
Disallow: /account/*
# Disallow legacy/unknown sections that should not be indexed
Disallow: /cms
Disallow: /cms/*
Disallow: /products
Disallow: /products/*
# Compare and image-analysis pages are served with server-side X-Robots-Tag noindex.
# Keep them crawlable so Google can observe the directive and drop stale indexed variants.
# Disallow payment/verification flows (not for indexing)
Disallow: /verification/payment-success
Disallow: /verification/payment-cancel
Disallow: /gethelp/success
Disallow: /gethelp/cancel
# Disallow API endpoints
Disallow: /api/
Disallow: /api/*
# Disallow unsubscribe pages
Disallow: /unsubscribe
# Crawl delay (be nice to our server)
Crawl-delay: 1
# Sitemap location (main index points to all sitemaps)
Sitemap: https://scamminder.com/sitemap_index.xml
# Special instructions for specific bots
User-agent: Googlebot
# IMPORTANT: Googlebot block previously allowed everything, overriding the Disallow rules above.
# Keep crawl delay tuning, but explicitly repeat the Disallow rules for Googlebot.
Disallow: /admin
Disallow: /admin/*
Disallow: /login
Disallow: /register
Disallow: /reset-password
Disallow: /auth/
Disallow: /dashboard
Disallow: /dashboard/*
Disallow: /account
Disallow: /account/*
Disallow: /cms
Disallow: /cms/*
Disallow: /products
Disallow: /products/*
Disallow: /verification/payment-success
Disallow: /verification/payment-cancel
Disallow: /gethelp/success
Disallow: /gethelp/cancel
Disallow: /api/
Disallow: /api/*
Disallow: /unsubscribe
Allow: /
Crawl-delay: 0.5
User-agent: Bingbot
Allow: /
Crawl-delay: 1
User-agent: archive.org
Allow: /
Crawl-delay: 2
# Block aggressive scrapers - prevents server OOM crashes
User-agent: SemrushBot
Crawl-delay: 10
User-agent: AhrefsBot
Crawl-delay: 10
User-agent: AhrefsSiteAudit
Crawl-delay: 30
User-agent: DotBot
Crawl-delay: 10
User-agent: Bytespider
Crawl-delay: 10
User-agent: MJ12bot
Crawl-delay: 10
User-agent: BLEXBot
Crawl-delay: 10
User-agent: DataForSeoBot
Crawl-delay: 10
| Pozíció | Kifejezés | oldal | Töredék |
|---|---|---|---|
| 11 | /websites/xvideos-ru... | ||
| 16 | /websites/theteenxxx... | ||
| 17 | /websites/narynoil.c... | ||
| 25 | /websites/ebalka.nl/ |