Domain Name: STEELONTHENET.COM
Registrar: Register SPA
Domain Status: client delete prohibited
Domain Status: client transfer prohibited
Domain Status: client update prohibited
Creation Date: 2000-09-25T09:35:38Z
Registry Expiry Date: 2028-09-25T09:35:38Z
Updated Date: 2024-09-22T09:30:04Z
Name Server: NS0.PHASE8.NET
Name Server: NS1.PHASE8.NET
Name Server: NS2.PHASE8.NET
REGISTRAR Contact: Register SPA
>>> Last update of RDAP database: 2026-05-29T11:13:59Z
# ============================================================================
# Steel on the Net - robots.txt
# https://www.steelonthenet.com/robots.txt
# Last updated: 2024-12-14
# ============================================================================
# ----------------------------------------------------------------------------
# Default Rules - All Crawlers
# ----------------------------------------------------------------------------
User-agent: *
# Allow Important Resources (must come before Disallow rules)
Allow: /css/
Allow: /js/
Allow: /images/
Allow: /uikit/
Allow: /logos/
Allow: /source_images/
# Admin & Database Access (Critical Security)
Disallow: /mysql/
Disallow: /phpmyadmin/
Disallow: /admin/
Disallow: /wp-admin/
# Analytics & Statistics
Disallow: /.stats/
Disallow: /analytics/
# Counter & Tracking Scripts
Disallow: /ccount/
Disallow: /count/
Disallow: /counter/
# Internal/Private Directories
Disallow: /archive/
Disallow: /buffer/
Disallow: /cache/
Disallow: /corey/
Disallow: /ISSB/
Disallow: /memo/
Disallow: /slpw/
Disallow: /temp/
Disallow: /tmp/
# Private Files & Account Information
Disallow: /mci-account-info.html
Disallow: /*?print=*
Disallow: /*?download=*
# Game/Entertainment Content (No Business Value)
Disallow: /crossword-1.html
Disallow: /crossword-2.html
Disallow: /crossword-3.html
Disallow: /crossword-4.html
# Search & Filter Parameters (Avoid Duplicate Content)
Disallow: /*?sort=*
Disallow: /*?filter=*
Disallow: /*?page=*
Disallow: /*?ref=*
Disallow: /*?utm_*
# Session IDs & Tracking Parameters
Disallow: /*?sid=*
Disallow: /*?session=*
Disallow: /*?PHPSESSID=*
# Login & Registration Pages (If Applicable)
Disallow: /login/
Disallow: /register/
Disallow: /account/
# Crawl Rate (1 second delay - gentle on server)
Crawl-delay: 1
# ----------------------------------------------------------------------------
# Google-Specific Rules
# ----------------------------------------------------------------------------
User-agent: Googlebot
# Google ignores Crawl-delay (use Search Console instead)
# Inherits all Disallow rules from User-agent: *
Disallow: /temp/
User-agent: Googlebot-Image
Allow: /images/
Allow: /source_images/
Allow: /logos/
Disallow: /temp/
# ----------------------------------------------------------------------------
# Bing-Specific Rules
# ----------------------------------------------------------------------------
User-agent: Bingbot
Crawl-delay: 1
# ----------------------------------------------------------------------------
# SEO Tools - Legitimate Research Bots
# ----------------------------------------------------------------------------
User-agent: AhrefsBot
Crawl-delay: 2
User-agent: SemrushBot
Crawl-delay: 2
User-agent: MJ12bot
Crawl-delay: 5
User-agent: DotBot
Crawl-delay: 5
User-agent: Seobility
Crawl-delay: 2
# ----------------------------------------------------------------------------
# Bad Bots (Complete Block)
# ----------------------------------------------------------------------------
User-agent: WebReaper
Disallow: /
User-agent: WebCopier
Disallow: /
User-agent: Offline Explorer
Disallow: /
User-agent: HTTrack
Disallow: /
User-agent: Microsoft.URL.Control
Disallow: /
User-agent: EmailCollector
Disallow: /
User-agent: EmailSiphon
Disallow: /
# ----------------------------------------------------------------------------
# XML Sitemaps
# ----------------------------------------------------------------------------
# Main sitemap
Sitemap: https://www.steelonthenet.com/sitemap.xml
# Additional sitemaps (add if you have them)
# Sitemap: https://www.steelonthenet.com/sitemap-news.xml
# Sitemap: https://www.steelonthenet.com/sitemap-images.xml
# Sitemap: https://www.steelonthenet.com/sitemap-pages.xml
| pozícia | Fráza | Stránka | Úryvok |
|---|---|---|---|
| 16 | / | ||
| 46 | / | ||
| 65 | / |