Domain Name: LSENGINEERS.CO.UK
Registrar: 123-Reg Limited t/a 123-reg
Domain Status: active
Creation Date: 2010-10-07T18:44:04Z
Updated Date: 2024-10-08T11:14:16.795014Z
Registry Expiry Date: 2026-10-07T18:44:04Z
Name Server: CHARLES.NS.CLOUDFLARE.COM.
Name Server: MELISSA.NS.CLOUDFLARE.COM.
REGISTRAR Contact: 123-Reg Limited t/a 123-reg
>>> Last update of RDAP database: 2026-05-18T13:54:23Z
# robots.txt for www.lsengineers.co.uk
# =========================================================
# General rules for all crawlers
# =========================================================
User-agent: *
Sitemap: https://www.lsengineers.co.uk/pub/media/sitemap.xml
# Do not crawl common Magento folders
Disallow: /index.php/
Disallow: /wishlist/
Disallow: /admin/
Disallow: /review/product/
Disallow: /sendfriend/
Disallow: /sendfriend/product/
Disallow: /enable-cookies/
Disallow: /directory/
# Do not crawl common Magento files
Disallow: /api.php
Disallow: /cron.php
Disallow: /cron.sh
Disallow: /error_log
Disallow: /get.php
Disallow: /install.php
Disallow: /LICENSE.html
Disallow: /LICENSE.txt
Disallow: /LICENSE_AFL.txt
Disallow: /README.txt
Disallow: /RELEASE_NOTES.txt
# Do not crawl checkout and user account pages
Disallow: /checkout/
Disallow: /onestepcheckout/
Disallow: /customer/
Disallow: /customer/account/
Disallow: /customer/account/login/
Disallow: /customer/account/create/
# Do not crawl search pages and non-SEO catalog links
Disallow: /catalogsearch/
Disallow: /catalog/product_compare/
Disallow: /catalog/category/view/
Disallow: /catalog/product/view/
# Blocking filter options, parameters and pagination
Disallow: /*?dir*
Disallow: /*?p=*
Disallow: /*?design=*
Disallow: /*?mode*
Disallow: /*is_next_day=*
Disallow: /*brand=*
#Disallow: /*?limit=all # optional
# Blocking CMS/system directories
Disallow: /app/
Disallow: /bin/
Disallow: /dev/
Disallow: /lib/
Disallow: /phpserver/
# Do not crawl common server technical files
Disallow: /cgi-bin/
Disallow: /cleanup.php
Disallow: /apc.php
Disallow: /memcache.php
Disallow: /phpinfo.php
# Do not crawl links with session IDs
Disallow: /*?SID=
# =========================================================
# Subdomain blocking
# Place separate robots.txt files on these subdomains:
# release.lsengineers.co.uk/robots.txt
# cdn.lsengineers.co.uk/robots.txt
# archive.lsengineers.co.uk/robots.txt
# =========================================================
# Content:
# User-agent: *
# Disallow: /
# =========================================================
# AI / LLM / GenAI crawlers (explicit allow)
# =========================================================
User-agent: GPTBot # OpenAI
Allow: /
User-agent: ChatGPT-User # OpenAI web crawler
Allow: /
User-agent: OAI-SearchBot # OpenAI search indexing
Allow: /
User-agent: ClaudeBot # Anthropic
Allow: /
User-agent: Claude-Web # Anthropic web crawler
Allow: /
User-agent: PerplexityBot # Perplexity.ai
Allow: /
User-agent: CCBot # Common Crawl
Allow: /
User-agent: Google-Extended # Google AI/extended use
Allow: /
User-agent: Bingbot # Microsoft
Allow: /
| Pozycja | Wyrażenie | Strona | Skrawek |
|---|---|---|---|
| 6 | /chainsaw-spare-part... |