Frequently Asked Questions
Need information on this product? Have questions? No worries we have put together some of our most commonly asked questions to skip the need of wondering or needing to call.
User-agent: * # Block private/system areas Disallow: /admin/ Disallow: /cart Disallow: /checkout Disallow: /orders Disallow: /account Disallow: /apps/ Disallow: /carts Disallow: /payments/ Disallow: /gift_cards Disallow: /apple-app-site-association # Block internal search (thin/duplicate pages) Disallow: /search Disallow: /search/ # Allow common assets & important resources Allow: /assets/ Allow: /files/ Allow: /cdn/ # Sitemaps Sitemap: https://dtfrushorders.com/sitemap.xml # Major search engine bots User-agent: Googlebot Allow: / User-agent: Googlebot-Image Allow: / User-agent: Googlebot-Mobile Allow: / User-agent: AdsBot-Google Allow: / User-agent: Bingbot Allow: / User-agent: Slurp Allow: / User-agent: DuckDuckBot Allow: / User-agent: Applebot Allow: / # Major AI/Research crawlers — explicitly allow User-agent: GPTBot Allow: / User-agent: Google-Extended Allow: / User-agent: CCBot Allow: / User-agent: PerplexityBot Allow: / User-agent: ClaudeBot Allow: / User-agent: Amazonbot Allow: / # Rate-limit aggressive non-Google crawlers that respect crawl-delay User-agent: AhrefsBot Crawl-delay: 10 User-agent: MJ12bot Crawl-delay: 10 User-agent: SemrushBot Crawl-delay: 10 User-agent: DotBot Crawl-delay: 10 # Yandex Host directive (harmless elsewhere) Host: dtfrushorders.com