# VIP Transport - robots.txt # https://viptransport.com # ============================================================================ # Wildcard rules — apply to every crawler that doesn't have its own block below. # # Important: when a crawler matches a more-specific User-agent block, it # IGNORES this wildcard block entirely (per Google's spec). So every per-bot # block below repeats the core Disallows. Don't add a per-bot section without # copying these in, or that bot will be free to crawl admin paths and WP junk. # ============================================================================ User-agent: * Allow: / # Sitemaps Sitemap: https://viptransport.com/sitemap.xml # Crawl-delay for politeness (Google ignores this; other crawlers honor it) Crawl-delay: 1 # Admin / internal paths Disallow: /api/ Disallow: /_next/ Disallow: /thank-you Disallow: /studio Disallow: /admin # WordPress-era leftovers — Google still hits these years later because they # were in the old index. Telling Googlebot to stop crawling them frees up # crawl budget for real pages. Most are also 301'd in netlify.toml. Disallow: /wp-admin/ Disallow: /wp-content/ Disallow: /wp-json/ Disallow: /wp-login.php Disallow: /cart Disallow: /cart/ Disallow: /checkout Disallow: /my-account Disallow: /edit-profile Disallow: /log-in Disallow: /register Disallow: /erp-subscription Disallow: /product/ Disallow: /product-category/ Disallow: /category/ Disallow: /tag/ Disallow: /faq_category/ # RSS feeds — never indexed by Google anyway, just wastes crawls Disallow: /feed Disallow: /feed/ Disallow: /*/feed Disallow: /*/feed/ # WP plugin and tracking-parameter duplicate URLs # (MailOptin alone generated ~80 dupe URLs in our GSC report) Disallow: /*?mailoptin-ajax= Disallow: /*?add-to-cart= Disallow: /*?remove_item= Disallow: /*?_wpnonce= Disallow: /*?share= Disallow: /*?wc-ajax= Disallow: /*?ref=aftership # Old WP team-page image gallery artifacts (each photo became a "page") Disallow: /team-vip/img_ Disallow: /team-vip/image- Disallow: /team-vip/15917_ Disallow: /team-vip/camppendletontrees # ============================================================================ # AI assistants — explicit allow for AEO (Answer Engine Optimization) # Listed explicitly because some AI crawlers default to opt-out and we want them in. # Each block repeats the core Disallows so AI bots don't waste cycles on WP junk. # ============================================================================ User-agent: GPTBot Allow: / Disallow: /api/ Disallow: /_next/ Disallow: /admin Disallow: /wp-admin/ Disallow: /wp-login.php Disallow: /cart Disallow: /checkout Disallow: /my-account Disallow: /product/ Disallow: /category/ User-agent: ChatGPT-User Allow: / Disallow: /api/ Disallow: /_next/ Disallow: /admin Disallow: /wp-admin/ User-agent: Claude-Web Allow: / Disallow: /api/ Disallow: /_next/ Disallow: /admin Disallow: /wp-admin/ User-agent: Anthropic-AI Allow: / Disallow: /api/ Disallow: /_next/ Disallow: /admin Disallow: /wp-admin/ User-agent: PerplexityBot Allow: / Disallow: /api/ Disallow: /_next/ Disallow: /admin Disallow: /wp-admin/ User-agent: Google-Extended Allow: / Disallow: /api/ Disallow: /_next/ Disallow: /admin Disallow: /wp-admin/