User-agent: * Disallow: /banner-icon.svg # ------ Legacy Next.js paths (no longer used, still crawled) ------ Disallow: /_next/ # ------ URL parameters: pagination & sorting (noindex via X-Robots-Tag, but waste crawl budget) ------ Disallow: /*?page= Disallow: /*?sort= Disallow: /*&page= Disallow: /*&sort= # ------ Internal API endpoints (JSON, not for indexing) ------ Disallow: /public/meta/ Disallow: /public/ctx/ Disallow: /public/faqs/ Disallow: /public/feedbacks/ Disallow: /public/suite-workers/ Disallow: /public/env/ Disallow: /public/notifications/ # ------ WordPress/scanner probes (not a WP site) ------ Disallow: /wp-admin/ Disallow: /wp-content/ Disallow: /wp-includes/ Disallow: /cgi-bin/ # ------ Block aggressive/useless bots ------ User-agent: Bytespider Disallow: / User-agent: Amazonbot Disallow: / User-agent: CCBot Disallow: / # Block aggressive SEO bots (no benefit, high volume) User-agent: AhrefsBot Disallow: / User-agent: SemrushBot Disallow: / User-agent: DotBot Disallow: / User-agent: MJ12bot Disallow: / User-agent: DataForSeoBot Disallow: / User-agent: BLEXBot Disallow: / User-agent: serpstatbot Disallow: / User-agent: Sogou Disallow: / User-agent: Baiduspider Disallow: / Sitemap: https://redsale.by/sitemap.xml