User-agent: * #internal search Disallow: */search #Images - Update to match site's DAM path Disallow: */cp-sites/ #Disallow Directives - Add any exclusion paths here - you can make them specific or recursive based on your needs Disallow: */content/ #sitemaps - All sitemaps on a multi-lang domain should be listed here, as robots.txt is domain based, not site based. Sitemap: [https://www.hillspet.co.hu][/sitemap.xml] User-agent: Swiftbot Crawl-delay: 5