##Please don't remove folders from disallow. ##The allows at the top allow any of the mimetypes listed to be crawled within any folder ##using long-tail wildcards, these ignore the disallows for the folders below. ##This gives full render for the search engines whilst preventing full crawls of system ##folders User-agent: * #THIS ALLOWS FULL RENDER AT ENGINES Allow: /*.js*************** Allow: /*.css************** Allow: /*.webp************** Allow: /*.png************** Allow: /*.jpg************** Allow: /*.jpeg************** Allow: /*.gif************** Allow: /*.eot************** Allow: /*.woff************** Allow: /*.woff2************** Allow: /*.ttf************** Allow: /*.svg************** Allow: /*.otf************** Allow: /*.pdf************** Allow: /*.PNG************** Allow: /*.JPG************** Allow: /*.JPEG************** Allow: /*.mp3************** Allow: /*.mp4************** Allow: /*.webm************** Allow: /*.pdf************** #THESE FOLDERS SHOULD NEVER BE CRAWLED Disallow: /administrator/ Disallow: /cache/ Disallow: /cgi-bin/ Disallow: /cli/ Disallow: /includes/ Disallow: /language/ Disallow: /libraries/ Disallow: /logs/ Disallow: /tmp/ Disallow: /test/ Disallow: /test/* Disallow: /*/edit User-agent: MJ12bot Disallow: / User-agent: AhrefsBot Disallow: / User-Agent: W3C_Validator Disallow: / User-Agent: W3C_CSS_Validator Disallow: / User-agent: YoudaoBot Disallow: / User-Agent: YodaoBot Disallow: / User-Agent: Yeti Disallow: / User-Agent: ichiro Disallow: / User-Agent: Acoon Disallow: / User-agent: Yandex Disallow: / User-agent: pingdom Disallow: / User-agent: proximic Disallow: / User-agent: Phantom.js bot Disallow: / User-agent: Offline Explorer Disallow: / User-agent: FavIconizer Disallow: / User-agent: yie8 Disallow: / User-agent: SeznamBot Disallow: / User-agent: A6-Indexer/1.0 Disallow: / User-agent: A6-Indexer Disallow: / User-agent: butterfly Disallow: / User-agent: TurnitinBot Disallow: / User-agent: voltron Disallow: / User-agent: oBot Disallow: / User-agent: BUbiNG Disallow: / User-agent: wotbox Disallow: / Sitemap:https://daftartelefon.com/index.php?option=com_jmap&view=sitemap&format=xml&dataset=1 Sitemap:https://daftartelefon.com/index.php?option=com_jmap&view=sitemap&format=xml&dataset=2 Sitemap:https://daftartelefon.com/index.php?option=com_jmap&view=sitemap&format=xml&dataset=6 Sitemap:https://daftartelefon.com/index.php?option=com_jmap&view=sitemap&format=xml&dataset=8