robots

User-agent: Baiduspider
Disallow: /w?
Allow: /

User-agent: Googlebot
Allow: /

User-agent: Googlebot-Mobile
Allow: /

User-agent: MSNBot
Allow: /

User-agent: Baiduspider-image
Allow: /

User-agent: YoudaoBot
Allow: /

User-agent: Sogou web spider
Allow: /

User-agent: Sogou inst spider
Allow: /

User-agent: Sogou spider2
Allow: /

User-agent: Sogou blog
Allow: /

User-agent: Sogou News Spider
Allow: /

User-agent: Sogou Orion spider
Allow: /

User-agent: ChinasoSpider
Allow: /

User-agent: Sosospider
Allow: /

User-agent: YYspider
Allow: /


User-agent: yisouspider
Allow: /

User-agent: EasouSpider
Allow: /

User-agent: *
Disallow: /
原文地址:https://www.cnblogs.com/freeze89/p/4860245.html