nginx 屏蔽爬虫

#禁止指定UA及UA为空的访问

if ($http_user_agent ~ "FeedDemon|JikeSpider|Indy Library|Alexa Toolbar|AskTbFXTV|AhrefsBot|CrawlDaddy|CoolpadWebkit|Java|Feedly|UniversalFeedParser|ApacheBench|Microsoft URL Control|Swiftbot|ZmEu|oBot|jaunty|Python-urllib|lightDeckReports Bot|YYSpider|DigExt|YisouSpider|HttpClient|MJ12bot|heritrix|EasouSpider|LinkpadBot|Ezooms|^$" )
{
return 403;
}

#在Nginx配置文件加入下面代码屏蔽指定 YisouSpider爬虫 多个可以用 | 分开 

if ($http_user_agent ~* (YisouSpider)) {
  return 403;
}

保存后重启Nginx

原文地址:https://www.cnblogs.com/LF-place/p/14681876.html