четверг, 20 июля 2017 г.

Блокируем ботом по юзер-агенту в nginx

if ($http_user_agent ~* "curl|007ac9|008|200PleaseBot|360Spider|360Spider-Image|360Spider-Video|AddThis|AhrefsBot|Alexabot|AMZNKAssocBot/4.0|archive.org_bot|baidu|Baiduspider|Baiduspider-image|BDCbot|BDCbot/1.0|BLEXBot|bsalsa|Butterfly|CCBot|ChangeDetection|coccoc|CrazyWebCrawler-Spider|daumoa|DoCoMo/2.0|dotbot|EasouSpider|Exabot|Ezooms|Findxbot|Findxbot/1.0|Flipboard|Genieo|Gluten Free Crawler|grapeshot|HaoSouSpider|HyperCrawl|ia_archiver|ichiro|ichiro/3.0|ichiro/4.0|InfoWeb|IstellaBot|libwww-perl|Linguee Bot|linkdexbot|linkdexbot/2.0|linkdexbot/2.1|linkdexbot/2.2|LinkpadBot|ltx71|meanpathbot|MegaIndex.ru|MegaIndex.ru/2.0|NaverBot|NerdyBot|netseer|Nutch Bots|oBot|PaperLiBot|phpservermon/3.1.1|PrivacyAwareBot/1.1|proximic|psbot|QuerySeekerSpider|Qwantify|rogerbot|ScreenerBot|SearchmetricsBot|SemrushBot-SA|SF/2 (skipfish)|SiteExplorer|Sogou head spider/3.0|Sogou Orion spider/3.0|Sogou Pic Agent|Sogou Pic Spider/3.0|SputnikBot|Sunrise|SurdotlyBot|Swiftbot|TDJBOT|trovitBot|TurnitinBot|tweetmeme|TweetmemeBot|TwengaBot|TwengaBot-2.0|TwengaBot-Discover|TwengaBot/2.0|UnwindFetchor|Uptimebot|Uptimebot/1.0|UptimeRobot|UptimeRobot/2.0|voltron|WBSearchBot|WeSEE|Wget|wget|wotbox|XoviBot|Y! J-ASR|yacybot|Yeti|YodaoBot|yoozBot-2.2|YoudaoBot/1.0|Yahoo! Slurp|yahoo-slurp|Anews/1.0|Anews|Kribrum|Kribrum/0.1|iskanie|PaperLiBot/2.1|special_archiver/3.1.1|special_archiver|CCBot/2.0|Exabot/3.0|SemrushBot/1.1~bl|SemrushBot/1.1|SemrushBot-SA/0.97|um-FC|um-FC/1.0|um-LN|um-LN/1.0|heritrix|DeuSu/5.0.2|bitlybot|bitlybot/3.0|MaxPointCrawler|GetIntent Crawler|Scrapy|Scrapy/1.2.2|Twitterbot|Twitterbot/1.0|SputnikBot/2.3|Trident/6.0|Trident/5.0|Trident|GuzzleHttp/6.1.0|GuzzleHttp|X11|MSNBot|bingbot|bingbot/2.0|StackRambler|Mail.RU|Mail.RU_Bot/2.0|Mail.RU_Bot|METASpider|Aport|Scooter|trendictionbot|trendictionbot0.5.0|Kraken|Kraken/0.1|DotBot/1.1|BUbiNG|Yeti/1.1|SeznamBot/3.2|WebartexBot|BLEXBot/1.0|spbot/5.0.3|um-IC|um-IC/1.0|GermCrawler|Sogou|GingerCrawler|nutch|Urlbot|ScoutJet|GroupHigh|GroupHigh/1.0|MJ12bot/v1.4.3|MJ12bot/v1.4.0|MJ12bot/v1.2.4|MJ12bot/v1.2.3|MJ12bot/v1.0.8|MJ12bot/v1.0.7|MJ12bot/v1.0.6|MJ12bot/v1.0.5|MJ12bot/v1.4.5|MJ12bot/v1.4.6|MJ12bot/v1.4.7|MJ12bot/v1.4.8|rogerbot/1.0|SemrushBot|MegaIndex|megaIndex.ru|megaindex|crawler4j|Steeler/3.5|Steeler|AhrefsBot/5.1|AhrefsBot/5.2|AhrefsBot/1.0|AhrefsBot/2.0|AhrefsBot/3.0|AhrefsBot/4.0|AhrefsBot/5.0|AhrefsBot/3.1") {
    return 403;
}

Комментариев нет:

Отправить комментарий