Handle WebMeUp bots (BLEXBot) (#158)

BLEXBot is a crawler that is responsible for a good chunk of bot
requests on a website I manage (>200k requests over multiple IPs), so
handling it with legitbot seems like a good idea.

The bot specs are available here: http://webmeup-crawler.com/

Let me know if any changes are needed.

Co-authored-by: Alexander Azarov <[email protected]>

README.md

@@ -55,6 +55,7 @@ end
 - [Applebot](https://support.apple.com/en-us/119829)
 - [Baidu spider](http://help.baidu.com/question?prod_en=master&class=498&id=1000973)
 - [Bingbot](https://blogs.bing.com/webmaster/2012/08/31/how-to-verify-that-bingbot-is-bingbot/)
+- [BLEXBot (WebMeUp)](http://webmeup-crawler.com/)
 - [DataForSEO](https://dataforseo.com/dataforseo-bot)
 - [DuckDuckGo bot](https://duckduckgo.com/duckduckbot)
 - [Google crawlers](https://support.google.com/webmasters/answer/1061943)

lib/legitbot.rb

@@ -8,6 +8,7 @@
 require_relative 'legitbot/apple'
 require_relative 'legitbot/baidu'
 require_relative 'legitbot/bing'
+require_relative 'legitbot/blexbot'
 require_relative 'legitbot/dataforseo'
 require_relative 'legitbot/duckduckgo'
 require_relative 'legitbot/facebook'

lib/legitbot/blexbot.rb

@@ -0,0 +1,10 @@
+# frozen_string_literal: true
+
+module Legitbot # :nodoc:
+  # http://webmeup-crawler.com/
+  class BLEXBot < BotMatch
+    domains 'webmeup.com.'
+  end
+
+  rule Legitbot::BLEXBot, %w[BLEXBot]
+end

test/blexbot_test.rb

@@ -0,0 +1,60 @@
+# frozen_string_literal: true
+
+require_relative 'test_helper'
+
+class BLEXBot < Minitest::Test
+  include Minitest::Hooks
+  include DnsServerMock
+
+  def test_malicious_ip
+    ip = '149.210.164.47'
+    match = Legitbot::BLEXBot.new ip
+
+    refute_predicate match, :valid?
+  end
+
+  def test_valid_ip
+    ip = '65.21.113.197'
+    match = Legitbot::BLEXBot.new ip
+
+    assert_predicate match, :valid?
+  end
+
+  def test_malicious_ua
+    bot = Legitbot.bot(
+      'Mozilla/5.0 (compatible; BLEXBot/1.0; +http://webmeup-crawler.com/)',
+      '149.210.164.47'
+    )
+
+    assert bot
+    refute_predicate bot, :valid?
+  end
+
+  def test_valid_ua
+    bot = Legitbot.bot(
+      'Mozilla/5.0 (compatible; BLEXBot/1.0; +http://webmeup-crawler.com/)',
+      '65.21.113.197'
+    )
+
+    assert bot
+    assert_predicate bot, :valid?
+  end
+
+  def test_valid_name
+    bot = Legitbot.bot(
+      'Mozilla/5.0 (compatible; BLEXBot/1.0; +http://webmeup-crawler.com/)',
+      '65.21.113.197'
+    )
+
+    assert_equal :blexbot, bot.detected_as
+  end
+
+  def test_fake_name
+    bot = Legitbot.bot(
+      'Mozilla/5.0 (compatible; BLEXBot/1.0; +http://webmeup-crawler.com/)',
+      '81.1.172.108'
+    )
+
+    assert_equal :blexbot, bot.detected_as
+  end
+end

test/lib/dns_server_mock.rb

@@ -44,6 +44,13 @@
     ptr: %w[17-58-98-60.applebot.apple.com]
   },
 
+  # BLEXBot (WebMeUp)
+  'pot22.webmeup.com' => {
+    a: %w[65.21.113.197]
+  },
+  '65.21.113.197' => {
+    ptr: %w[pot22.webmeup.com]
+  },
   # DataForSEO
   'crawling-gateway-136-243-228-176.dataforseo.com' => {
     a: %w[136.243.228.176]