Details
-
Type:
Bug
-
Status: Unverified Fix
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: 3.0.10
-
Fix Version/s: 3.0.12-RC1
-
Component/s: Other
-
Labels:None
-
GitHub Pull Request URL:
Description
We currently match "Baiduspider+(" but it uses a user agent "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)" which is not matched.
We do not have Yandex at all, e.g. "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)"
We do not have EventGutu at all, e.g. "Mozilla/5.0 (compatible; EventGuruBot/1.0; +http://www.eventguru.com/spider.html)"
We should probably also review all other entries to check if they are still up to date.
Activity
What might be a good idea is to check the bots/users table of phpbb.com to see the list of bots and when (if ever) they last visited.
I think it would be safe to remove any that have not visited phpbb.com within the last 2 years.
Also what might be good to find new bots is to get a dump of the sessions table from phpbb.com with just the unique user agents of all guests.
Bots, last visit, and active status from phpbb.com:
mysql> SELECT b.bot_name, FROM_UNIXTIME(u.user_lastvisit) lastvisit, b.bot_agent, b.bot_active
|
FROM users u
|
JOIN bots b ON b.user_id = u.user_id
|
ORDER BY u.user_lastvisit;
|
+---------------------------+---------------------+---------------------------------------------+------------+
|
| bot_name | lastvisit | bot_agent | bot_active |
|
+---------------------------+---------------------+---------------------------------------------+------------+
|
| Orange [Spider] | 1969-12-31 16:00:00 | OrangeSpider | 1 |
|
| SEO Crawler | 1969-12-31 16:00:00 | SEO search Crawler/ | 1 |
|
| Gonzo2 [Bot] | 1969-12-31 16:00:00 | gonzo2[P] | 1 |
|
| ContextAd | 1969-12-31 16:00:00 | contextad live parser | 1 |
|
| Seoma [Crawler] | 1969-12-31 16:00:00 | Seoma [SEO Crawler] | 1 |
|
| WebAlta | 1969-12-31 16:00:00 | WebAlta Crawler | 1 |
|
| McSeek [Bot] | 1969-12-31 16:00:00 | powered by www.McSeek.de | 1 |
|
| SEOSearch [Crawler] | 1969-12-31 16:00:00 | SEOsearch/ | 1 |
|
| 123metaspider [Bot] | 1969-12-31 16:00:00 | 123metaspider-Bot | 1 |
|
| miniRank | 1969-12-31 16:00:00 | miniRank/ | 1 |
|
| Crawler Search [Crawler] | 1969-12-31 16:00:00 | .Crawler-Search.de | 1 |
|
| WissenOnline [Bot] | 1969-12-31 16:00:00 | WissenOnline-Bot | 1 |
|
| 123spider [Bot] | 1969-12-31 16:00:00 | 123spider-Bot | 1 |
|
| Mirago [Bot] | 1969-12-31 16:00:00 | HeinrichderMiragoRobot ( | 1 |
|
| ICCenter [Bot] | 1969-12-31 16:00:00 | iCCrawler ( | 1 |
|
| WWWeasel [Bot] | 1969-12-31 16:00:00 | WWWeasel Robot v | 1 |
|
| 24spider [Bot] | 1969-12-31 16:00:00 | 24spider-Robot | 1 |
|
| MMSBot [Bot] | 1969-12-31 16:00:00 | http://www.mmsweb.at/bot.html | 1 |
|
| MP3-Bot [Bot] | 1969-12-31 16:00:00 | MP3-Bot | 1 |
|
| Scumbot [Bot] | 1969-12-31 16:00:00 | Scumbot/ | 1 |
|
| IDBot | 1969-12-31 16:00:00 | id-search.org | 1 |
|
| Esperanza [Bot] | 1969-12-31 16:00:00 | EsperanzaBot | 1 |
|
| Insuranco [Bot] | 1969-12-31 16:00:00 | InsurancoBot | 1 |
|
| Yoono | 1969-12-31 16:00:00 | Yoono; http://www.yoono.com/ | 1 |
|
| MyFamily [Bot] | 1969-12-31 16:00:00 | MyFamilyBot/ | 1 |
|
| Sproose [Bot] | 1969-12-31 16:00:00 | http://www.sproose.com/bot | 1 |
|
| IPG [Bot] | 1969-12-31 16:00:00 | internet-provider-guenstig.de-Bot | 1 |
|
| Zerx [Bot] | 1969-12-31 16:00:00 | zerxbot/ | 1 |
|
| Advista AdBot | 1969-12-31 16:00:00 | Advista AdBot | 1 |
|
| suchpad [Bot] | 1969-12-31 16:00:00 | http://www.suchpad.de/bot/ | 1 |
|
| Nebulla [Bot] | 1969-12-31 16:00:00 | Nebullabot/ | 1 |
|
| g2 [Crawler] | 1969-12-31 16:00:00 | g2Crawler | 1 |
|
| GBSpider [Spider] | 1969-12-31 16:00:00 | GBSpider v | 1 |
|
| Asterias [Spider] | 1969-12-31 16:00:00 | http://www.singingfish.com/help/spider.html | 1 |
|
| GeigerzaehlerBot [Bot] | 1969-12-31 16:00:00 | http://www.geigerzaehler.org/bot.html | 1 |
|
| jurabot | 1969-12-31 16:00:00 | jurabot/Metasuche | 1 |
|
| BuildCMS [Crawler] | 1969-12-31 16:00:00 | BuildCMS crawler | 1 |
|
| GeoBot [Bot] | 1969-12-31 16:00:00 | GeoBot/version | 1 |
|
| Bunnybot [Bot] | 1969-12-31 16:00:00 | powered by www.buncat.de | 1 |
|
| GeonaBot | 1969-12-31 16:00:00 | GeonaBot/ | 1 |
|
| Onsearch [Bot] | 1969-12-31 16:00:00 | onCHECK-Robot | 1 |
|
| VisBot | 1969-12-31 16:00:00 | VisBot/2 | 1 |
|
| LiteFinder | 1969-12-31 16:00:00 | LiteFinder/ | 1 |
|
| Sensis [Crawler] | 2007-10-12 09:48:39 | Sensis Web Crawler | 1 |
|
| Telekom [Bot] | 2008-03-03 01:08:26 | crawleradmin.t-info@telekom.de | 1 |
|
| Snappy [Bot] | 2008-08-27 03:33:38 | Snappy/1 | 1 |
|
| PDFBot | 2008-09-14 09:31:08 | PDFBot crawler | 1 |
|
| Heise IT-Markt [Crawler] | 2008-10-18 14:34:48 | heise-IT-Markt-Crawler | 1 |
|
| REAP Crawler | 2008-12-08 08:38:08 | REAP-crawler Nutch | 1 |
|
| PlantyNet [Bot] | 2008-12-29 17:30:15 | PlantyNet_WebRobot_V | 1 |
|
| ISC Systems [Bot] | 2009-01-23 07:03:19 | ISC Systems iRc Search | 1 |
|
| IPBot | 2009-02-17 10:17:51 | | 0 |
|
| Pagebull | 2009-03-29 02:25:46 | Pagebull http | 1 |
|
| Orbiter [Bot] | 2009-05-13 05:58:19 | http://www.dailyorbit.com/bot.htm | 1 |
|
| Houxou [Crawler] | 2009-06-01 17:01:42 | HouxouCrawler/ | 1 |
|
| MaSagool | 2009-07-08 12:03:20 | MaSagool/ | 1 |
|
| Gonzo [Bot] | 2009-08-14 11:45:58 | gonzo1[P] | 1 |
|
| Google Sitemaps | 2009-10-05 00:29:11 | Google-Sitemaps/ | 1 |
|
| Synoo [Bot] | 2010-02-02 15:24:47 | SynooBot/ | 1 |
|
| PageFetcher [Google] | 2010-02-06 18:49:25 | PageFetcher-Google-CoOp; | 1 |
|
| FeedHub | 2010-02-10 18:05:18 | FeedHub FeedFetcher/ | 1 |
|
| genie [Bot] | 2010-03-10 09:57:07 | genieBot | 1 |
|
| IBM Research [Bot] | 2010-05-22 06:58:54 | ibm.com/cs/crawler | 1 |
|
| ICCrawler - ICjobs | 2010-08-20 21:50:10 | ICCrawler - ICjobs | 1 |
|
| Metager [Bot] | 2011-01-09 01:39:03 | MetagerBot/ | 1 |
|
| Exabot Images | 2011-03-01 07:21:23 | Exabot-Images/3 | 1 |
|
| Steeler [Crawler] | 2011-04-19 02:21:17 | http://www.tkl.iis.u-tokyo.ac.jp/~crawler/ | 1 |
|
| OmniExplorer [Bot] | 2011-04-24 18:31:03 | OmniExplorer_Bot/ | 1 |
|
| Nutch/CVS [Bot] | 2011-05-10 08:59:34 | NutchCVS/ | 1 |
|
| Twiceler [Bot] | 2011-05-16 18:11:49 | Twiceler-0 | 1 |
|
| Sygol [Bot] | 2011-09-13 00:33:45 | SygolBot | 1 |
|
| NetResearchServer | 2011-09-13 05:06:29 | NetResearchServer/ | 1 |
|
| CazoodleBot | 2011-10-02 03:22:04 | CazoodleBot | 1 |
|
| Yahoo MMCrawler [Bot] | 2011-10-12 19:40:56 | Yahoo-MMCrawler/ | 1 |
|
| Alta Vista [Bot] | 2011-10-12 19:41:00 | Scooter/ | 1 |
|
| Yahoo Slurp [Bot] | 2011-10-13 06:44:38 | Yahoo! DE Slurp | 1 |
|
| SnapBot | 2011-10-22 10:56:04 | Snapbot/ | 1 |
|
| Bloglines | 2011-10-25 21:15:34 | Bloglines/ | 1 |
|
| Kraehe [Metasuche] | 2011-11-17 18:16:01 | -DIE-KRAEHE- META-SEARCH-ENGINE/ | 1 |
|
| Francis [Bot] | 2011-11-19 00:49:39 | http://www.neomo.de/ | 1 |
|
| Webmeasurement [Bot] | 2011-12-09 04:13:53 | webmeasurement-bot, | 1 |
|
| IRLbot [Bot] | 2012-01-03 03:00:38 | http://irl.cs.tamu.edu/crawler | 1 |
|
| NG-Search [Bot] | 2012-02-18 15:06:09 | NG-Search/ | 1 |
|
| LinkWalker | 2012-03-01 08:13:24 | LinkWalker | 1 |
|
| Acoon-Robot | 2012-03-04 01:59:12 | Acoon-Robot | 1 |
|
| Seekport [Bot] | 2012-03-29 06:29:20 | Seekbot/ | 1 |
|
| Cometrics [Bot] | 2012-04-12 16:31:22 | cometrics-bot, | 1 |
|
| Walhello [Bot] | 2012-04-20 06:14:12 | appie 1.1 (www.walhello.com) | 1 |
|
| Online link [Validator] | 2012-04-22 12:05:22 | online link validator | 1 |
|
| Findexa [Crawler] | 2012-04-24 23:08:48 | Findexa Crawler ( | 1 |
|
| EliteSys [Bot] | 2012-04-29 06:51:42 | EliteSys SuperBot/ | 1 |
|
| ASPseek [Crawler] | 2012-05-09 00:45:26 | ASPseek/ | 1 |
|
| WiseNut [Bot] | 2012-05-21 22:31:32 | http://www.WISEnutbot.com | 1 |
|
| Speedy Spider | 2012-05-24 02:29:21 | Speedy Spider | 1 |
|
| Naver [Bot] | 2012-06-01 08:31:42 | nhnbot@naver.com) | 1 |
|
| Asterias [Bot] | 2012-06-05 22:36:33 | asterias/ | 1 |
|
| Ocelli [Bot] | 2012-06-06 05:34:08 | Ocelli/ | 1 |
|
| Disco Nutch [Crawler] | 2012-06-11 19:08:36 | disco/Nutch | 1 |
|
| Cosmix [Bot] | 2012-06-23 03:09:24 | cfetch/ | 1 |
|
| Nimble [Crawler] | 2012-06-23 17:30:04 | NimbleCrawler | 1 |
|
| Voyager [Bot] | 2012-07-07 11:00:56 | voyager/1.0 | 1 |
|
| Hogsearch [Bot] | 2012-07-12 23:22:35 | oegp v. 1.3.0 | 1 |
|
| Netluchs | 2012-07-14 21:10:42 | Netluchs/Nutch | 1 |
|
| Archive Org | 2012-07-17 05:39:35 | archive.org_bot/ | 1 |
|
| Nutch [Bot] | 2012-07-20 00:01:13 | http://lucene.apache.org/nutch/ | 1 |
|
| Java 1 | 2012-07-21 11:58:12 | Java/1 | 1 |
|
| Jyxobot [Bot] | 2012-07-23 19:31:01 | Jyxobot/ | 1 |
|
| Accoona [Bot] | 2012-07-27 01:09:16 | Accoona-AI-Agent/ | 1 |
|
| MSN NewsBlogs | 2012-07-27 19:52:52 | msnbot-NewsBlogs/ | 1 |
|
| Boitho [Crawler] | 2012-07-28 14:47:13 | boitho.com-dc/ | 1 |
|
| Thumbshots [Capture] | 2012-07-28 21:00:29 | thumbshots-de-Bot | 1 |
|
| proximic | 2012-07-31 03:55:02 | proximic; | 1 |
|
| MSRBOT | 2012-07-31 04:27:33 | MSRBOT | 1 |
|
| TurnitinBot [Bot] | 2012-07-31 10:34:33 | TurnitinBot/ | 1 |
|
| Ask Jeeves [Bot] | 2012-07-31 13:55:49 | Ask Jeeves | 1 |
|
| FAST WebCrawler [Crawler] | 2012-08-01 07:25:04 | FAST-WebCrawler/ | 1 |
|
| Xaldon [Spider] | 2012-08-01 14:55:41 | Xaldon WebSpider | 1 |
|
| W3 [Sitesearch] | 2012-08-04 13:04:07 | W3 SiteSearch Crawler | 1 |
|
| YaCy [Bot] | 2012-08-05 02:33:34 | yacybot | 1 |
|
| Susie [Sync] | 2012-08-05 05:38:33 | !Susie (http://www.sync2it.com/susie) | 1 |
|
| W3C [Linkcheck] | 2012-08-05 12:11:00 | W3C-checklink/ | 1 |
|
| psbot [Picsearch] | 2012-08-05 18:45:20 | psbot/0 | 1 |
|
| YahooSeeker [Bot] | 2012-08-06 10:23:37 | YahooSeeker/ | 1 |
|
| FAST Enterprise [Crawler] | 2012-08-06 20:59:18 | FAST Enterprise Crawler | 1 |
|
| Vagabondo [Crawler] | 2012-08-06 21:22:08 | http://webagent.wise-guys.nl/ | 1 |
|
| Baidu [Spider] | 2012-08-07 03:34:56 | Baiduspider+( | 1 |
|
| AdsBot [Google] | 2012-08-07 08:02:26 | AdsBot-Google | 1 |
|
| W3C [Validator] | 2012-08-07 09:24:38 | W3C_*Validator | 1 |
|
| Heritrix [Crawler] | 2012-08-07 09:41:57 | heritrix/1. | 1 |
|
| Google Adsense [Bot] | 2012-08-07 12:43:00 | mediapartners-google | 1 |
|
| Alexa [Bot] | 2012-08-07 13:23:27 | ia_archiver | 1 |
|
| Sogou [Spider] | 2012-08-07 13:23:42 | Sogou web spider | 1 |
|
| MSNbot Media | 2012-08-07 13:32:05 | msnbot-media/ | 1 |
|
| Magpie [RSS] | 2012-08-07 13:44:57 | MagpieRSS/ | 1 |
|
| Exabot [Bot] | 2012-08-07 13:57:46 | Exabot/ | 1 |
|
| omgili [Bot] | 2012-08-07 14:18:04 | omgilibot/ | 1 |
|
| ichiro [Crawler] | 2012-08-07 14:27:44 | ichiro/ | 1 |
|
| MSN [Bot] | 2012-08-07 14:31:59 | msnbot/ | 1 |
|
| Gigabot [Bot] | 2012-08-07 14:32:12 | Gigabot/ | 1 |
|
| NetNewsWire | 2012-08-07 14:33:08 | NetNewsWire/ | 1 |
|
| Google Desktop | 2012-08-07 14:34:56 | Google Desktop | 1 |
|
| NewsFire | 2012-08-07 14:49:02 | NewsFire/ | 1 |
|
| Google Feedfetcher | 2012-08-07 14:50:56 | Feedfetcher-Google | 1 |
|
| Tbot [Bot] | 2012-08-07 14:51:05 | Tbot/ | 1 |
|
| Yandex [Bot] | 2012-08-07 14:51:13 | Yandex | 1 |
|
| oBot | 2012-08-07 14:51:16 | oBot | 1 |
|
| FeedReader | 2012-08-07 14:51:38 | Feedreader | 1 |
|
| Bing [Bot] | 2012-08-07 14:51:38 | bingbot/ | 1 |
|
| Yahoo [Bot] | 2012-08-07 14:51:41 | Yahoo! Slurp | 1 |
|
| Majestic-12 [Bot] | 2012-08-07 14:51:55 | MJ12bot/ | 1 |
|
| Google [Bot] | 2012-08-07 14:51:56 | Googlebot | 1 |
|
+---------------------------+---------------------+---------------------------------------------+------------+
|
151 rows in set (0.01 sec)
|
Excluding Mozilla and Opera as browser agents for current sessions on phpbb.com:
mysql> SELECT DISTINCT session_browser FROM sessions
|
WHERE session_browser NOT LIKE 'Mozilla/%' AND session_browser NOT LIKE 'Opera/%'
|
ORDER BY session_browser;
|
+-------------------------------------------------------------------------------------------------------------------------------------------------------+
|
| session_browser |
|
+-------------------------------------------------------------------------------------------------------------------------------------------------------+
|
| |
|
| Akregator/1.6.5; syndication |
|
| Akregator/4.8.3; syndication |
|
| Akregator/4.8.4; syndication SUSE |
|
| Akregator/4.9; syndication |
|
| AppEngine-Google; (+http://code.google.com/appengine; appid: varlopie) |
|
| AppEngine-Google; (+http://code.google.com/appengine; appid: webproxy8-9) |
|
| Apple-PubSub/28 |
|
| Apple-PubSub/65.21 |
|
| Apple-PubSub/65.23 |
|
| Apple-PubSub/65.28 |
|
| Apple-PubSub/65.32 |
|
| AppleSyndication/56.1 |
|
| Baiduspider-image+(+http://www.baidu.com/search/spider.htm) |
|
| BlackBerry8520/5.0.0.1036 Profile/MIDP-2.1 Configuration/CLDC-1.1 VendorID/132 |
|
| BlogPulse (ISSpider-3.0; http://www.blogpulse.com; compatible; MSIE 6.0) |
|
| Blogtrottr/2.0 |
|
| COMODOSpider/Nutch-1.2 |
|
| ColorizeIt |
|
| DoCoMo/2.0 SH902i (compatible; Y!J-SRD/1.0; http://help.yahoo.co.jp/help/jp/search/indexing/indexing-27.html) |
|
| ELinks/0.11.1 (textmode; Linux; -) |
|
| FeedBurner/1.0 (http://www.FeedBurner.com) |
|
| FeedDemon/4.0 (http://www.feeddemon.com/; Microsoft Windows) |
|
| Feedfetcher-Google; (+http://www.google.com/feedfetcher.html; 1 subscribers; feed-id=1155424950123321032) |
|
| Feedreader 3.14 (Powered by Newsbrain) |
|
| Fever/1.25 (Feed Parser; http://feedafever.com; Allow like Gecko) |
|
| GG PeekBot 1.0 ( http://gg.pl/ http://info.gadu-gadu.pl/praca ) |
|
| Gigabot/3.0 (http://www.gigablast.com/spider.html) |
|
| GreatNews/1.0 |
|
| InetURL:/1.0 |
|
| Installatron Plugin/8.0.3 |
|
| Liferea/1.6.6b (Linux; en_US.UTF-8; http://liferea.sf.net/) |
|
| Liferea/1.8.6 (Linux; de_DE.UTF-8; http://liferea.sf.net/) |
|
| LiveJournal.com (webmaster@livejournal.com; for http://www.livejournal.com/users/phpbb_rss/; 1 readers) |
|
| MagpieRSS/0.72 (+http://magpierss.sf.net) |
|
| Microsoft URL Control - 6.00.8862 |
|
| Microsoft-WebDAV-MiniRedir/6.1.7601 |
|
| Motorola |
|
| Mozila/4.0 (compatible; MSIE 6.0; Windows NT 5.1;) |
|
| NetNewsWire/3.1.6 (Mac OS X; http://www.newsgator.com/Individuals/NetNewsWire/) |
|
| Netvibes (http://www.netvibes.com/; 1 subscribers; feedID: 1750138) |
|
| Netvibes (http://www.netvibes.com/; 1 subscribers; feedID: 6056160) |
|
| Netvibes (http://www.netvibes.com/; 2 subscribers; feedID: 5285318) |
|
| NewsFire/85 CFNetwork/596.0.1 Darwin/12.0.0 (x86_64) (MacBookPro10%2C1) |
|
| NewspointBot/1.0 (http://www.netsprint.eu/) |
|
| Nokia7610/2.0 (7.0642.0) SymbianOS/7.0s Series60/2.1 Profile/MIDP-2.0 Configuration/CLDC-1.0/UC Browser7.9.1.120/27/350/UCWEB |
|
| NokiaC1-01/2.0 (05.45) Profile/MIDP-2.1 Configuration/CLDC-1.1 |
|
| NokiaE63 |
|
| PHP/5.3.10 |
|
| Planeta Software Libre +http://planetasl.bilson.cl Planet/2.0 +http://www.planetplanet.org UniversalFeedParser/4.1 +http://feedparser.org/ |
|
| Python-httplib2/0.7.2 (gzip) |
|
| RSS Menu/2.1.3 (Mac OS X; http://www.edotstudios.com) |
|
| RSSMirandaPlugin/1.0 |
|
| RSSOwl/2.1.2.201108131746 (Windows; U; en) |
|
| RSSOwl/2.1.2.201108151941 (Windows; U; en) |
|
| RSSOwl/2.1.4.201207281409 (Windows; U; en) |
|
| RSSOwl/2.1.4.201207281417 (Windows; U; en) |
|
| Read ahead agent 1.0 |
|
| Reeder/1.5.5 CFNetwork/548.1.4 Darwin/11.0.0 |
|
| Reeder/1010.79.00 CFNetwork/596.0.1 Darwin/12.0.0 (x86_64) (MacBook7%2C1) |
|
| SAMSUNG-SGH-B130/B130XEHF1 UP.Browser/6.2.3.3.c.1.101 (GUI) MMP/2.0 |
|
| SimplePie/1.0.1 (Feed Parser; http://simplepie.org/; Allow like Gecko) Build/20070719221955 |
|
| SimplePie/1.1.3 (Feed Parser; http://simplepie.org; Allow like Gecko) Build/20081219 |
|
| SimplePie/1.2 (Feed Parser; http://simplepie.org; Allow like Gecko) Build/20090627192103 |
|
| SimplePie/1.2.1-dev (Feed Parser; http://simplepie.org; Allow like Gecko) Build/20110723163749 |
|
| Snarfer/1.0.2 (http://www.snarfware.com/) |
|
| Sosospider+(+http://help.soso.com/webspider.htm) |
|
| Superfeedr: Superparser bot/1.1 http://superfeedr.com - Please read this http://blog.superfeedr.com/publishers.html or get in touch if we are polling |
|
| Tiny Tiny RSS/1.5.11 (http://tt-rss.org/) |
|
| Tiny Tiny RSS/1.5.8.1 (http://tt-rss.org/) |
|
| TosCrawler/Nutch-1.4 (http://www.toshiba.co.jp/rdc/about/crawl_info.htm; 'Rdc-crawler at ml dot toshiba dot co dot jp') |
|
| TwitterFeed 3 |
|
| Vienna/2.6.0.2601 |
|
| Wget/1.12 (linux-gnu) |
|
| Wget/1.13.4 (linux-gnu) |
|
| Windows-Live-Social-Object-Extractor-Engine/1.0 |
|
| Windows-RSS-Platform/2.0 (MSIE 9.0; Windows NT 6.1) |
|
| Xydo v0.1; (Contact: Xydo <info@xydo.com>) |
|
| Yahoo Pipes 2.0 |
|
| Z-Add Link Checker (http://w3.z-add.co.uk/linkcheck/) |
|
| check_http/v1.4.15 (nagios-plugins 1.4.15) |
|
| checks.panopta.com |
|
| coccoc/1.0 () |
|
| curl/7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5 |
|
| curl/7.21.2 (i386-pc-solaris2.11) libcurl/7.21.2 OpenSSL/1.0.0e zlib/1.2.3 |
|
| facebookexternalhit/1.0 (+http://www.facebook.com/externalhit_uatext.php) |
|
| facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php) |
|
| ichiro/3.0 (http://help.goo.ne.jp/door/crawler.html) |
|
| lastRSS |
|
| linkdex.com/v2.0 |
|
| livedoor FeedFetcher/0.01 (http://reader.livedoor.com/; 1 subscriber) |
|
| magpie-crawler/1.1 (U; Linux amd64; en-GB; +http://www.brandwatch.net) |
|
| msnbot-UDiscovery/2.0b (+http://search.msn.com/msnbot.htm) |
|
| msnbot-media/1.1 (+http://search.msn.com/msnbot.htm) |
|
| msnbot/2.0b (+http://search.msn.com/msnbot.htm) |
|
| newsbeuter/2.5 (Linux 3.4.2; armv5tel; http://www.newsbeuter.org/) libcurl/7.25.0 OpenSSL/1.0.0j zlib/1.2.5.1 libidn/1.25 |
|
| omgilibot/0.3 +http://www.omgili.com/Crawler.html |
|
| rss2email/2.71 +http://www.allthingsrss.com/rss2email/ |
|
| ssearch_bot (sSearch Crawler; http://www.semantissimo.de) |
|
| vBulletin via cURL/PHP |
|
+-------------------------------------------------------------------------------------------------------------------------------------------------------+
|
100 rows in set (0.02 sec)
|
In phpBB.com bot list there are some bots that there aren't in phpBB bot list.
In the bot list of phpBB there are also bots that operate only in some countries. See German bots/crawlers.
So should I remove my discretion bots who have never visited phpBB.com from the list?
Opened new pull request for merging request on develop-olympus instead of on develop.
The existing list has been updated.
Yandex and EventGutu aren't in the list. Should I add them?