Details

    • Type: Bug
    • Status: Unverified Fix
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.10
    • Fix Version/s: 3.0.12-RC1
    • Component/s: Other
    • Labels:
      None

      Description

      We currently match "Baiduspider+(" but it uses a user agent "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)" which is not matched.

      We do not have Yandex at all, e.g. "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)"

      We do not have EventGutu at all, e.g. "Mozilla/5.0 (compatible; EventGuruBot/1.0; +http://www.eventguru.com/spider.html)"

      We should probably also review all other entries to check if they are still up to date.

        Activity

        Hide
        carlino1994 carlino1994 added a comment -

        The existing list has been updated.

        Yandex and EventGutu aren't in the list. Should I add them?

        Show
        carlino1994 carlino1994 added a comment - The existing list has been updated. Yandex and EventGutu aren't in the list. Should I add them?
        Hide
        EXreaction EXreaction [X] (Inactive) added a comment -

        What might be a good idea is to check the bots/users table of phpbb.com to see the list of bots and when (if ever) they last visited.

        I think it would be safe to remove any that have not visited phpbb.com within the last 2 years.

        Also what might be good to find new bots is to get a dump of the sessions table from phpbb.com with just the unique user agents of all guests.

        Show
        EXreaction EXreaction [X] (Inactive) added a comment - What might be a good idea is to check the bots/users table of phpbb.com to see the list of bots and when (if ever) they last visited. I think it would be safe to remove any that have not visited phpbb.com within the last 2 years. Also what might be good to find new bots is to get a dump of the sessions table from phpbb.com with just the unique user agents of all guests.
        Hide
        DavidIQ David Colón added a comment -

        Bots, last visit, and active status from phpbb.com:

        mysql> SELECT b.bot_name, FROM_UNIXTIME(u.user_lastvisit) lastvisit, b.bot_agent, b.bot_active
        FROM users u
        JOIN bots b ON b.user_id = u.user_id
        ORDER BY u.user_lastvisit;
        +---------------------------+---------------------+---------------------------------------------+------------+
        | bot_name                  | lastvisit           | bot_agent                                   | bot_active |
        +---------------------------+---------------------+---------------------------------------------+------------+
        | Orange [Spider]           | 1969-12-31 16:00:00 | OrangeSpider                                |          1 |
        | SEO Crawler               | 1969-12-31 16:00:00 | SEO search Crawler/                         |          1 |
        | Gonzo2 [Bot]              | 1969-12-31 16:00:00 | gonzo2[P]                                   |          1 |
        | ContextAd                 | 1969-12-31 16:00:00 | contextad live parser                       |          1 |
        | Seoma [Crawler]           | 1969-12-31 16:00:00 | Seoma [SEO Crawler]                         |          1 |
        | WebAlta                   | 1969-12-31 16:00:00 | WebAlta Crawler                             |          1 |
        | McSeek [Bot]              | 1969-12-31 16:00:00 | powered by www.McSeek.de                    |          1 |
        | SEOSearch [Crawler]       | 1969-12-31 16:00:00 | SEOsearch/                                  |          1 |
        | 123metaspider [Bot]       | 1969-12-31 16:00:00 | 123metaspider-Bot                           |          1 |
        | miniRank                  | 1969-12-31 16:00:00 | miniRank/                                   |          1 |
        | Crawler Search [Crawler]  | 1969-12-31 16:00:00 | .Crawler-Search.de                          |          1 |
        | WissenOnline [Bot]        | 1969-12-31 16:00:00 | WissenOnline-Bot                            |          1 |
        | 123spider [Bot]           | 1969-12-31 16:00:00 | 123spider-Bot                               |          1 |
        | Mirago [Bot]              | 1969-12-31 16:00:00 | HeinrichderMiragoRobot (                    |          1 |
        | ICCenter [Bot]            | 1969-12-31 16:00:00 | iCCrawler (                                 |          1 |
        | WWWeasel [Bot]            | 1969-12-31 16:00:00 | WWWeasel Robot v                            |          1 |
        | 24spider [Bot]            | 1969-12-31 16:00:00 | 24spider-Robot                              |          1 |
        | MMSBot [Bot]              | 1969-12-31 16:00:00 | http://www.mmsweb.at/bot.html               |          1 |
        | MP3-Bot [Bot]             | 1969-12-31 16:00:00 | MP3-Bot                                     |          1 |
        | Scumbot [Bot]             | 1969-12-31 16:00:00 | Scumbot/                                    |          1 |
        | IDBot                     | 1969-12-31 16:00:00 | id-search.org                               |          1 |
        | Esperanza [Bot]           | 1969-12-31 16:00:00 | EsperanzaBot                                |          1 |
        | Insuranco [Bot]           | 1969-12-31 16:00:00 | InsurancoBot                                |          1 |
        | Yoono                     | 1969-12-31 16:00:00 | Yoono; http://www.yoono.com/                |          1 |
        | MyFamily [Bot]            | 1969-12-31 16:00:00 | MyFamilyBot/                                |          1 |
        | Sproose [Bot]             | 1969-12-31 16:00:00 | http://www.sproose.com/bot                  |          1 |
        | IPG [Bot]                 | 1969-12-31 16:00:00 | internet-provider-guenstig.de-Bot           |          1 |
        | Zerx [Bot]                | 1969-12-31 16:00:00 | zerxbot/                                    |          1 |
        | Advista AdBot             | 1969-12-31 16:00:00 | Advista AdBot                               |          1 |
        | suchpad [Bot]             | 1969-12-31 16:00:00 | http://www.suchpad.de/bot/                  |          1 |
        | Nebulla [Bot]             | 1969-12-31 16:00:00 | Nebullabot/                                 |          1 |
        | g2 [Crawler]              | 1969-12-31 16:00:00 | g2Crawler                                   |          1 |
        | GBSpider [Spider]         | 1969-12-31 16:00:00 | GBSpider v                                  |          1 |
        | Asterias [Spider]         | 1969-12-31 16:00:00 | http://www.singingfish.com/help/spider.html |          1 |
        | GeigerzaehlerBot [Bot]    | 1969-12-31 16:00:00 | http://www.geigerzaehler.org/bot.html       |          1 |
        | jurabot                   | 1969-12-31 16:00:00 | jurabot/Metasuche                           |          1 |
        | BuildCMS [Crawler]        | 1969-12-31 16:00:00 | BuildCMS crawler                            |          1 |
        | GeoBot [Bot]              | 1969-12-31 16:00:00 | GeoBot/version                              |          1 |
        | Bunnybot [Bot]            | 1969-12-31 16:00:00 | powered by www.buncat.de                    |          1 |
        | GeonaBot                  | 1969-12-31 16:00:00 | GeonaBot/                                   |          1 |
        | Onsearch [Bot]            | 1969-12-31 16:00:00 | onCHECK-Robot                               |          1 |
        | VisBot                    | 1969-12-31 16:00:00 | VisBot/2                                    |          1 |
        | LiteFinder                | 1969-12-31 16:00:00 | LiteFinder/                                 |          1 |
        | Sensis [Crawler]          | 2007-10-12 09:48:39 | Sensis Web Crawler                          |          1 |
        | Telekom [Bot]             | 2008-03-03 01:08:26 | crawleradmin.t-info@telekom.de              |          1 |
        | Snappy [Bot]              | 2008-08-27 03:33:38 | Snappy/1                                    |          1 |
        | PDFBot                    | 2008-09-14 09:31:08 | PDFBot crawler                              |          1 |
        | Heise IT-Markt [Crawler]  | 2008-10-18 14:34:48 | heise-IT-Markt-Crawler                      |          1 |
        | REAP Crawler              | 2008-12-08 08:38:08 | REAP-crawler Nutch                          |          1 |
        | PlantyNet [Bot]           | 2008-12-29 17:30:15 | PlantyNet_WebRobot_V                        |          1 |
        | ISC Systems [Bot]         | 2009-01-23 07:03:19 | ISC Systems iRc Search                      |          1 |
        | IPBot                     | 2009-02-17 10:17:51 |                                             |          0 |
        | Pagebull                  | 2009-03-29 02:25:46 | Pagebull http                               |          1 |
        | Orbiter [Bot]             | 2009-05-13 05:58:19 | http://www.dailyorbit.com/bot.htm           |          1 |
        | Houxou [Crawler]          | 2009-06-01 17:01:42 | HouxouCrawler/                              |          1 |
        | MaSagool                  | 2009-07-08 12:03:20 | MaSagool/                                   |          1 |
        | Gonzo [Bot]               | 2009-08-14 11:45:58 | gonzo1[P]                                   |          1 |
        | Google Sitemaps           | 2009-10-05 00:29:11 | Google-Sitemaps/                            |          1 |
        | Synoo [Bot]               | 2010-02-02 15:24:47 | SynooBot/                                   |          1 |
        | PageFetcher [Google]      | 2010-02-06 18:49:25 | PageFetcher-Google-CoOp;                    |          1 |
        | FeedHub                   | 2010-02-10 18:05:18 | FeedHub FeedFetcher/                        |          1 |
        | genie [Bot]               | 2010-03-10 09:57:07 | genieBot                                    |          1 |
        | IBM Research [Bot]        | 2010-05-22 06:58:54 | ibm.com/cs/crawler                          |          1 |
        | ICCrawler - ICjobs        | 2010-08-20 21:50:10 | ICCrawler - ICjobs                          |          1 |
        | Metager [Bot]             | 2011-01-09 01:39:03 | MetagerBot/                                 |          1 |
        | Exabot Images             | 2011-03-01 07:21:23 | Exabot-Images/3                             |          1 |
        | Steeler [Crawler]         | 2011-04-19 02:21:17 | http://www.tkl.iis.u-tokyo.ac.jp/~crawler/  |          1 |
        | OmniExplorer [Bot]        | 2011-04-24 18:31:03 | OmniExplorer_Bot/                           |          1 |
        | Nutch/CVS [Bot]           | 2011-05-10 08:59:34 | NutchCVS/                                   |          1 |
        | Twiceler [Bot]            | 2011-05-16 18:11:49 | Twiceler-0                                  |          1 |
        | Sygol [Bot]               | 2011-09-13 00:33:45 | SygolBot                                    |          1 |
        | NetResearchServer         | 2011-09-13 05:06:29 | NetResearchServer/                          |          1 |
        | CazoodleBot               | 2011-10-02 03:22:04 | CazoodleBot                                 |          1 |
        | Yahoo MMCrawler [Bot]     | 2011-10-12 19:40:56 | Yahoo-MMCrawler/                            |          1 |
        | Alta Vista [Bot]          | 2011-10-12 19:41:00 | Scooter/                                    |          1 |
        | Yahoo Slurp [Bot]         | 2011-10-13 06:44:38 | Yahoo! DE Slurp                             |          1 |
        | SnapBot                   | 2011-10-22 10:56:04 | Snapbot/                                    |          1 |
        | Bloglines                 | 2011-10-25 21:15:34 | Bloglines/                                  |          1 |
        | Kraehe [Metasuche]        | 2011-11-17 18:16:01 | -DIE-KRAEHE- META-SEARCH-ENGINE/            |          1 |
        | Francis [Bot]             | 2011-11-19 00:49:39 | http://www.neomo.de/                        |          1 |
        | Webmeasurement [Bot]      | 2011-12-09 04:13:53 | webmeasurement-bot,                         |          1 |
        | IRLbot [Bot]              | 2012-01-03 03:00:38 | http://irl.cs.tamu.edu/crawler              |          1 |
        | NG-Search [Bot]           | 2012-02-18 15:06:09 | NG-Search/                                  |          1 |
        | LinkWalker                | 2012-03-01 08:13:24 | LinkWalker                                  |          1 |
        | Acoon-Robot               | 2012-03-04 01:59:12 | Acoon-Robot                                 |          1 |
        | Seekport [Bot]            | 2012-03-29 06:29:20 | Seekbot/                                    |          1 |
        | Cometrics [Bot]           | 2012-04-12 16:31:22 | cometrics-bot,                              |          1 |
        | Walhello [Bot]            | 2012-04-20 06:14:12 | appie 1.1 (www.walhello.com)                |          1 |
        | Online link [Validator]   | 2012-04-22 12:05:22 | online link validator                       |          1 |
        | Findexa [Crawler]         | 2012-04-24 23:08:48 | Findexa Crawler (                           |          1 |
        | EliteSys [Bot]            | 2012-04-29 06:51:42 | EliteSys SuperBot/                          |          1 |
        | ASPseek [Crawler]         | 2012-05-09 00:45:26 | ASPseek/                                    |          1 |
        | WiseNut [Bot]             | 2012-05-21 22:31:32 | http://www.WISEnutbot.com                   |          1 |
        | Speedy Spider             | 2012-05-24 02:29:21 | Speedy Spider                               |          1 |
        | Naver [Bot]               | 2012-06-01 08:31:42 | nhnbot@naver.com)                           |          1 |
        | Asterias [Bot]            | 2012-06-05 22:36:33 | asterias/                                   |          1 |
        | Ocelli [Bot]              | 2012-06-06 05:34:08 | Ocelli/                                     |          1 |
        | Disco Nutch [Crawler]     | 2012-06-11 19:08:36 | disco/Nutch                                 |          1 |
        | Cosmix [Bot]              | 2012-06-23 03:09:24 | cfetch/                                     |          1 |
        | Nimble [Crawler]          | 2012-06-23 17:30:04 | NimbleCrawler                               |          1 |
        | Voyager [Bot]             | 2012-07-07 11:00:56 | voyager/1.0                                 |          1 |
        | Hogsearch [Bot]           | 2012-07-12 23:22:35 | oegp v. 1.3.0                               |          1 |
        | Netluchs                  | 2012-07-14 21:10:42 | Netluchs/Nutch                              |          1 |
        | Archive Org               | 2012-07-17 05:39:35 | archive.org_bot/                            |          1 |
        | Nutch [Bot]               | 2012-07-20 00:01:13 | http://lucene.apache.org/nutch/             |          1 |
        | Java 1                    | 2012-07-21 11:58:12 | Java/1                                      |          1 |
        | Jyxobot [Bot]             | 2012-07-23 19:31:01 | Jyxobot/                                    |          1 |
        | Accoona [Bot]             | 2012-07-27 01:09:16 | Accoona-AI-Agent/                           |          1 |
        | MSN NewsBlogs             | 2012-07-27 19:52:52 | msnbot-NewsBlogs/                           |          1 |
        | Boitho [Crawler]          | 2012-07-28 14:47:13 | boitho.com-dc/                              |          1 |
        | Thumbshots [Capture]      | 2012-07-28 21:00:29 | thumbshots-de-Bot                           |          1 |
        | proximic                  | 2012-07-31 03:55:02 | proximic;                                   |          1 |
        | MSRBOT                    | 2012-07-31 04:27:33 | MSRBOT                                      |          1 |
        | TurnitinBot [Bot]         | 2012-07-31 10:34:33 | TurnitinBot/                                |          1 |
        | Ask Jeeves [Bot]          | 2012-07-31 13:55:49 | Ask Jeeves                                  |          1 |
        | FAST WebCrawler [Crawler] | 2012-08-01 07:25:04 | FAST-WebCrawler/                            |          1 |
        | Xaldon [Spider]           | 2012-08-01 14:55:41 | Xaldon WebSpider                            |          1 |
        | W3 [Sitesearch]           | 2012-08-04 13:04:07 | W3 SiteSearch Crawler                       |          1 |
        | YaCy [Bot]                | 2012-08-05 02:33:34 | yacybot                                     |          1 |
        | Susie [Sync]              | 2012-08-05 05:38:33 | !Susie (http://www.sync2it.com/susie)       |          1 |
        | W3C [Linkcheck]           | 2012-08-05 12:11:00 | W3C-checklink/                              |          1 |
        | psbot [Picsearch]         | 2012-08-05 18:45:20 | psbot/0                                     |          1 |
        | YahooSeeker [Bot]         | 2012-08-06 10:23:37 | YahooSeeker/                                |          1 |
        | FAST Enterprise [Crawler] | 2012-08-06 20:59:18 | FAST Enterprise Crawler                     |          1 |
        | Vagabondo [Crawler]       | 2012-08-06 21:22:08 | http://webagent.wise-guys.nl/               |          1 |
        | Baidu [Spider]            | 2012-08-07 03:34:56 | Baiduspider+(                               |          1 |
        | AdsBot [Google]           | 2012-08-07 08:02:26 | AdsBot-Google                               |          1 |
        | W3C [Validator]           | 2012-08-07 09:24:38 | W3C_*Validator                              |          1 |
        | Heritrix [Crawler]        | 2012-08-07 09:41:57 | heritrix/1.                                 |          1 |
        | Google Adsense [Bot]      | 2012-08-07 12:43:00 | mediapartners-google                        |          1 |
        | Alexa [Bot]               | 2012-08-07 13:23:27 | ia_archiver                                 |          1 |
        | Sogou [Spider]            | 2012-08-07 13:23:42 | Sogou web spider                            |          1 |
        | MSNbot Media              | 2012-08-07 13:32:05 | msnbot-media/                               |          1 |
        | Magpie [RSS]              | 2012-08-07 13:44:57 | MagpieRSS/                                  |          1 |
        | Exabot [Bot]              | 2012-08-07 13:57:46 | Exabot/                                     |          1 |
        | omgili [Bot]              | 2012-08-07 14:18:04 | omgilibot/                                  |          1 |
        | ichiro [Crawler]          | 2012-08-07 14:27:44 | ichiro/                                     |          1 |
        | MSN [Bot]                 | 2012-08-07 14:31:59 | msnbot/                                     |          1 |
        | Gigabot [Bot]             | 2012-08-07 14:32:12 | Gigabot/                                    |          1 |
        | NetNewsWire               | 2012-08-07 14:33:08 | NetNewsWire/                                |          1 |
        | Google Desktop            | 2012-08-07 14:34:56 | Google Desktop                              |          1 |
        | NewsFire                  | 2012-08-07 14:49:02 | NewsFire/                                   |          1 |
        | Google Feedfetcher        | 2012-08-07 14:50:56 | Feedfetcher-Google                          |          1 |
        | Tbot [Bot]                | 2012-08-07 14:51:05 | Tbot/                                       |          1 |
        | Yandex [Bot]              | 2012-08-07 14:51:13 | Yandex                                      |          1 |
        | oBot                      | 2012-08-07 14:51:16 | oBot                                        |          1 |
        | FeedReader                | 2012-08-07 14:51:38 | Feedreader                                  |          1 |
        | Bing [Bot]                | 2012-08-07 14:51:38 | bingbot/                                    |          1 |
        | Yahoo [Bot]               | 2012-08-07 14:51:41 | Yahoo! Slurp                                |          1 |
        | Majestic-12 [Bot]         | 2012-08-07 14:51:55 | MJ12bot/                                    |          1 |
        | Google [Bot]              | 2012-08-07 14:51:56 | Googlebot                                   |          1 |
        +---------------------------+---------------------+---------------------------------------------+------------+
        151 rows in set (0.01 sec)

        Show
        DavidIQ David Colón added a comment - Bots, last visit, and active status from phpbb.com: mysql> SELECT b.bot_name, FROM_UNIXTIME(u.user_lastvisit) lastvisit, b.bot_agent, b.bot_active FROM users u JOIN bots b ON b.user_id = u.user_id ORDER BY u.user_lastvisit; +---------------------------+---------------------+---------------------------------------------+------------+ | bot_name | lastvisit | bot_agent | bot_active | +---------------------------+---------------------+---------------------------------------------+------------+ | Orange [Spider] | 1969-12-31 16:00:00 | OrangeSpider | 1 | | SEO Crawler | 1969-12-31 16:00:00 | SEO search Crawler/ | 1 | | Gonzo2 [Bot] | 1969-12-31 16:00:00 | gonzo2[P] | 1 | | ContextAd | 1969-12-31 16:00:00 | contextad live parser | 1 | | Seoma [Crawler] | 1969-12-31 16:00:00 | Seoma [SEO Crawler] | 1 | | WebAlta | 1969-12-31 16:00:00 | WebAlta Crawler | 1 | | McSeek [Bot] | 1969-12-31 16:00:00 | powered by www.McSeek.de | 1 | | SEOSearch [Crawler] | 1969-12-31 16:00:00 | SEOsearch/ | 1 | | 123metaspider [Bot] | 1969-12-31 16:00:00 | 123metaspider-Bot | 1 | | miniRank | 1969-12-31 16:00:00 | miniRank/ | 1 | | Crawler Search [Crawler] | 1969-12-31 16:00:00 | .Crawler-Search.de | 1 | | WissenOnline [Bot] | 1969-12-31 16:00:00 | WissenOnline-Bot | 1 | | 123spider [Bot] | 1969-12-31 16:00:00 | 123spider-Bot | 1 | | Mirago [Bot] | 1969-12-31 16:00:00 | HeinrichderMiragoRobot ( | 1 | | ICCenter [Bot] | 1969-12-31 16:00:00 | iCCrawler ( | 1 | | WWWeasel [Bot] | 1969-12-31 16:00:00 | WWWeasel Robot v | 1 | | 24spider [Bot] | 1969-12-31 16:00:00 | 24spider-Robot | 1 | | MMSBot [Bot] | 1969-12-31 16:00:00 | http://www.mmsweb.at/bot.html | 1 | | MP3-Bot [Bot] | 1969-12-31 16:00:00 | MP3-Bot | 1 | | Scumbot [Bot] | 1969-12-31 16:00:00 | Scumbot/ | 1 | | IDBot | 1969-12-31 16:00:00 | id-search.org | 1 | | Esperanza [Bot] | 1969-12-31 16:00:00 | EsperanzaBot | 1 | | Insuranco [Bot] | 1969-12-31 16:00:00 | InsurancoBot | 1 | | Yoono | 1969-12-31 16:00:00 | Yoono; http://www.yoono.com/ | 1 | | MyFamily [Bot] | 1969-12-31 16:00:00 | MyFamilyBot/ | 1 | | Sproose [Bot] | 1969-12-31 16:00:00 | http://www.sproose.com/bot | 1 | | IPG [Bot] | 1969-12-31 16:00:00 | internet-provider-guenstig.de-Bot | 1 | | Zerx [Bot] | 1969-12-31 16:00:00 | zerxbot/ | 1 | | Advista AdBot | 1969-12-31 16:00:00 | Advista AdBot | 1 | | suchpad [Bot] | 1969-12-31 16:00:00 | http://www.suchpad.de/bot/ | 1 | | Nebulla [Bot] | 1969-12-31 16:00:00 | Nebullabot/ | 1 | | g2 [Crawler] | 1969-12-31 16:00:00 | g2Crawler | 1 | | GBSpider [Spider] | 1969-12-31 16:00:00 | GBSpider v | 1 | | Asterias [Spider] | 1969-12-31 16:00:00 | http://www.singingfish.com/help/spider.html | 1 | | GeigerzaehlerBot [Bot] | 1969-12-31 16:00:00 | http://www.geigerzaehler.org/bot.html | 1 | | jurabot | 1969-12-31 16:00:00 | jurabot/Metasuche | 1 | | BuildCMS [Crawler] | 1969-12-31 16:00:00 | BuildCMS crawler | 1 | | GeoBot [Bot] | 1969-12-31 16:00:00 | GeoBot/version | 1 | | Bunnybot [Bot] | 1969-12-31 16:00:00 | powered by www.buncat.de | 1 | | GeonaBot | 1969-12-31 16:00:00 | GeonaBot/ | 1 | | Onsearch [Bot] | 1969-12-31 16:00:00 | onCHECK-Robot | 1 | | VisBot | 1969-12-31 16:00:00 | VisBot/2 | 1 | | LiteFinder | 1969-12-31 16:00:00 | LiteFinder/ | 1 | | Sensis [Crawler] | 2007-10-12 09:48:39 | Sensis Web Crawler | 1 | | Telekom [Bot] | 2008-03-03 01:08:26 | crawleradmin.t-info@telekom.de | 1 | | Snappy [Bot] | 2008-08-27 03:33:38 | Snappy/1 | 1 | | PDFBot | 2008-09-14 09:31:08 | PDFBot crawler | 1 | | Heise IT-Markt [Crawler] | 2008-10-18 14:34:48 | heise-IT-Markt-Crawler | 1 | | REAP Crawler | 2008-12-08 08:38:08 | REAP-crawler Nutch | 1 | | PlantyNet [Bot] | 2008-12-29 17:30:15 | PlantyNet_WebRobot_V | 1 | | ISC Systems [Bot] | 2009-01-23 07:03:19 | ISC Systems iRc Search | 1 | | IPBot | 2009-02-17 10:17:51 | | 0 | | Pagebull | 2009-03-29 02:25:46 | Pagebull http | 1 | | Orbiter [Bot] | 2009-05-13 05:58:19 | http://www.dailyorbit.com/bot.htm | 1 | | Houxou [Crawler] | 2009-06-01 17:01:42 | HouxouCrawler/ | 1 | | MaSagool | 2009-07-08 12:03:20 | MaSagool/ | 1 | | Gonzo [Bot] | 2009-08-14 11:45:58 | gonzo1[P] | 1 | | Google Sitemaps | 2009-10-05 00:29:11 | Google-Sitemaps/ | 1 | | Synoo [Bot] | 2010-02-02 15:24:47 | SynooBot/ | 1 | | PageFetcher [Google] | 2010-02-06 18:49:25 | PageFetcher-Google-CoOp; | 1 | | FeedHub | 2010-02-10 18:05:18 | FeedHub FeedFetcher/ | 1 | | genie [Bot] | 2010-03-10 09:57:07 | genieBot | 1 | | IBM Research [Bot] | 2010-05-22 06:58:54 | ibm.com/cs/crawler | 1 | | ICCrawler - ICjobs | 2010-08-20 21:50:10 | ICCrawler - ICjobs | 1 | | Metager [Bot] | 2011-01-09 01:39:03 | MetagerBot/ | 1 | | Exabot Images | 2011-03-01 07:21:23 | Exabot-Images/3 | 1 | | Steeler [Crawler] | 2011-04-19 02:21:17 | http://www.tkl.iis.u-tokyo.ac.jp/~crawler/ | 1 | | OmniExplorer [Bot] | 2011-04-24 18:31:03 | OmniExplorer_Bot/ | 1 | | Nutch/CVS [Bot] | 2011-05-10 08:59:34 | NutchCVS/ | 1 | | Twiceler [Bot] | 2011-05-16 18:11:49 | Twiceler-0 | 1 | | Sygol [Bot] | 2011-09-13 00:33:45 | SygolBot | 1 | | NetResearchServer | 2011-09-13 05:06:29 | NetResearchServer/ | 1 | | CazoodleBot | 2011-10-02 03:22:04 | CazoodleBot | 1 | | Yahoo MMCrawler [Bot] | 2011-10-12 19:40:56 | Yahoo-MMCrawler/ | 1 | | Alta Vista [Bot] | 2011-10-12 19:41:00 | Scooter/ | 1 | | Yahoo Slurp [Bot] | 2011-10-13 06:44:38 | Yahoo! DE Slurp | 1 | | SnapBot | 2011-10-22 10:56:04 | Snapbot/ | 1 | | Bloglines | 2011-10-25 21:15:34 | Bloglines/ | 1 | | Kraehe [Metasuche] | 2011-11-17 18:16:01 | -DIE-KRAEHE- META-SEARCH-ENGINE/ | 1 | | Francis [Bot] | 2011-11-19 00:49:39 | http://www.neomo.de/ | 1 | | Webmeasurement [Bot] | 2011-12-09 04:13:53 | webmeasurement-bot, | 1 | | IRLbot [Bot] | 2012-01-03 03:00:38 | http://irl.cs.tamu.edu/crawler | 1 | | NG-Search [Bot] | 2012-02-18 15:06:09 | NG-Search/ | 1 | | LinkWalker | 2012-03-01 08:13:24 | LinkWalker | 1 | | Acoon-Robot | 2012-03-04 01:59:12 | Acoon-Robot | 1 | | Seekport [Bot] | 2012-03-29 06:29:20 | Seekbot/ | 1 | | Cometrics [Bot] | 2012-04-12 16:31:22 | cometrics-bot, | 1 | | Walhello [Bot] | 2012-04-20 06:14:12 | appie 1.1 (www.walhello.com) | 1 | | Online link [Validator] | 2012-04-22 12:05:22 | online link validator | 1 | | Findexa [Crawler] | 2012-04-24 23:08:48 | Findexa Crawler ( | 1 | | EliteSys [Bot] | 2012-04-29 06:51:42 | EliteSys SuperBot/ | 1 | | ASPseek [Crawler] | 2012-05-09 00:45:26 | ASPseek/ | 1 | | WiseNut [Bot] | 2012-05-21 22:31:32 | http://www.WISEnutbot.com | 1 | | Speedy Spider | 2012-05-24 02:29:21 | Speedy Spider | 1 | | Naver [Bot] | 2012-06-01 08:31:42 | nhnbot@naver.com) | 1 | | Asterias [Bot] | 2012-06-05 22:36:33 | asterias/ | 1 | | Ocelli [Bot] | 2012-06-06 05:34:08 | Ocelli/ | 1 | | Disco Nutch [Crawler] | 2012-06-11 19:08:36 | disco/Nutch | 1 | | Cosmix [Bot] | 2012-06-23 03:09:24 | cfetch/ | 1 | | Nimble [Crawler] | 2012-06-23 17:30:04 | NimbleCrawler | 1 | | Voyager [Bot] | 2012-07-07 11:00:56 | voyager/1.0 | 1 | | Hogsearch [Bot] | 2012-07-12 23:22:35 | oegp v. 1.3.0 | 1 | | Netluchs | 2012-07-14 21:10:42 | Netluchs/Nutch | 1 | | Archive Org | 2012-07-17 05:39:35 | archive.org_bot/ | 1 | | Nutch [Bot] | 2012-07-20 00:01:13 | http://lucene.apache.org/nutch/ | 1 | | Java 1 | 2012-07-21 11:58:12 | Java/1 | 1 | | Jyxobot [Bot] | 2012-07-23 19:31:01 | Jyxobot/ | 1 | | Accoona [Bot] | 2012-07-27 01:09:16 | Accoona-AI-Agent/ | 1 | | MSN NewsBlogs | 2012-07-27 19:52:52 | msnbot-NewsBlogs/ | 1 | | Boitho [Crawler] | 2012-07-28 14:47:13 | boitho.com-dc/ | 1 | | Thumbshots [Capture] | 2012-07-28 21:00:29 | thumbshots-de-Bot | 1 | | proximic | 2012-07-31 03:55:02 | proximic; | 1 | | MSRBOT | 2012-07-31 04:27:33 | MSRBOT | 1 | | TurnitinBot [Bot] | 2012-07-31 10:34:33 | TurnitinBot/ | 1 | | Ask Jeeves [Bot] | 2012-07-31 13:55:49 | Ask Jeeves | 1 | | FAST WebCrawler [Crawler] | 2012-08-01 07:25:04 | FAST-WebCrawler/ | 1 | | Xaldon [Spider] | 2012-08-01 14:55:41 | Xaldon WebSpider | 1 | | W3 [Sitesearch] | 2012-08-04 13:04:07 | W3 SiteSearch Crawler | 1 | | YaCy [Bot] | 2012-08-05 02:33:34 | yacybot | 1 | | Susie [Sync] | 2012-08-05 05:38:33 | !Susie (http://www.sync2it.com/susie) | 1 | | W3C [Linkcheck] | 2012-08-05 12:11:00 | W3C-checklink/ | 1 | | psbot [Picsearch] | 2012-08-05 18:45:20 | psbot/0 | 1 | | YahooSeeker [Bot] | 2012-08-06 10:23:37 | YahooSeeker/ | 1 | | FAST Enterprise [Crawler] | 2012-08-06 20:59:18 | FAST Enterprise Crawler | 1 | | Vagabondo [Crawler] | 2012-08-06 21:22:08 | http://webagent.wise-guys.nl/ | 1 | | Baidu [Spider] | 2012-08-07 03:34:56 | Baiduspider+( | 1 | | AdsBot [Google] | 2012-08-07 08:02:26 | AdsBot-Google | 1 | | W3C [Validator] | 2012-08-07 09:24:38 | W3C_*Validator | 1 | | Heritrix [Crawler] | 2012-08-07 09:41:57 | heritrix/1. | 1 | | Google Adsense [Bot] | 2012-08-07 12:43:00 | mediapartners-google | 1 | | Alexa [Bot] | 2012-08-07 13:23:27 | ia_archiver | 1 | | Sogou [Spider] | 2012-08-07 13:23:42 | Sogou web spider | 1 | | MSNbot Media | 2012-08-07 13:32:05 | msnbot-media/ | 1 | | Magpie [RSS] | 2012-08-07 13:44:57 | MagpieRSS/ | 1 | | Exabot [Bot] | 2012-08-07 13:57:46 | Exabot/ | 1 | | omgili [Bot] | 2012-08-07 14:18:04 | omgilibot/ | 1 | | ichiro [Crawler] | 2012-08-07 14:27:44 | ichiro/ | 1 | | MSN [Bot] | 2012-08-07 14:31:59 | msnbot/ | 1 | | Gigabot [Bot] | 2012-08-07 14:32:12 | Gigabot/ | 1 | | NetNewsWire | 2012-08-07 14:33:08 | NetNewsWire/ | 1 | | Google Desktop | 2012-08-07 14:34:56 | Google Desktop | 1 | | NewsFire | 2012-08-07 14:49:02 | NewsFire/ | 1 | | Google Feedfetcher | 2012-08-07 14:50:56 | Feedfetcher-Google | 1 | | Tbot [Bot] | 2012-08-07 14:51:05 | Tbot/ | 1 | | Yandex [Bot] | 2012-08-07 14:51:13 | Yandex | 1 | | oBot | 2012-08-07 14:51:16 | oBot | 1 | | FeedReader | 2012-08-07 14:51:38 | Feedreader | 1 | | Bing [Bot] | 2012-08-07 14:51:38 | bingbot/ | 1 | | Yahoo [Bot] | 2012-08-07 14:51:41 | Yahoo! Slurp | 1 | | Majestic-12 [Bot] | 2012-08-07 14:51:55 | MJ12bot/ | 1 | | Google [Bot] | 2012-08-07 14:51:56 | Googlebot | 1 | +---------------------------+---------------------+---------------------------------------------+------------+ 151 rows in set (0.01 sec)
        Hide
        DavidIQ David Colón added a comment -

        Excluding Mozilla and Opera as browser agents for current sessions on phpbb.com:

        mysql> SELECT DISTINCT session_browser FROM sessions
        WHERE session_browser NOT LIKE 'Mozilla/%' AND session_browser NOT LIKE 'Opera/%'
        ORDER BY session_browser;
        +-------------------------------------------------------------------------------------------------------------------------------------------------------+
        | session_browser                                                                                                                                       |
        +-------------------------------------------------------------------------------------------------------------------------------------------------------+
        |                                                                                                                                                       |
        | Akregator/1.6.5; syndication                                                                                                                          |
        | Akregator/4.8.3; syndication                                                                                                                          |
        | Akregator/4.8.4; syndication SUSE                                                                                                                     |
        | Akregator/4.9; syndication                                                                                                                            |
        | AppEngine-Google; (+http://code.google.com/appengine; appid: varlopie)                                                                                |
        | AppEngine-Google; (+http://code.google.com/appengine; appid: webproxy8-9)                                                                             |
        | Apple-PubSub/28                                                                                                                                       |
        | Apple-PubSub/65.21                                                                                                                                    |
        | Apple-PubSub/65.23                                                                                                                                    |
        | Apple-PubSub/65.28                                                                                                                                    |
        | Apple-PubSub/65.32                                                                                                                                    |
        | AppleSyndication/56.1                                                                                                                                 |
        | Baiduspider-image+(+http://www.baidu.com/search/spider.htm)                                                                                           |
        | BlackBerry8520/5.0.0.1036 Profile/MIDP-2.1 Configuration/CLDC-1.1 VendorID/132                                                                        |
        | BlogPulse (ISSpider-3.0; http://www.blogpulse.com; compatible; MSIE 6.0)                                                                              |
        | Blogtrottr/2.0                                                                                                                                        |
        | COMODOSpider/Nutch-1.2                                                                                                                                |
        | ColorizeIt                                                                                                                                            |
        | DoCoMo/2.0 SH902i (compatible; Y!J-SRD/1.0; http://help.yahoo.co.jp/help/jp/search/indexing/indexing-27.html)                                         |
        | ELinks/0.11.1 (textmode; Linux; -)                                                                                                                    |
        | FeedBurner/1.0 (http://www.FeedBurner.com)                                                                                                            |
        | FeedDemon/4.0 (http://www.feeddemon.com/; Microsoft Windows)                                                                                          |
        | Feedfetcher-Google; (+http://www.google.com/feedfetcher.html; 1 subscribers; feed-id=1155424950123321032)                                             |
        | Feedreader 3.14 (Powered by Newsbrain)                                                                                                                |
        | Fever/1.25 (Feed Parser; http://feedafever.com; Allow like Gecko)                                                                                     |
        | GG PeekBot 1.0 ( http://gg.pl/ http://info.gadu-gadu.pl/praca )                                                                                       |
        | Gigabot/3.0 (http://www.gigablast.com/spider.html)                                                                                                    |
        | GreatNews/1.0                                                                                                                                         |
        | InetURL:/1.0                                                                                                                                          |
        | Installatron Plugin/8.0.3                                                                                                                             |
        | Liferea/1.6.6b (Linux; en_US.UTF-8; http://liferea.sf.net/)                                                                                           |
        | Liferea/1.8.6 (Linux; de_DE.UTF-8; http://liferea.sf.net/)                                                                                            |
        | LiveJournal.com (webmaster@livejournal.com; for http://www.livejournal.com/users/phpbb_rss/; 1 readers)                                               |
        | MagpieRSS/0.72 (+http://magpierss.sf.net)                                                                                                             |
        | Microsoft URL Control - 6.00.8862                                                                                                                     |
        | Microsoft-WebDAV-MiniRedir/6.1.7601                                                                                                                   |
        | Motorola                                                                                                                                              |
        | Mozila/4.0 (compatible; MSIE 6.0; Windows NT 5.1;)                                                                                                    |
        | NetNewsWire/3.1.6 (Mac OS X; http://www.newsgator.com/Individuals/NetNewsWire/)                                                                       |
        | Netvibes (http://www.netvibes.com/; 1 subscribers; feedID: 1750138)                                                                                   |
        | Netvibes (http://www.netvibes.com/; 1 subscribers; feedID: 6056160)                                                                                   |
        | Netvibes (http://www.netvibes.com/; 2 subscribers; feedID: 5285318)                                                                                   |
        | NewsFire/85 CFNetwork/596.0.1 Darwin/12.0.0 (x86_64) (MacBookPro10%2C1)                                                                               |
        | NewspointBot/1.0 (http://www.netsprint.eu/)                                                                                                           |
        | Nokia7610/2.0 (7.0642.0) SymbianOS/7.0s Series60/2.1 Profile/MIDP-2.0 Configuration/CLDC-1.0/UC Browser7.9.1.120/27/350/UCWEB                         |
        | NokiaC1-01/2.0 (05.45) Profile/MIDP-2.1 Configuration/CLDC-1.1                                                                                        |
        | NokiaE63                                                                                                                                              |
        | PHP/5.3.10                                                                                                                                            |
        | Planeta Software Libre +http://planetasl.bilson.cl Planet/2.0 +http://www.planetplanet.org UniversalFeedParser/4.1 +http://feedparser.org/            |
        | Python-httplib2/0.7.2 (gzip)                                                                                                                          |
        | RSS Menu/2.1.3 (Mac OS X; http://www.edotstudios.com)                                                                                                 |
        | RSSMirandaPlugin/1.0                                                                                                                                  |
        | RSSOwl/2.1.2.201108131746 (Windows; U; en)                                                                                                            |
        | RSSOwl/2.1.2.201108151941 (Windows; U; en)                                                                                                            |
        | RSSOwl/2.1.4.201207281409 (Windows; U; en)                                                                                                            |
        | RSSOwl/2.1.4.201207281417 (Windows; U; en)                                                                                                            |
        | Read ahead agent 1.0                                                                                                                                  |
        | Reeder/1.5.5 CFNetwork/548.1.4 Darwin/11.0.0                                                                                                          |
        | Reeder/1010.79.00 CFNetwork/596.0.1 Darwin/12.0.0 (x86_64) (MacBook7%2C1)                                                                             |
        | SAMSUNG-SGH-B130/B130XEHF1 UP.Browser/6.2.3.3.c.1.101 (GUI) MMP/2.0                                                                                   |
        | SimplePie/1.0.1 (Feed Parser; http://simplepie.org/; Allow like Gecko) Build/20070719221955                                                           |
        | SimplePie/1.1.3 (Feed Parser; http://simplepie.org; Allow like Gecko) Build/20081219                                                                  |
        | SimplePie/1.2 (Feed Parser; http://simplepie.org; Allow like Gecko) Build/20090627192103                                                              |
        | SimplePie/1.2.1-dev (Feed Parser; http://simplepie.org; Allow like Gecko) Build/20110723163749                                                        |
        | Snarfer/1.0.2 (http://www.snarfware.com/)                                                                                                             |
        | Sosospider+(+http://help.soso.com/webspider.htm)                                                                                                      |
        | Superfeedr: Superparser bot/1.1 http://superfeedr.com - Please read this http://blog.superfeedr.com/publishers.html or get in touch if we are polling |
        | Tiny Tiny RSS/1.5.11 (http://tt-rss.org/)                                                                                                             |
        | Tiny Tiny RSS/1.5.8.1 (http://tt-rss.org/)                                                                                                            |
        | TosCrawler/Nutch-1.4 (http://www.toshiba.co.jp/rdc/about/crawl_info.htm; 'Rdc-crawler at ml dot toshiba dot co dot jp')                               |
        | TwitterFeed 3                                                                                                                                         |
        | Vienna/2.6.0.2601                                                                                                                                     |
        | Wget/1.12 (linux-gnu)                                                                                                                                 |
        | Wget/1.13.4 (linux-gnu)                                                                                                                               |
        | Windows-Live-Social-Object-Extractor-Engine/1.0                                                                                                       |
        | Windows-RSS-Platform/2.0 (MSIE 9.0; Windows NT 6.1)                                                                                                   |
        | Xydo v0.1; (Contact: Xydo <info@xydo.com>)                                                                                                      |
        | Yahoo Pipes 2.0                                                                                                                                       |
        | Z-Add Link Checker (http://w3.z-add.co.uk/linkcheck/)                                                                                                 |
        | check_http/v1.4.15 (nagios-plugins 1.4.15)                                                                                                            |
        | checks.panopta.com                                                                                                                                    |
        | coccoc/1.0 ()                                                                                                                                         |
        | curl/7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5                                                           |
        | curl/7.21.2 (i386-pc-solaris2.11) libcurl/7.21.2 OpenSSL/1.0.0e zlib/1.2.3                                                                            |
        | facebookexternalhit/1.0 (+http://www.facebook.com/externalhit_uatext.php)                                                                             |
        | facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)                                                                             |
        | ichiro/3.0 (http://help.goo.ne.jp/door/crawler.html)                                                                                                  |
        | lastRSS                                                                                                                                               |
        | linkdex.com/v2.0                                                                                                                                      |
        | livedoor FeedFetcher/0.01 (http://reader.livedoor.com/; 1 subscriber)                                                                                 |
        | magpie-crawler/1.1 (U; Linux amd64; en-GB; +http://www.brandwatch.net)                                                                                |
        | msnbot-UDiscovery/2.0b (+http://search.msn.com/msnbot.htm)                                                                                            |
        | msnbot-media/1.1 (+http://search.msn.com/msnbot.htm)                                                                                                  |
        | msnbot/2.0b (+http://search.msn.com/msnbot.htm)                                                                                                       |
        | newsbeuter/2.5 (Linux 3.4.2; armv5tel; http://www.newsbeuter.org/) libcurl/7.25.0 OpenSSL/1.0.0j zlib/1.2.5.1 libidn/1.25                             |
        | omgilibot/0.3 +http://www.omgili.com/Crawler.html                                                                                                     |
        | rss2email/2.71 +http://www.allthingsrss.com/rss2email/                                                                                                |
        | ssearch_bot (sSearch Crawler; http://www.semantissimo.de)                                                                                             |
        | vBulletin via cURL/PHP                                                                                                                                |
        +-------------------------------------------------------------------------------------------------------------------------------------------------------+
        100 rows in set (0.02 sec)

        Show
        DavidIQ David Colón added a comment - Excluding Mozilla and Opera as browser agents for current sessions on phpbb.com: mysql> SELECT DISTINCT session_browser FROM sessions WHERE session_browser NOT LIKE 'Mozilla/%' AND session_browser NOT LIKE 'Opera/%' ORDER BY session_browser; +-------------------------------------------------------------------------------------------------------------------------------------------------------+ | session_browser | +-------------------------------------------------------------------------------------------------------------------------------------------------------+ | | | Akregator/1.6.5; syndication | | Akregator/4.8.3; syndication | | Akregator/4.8.4; syndication SUSE | | Akregator/4.9; syndication | | AppEngine-Google; (+http://code.google.com/appengine; appid: varlopie) | | AppEngine-Google; (+http://code.google.com/appengine; appid: webproxy8-9) | | Apple-PubSub/28 | | Apple-PubSub/65.21 | | Apple-PubSub/65.23 | | Apple-PubSub/65.28 | | Apple-PubSub/65.32 | | AppleSyndication/56.1 | | Baiduspider-image+(+http://www.baidu.com/search/spider.htm) | | BlackBerry8520/5.0.0.1036 Profile/MIDP-2.1 Configuration/CLDC-1.1 VendorID/132 | | BlogPulse (ISSpider-3.0; http://www.blogpulse.com; compatible; MSIE 6.0) | | Blogtrottr/2.0 | | COMODOSpider/Nutch-1.2 | | ColorizeIt | | DoCoMo/2.0 SH902i (compatible; Y!J-SRD/1.0; http://help.yahoo.co.jp/help/jp/search/indexing/indexing-27.html) | | ELinks/0.11.1 (textmode; Linux; -) | | FeedBurner/1.0 (http://www.FeedBurner.com) | | FeedDemon/4.0 (http://www.feeddemon.com/; Microsoft Windows) | | Feedfetcher-Google; (+http://www.google.com/feedfetcher.html; 1 subscribers; feed-id=1155424950123321032) | | Feedreader 3.14 (Powered by Newsbrain) | | Fever/1.25 (Feed Parser; http://feedafever.com; Allow like Gecko) | | GG PeekBot 1.0 ( http://gg.pl/ http://info.gadu-gadu.pl/praca ) | | Gigabot/3.0 (http://www.gigablast.com/spider.html) | | GreatNews/1.0 | | InetURL:/1.0 | | Installatron Plugin/8.0.3 | | Liferea/1.6.6b (Linux; en_US.UTF-8; http://liferea.sf.net/) | | Liferea/1.8.6 (Linux; de_DE.UTF-8; http://liferea.sf.net/) | | LiveJournal.com (webmaster@livejournal.com; for http://www.livejournal.com/users/phpbb_rss/; 1 readers) | | MagpieRSS/0.72 (+http://magpierss.sf.net) | | Microsoft URL Control - 6.00.8862 | | Microsoft-WebDAV-MiniRedir/6.1.7601 | | Motorola | | Mozila/4.0 (compatible; MSIE 6.0; Windows NT 5.1;) | | NetNewsWire/3.1.6 (Mac OS X; http://www.newsgator.com/Individuals/NetNewsWire/) | | Netvibes (http://www.netvibes.com/; 1 subscribers; feedID: 1750138) | | Netvibes (http://www.netvibes.com/; 1 subscribers; feedID: 6056160) | | Netvibes (http://www.netvibes.com/; 2 subscribers; feedID: 5285318) | | NewsFire/85 CFNetwork/596.0.1 Darwin/12.0.0 (x86_64) (MacBookPro10%2C1) | | NewspointBot/1.0 (http://www.netsprint.eu/) | | Nokia7610/2.0 (7.0642.0) SymbianOS/7.0s Series60/2.1 Profile/MIDP-2.0 Configuration/CLDC-1.0/UC Browser7.9.1.120/27/350/UCWEB | | NokiaC1-01/2.0 (05.45) Profile/MIDP-2.1 Configuration/CLDC-1.1 | | NokiaE63 | | PHP/5.3.10 | | Planeta Software Libre +http://planetasl.bilson.cl Planet/2.0 +http://www.planetplanet.org UniversalFeedParser/4.1 +http://feedparser.org/ | | Python-httplib2/0.7.2 (gzip) | | RSS Menu/2.1.3 (Mac OS X; http://www.edotstudios.com) | | RSSMirandaPlugin/1.0 | | RSSOwl/2.1.2.201108131746 (Windows; U; en) | | RSSOwl/2.1.2.201108151941 (Windows; U; en) | | RSSOwl/2.1.4.201207281409 (Windows; U; en) | | RSSOwl/2.1.4.201207281417 (Windows; U; en) | | Read ahead agent 1.0 | | Reeder/1.5.5 CFNetwork/548.1.4 Darwin/11.0.0 | | Reeder/1010.79.00 CFNetwork/596.0.1 Darwin/12.0.0 (x86_64) (MacBook7%2C1) | | SAMSUNG-SGH-B130/B130XEHF1 UP.Browser/6.2.3.3.c.1.101 (GUI) MMP/2.0 | | SimplePie/1.0.1 (Feed Parser; http://simplepie.org/; Allow like Gecko) Build/20070719221955 | | SimplePie/1.1.3 (Feed Parser; http://simplepie.org; Allow like Gecko) Build/20081219 | | SimplePie/1.2 (Feed Parser; http://simplepie.org; Allow like Gecko) Build/20090627192103 | | SimplePie/1.2.1-dev (Feed Parser; http://simplepie.org; Allow like Gecko) Build/20110723163749 | | Snarfer/1.0.2 (http://www.snarfware.com/) | | Sosospider+(+http://help.soso.com/webspider.htm) | | Superfeedr: Superparser bot/1.1 http://superfeedr.com - Please read this http://blog.superfeedr.com/publishers.html or get in touch if we are polling | | Tiny Tiny RSS/1.5.11 (http://tt-rss.org/) | | Tiny Tiny RSS/1.5.8.1 (http://tt-rss.org/) | | TosCrawler/Nutch-1.4 (http://www.toshiba.co.jp/rdc/about/crawl_info.htm; 'Rdc-crawler at ml dot toshiba dot co dot jp') | | TwitterFeed 3 | | Vienna/2.6.0.2601 | | Wget/1.12 (linux-gnu) | | Wget/1.13.4 (linux-gnu) | | Windows-Live-Social-Object-Extractor-Engine/1.0 | | Windows-RSS-Platform/2.0 (MSIE 9.0; Windows NT 6.1) | | Xydo v0.1; (Contact: Xydo <info@xydo.com>) | | Yahoo Pipes 2.0 | | Z-Add Link Checker (http://w3.z-add.co.uk/linkcheck/) | | check_http/v1.4.15 (nagios-plugins 1.4.15) | | checks.panopta.com | | coccoc/1.0 () | | curl/7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5 | | curl/7.21.2 (i386-pc-solaris2.11) libcurl/7.21.2 OpenSSL/1.0.0e zlib/1.2.3 | | facebookexternalhit/1.0 (+http://www.facebook.com/externalhit_uatext.php) | | facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php) | | ichiro/3.0 (http://help.goo.ne.jp/door/crawler.html) | | lastRSS | | linkdex.com/v2.0 | | livedoor FeedFetcher/0.01 (http://reader.livedoor.com/; 1 subscriber) | | magpie-crawler/1.1 (U; Linux amd64; en-GB; +http://www.brandwatch.net) | | msnbot-UDiscovery/2.0b (+http://search.msn.com/msnbot.htm) | | msnbot-media/1.1 (+http://search.msn.com/msnbot.htm) | | msnbot/2.0b (+http://search.msn.com/msnbot.htm) | | newsbeuter/2.5 (Linux 3.4.2; armv5tel; http://www.newsbeuter.org/) libcurl/7.25.0 OpenSSL/1.0.0j zlib/1.2.5.1 libidn/1.25 | | omgilibot/0.3 +http://www.omgili.com/Crawler.html | | rss2email/2.71 +http://www.allthingsrss.com/rss2email/ | | ssearch_bot (sSearch Crawler; http://www.semantissimo.de) | | vBulletin via cURL/PHP | +-------------------------------------------------------------------------------------------------------------------------------------------------------+ 100 rows in set (0.02 sec)
        Hide
        carlino1994 carlino1994 added a comment -

        In phpBB.com bot list there are some bots that there aren't in phpBB bot list.

        In the bot list of phpBB there are also bots that operate only in some countries. See German bots/crawlers.

        So should I remove my discretion bots who have never visited phpBB.com from the list?

        Show
        carlino1994 carlino1994 added a comment - In phpBB.com bot list there are some bots that there aren't in phpBB bot list. In the bot list of phpBB there are also bots that operate only in some countries. See German bots/crawlers. So should I remove my discretion bots who have never visited phpBB.com from the list?
        Hide
        carlino1994 carlino1994 added a comment -

        Opened new pull request for merging request on develop-olympus instead of on develop.

        Show
        carlino1994 carlino1994 added a comment - Opened new pull request for merging request on develop-olympus instead of on develop.

          People

          • Assignee:
            bantu Andreas Fischer
            Reporter:
            naderman Nils Adermann
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development