Unidentified BROWSERS |
|
| WhizBang! Lab |
| WebCopier Session 3 |
| Mozilla/3.0 (Win95; I) |
| Microsoft URL Control - 6.00.8169 |
| Mozilla/3.0 (Win16; I) |
| VPF 5.0 |
| Mozilla/5.0 (X11; N; Linux 2.2.14 i686; en-US; m14) |
| Teleport Pro/1.29 |
| Mozilla |
| Mozilla/3.01Gold (Macintosh; I; 68K) |
| LinkWalker |
| Mozilla/5.0 |
| Wget/1.5.3 |
| NG/1.0 |
| Mozilla/8.0 |
| RPT-HTTPClient/0.3-2 |
| NG/0.9 |
| TEAR - MFC Sample App |
| Mozilla (X11; I; Linux 2.0.32 i586) |
| Lycos-Proxy |
| Links SQL (http://gossamer-threads.com/scripts/links-sql/) |
| Bobby 3.0 |
| Mozilla/3.0b3Gold (WinNT; I) |
| Java1.2.2 |
| Mozilla/3.0 (compatible) |
| webbandit/4.05.0 |
| scrubtheweb.com |
| Cold Fusion 3.1 |
| Mozilla/3.Mozilla/2.01 (Win95; I) |
| EchO!/2.0 |
| Mozilla/3.0 (compatible; WebTrack Tracker/2.0B3) |
| utopy |
| KE_1.0/2.0 libwww/5.2.8 |
| Hubater (http://www.hubat.com) |
| Oxxbot1 :: www.oxxfordinfo.com |
| TeraCrawl/1.0 |
| Java1.3.0 |
| oBot ((compatible;Win32)) |
| ZyBorg/1.0 (ZyBorg@WISEnut.com; http://www.WISEnut.com) |
| None |
| InfoSeek Sidewinder/0.9 |
| ZyBorg/1.0 (ZyBorg@WISEnutbot.com; http://www.WISEnutbot.com) |
| MindCrawler v4.0, support@mindpass.com |
| Mozilla/2.0 (compatible; Ask Jeeves) |
| Mozilla/3.01 (compatible;) |
| ia_archiver |
| Mozilla/2.0 (compatible; T-H-U-N-D-E-R-S-T-O-N-E) |
| DeMozulator 1.0 (MacOS, dMoz URL Check Agent, trebor@animeigo.com) |
| Mozilla/5.0 (X11; U; Linux 2.4.0-test6 i686; en-US; m16) |
| UpdateHTML |
| www.first-search.com |
| - |
| ZBot/1.00 (zcraft-query@zeus.com, checking notionlogic.com) |
| EmailSiphon |
| ZBot/1.00 (zcraft-query@zeus.com, checking www.notionlogic.com) |
| Mozilla/5.0 (X11; U; Linux 2.4.0-test6 i686; en-US; m18) |
| Jack |
| crawler@utopy.com |
| user0.ip3000.com |
| user1.ip3000.com |
| user2.ip3000.com |
| user3.ip3000.com |
| user4.ip3000.com |
| user5.ip3000.com |
| user6.ip3000.com |
| user7.ip3000.com |
| user8.ip3000.com |
| ru-robot/d_0.1 [hseo@cs.rutgers.edu] |
| LexiBot/1.00 |
| DMOZ SafeSearch |
| Schmozilla/v9.14 Platinum via aVirt Gateway Server v3.5 |
| Microsoft URL Control - 6.00.8862 |
| Mozilla/3.0 (compatible; Indy Library) |
| teoma_agent1 [crawler_admin@teoma.com] |
| Crawl_Application |
| Gulper Web Bot 0.2.4 (www.ecsl.cs.sunysb.edu/~maxim/cgi-bin/Link/GulperBot) |
| Bjaaland/0.6 (bjaaland@antarcti.ca) |
| asterias/2.0 |
| Java1.3.0_01 |
| Surfairy |
| Openfind piranha,Shark/0.95h+(response@bbs.openfind.com.tw;+http://www.openfind.com/) |
| teoma_agent3 crawler-admin@teoma.com |
| Openfind piranha,Shark/0.95h+(robot-response@openfind.com.tw;+http://www.openfind.com/) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; m18) Gecko/20010131 Netscape6/6.01 |
| test |
| appie/1.1 |
| PCBeaconBot/1.0 |
| teomaagent crawler-admin@teoma.com |
| IP*Works! HTTP Component - www.dev-soft.com |
| Wget/1.6 |
| polybot 1.0 |
| IE 5.5 Compatible Browser |
| (W3MATTER; info@w3matter.com; http://www.w3matter.com/about.html) |
| Bjaaland/0.9 (bjaaland@antarcti.ca) |
| htdig/3.1.5 (bogus@configured.htdig.user) |
| psbot/0.1 (+http://www.picsearch.org/bot.html) |
| SEPT/1.0 |
| teoma_agent1_[crawler_admin_at_teoma.com] |
| Mozilla/5.0 (Windows; U; Win 9x 4.90; en-GB; m18) Gecko/20010131 Netscape6/6.01 |
| DBrowse 1.4b |
| bumblebee@relevare.com |
| Sqworm/2.9.67-BETA (beta_release; 20010604-714; i686-pc-linux-gnu) |
| ScoutAbout |
| toCrawl/UrlDispatcher |
| netalert |
| psbot/0.1 (+http://www.picsearch.com/bot.html) |
| Lynx/2.8.3dev.9 libwww-FM/2.14 SSL-MM/1.4.1 OpenSSL/0.9.6 |
| Harvest/HARVEST_VERSION |
| MultiText/0.1 |
| IncyWincy page crawler(webmaster@loopimprovements.com,http://www.loopimprovements.com/robot.html) |
| Lifeboat |
| flunky |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.4+) Gecko/20010915 |
| Openfind data gatherer, Openbot/3.0+(robot-response@openfind.com.tw;+http://www.openfind.com.tw/robot.html) |
| larbin_2.2.0 crawl@compete.com |
| IncyWincy(http://www.look.com) |
| LWP::Simple/5.48 |
| Slarp/0.1 |
| gigabaz/3.14 (baz@gigabaz.com; http://gigabaz.com/gigabaz/) |
| Wget/1.7 |
| ColdFusion |
| Mozilla/3.01-C-MACOS8 (Macintosh; I; PPC) |
| Crawler |
| Microsoft Internet Explorer/4.40.426 (Windows 95) |
| RRC crawler_admin@bigfoot.com |
| Mozilla/2.0 (compatible; NEWT ActiveX; Win32) |
| Mozilla 7.1 |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-GB; rv:0.9.4) Gecko/20011019 Netscape6/6.2 |
| Marvin v0.5 |
| IncyWincy(http://www.loopimprovements.com/robot.html) |
| larbin_2.2.1_de_Viennot Laurent.Viennot@inria.fr |
| TE |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:0.9.4) Gecko/20011019 Netscape6/6.2 |
| Lynx/2.6 libwww-FM/2.14 |
| PingALink Monitoring Services 1.0 |
| Mozilla/5.0 (compatible; Konqueror/2.2.1; Linux) |
| JoBo/@JOBO_VERSION@(http://www.matuschek.net/jobo.html) |
| xŠ http |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.4) Gecko/20011126 Netscape6/6.2.1 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-US; rv:0.9.4) Gecko/20011022 Netscape6/6.2 |
| Ad Muncher v4.3d |
| appie 1.1 (www.walhello.com) |
| IncyWincy(loopimprovements.com/robot.html) |
| ø<)http |
| NICO/1.0 |
| Teleport Pro/1.29.1820 |
| Sqworm/2.9.89-BETA (beta_release; 20020130-839; i686-pc-linux-gnu) |
| Xenu Link Sleuth 1.2b |
| bigbrain/3.3 |
| Mozi! |
| Mozilla/5.0 (Windows; U; Win98; en-US; rv:0.9.4) Gecko/20011128 Netscape6/6.2.1 |
| UIowaCrawler/1.0 |
| multiBlocker browser - IP blocker for Spam, Fraud + Snoop Protection |
| Pompos/1.0 pompos@iliad.fr |
| @«http |
| IncyWincy/2.1(loopimprovements.com/robot.html) |
| LWP::Simple/5.53 |
| Spida/0.1 |
| WebCopier v2.8a |
| indexer0.1b |
| Mozilla/2.02 |
| webbandit/4.52.0 |
| Xenu Link Sleuth 1.2a |
| Mozilla/5.0 (Windows; U; Win98; de-DE; rv:0.9.4) Gecko/20011128 Netscape6/6.2.1 |
| Mozilla/5.0 (Windows; U; Win 9x 4.90; en-US; rv:0.9.4.1) Gecko/20020314 Netscape6/6.2.2 |
| WEBMASTERS! AUTOMATICALLY SUBMIT EVERY MONTH to over 500k Sites! |
| Linkbot 3.0 |
| larbin_2.5.0 larbin2.5.0@unspecified.mail |
| RPT-HTTPClient/0.3-3 |
| ¸ |
| NetResearchServer/2.2(loopimprovements.com/robot.html) |
| Java1.3.1_02 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.3) Gecko/20010815 |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0rc1) Gecko/20020417 |
| larbin_2.6.2 larbin2.6.2@unspecified.mail |
| NCSA Beta 1 (http://vias.ncsa.uiuc.edu/viasarchivinginformation.html) |
| Mozilla/5.0 (Windows; U; Win 9x 4.90; en-US; rv:0.9.4) Gecko/20011019 Netscape6/6.2 |
| IE6 |
| Mozilla/3.01+(compatible;) |
| isurf (tszhu@canada.com) |
| Gigabot/1.0 |
| NetResearchServer/2.3(loopimprovements.com/robot.html) |
| (Teradex Mapper; mapper@teradex.com; http://www.teradex.com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0rc2) Gecko/20020512 Netscape/7.0b1 |
| Szukacz/1.5 (robot; www.szukacz.pl/jakdzialarobot.html; info@szukacz.pl) |
| bumblebee/1.0 (bumblebee@relevare.com; http://www.relevare.com/) |
| Wget/1.8 |
| lwp-request/2.01 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.8) Gecko/20020328 |
| Python-urllib/1.13 |
| Sqworm/2.9.85-BETA (beta_release; 20011115-775; i686-pc-linux-gnu) |
| @ÖT |
| ah-ha.com crawler (crawler@ah-ha.com) |
| Mozilla/5.0 (X11; Linux i686; en-US; rv:1.0rc5; OBJR) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.2.1) Gecko/20010901 |
| rico/0.1 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-US; rv:0.9.4.1) Gecko/20020315 Netscape6/6.2.2 |
| AbrexaBot/1.1 (http://www.abrexa.co.uk - *The Great British search engine* ) |
| OrangeBot |
| TulipChain/5.3 (http://ostermiller.org/tulipchain/) Java/1.3.1 (http://apple.com/) Mac_OS_X/10.1.5 |
| curl/7.9.1 (win32) libcurl 7.9.1 |
| TurnitinBot/1.4 http://www.turnitin.com/robot/crawlerinfo.html |
| Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.1b) Gecko/20020721 |
| NetResearchServer/2.4(loopimprovements.com/robot.html) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:0.9.4) Gecko/20011128 Netscape6/6.2.1 |
| Lachesis |
| TITAN/6.0 |
| Mozilla/5.0 [en] (compatible; Gulper Web Bot 0.2.4 www.ecsl.cs.sunysb.edu/~maxim/cgi-bin/Link/GulperBot) |
| TurnitinBot/1.5 http://www.turnitin.com/robot/crawlerinfo.html |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.1b) Gecko/20020721 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-US; rv:1.1) Gecko/20020826 |
| Mozilla/5.0 (Macintosh; N; PPC; en-US; m18) Gecko/20010131 Netscape6/6.01 |
| Scrubby/2.2 (http://www.scrubtheweb.com/) |
| antibot-V1.1.9/i586-linux-2.2 |
| Mozilla/5.0 (X11; U; SunOS sun4u; en-US; rv:1.0.1) Gecko/20020729 Netscape/7.0 |
| Robot: NutchCrawler, Owner: wdavies@acm.org |
| Mozilla/3.0 (compatible; websqueezer 1.0) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.8) Gecko/20020204 |
| Scrubby/2.1 (http://www.scrubtheweb.com/) |
| pavuk/0.9pl29b i686-pc-linux-gnu |
| Website Quester - www.asona.org |
| Pompos/1.1 http://pompos.iliad.fr |
| rabaz (rabaz at gigabaz dot com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.1) Gecko/20020823 Netscape/7.0 |
| Python-urllib/1.15 |
| grub-client |
| Lite Bot 0916b |
| curl/7.7.1 (i386--freebsd4.3) libcurl 7.7.1 (SSL 0.9.6) (ipv6 enabled) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.0.1) Gecko/20020823 Netscape/7.0 |
| Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.1) Gecko/20020826 |
| http://www.almaden.ibm.com/cs/crawler [c01] |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.0.1) Gecko/20020823 Netscape/7.0 |
| Internet Explore 5.x |
| NetResearchServer/2.5(loopimprovements.com/robot.html) |
| Xenu Link Sleuth 1.2d |
| Mozilla/5.0 (Windows; U; Win 9x 4.90; en-US; rv:1.0.1) Gecko/20020823 Netscape/7.0 |
| Szukacz/1.5 |
| edake.com |
| JoBo/1.3 (http://www.matuschek.net/jobo.html) |
| Pompos/1.2 http://pompos.iliad.fr |
| Wget/1.8.2 |
| htdig/3.1.6 (http://computerorgs.com) |
| Java1.2.1 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.1) Gecko/20020826 |
| Mozilla/2.0 (compatible; Ask Jeeves/Teoma) |
| UKCompassBot/1.0+(http://www.ukcompass.com/bot.html) |
| DeepIndex (www.en.deepindex.com) |
| hget/0.3 |
| Java1.4.0_02 |
| Zeus 2.6 |
| lcabotAccept: */* |
| Teleport Pro/1.28 |
| NetResearchServer(http://www.look.com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.2) Gecko/20021120 Netscape/7.01 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-us) AppleWebKit/51 (like Gecko) Safari/51 |
| Internet Ninja 6.0 |
| OWR_Crawler 0.1 |
| potbot 1.0 |
| Wget/1.8.1 |
| WebCopier v2.7 |
| pavuk/0.9pl28 i586-pc-cygwin |
| sitecheck.internetseer.com (For more info see: http://sitecheck.internetseer.com) |
| NPBot-1/2.0 (http://www.nameprotect.com/botinfo.html) |
| Pompos/1.3 http://dir.com/pompos.html |
| Production Bot 2716B |
| GetRight/4.5d |
| Nutch |
| IlTrovatore-Setaccio (+http://www.iltrovatore.it) |
| Mozilla/3.0 (compatible; AvantGo 3.2) |
| Dart Communications PowerTCP |
| dloader(NaverRobot)/1.0 |
| NetResearchServer/2.7(loopimprovements.com/robot.html) |
| Agense Assist v0.8 (http://www.agense.com) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20021202) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020418) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020716) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020606) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20020805) |
| Zeus 5242 Webster Pro V2.9 Win32 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.0.1) Gecko/20021003 |
| dumbBot |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2.1) Gecko/20021130 |
| semanticdiscovery/0.1 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20021108) |
| Irvine/1.0.5b |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.3a) Gecko/20021212 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-us) AppleWebKit/60 (like Gecko) Safari/60 |
| curl/7.8 (i386-redhat-linux-gnu) libcurl 7.8 (OpenSSL 0.9.6b) (ipv6 enabled) |
| Java/1.4.1_02 |
| LARBIN-EXPERIMENTAL efp@gmx.net |
| Faerie/1.0 - ** Brynyth says thank you ** (Please visit http://www.sminkey.co.uk for more information) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.3) Gecko/20030312 |
| DeepIndex |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020617) |
| Faerie/1.0 - ** Clanaph says thank you ** (Please visit http://www.sminkey.co.uk for more information) |
| ask.24x.info |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20020910) |
| Openbot/3.0+(robot-response@openfind.com.tw;+http://www.openfind.com.tw/robot.html) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020302) |
| NutchOrg/0.03-dev (Nutch; http://www.nutch.org/docs/bot.html; nutch-agent@lists.sourceforge.net) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20020503) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20020202) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20020902) |
| Openfind data gatherer, Openbot/3.0+(robot-response@openfind.com.tw;+http://www.openfind.com/robot.html) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20020818) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20021227) |
| Mozilla/5.0 Galeon/1.2.1 (X11; Linux i686; U;) Gecko/20020421 Debian/1.2.1-2 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20021109) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; fr-fr) AppleWebKit/73 (KHTML, like Gecko) Safari/73 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020627) |
| Mozilla/3.0 (compatible; WebCapture 2.0; Auto; Windows) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020406) |
| ProWebGuide Link Checker (http://www.prowebguide.com) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020211) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20021227) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020413) |
| Zeus 41171 Webster Pro V2.9 Win32 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020221) |
| Generic |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020714) |
| Seeker.lookseek.com |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20020328) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020525) |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020404) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-us) AppleWebKit/67 (KHTML, like Gecko) Safari/67 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20020606) |
| lwp-trivial/1.35 |
| InstantSSL Browser: low cost fully validated SSL + free trial |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.3.1) Gecko/20030425 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20021011) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20020409) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20020407) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20020514) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20021128) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20021014) |
| boitho.com-robot/1.0 |
| Mozilla/5.0 (OS/2; U; Warp 4.5; en-US; rv:1.3) Gecko/20030313 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20020222) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20020811) |
| Java/1.4.1_01 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20020907) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20020810) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020322) |
| http://Ask.24x.Info |
| PHP/4.1.2 |
| curl/7.10.4 (i386-redhat-linux-gnu) libcurl/7.10.4 OpenSSL/0.9.6b zlib/1.1.3 |
| Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.4) Gecko/20030529 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020123) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20021122) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20020403) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020406) |
| Vivante Link Checker (http://www.vivante.com) |
| MSNBOT/0.1 (http://search.msn.com/msnbot.htm) |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20021111) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20021015) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20021117) |
| DeepIndex (http://www.deepindex.com) |
| Phantom/2.2.5 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020512) |
| Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.0.2) Gecko/20030208 Netscape/7.02 |
| WebStripper/2.10 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020314) |
| Mac Finder 1.0.35 |
| Mozilla/6.0 [en] (Win32; I) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020712) |
| semanticdiscovery/0.2(http://www.semanticdiscovery.com/sd/robot.html) |
| eseek-larbin_2.6.2 crawler@exactseek.com |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20020303) |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020420) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20020605) |
| WMWWebBot |
| dloader(NaverRobot)/1.5 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020306) |
| SurveyBot/2.3 (Whois Source) |
| Gaisbot/3.0+(robot@gais.cs.ccu.edu.tw;+http://gais.cs.ccu.edu.tw/robot.php) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.5a) Gecko/20030624 Mozilla Firebird/0.6 |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20021109) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020404) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20020409) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20021021) |
| minibot(NaverRobot)/1.0 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020517) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20021006) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.0.2) Gecko/20030208 Netscape/7.02 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20020304) |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020108) |
| boitho.com-robot/1.1 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020828) |
| Program Shareware 1.0.3 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20020823) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20020322) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20021018) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020820) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020222) |
| TulipChain/5.93 (http://ostermiller.org/tulipchain/) Java/1.4.1_02 (http://java.sun.com/) Windows_98/4.10 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020214) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.4b) Gecko/20030516 Mozilla Firebird/0.6 |
| Tcl http client package 2.3 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20021025) |
| Java1.3.1 |
| Franklin Locator 1.8 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020105) |
| http://Ask.24x.Info/ (http://narres.it/) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020204) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020622) |
| Tagword (http://tagword.com/dmoz_survey.php) |
| AOLserver-Tcl/3.5.6 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020717) |
| (Teradex Mapper; mapper@teradex.com; http://www.teradex.com/mapper.html) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20020409) |
| Java/1.4.1-p3 |
| NPBot (http://www.nameprotect.com/botinfo.html) |
| Look.com |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20021024) |
| Overture-WebCrawler/3.8/Fresh (atw-crawler at fast dot no; http://fast.no/support/crawler.asp) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20020603) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20021005) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020516) |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20021115) |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020820) |
| NutchCVS/0.03-dev (Nutch; http://www.nutch.org/docs/bot.html; nutch-agent@lists.sourceforge.net) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20021222) |
| Exalead NG/MimeLive Client (convert/http/0.146) |
| kuloko-bot/0.2 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020811) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020921) |
| Zao/0.1 (http://www.kototoi.org/zao/) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20020816) |
| Mozilla5/0 |
| NetResearchServer/2.8(loopimprovements.com/robot.html) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20020623) |
| Xbot |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20021103) |
| Mozilla(IE Compatible) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020716) |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020502) |
| Jakarta Commons-HttpClient/2.0rc1 |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20021025) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20020522) |
| icsbot-0.1 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20020410) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.0.1) Gecko/20030306 Camino/0.7 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020512) |
| gnome-vfs/2.2.5 |
| Zeus 16379 Webster Pro V2.9 Win32 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020315) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20020905) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020527) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020614) |
| Cowbot-0.1 (NHN Corp. / +82-2-3011-1954 / nhnbot@naver.com) |
| Xenu Link Sleuth 1.2e |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20021127) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-us) AppleWebKit/103u (KHTML, like Gecko) Safari/100 |
| Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax) |
| IP*Works! V5 HTTP/S Component - by /n software - www.nsoftware.com |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020924) |
| NutchOrg/0.04-dev (Nutch; http://www.nutch.org/docs/en/bot.html; nutch-agent@lists.sourceforge.net) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020524) |
| PanopeaBot/1.0 (UCLA CS Dpt.) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20021227) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-us) AppleWebKit/85.7 (KHTML, like Gecko) Safari/85.5 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20021107) |
| Mozilla/5.001 (windows; U; NT4.0; en-us) Gecko/25250101 |
| Exalead NG/MimeLive Client (convert/http/0.147) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.4) Gecko/20030624 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20021025) |
| SearchGuild_DMOZ_Experiment chris@searchguild.com |
| msnbot/0.11 (+http://search.msn.com/msnbot.htm) |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020411) |
| WinScripter iNet Tools |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20020312) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020208) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020320) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20020703) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20020307) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020724) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20021003) |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020825) |
| CerberianDrtrs/Version-3.0-Build-37 |
| TygoBot |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20021021) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020904) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.5) Gecko/20031007 Firebird/0.7 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020609) |
| Art-Online.com 0.9(Beta) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020924) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20021017) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.5b) Gecko/20030722 Mozilla Firebird/0.6Connection: close |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020810) |
| NG/2.0 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020125) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-us) AppleWebKit/85.7 (KHTML, like Gecko) Safari/85.6 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20020318) |
| ASPseek/1.2.10 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20021027) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20020221) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20020708) |
| Missigua Locator 1.9 |
| Mozilla/5.0 (compatible; Konqueror/3) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20020625) |
| NetResearchServer/2.9(loopimprovements.com/robot.html) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20020514) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20021226) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax) |
| Mozilla/5.0 (Windows; U; Win 9x 4.90; en-US; rv:1.0.2) Gecko/20030208 Netscape/7.02 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.5) Gecko/20031007 Firebird/0.7 |
| Pita |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.5) Gecko/20031007 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020609) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.5b) Gecko/20030827 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.4) Gecko/20030624 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.5) Gecko/20031007 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20020525) |
| citenikbot/0.2 (albert; http://www.citenik.co.uk/bot.html) |
| WorQmada/1.0 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020328) |
| Mozilla/5.0 (compatible; Vagabondo/2.1; webcrawler at wise-guys dot nl; http://webagent.wise-guys.nl/) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2.1) Gecko/20030225 |
| Mozilla/5.0 (Macintosh; U; PPC; en-US; rv:1.0.2) Gecko/20030208 Netscape/7.02 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.5) Gecko/20031007 |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.5a) Gecko/20030728 Mozilla Firebird/0.6.1 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20020221) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20021123) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax) |
| Mozilla/5.0 (X11; U; SunOS sun4u; en-US; rv:1.5a) Gecko/20030729 Mozilla Firebird/0.6.1 |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020610) |
| Mozilla/5.0 (Macintosh; U; PPC;) Gecko DEVONtech |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-us) AppleWebKit/106.2 (KHTML, like Gecko) Safari/100.1 |
| Mozilla/5.0 (Windows; U; Win 9x 4.90; en-US; rv:1.4) Gecko/20030612 |
| Mozilla/5.0 (Windows; U; Win98; de-DE; rv:1.5) Gecko/20031007 Firebird/0.7 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; fr) AppleWebKit/106.2 (KHTML, like Gecko) Safari/100.1 |
| NaverBot-1.0 (NHN Corp. / +82-2-3011-1954 / nhnbot@naver.com) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20020521) |
| NutchCVS/0.03-dev (Nutch; http://www.nutch.org/docs/en/bot.html; nutch-agent@lists.sourceforge.net) |
| exactseek-crawler-2.63 crawler@exactseek.com |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.3) Gecko/20030313 |
| http://www.planethosting.com |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20021107) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/106.2 (KHTML, like Gecko) Safari/100.1 |
| Mozilla/5.0 Galeon/1.2.7 (X11; Linux i686; U;) Gecko/20030131 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr-FR; rv:1.0.2) Gecko/20030208 Netscape/7.02 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.4) Gecko/20030630 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de-DE; rv:1.5) Gecko/20031007 Firebird/0.7 |
| : findx_robot, Java1.4.0 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/85.7 (KHTML, like Gecko) Safari/85.6 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020821) |
| Mozilla/5.0 (Windows; U; Win98; PL; rv:1.5) Gecko/20031007 Firebird/0.7 |
| your_agentname |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020113) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4b) Gecko/20030516 Mozilla Firebird/0.6 |
| Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.4) Gecko/20030624 |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; de-DE; rv:1.5) Gecko/20031007 Firebird/0.7 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 |
| Mozilla/5.0 (X11; U; Linux i686; es-ES; rv:1.2.1) Gecko/20030225 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020109) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020604) |
| Mozilla/5.0 (Windows; U; Win98; es-ES; rv:1.5) Gecko/20031007 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.6) Gecko/20040113 |
| GeonaBot 1.0; http://www.geona.com/ |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6) Gecko/20040116 |
| Mozilla/5.0 (Windows; U; Win 9x 4.90; en-US; rv:1.5) Gecko/20031007 Firebird/0.7 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020124) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de-DE; rv:1.4) Gecko/20030619 Netscape/7.1 (ax) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 |
| Mozilla/5.0 (Windows; U; Win 9x 4.90; en-US; rv:1.6) Gecko/20040206 Firefox/0.8 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-us) AppleWebKit/124 (KHTML, like Gecko) Safari/125 |
| VSE/1.0 (vsecrawler@hotmail.com) |
| Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.3) Gecko/20030312 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20021120) |
| NaverBot_dloader/1.5 |
| Mozilla/5.0 (Windows; U; Win98; fr-FR; rv:1.0.2) Gecko/20030208 Netscape/7.02 |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; de-AT; rv:1.6b) Gecko/20031208 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; pt-BR; rv:1.0.2) Gecko/20030208 Netscape/7.02 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.6) Gecko/20040206 Firefox/0.8 |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6) Gecko/20040113 |
| WebRACE/1.1 (University of Cyprus, Distributed Crawler) |
| Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.5) Gecko/20031007 Firebird/0.7 |
| Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20021221) |
| TSW Bot 1.0.1 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20020506) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; it-it) AppleWebKit/124 (KHTML, like Gecko) Safari/125 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/124 (KHTML, like Gecko) Safari/125 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020325) |
| semanticdiscovery/0.2(http://www.semanticdiscovery.com/sd/robot.html |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20021128) |
| Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.6) Gecko/20040113 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020312) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20020819) |
| Mozilla/5.0 (Windows; U; Windows NT 5.2; pl-PL; rv:1.6) Gecko/20040206 Firefox/0.8 |
| Zao/0.2 (http://www.kototoi.org/zao/) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.5) Gecko/20031007 Firebird/0.7 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20020803) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7b) Gecko/20040303 Firefox/0.8.0+ (BlueFyre) |
| Tutorial Crawler 1.4 (http://www.tutorgig.com/crawler) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de-DE; rv:1.6) Gecko/20040206 Firefox/0.8 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; de-de) AppleWebKit/85 (KHTML, like Gecko) Safari/85 |
| Xenu Link Sleuth 1.2c |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20020218) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020704) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6) Gecko/20040206 Firefox/0.8 |
| Mozilla/3.x (Linkbot) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20021110) |
| Homerbot: www.homerweb.com |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020920) |
| Mozilla/5.0 (X11; U; Linux i586; en-US; rv:1.6) Gecko/20040207 Firefox/0.8 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; zh-tw) AppleWebKit/124 (KHTML, like Gecko) Safari/125 |
| Internet Explorer 5.5 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.4) Gecko/20011022 Netscape6/6.2 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.6b) Gecko/20031208 |
| Yahoo-VerticalCrawler-FormerWebCrawler/3.9 crawler at trd dot overture dot com; http://www.alltheweb.com/help/webmaster/crawler |
| http://www.almaden.ibm.com/cs/crawler [wf134] |
| Mozilla/5.0 (Macintosh; U; PPC; en-US; rv:1.0.1) Gecko/20020826 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/124 (KHTML, like Gecko) Safari/125.1 |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020303) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020813) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20021102) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/85 (KHTML, like Gecko) Safari/85 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20020815) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; it-IT; rv:1.6) Gecko/20040206 Firefox/0.8 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-us) AppleWebKit/124 (KHTML, like Gecko) Safari/125.1 |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20021228) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-US; rv:1.0.2) Gecko/20030208 Netscape/7.02 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6) Gecko/20040207 Firefox/0.8 |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.1) Gecko/20020823 Netscape/7.0 (CK-TxRC-70-en) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de-AT; rv:1.6) Gecko/20040113 |
| TurnitinBot/2.0 http://www.turnitin.com/robot/crawlerinfo.html |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20021209) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020913) |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020704) |
| Jyxobot/1 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20021219) |
| UptimeBot(www.uptimebot.com) |
| Infoseek SideWinder/2.0B (Linux 2.4 i686) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20020304) |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020522) |
| Mozilla/5.0 (X11; U; Linux i686; it-IT; rv:1.6) Gecko/20040324 |
| test/0.1 |
| QuepasaCreep ( crawler@quepasacorp.com ) |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020225) |
| ObjectsSearch/0.01 (ObjectsSearch; http://www.ObjectsSearch.com/bot.html; support@thesoftwareobjects.com) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20020209) |
| lwp-trivial/1.36 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.2b) Gecko/20021016 K-Meleon 0.7 |
| Lynx/2.8.5rel.1 libwww-FM/2.14 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20021222) |
| UptimeBot |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7b) Gecko/20040316 |
| Search-Channel |
| Custo 2.0 (www.netwu.com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.2; PL; rv:1.6) Gecko/20040206 Firefox/0.8 |
| Iltrovatore-Setaccio/0.3-dev (Indexing; http://www.iltrovatore.it/bot.html; info@iltrovatore.it) |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020920) |
| Wget/1.9.1 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/103u (KHTML, like Gecko) Safari/100 |
| GoForIt.com |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.4.1) Gecko/20031114 |
| LWP::Simple/5.69 |
| Reaper/2.07 (+http://www.sitesearch.ca/reaper) |
| Mozilla/5.0 (compatible; Konqueror/3.2; Linux) (KHTML, like Gecko) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20020406) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020402) |
| Mozilla/5.0 (Windows; U; Win 9x 4.90; fr-FR; rv:1.0.2) Gecko/20030208 Netscape/7.02 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; fr-fr) AppleWebKit/124 (KHTML, like Gecko) Safari/125.1 |
| Mozilla/5.0 (X11; U; Linux i586; en-US; rv:1.6) Gecko/20040227 Firefox/0.8 |
| Mozilla/5.0 (X11; U; Linux i686; fr; rv:1.4.1) Gecko/20031114 Epiphany/1.0.4 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; es-es) AppleWebKit/124 (KHTML, like Gecko) Safari/125.1 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20021013) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20020713) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.7b) Gecko/20040317 Firefox/0.8.0+ |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8a) Gecko/20040418 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6) Gecko/20040227 Firefox/0.8 |
| Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.3a) Gecko/20021207 Phoenix/0.5 |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; cs-CZ; rv:1.6) Gecko/20040206 Firefox/0.8 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-us) AppleWebKit/85.7 (KHTML, like Gecko) Safari/85.7 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20021019) |
| larbin_2.6.3_for_(http://cosco.hiit.fi/search/) tsilande@hiit.fi |
| IlTrovatore-Setaccio/1.2-dev (Indexing; http://www.iltrovatore.it/bot.html; info@iltrovatore.it) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20021101) |
| citenikbot/0.3 (albert; http://www.citenik.co.uk/bot.html) |
| CSUofA |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de-DE; rv:1.0.2) Gecko/20030208 Netscape/7.02 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.6) Gecko/20040206 Firefox/0.8 StumbleUpon/1.908 |
| ZIPPPCVS/0.24 (ZipppBot/.24; http://www.zippp.net; webmaster@zippp.net) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.6) Gecko/20040113 |
| Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.6) Gecko/20040206 Firefox/0.8 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20020324) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6) Gecko/20040413 Epiphany/1.2.3 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20021026) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; de-de) AppleWebKit/106.2 (KHTML, like Gecko) Safari/100.1 |
| google |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-US) AppleWebKit/85 (KHTML, like Gecko) OmniWeb/v558 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20021210) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020712) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20020706) |
| PHP/4.2.2 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.0.1) Gecko/20020823 Netscape/7.0 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6) Gecko/20040506 Firefox/0.8 |
| Wotbox/alpha0.6 (bot@wotbox.com; http://www.wotbox.com) |
| IlTrovatore-Setaccio/1.2 (Indexing; http://www.iltrovatore.it/bot.html; info@iltrovatore.it) |
| trexmod |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20020826) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020523) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.3) Gecko/20030708 Debian/1.3-4.lindows43 |
| LinkLint-checkonly/2.3.5 |
| LookBot |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20020923) |
| Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.0.2) Gecko/20021120 Netscape/7.01 |
| TestCrawler/1.0 (testcrawler@hotmail.com) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-us) AppleWebKit/125.2 (KHTML, like Gecko) Safari/125.7 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.6) Gecko/20040206 Firefox/0.8 StumbleUpon/1.906 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020504) |
| iexplore.exe |
| curl/7.10.6 (i386-redhat-linux-gnu) libcurl/7.10.6 OpenSSL/0.9.7a ipv6 zlib/1.2.0.7 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020927) |
| Openbot/3.0+(robot@monkia.com.tw;+http://gais.cs.ccu.edu.tw/robot.php) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20020626) |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020112) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020528) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6) Gecko/20040430 Firefox/0.8 |
| IlTrovatore-Setaccio/0.03-dev (Indexing; http://www.iltrovatore.it/bot.html; info@iltrovatore.it) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax; PROMO) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20021115) |
| WDG_Validator/1.5.6 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-us) AppleWebKit/125.2 (KHTML, like Gecko) Safari/125.8 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6) Gecko/20040413 Debian/1.6-5 |
| CosmixCrawler/0.1 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20020423) |
| GetURL11/1.0 |
| Mozilla/5.0 (000000000; 0; 000 000 00 0; 00000) 00000000000000000 0000000 0000 000000 000000000000 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; PL; rv:1.6) Gecko/20040113 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20020926) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.2b) Gecko/20021016 K-Meleon 0.7 |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20021002) |
| GT::WWW/1.022 |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; pl-PL; rv:1.6) Gecko/20040206 Firefox/0.8 |
| Anonymized by Steganos Internet Anonym Pro 6 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20020310) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20021119) |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020410) |
| NutchCVS/0.05-dev (Nutch; http://www.nutch.org/docs/en/bot.html; nutch-agent@lists.sourceforge.net) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020920) |
| ZipppBot/0.25 (Zippp; http://www.zippp.net; webmaster@zippp.net) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7) Gecko/20040614 Firefox/0.8 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20020307) |
| BlackMask.Net Search Engine |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020519) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20020828) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020206) |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20021125) |
| Mozilla/5.0 (Windows; U; Win98; en-US; rv:0.9.4) Gecko/20011128 Netscape6/6.2.1 (CK-VSNL) |
| Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.7) Gecko/20040624 Firefox/0.9 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20020113) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7) Gecko/20040614 Firefox/0.9 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020923) |
| Website Downloader http://www.notionlogic.com/ |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020624) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/125.2 (KHTML, like Gecko) Safari/125.8 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7) Gecko/20040616 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7) Gecko/20040614 Firefox/0.9 |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020514) |
| IlTrovatore-Setaccio/1.2 (Indexing; http://www.iltrovatore.it/bot.html; bot@iltrovatore.it) |
| Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.7) Gecko/20040626 Firefox/0.9.1 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; nl-nl) AppleWebKit/125.2 (KHTML, like Gecko) Safari/125.8 |
| Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.7) Gecko/20040514 |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7) Gecko/20040614 Firefox/0.8 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de-AT; rv:1.5) Gecko/20031007 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6) Gecko/20040510 Epiphany/1.2.4 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7) Gecko/20040626 Firefox/0.9.1 |
| Java/1.4.2_01 |
| ELinks (0.4.2; Linux; ) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8a2) Gecko/20040709 Firefox/0.9.0+ (MOOX-TK) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6) Gecko/20040115 Epiphany/1.0.7 |
| boitho.com-dc/0.51 ( http://www.boitho.com/dcbot.html ) |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020918) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7) Gecko/20040707 Firefox/0.9.2 |
| Mozilla/5.0 (Firefox 0.9; Windows NT 5.1 Service Pack 3; Geoshell 4.11.7R; H51 MEDIA [http://h51.darkistfearz.net/]) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7) Gecko/20040616 |
| Links (0.98; Darwin 7.4.0 Power Macintosh) |
| Omni-Explorer Search Interface Agent (support@omni-explorer.com) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20020923) |
| Mozilla/5.0 (compatible; Konqueror/3.1; Linux) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; cs-CZ; rv:1.6) Gecko/20040206 Firefox/0.8 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20021113) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.1) Gecko/20040707 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020603) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7) Gecko/20040626 Firefox/0.9.1 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; de-de) AppleWebKit/125.2 (KHTML, like Gecko) Safari/125.8 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7) Gecko/20040626 Firefox/0.8 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7b) Gecko/20040421 MultiZilla/1.6.3.2d |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020203) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020205) |
| Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20020507) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; fr-fr) AppleWebKit/125.2 (KHTML, like Gecko) Safari/125.8 |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7) Gecko/20040707 Firefox/0.9.2 |
| Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.7) Gecko/20040707 Firefox/0.8 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20021215) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20021123) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20020622) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de-DE; rv:1.7) Gecko/20040707 Firefox/0.9.2 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.5a) Gecko/20030728 Mozilla Firebird/0.6.1 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.6) Gecko/20040210 Firefox/0.8 |
| Iltrovatore-Setaccio/1.2 (It-bot; http://www.iltrovatore.it/bot.html; info@iltrovatore.it) |
| GeonaBot/1.1; http://www.geona.com/ |
| ObjectsSearch/0.05 (ObjectsSearch; http://www.ObjectsSearch.com/bot.html; support@thesoftwareobjects.com) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20021010) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020615) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/146.1 (KHTML, like Gecko) Safari/146 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.7) Gecko/20040803 Firefox/0.9.3 |
| SOFT411 Directory http://www.soft411.com |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020301) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.7) Gecko/20040517 Camino/0.8b |
| Mozilla/5.0 (Windows; U; Win 9x 4.90; en-US; rv:1.7) Gecko/20040707 Firefox/0.9.2 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020811) |
| Gigabot/2.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.1) Gecko/20040707 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020213) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020920) |
| Java1.4.0_03 |
| PEERbot www.peerbot.com |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020609) |
| sohu-search |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7) Gecko/20040803 Firefox/0.9.3 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20021201) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20020928) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/125.4 (KHTML, like Gecko) Safari/125.9 |
| Mozilla/5.0 (Windows; U; Win 9x 4.90; en-US; rv:1.7) Gecko/20040803 Firefox/0.9.3 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020819) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20021224) |
| MSNPTC/1.0 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20020326) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020713) |
| PHP version tracker |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020418) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20020219) |
| Projekt Pitchfork 3.1 - *NON-PUBLIC RELEASE* (MAC OS XI - Coyote;) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7) Gecko/20040707 Firefox/0.8 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.2) Gecko/20040803 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.0.1) Gecko/20020918 |
| ZipppBot/0.11 (ZipppBot; http://www.zippp.net; webmaster@zippp.net) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7) Gecko/20040803 Firefox/0.9.3 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20020415) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020507) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20020501) |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020102) |
| Mozilla/5.0 (Clustered-Search-Bot/1.0; support@clush.com; http://www.clush.com/) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7) Gecko/20040803 Firefox/0.8 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-us) AppleWebKit/85.8.2 (KHTML, like Gecko) Safari/85.8 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; hu-hu) AppleWebKit/103u (KHTML, like Gecko) Safari/100 |
| falcon/1.0 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020606) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020119) |
| Clushbot/2.1 (+http://www.clush.com/bot.html) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/125.4.2 (KHTML, like Gecko) Safari/125.9 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20020319) |
| IconSurf/2.0 favicon finder (see http://iconsurf.com/robot.html) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7) Gecko/20040803 Firefox/0.8 |
| Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.7) Gecko/20040803 Firefox/0.9.3 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20020808) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20020306) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020610) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20020928) |
| Mozilla/5.0 (compatible; Konqueror/3.1) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; fr-fr) AppleWebKit/125.5 (KHTML, like Gecko) Safari/125.9 |
| Wotbox/0.7-alpha (bot@wotbox.com; http://www.wotbox.com) |
| LinkExplore.com Link Popularity Software - Top 10 Rankings in Google Guaranteed |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; de-de) AppleWebKit/125.5 (KHTML, like Gecko) Safari/125.9 |
| Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.5a) Gecko/20030728 Mozilla Firebird/0.6.1 |
| Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.3.1) Gecko/20030425 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20020321) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020206) |
| Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7) Gecko/20040914 Firefox/0.9.3 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20020823) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020309) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; rv:1.7.3) Gecko/20040913 Firefox/0.10 |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020618) |
| msnbot/0.3 (+http://search.msn.com/msnbot.htm) |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020601) |
| Java/1.4.2_04 |
| lwp-trivial/1.40 |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020113) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; fr) AppleWebKit/125.5 (KHTML, like Gecko) Safari/125.9 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20020119) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20020615) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20020722) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.2) Gecko/20040803 |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020212) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020117) |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20021116) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20020413) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20021113) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20020419) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20021023) |
| webcrawl.net |
| Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.7) Gecko/20040626 Firefox/0.9.1 |
| Xenu Link Sleuth 1.2f |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20021128) |
| stat statcrawler@gmail.com |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020601) |
| ActiveBookmark 1.0 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020106) |
| Mozilla/5.0 (Windows; U; Win 9x 4.90; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; fr; rv:1.6) Gecko/20040206 Firefox/0.8 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20021111) |
| Faxobot/1.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.7) Gecko/20040803 Firefox/0.9.3 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/125.5 (KHTML, like Gecko) Safari/125.9 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020926) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20020519) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20021012) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; rv:1.7.3) Gecko/20040913 Firefox/0.10.1 |
| Blinkx/DFS-Fetch |
| Mozilla/5.0 (compatible; Konqueror/2.2.2; Linux) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020918) |
| WorldLight |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; rv:1.7.3) Gecko/20041001 Firefox/0.10.1 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20020227) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020622) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-us) AppleWebKit/125.5 (KHTML, like Gecko) Safari/125.9 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20020104) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020407) |
| Jetbot/1.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.3) Gecko/20040910 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6) Gecko/20040518 Firefox/0.8 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020526) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20021117) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020903) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; rv:1.7.3) Gecko/20040913 Firefox/0.10.1 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/85.8.2 (KHTML, like Gecko) Safari/85.8 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20020525) |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020608) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20020613) |
| SSM Agent 1.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.7) Gecko/20040803 Firefox/0.8 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020613) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7) Gecko/20040917 Firefox/0.9.3 |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020424) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20020306) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20020318) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20020320) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; rv:1.7.3) Gecko/20041001 Firefox/0.10.1 |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; rv:1.7.3) Gecko/20040913 Firefox/0.10 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020101) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7) Gecko/20040615 Firefox/0.9 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20020614) |
| MJ12bot/v0.5.0 (http://www.majestic12.co.uk/projects/dsearch/mj12bot.php) run by PeerID=F43AA089D42A3C7610E3778C4E73A95E MemberID=C9403BB515387FBB631AC512950E6F0E |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.7) Gecko/20040628 Firefox/0.9.1 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020914) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.3) Gecko/20041026 Firefox/1.0RC1 |
| MojeekBot/0.1 (galileo; http://www.mojeek.com/bot.html) |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020818) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20021004) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.2) Gecko/20040804 Netscape/7.2 (ax) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20020512) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020221) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020922) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020702) |
| SOFT411 Directory http://www.soft411.com/ |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7) Gecko/20040803 Firefox/0.9.3 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020219) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; fr-fr) AppleWebKit/125.4 (KHTML, like Gecko) Safari/125.9 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020620) |
| wbdbot |
| http://www.almaden.ibm.com/cs/crawler [wf223] |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6) Gecko/20040114 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20020222) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20020401) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020626) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20021113) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020309) |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20021007) |
| semanticdiscovery/0.3(http://www.semanticdiscovery.com/sd/robot.html |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.5) Gecko/20041107 Firefox/1.0 |
| NokodoBot/1.2 (+http://nokodo.com/bot.htm) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.5) Gecko/20041107 Firefox/1.0 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020324) |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20021226) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20020821) |
| Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.7.2) Gecko/20040804 Netscape/7.2 (ax) |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020721) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20021023) |
| Mozilla/5.0 (Windows; U; Win98; rv:1.7.3) Gecko/20040913 Firefox/0.10.1 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.4) Gecko/20030821 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.2) Gecko/20040805 Netscape/7.2 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020514) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-AR; rv:1.7.5) Gecko/20041108 Firefox/1.0 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20020228) |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20021124) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.2) Gecko/20040804 Netscape/7.2 (ax) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20021106) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/125.2 (KHTML, like Gecko) Safari/125.7 |
| NutchCVS/0.05 (Nutch; http://www.nutch.org/docs/en/bot.html; nutch-agent@lists.sourceforge.net) |
| NutchCVS/0.06-dev (Nutch; http://www.nutch.org/docs/en/bot.html; nutch-agent@lists.sourceforge.net) |
| SearchByUsa/2 (SearchByUsa; http://www.SearchByUsa.com/bot.html; info@SearchByUsa.com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de-DE; rv:1.7.5) Gecko/20041108 Firefox/1.0 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20021222) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/125.5.5 (KHTML, like Gecko) Safari/125.11 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020925) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020610) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20021226) |
| Mozilla/5.0 (compatible; BecomeBot/1.23; +http://www.become.com/webmasters.html) |
| Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.7.5) Gecko/20041107 Firefox/1.0 |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020124) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020113) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020124) |
| Mozilla/5.0 (Macintosh; U; PPC; en-US; rv:1.2.1) Gecko/20021130 |
| Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.7) Gecko/20040803 Firefox/0.9.3 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020311) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20020514) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20021113) |
| Mozilla/5.0 (compatible; BecomeBot/2.0beta; +http://www.become.com/webmasters.html) |
| Java(TM) 2 Runtime Environment, Standard Edition |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020712) |
| france.misesajour.com |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.7.5) Gecko/20041110 Firefox/1.0 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; fr-fr) AppleWebKit/125.5.5 (KHTML, like Gecko) Safari/125.11 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020516) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020403) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-us) AppleWebKit/125.5.5 (KHTML, like Gecko) Safari/125.12 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20021213) |
| Amfibibot/0.06 (Amfibi Robot; http://www.amfibi.com; agent@amfibi.com) |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020227) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20021217) |
| YottaCars_Bot/4.12 (+http://www.yottacars.com) Car Search Engine |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020823) |
| Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.7.5) Gecko/20041107 Firefox/1.0 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6) Gecko/20040510 |
| Zao-Crawler 0.1b |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de-DE; rv:1.7.5) Gecko/20041122 Firefox/1.0 |
| gazz/5.0 (gazz@nttr.co.jp) |
| http://www.almaden.ibm.com/cs/crawler [bc34] |
| Mozilla/5.0 (X11; U; Linux i686; rv:1.7.3) Gecko/20041020 Firefox/0.10.1 |
| ObjectsSearch/0.06 (ObjectsSearch; http://www.ObjectsSearch.com/bot.html; support@thesoftwareobjects.com) |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020719) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20020918) |
| Holmes/1.0 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20020706) |
| Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.7) Gecko/20040614 Firefox/0.8 |
| Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.6) Gecko/20040206 Firefox/0.8 |
| Mozilla/5.0 (compatible; Konqueror/3.3; Linux 2.6.8.1-3) (KHTML, like Gecko) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; el-GR; rv:1.7.5) Gecko/20041108 Firefox/1.0 |
| Mozilla/5.0 (compatible; Konqueror/3.2; Linux) KHTML/3.2.3 (like Gecko) |
| SeznamBot/1.0 |
| sherlock/1.0 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.7.5) Gecko/20041107 Firefox/1.0 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020503) |
| smartwit.com |
| Mozilla/5.0 (compatible; BecomeBot/2.0.1; +http://www.become.com/webmasters.html) |
| Knowledge.com/0.3 |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; it-IT; rv:1.7.5) Gecko/20041108 Firefox/1.0 |
| IRLbot/1.0 (+http://irl.cs.tamu.edu/crawler) |
| Mozilla/5.0 (compatible; Konqueror; FreeBSD; http://www.cartedaffaire.net/) |
| ConveraCrawler/0.4 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020715) |
| Zao-Crawler 0.2b |
| Mozilla/5.0 (compatible; BecomeBot/2.0.3; +http://www.become.com/webmasters.html) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20021001) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; rv:1.7) Gecko/20040803 Firefox/0.9.3 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.5) Gecko/20041107 Firefox/1.0 StumbleUpon/1.999 |
| InnerpriseBot/1.0 (http://www.innerprise.com/) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.5) Gecko/20050102 Firefox/1.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; tr-TR; rv:1.7.5) Gecko/20041208 Firefox/1.0 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.5) Gecko/20041107 Firefox/1.0 |
| GT::WWW/1.023 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-US) AppleWebKit/125.4 (KHTML, like Gecko, Safari) OmniWeb/v563.34 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr-FR; rv:1.7.5) Gecko/20041108 Firefox/1.0 |
| Java/1.4.2_06 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020615) |
| YottaShopping_Bot/4.12 (+http://www.yottashopping.com) Shopping Search Engine |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020509) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/125.5.5 (KHTML, like Gecko) Safari/125.12 |
| Homerweb |
| inetbot/0.0alpha (+http://www.inetbot.com/bot.html) |
| Clushbot/3.5-BinaryFury (+http://www.clush.com/bot.html) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.5) Gecko/20041128 Firefox/1.0 (Debian package 1.0-4) |
| Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.7.5) Gecko/20041103 Firefox/1.0RC2 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20021204) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20021008) |
| InelaBot/0.2 (+http://inelegant.org/bot) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; nl-NL; rv:1.7.5) Gecko/20041202 Firefox/1.0 |
| updated/0.1beta (updated.com; http://www.updated.com; crawler@updated.om) |
| Clushbot/3.9-BinaryFury (+http://www.clush.com/bot.html) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20020118) |
| msnbot/1.0 (+http://search.msn.com/msnbot.htm) |
| Mozilla/5.0 (Windows; U; Windows NT 5.2; zh-CN; rv:1.7.5) Gecko/20041124 Firefox/1.0 (ax) |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020216) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20020211) |
| Zao-Crawler |
| boitho.com-dc/0.66 ( http://www.boitho.com/dcbot.html ) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) |
| NetResearchServer/3.4(loopimprovements.com/robot.html) |
| OmniExplorer_Bot/1.07 (+http://www.omni-explorer.com) Internet Categorizer |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20020621) |
| Mozilla/5.0 (X11; U; Linux i686; pl-PL; rv:1.7) Gecko/20040616 |
| agent |
| Mozilla/5.0 (Windows; U; Win98; tr-TR; rv:1.7.5) Gecko/20041208 Firefox/1.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; pl-PL; rv:1.7.5) Gecko/20041108 Firefox/1.0 |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020813) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.3) Gecko/20040910 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20020113) |
| lwp-trivial/1.41 |
| HenryTheMiragoRobot (http://www.miragorobot.com/scripts/mrinfo.asp) |
| Mozilla/5.0 (compatible; Konqueror/3.1; FreeBSD) |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020224) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; it-IT; rv:1.7.5) Gecko/20041110 Firefox/1.0 |
| OmniExplorer_Bot/1.07 (+http://www.omni-explorer.com) Job Crawler |
| Crawler (cometsearch@cometsystems.com) |
| Clushbot/3.21-BinaryFury (+http://www.clush.com/bot.html) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; fr-FR; rv:1.7.5) Gecko/20041108 Firefox/1.0 |
| RS |
| Mozilla/5.0 (Windows; U; Win 9x 4.90; es-ES; rv:1.7.5) Gecko/20041108 Firefox/1.0 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020928) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; pt-BR; rv:1.7.5) Gecko/20041118 Firefox/1.0 |
| newsg8/0.1 |
| Mozilla/5.0 (X11; U; Linux i686; fr; rv:1.7.2) Gecko/20040804 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020521) |
| Aberja Checkomat |
| Mozilla/5.0 (Windows; U; Win 9x 4.90; en-US; rv:1.7.5) Gecko/20041107 Firefox/1.0 |
| Clushbot/3.23-BinaryFury (+http://www.clush.com/bot.html) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; nl-NL; rv:1.7.5) Gecko/20041202 Firefox/1.0 (ax) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020420) |
| Mozilla/5.0 (compatible; Konqueror/3.1; Linux; i686) |
| Mozilla/5.0 (compatible; Konqueror/3.0; Darwin) |
| Mozilla/5.0 (compatible; Konqueror/3; Linux) |
| Mozilla/5.5 (compatible; alpha/06; AmigaOS) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.5) Gecko/20041103 Firefox/1.0RC2 |
| Jakarta Commons-HttpClient/3.0-beta1 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; cs-CZ; rv:1.7.5) Gecko/20041108 Firefox/1.0 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20021120) |
| Mozilla/5.0 (compatible; Konqueror/2.2.2; FreeBSD) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8a4) Gecko/20040927 |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020305) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20021110) |
| boitho.com-dc/0.71 ( http://www.boitho.com/dcbot.html ) |
| Mozilla/5.0 (compatible; AnsearchBot/1.1; +http://www.ansearch.com/) |
| Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.7.6) Gecko/20050223 Firefox/1.0.1 |
| Clushbot/3.24-BinaryFury (+http://www.clush.com/bot.html) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7) Gecko/20041013 Firefox/0.9.3 (Ubuntu) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/125.5.6 (KHTML, like Gecko) Safari/125.12 |
| Mozilla/2.0 (compatible; Ask Jeeves/Teoma; +http://sp.ask.com/docs/about/tech_crawling.html) |
| savvybot/0.2 |
| htdig |
| Sopheus Project/0.01 (Nutch; http://www.thenetplanet.com; info@thenetplanet.com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.6) Gecko/20050225 Firefox/1.0.1 |
| Java/1.5.0_01 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20020227) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.5) Gecko/20041217 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20020316) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.6) Gecko/20050225 Firefox/1.0.1 |
| Jakarta Commons-HttpClient/2.0.2 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.7.6) Gecko/20050223 Firefox/1.0.1 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-AR; rv:1.7.6) Gecko/20050223 Firefox/1.0.1 |
| MnoGoSearch/3.2.31 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; nl-NL; rv:1.7.6) Gecko/20050226 Firefox/1.0.1 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20021004) |
| HooWWWer/2.1.0 (+http://cosco.hiit.fi/search/hoowwwer/ | mailto:crawler-infohiit.fi) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-us) AppleWebKit/125.5.6 (KHTML, like Gecko) Safari/125.12 |
| ObjectsSearch/0.07 (ObjectsSearch; http://www.ObjectsSearch.com/bot.html; support@thesoftwareobjects.com) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20020209) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20020717) |
| Mozilla/5.0 (compatible; Konqueror/3.3; Linux 2.6.8-2-686-smp) KHTML/3.3.2 (like Gecko) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.1) Gecko/20040707 WebWasher 3.4 |
| DataFountains/DMOZ Downloader |
| Clushbot/3.31-BinaryFury (+http://www.clush.com/bot.html) |
| NP/0.1 (NP; http://www.nameprotect.com; npbot@nameprotect.com) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-us) AppleWebKit/85.8.5 (KHTML, like Gecko) Safari/85.8.1 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020806) |
| Dudu v1.4 Platinum |
| Mozilla/5.0 (Windows; U; Win98; es-AR; rv:1.7.5) Gecko/20041108 Firefox/1.0 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; nl-nl) AppleWebKit/125.5.6 (KHTML, like Gecko) Safari/125.12 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20021220) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.6) Gecko/20050223 Firefox/1.0.1 |
| Mozilla/5.0 (Windows; U; Win98; pl-PL; rv:1.7.5) Gecko/20041108 Firefox/1.0 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; it-IT; rv:1.7.5) Gecko/20041110 Firefox/1.0 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020709) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.6) Gecko/20050302 Firefox/1.0.1 Fedora/1.0.1-1.3.2 |
| SeznamBot/1.0 (+http://fulltext.seznam.cz/) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-us) AppleWebKit/125.5.7 (KHTML, like Gecko) Safari/125.12 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20020724) |
| muscopeBOT/0.01 (muscope; http://www.muscope.com/bot.html; bot@muscope.com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.6) Gecko/20050317 Firefox/1.0.2 |
| EmeraldShield.com WebBot (http://www.emeraldshield.com/webbot.aspx) |
| SCrawlTest/CR1 (CWD; http://sproose.com; crawl@sproose.com) |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020501) |
| Clushbot/3.38-BinaryFury (+http://www.clush.com/bot.html) |
| aipbot/1.0 (aipbot; http://www.aipbot.com; aipbot@aipbot.com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.6) Gecko/20050317 Firefox/1.0.2 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20020218) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.7.6) Gecko/20050317 Firefox/1.0.2 |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-GB; rv:1.7.5) Gecko/20041110 Firefox/1.0 |
| InternetSupervision (http://internetsupervision.com/urlmonitor/3) |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20021220) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20020112) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; de-DE; rv:1.7.6) Gecko/20050321 Firefox/1.0.2 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.7.6) Gecko/20050321 Firefox/1.0.2 |
| semanticdiscovery/0.4(http://www.semanticdiscovery.com/sd/robot.html |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/125.5.7 (KHTML, like Gecko) Safari/125.12 |
| Python-urllib/1.16 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; pl-PL; rv:1.7.6) Gecko/20050223 Firefox/1.0.1 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; it-IT; rv:1.7.6) Gecko/20050318 Firefox/1.0.2 |
| Knowledge.com/0.2 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr-FR; rv:1.7.6) Gecko/20050318 Firefox/1.0.2 |
| Clushbot/3.41-BinaryFury (+http://www.clush.com/bot.html) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.6) Gecko/20050324 Firefox/1.0.2 Red Hat/1.0.2-1.4.1.centos4 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20020106) |
| McBot/5.001 (windows; U; NT4.0; en-us) |
| NutchCVS/0.06-dev (Nutch; http://www.s14g.com/en/search.html; travelbot@s14g.com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.2; es-ES; rv:1.7.6) Gecko/20050321 Firefox/1.0.2 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.7) Gecko/20050414 Firefox/1.0.3 |
| Mozilla/5.0 (Windows; U; Win98; en-GB; rv:1.7.6) Gecko/20050321 Firefox/1.0.2 |
| Clushbot/3.42-BinaryFury (+http://www.clush.com/bot.html) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020708) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr-FR; rv:1.7.7) Gecko/20050414 Firefox/1.0.3 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; tr-TR; rv:1.7.7) Gecko/20050414 Firefox/1.0.3 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; tr-TR; rv:1.7.6) Gecko/20050321 Firefox/1.0.2 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20021222) |
| Schmozilla/v9.14 Platinum |
| snap.com beta crawler v0 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-us) AppleWebKit/312.1 (KHTML, like Gecko) Safari/312 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7b) Gecko/20040421 |
| Wget/1.9 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20020416) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.6) Gecko/20050406 Firefox/1.0.2 (Debian package 1.0.2-3) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.7.6) Gecko/20050226 Firefox/1.0.1 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de-DE; rv:1.7.6) Gecko/20050321 Firefox/1.0.2 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20020214) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; rv:1.7.3) Gecko/20041001 Firefox/0.10.1 StumbleUpon/1.9993 |
| NextGenSearchBot 1 (for information visit http://www.eliyon.com/NextGenSearchBot) |
| Phantom/2.5.1 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.7) Gecko/20050414 Firefox/1.0.3 StumbleUpon/1.9995 |
| Regiochannel/1.0 (http://www.regiochannel.de/) |
| Clushbot/3.48-BinaryFury (+http://www.clush.com/bot.html) |
| Mozilla/5.0 (Windows; U; Windows NT 5.2; es-AR; rv:1.7.6) Gecko/20050321 Firefox/1.0.2 |
| Java/1.5.0_02 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/412 (KHTML, like Gecko) Safari/412 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20020102) |
| Lynx/2.8.4rel.1 libwww-FM/2.14 SSL-MM/1.4.1 OpenSSL/0.9.6b |
| NetResearchServer/3.5(loopimprovements.com/robot.html) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; it-IT; rv:1.7.7) Gecko/20050414 Firefox/1.0.3 |
| ConveraCrawler/0.7 (+http://www.authoritativeweb.com/crawl) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; hu-HU; rv:1.7.7) Gecko/20050414 Firefox/1.0.3 |
| GOFORITBOT ( http://www.goforit.com/about/ ) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; es) AppleWebKit/312.1 (KHTML, like Gecko) Safari/312 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; pt-BR; rv:1.7.7) Gecko/20050414 Firefox/1.0.3 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020411) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20020913) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.8) Gecko/20050511 Firefox/1.0.4 |
| DTAAgent |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; sv-SE; rv:1.7.7) Gecko/20050414 Firefox/1.0.3 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20021217) |
| Clushbot/3.49-BinaryFury (+http://www.clush.com/bot.html) |
| Amfibibot/0.07 (Amfibi Robot; http://www.amfibi.com; agent@amfibi.com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de-DE; rv:1.7.7) Gecko/20050414 Firefox/1.0.3 |
| trafficbloater/1.0+(TrafficBloater;+http://www.trafficbloater.com) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20020914) |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020211) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.8) Gecko/20050511 Firefox/1.0.4 |
| ConveraCrawler/0.8 (+http://www.authoritativeweb.com/crawl) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; ja-JP; rv:1.7.7) Gecko/20050414 Firefox/1.0.3 |
| AtlocalBot/1.1 +(http://www.atlocal.com/local-web-site-owner.html) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8b2) Gecko/20050419 Firefox/1.0+ |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20021214) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.7) Gecko/20050414 Firefox/1.0.3 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; he-IL; rv:1.7.5) Gecko/20041108 Firefox/1.0 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020128) |
| SiteBar/3.3 (Bookmark Server; http://sitebar.org/) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; pl-PL; rv:1.7.8) Gecko/20050511 Firefox/1.0.4 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20020401) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020112) |
| http://www.almaden.ibm.com/cs/crawler [c11] |
| Mozilla/5.0 (compatible; heritrix/1.3.0 +http://www.l3s.de/) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020912) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020912) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.6) Gecko/20050317 Firefox/1.0.2 |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; fr-FR; rv:1.7.6) Gecko/20050318 Firefox/1.0.2 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fi-FI; rv:1.7.8) Gecko/20050511 Firefox/1.0.4 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; ru-RU; rv:1.7.6) Gecko/20050318 Firefox/1.0.2 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.7) Gecko/20050414 |
| Clushbot/3.53-BinaryFury (+http://www.clush.com/bot.html) |
| Twiceler www.cuill.com/robots.html |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20020728) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.7.7) Gecko/20050414 Firefox/1.0.3 |
| Mozilla 4.0 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; fr-fr) AppleWebKit/312.1 (KHTML, like Gecko) Safari/312 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020606) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.7.8) Gecko/20050427 Camino/0.8.4 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20020503) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20021005) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.8) Gecko/20050511 Firefox/1.0.4 (ax) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020503) |
| BruinBot (+http://webarchive.cs.ucla.edu/bruinbot.html) |
| Mozilla/5.0 (X11; U; Linux i686; pl-PL; rv:1.7.8) Gecko/20050511 Firefox/1.0.4 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.8) Gecko/20050511 |
| ActiveBookmark 1.1 |
| Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.7.8) Gecko/20050511 Firefox/1.0.4 |
| Java/1.5.0_03 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de-DE; rv:1.7.8) Gecko/20050511 Firefox/1.0.4 |
| Mozilla/5.0 (compatible; heritrix/1.5.0 +http://www.l3s.de/~kohlschuetter/projects/crawling/) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; it-IT; rv:1.7.8) Gecko/20050511 Firefox/1.0.4 |
| Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.7) Gecko/20040707 Firefox/0.9.2 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/312.1 (KHTML, like Gecko) Safari/312 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.6) Gecko/20050524 Firefox/1.0 (Ubuntu package 1.0.2 MFSA2005-44) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; fi-FI; rv:1.7.6) Gecko/20050318 Firefox/1.0.2 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20021003) |
| updated/0.1beta (updated.com; http://www.updated.com; crawler@updated.com) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.7.8) Gecko/20050511 Firefox/1.0.4 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr-FR; rv:1.7.8) Gecko/20050511 Firefox/1.0.4 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020722) |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020319) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020813) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.8) Gecko/20050511 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-AR; rv:1.7.6) Gecko/20050226 Firefox/1.0.1 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.7.8) Gecko/20050511 Firefox/1.0.4 |
| ccubee/3.0 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020715) |
| cfetch/1.0 |
| http://www.sygol.it |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020105) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020216) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020801) |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020803) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020511) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20021028) |
| NutchTestBot/0.0 (Nutch Test; ; crawler.support@gmail.com) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20020428) |
| OmniExplorer_Bot/1.09 (+http://www.omni-explorer.com) Internet Categorizer |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; nl-NL; rv:1.7.7) Gecko/20050414 Firefox/1.0.3 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20020105) |
| Xenu Link Sleuth 1.2g |
| Wget/1.9+cvs-stable (Red Hat modified) |
| LWP::Simple/5.800 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.5) Gecko/20050603 Netscape/8.0.2 |
| Mozilla/5.0 (Windows; U; Windows NT 5.2; en-GB; rv:1.7.8) Gecko/20050511 Firefox/1.0.4 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20020501) |
| WebIndexer/1-dev (WebIndexer; webindexerv1@yahoo.com; webindexerv1@yahoo.com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-AR; rv:1.7.8) Gecko/20050511 Firefox/1.0.4 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20020902) |
| Miva (AlgoFeedback@miva.com) |
| Java/1.5.0_04 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20020718) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020203) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020616) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20021013) |
| Links (0.95; Darwin 7.3.1 Power Macintosh) |
| DomainsDB.net MetaCrawler v.0.9.7b (http://domainsdb.net/) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.8) Gecko/20050511 Firefox/1.0.4 StumbleUpon/1.9992 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20021203) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.10) Gecko/20050716 Firefox/1.0.6 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020819) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-GB; rv:1.7.8) Gecko/20050511 Firefox/1.0.4 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020811) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020827) |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020408) |
| DomainsDB.net MetaCrawler v.0.9.7c (http://domainsdb.net/) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020217) |
| Mozilla/5.0 (compatible; nextthing.org/1.0; +http://www.nextthing.org/bot) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20021215) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20020915) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20021102) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20020115) |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20021223) |
| Dumbot(version 0.1 beta - http://www.dumbfind.com/dumbot.html) |
| ConveraCrawler/0.9 (+http://www.authoritativeweb.com/crawl) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; nl-NL; rv:1.7.8) Gecko/20050511 Firefox/1.0.4 |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.9) Gecko/20050711 Firefox/1.0.5 |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20021120) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020718) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20021125) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr-FR; rv:1.7.10) Gecko/20050717 Firefox/1.0.6 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de-DE; rv:1.7.10) Gecko/20050717 Firefox/1.0.6 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20021209) |
| Gaisbot/3.0+(robot05@gais.cs.ccu.edu.tw;+http://gais.cs.ccu.edu.tw/robot.php) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020220) |
| $BotName/$BotVersion [en] (UNIX; U) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.7.10) Gecko/20050717 Firefox/1.0.6 |
| ConveraCrawler/0.9c (+http://www.authoritativeweb.com/crawl) |
| wwwster/1.4 (Beta, mailto:gue@cis.uni-muenchen.de) |
| GT::WWW/1.025 |
| Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.7.6) Gecko/20050317 Firefox/1.0.6 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20021002) |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020306) |
| Website Monitoring Bot (http://InternetSupervision.com/UrlMonitor/3/) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20020913) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; pl-PL; rv:1.7.10) Gecko/20050717 Firefox/1.0.6 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20020618) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20021205) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020118) |
| Mozilla/5.0 (compatible; Konqueror/3.3; Linux) KHTML/3.3.2 (like Gecko) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.8) Gecko/20050517 Firefox/1.0.4 (Debian package 1.0.4-2) |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020924) |
| Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.7.6) Gecko/20050317 Firefox/1.0.2 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20021223) |
| Clushbot/3.57-Ajax (+http://www.clush.com/bot.html) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.10) Gecko/20050716 Firefox/1.0.6 (ax) |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020716) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.7.10) Gecko/20050716 Firefox/1.0.6 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020513) |
| OmniExplorer_Bot/3.28 (+http://www.omni-explorer.com) WorldIndexer |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20020508) |
| MojeekBot/0.2 (archi; http://www.mojeek.com/bot.html) |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020206) |
| http://www.chrisranjana.com |
| Mozilla/5.0 (X11; U; Linux i686; it-IT; rv:1.2.1) Gecko/20030225 |
| findlinks/0.961 (+http://wortschatz.uni-leipzig.de/findlinks/) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.7.7) Gecko/20050414 Firefox/1.0.3 |
| NutchCVS/0.8-dev (Nutch; http://lucene.apache.org/nutch/bot.html; nutch-agent@lucene.apache.org) |
| Jayde Crawler. http://www.jayde.com |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; sk-SK; rv:1.7.10) Gecko/20050717 Firefox/1.0.6 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020104) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20021209) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020524) |
| Clushbot/3.58-Ajax (+http://www.clush.com/bot.html) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020417) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; es-AR; rv:1.7.10) Gecko/20050717 Firefox/1.0.6 |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020720) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20020321) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fi-FI; rv:1.7.10) Gecko/20050717 Firefox/1.0.6 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20021210) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; ru-RU; rv:1.7.10) Gecko/20050717 Firefox/1.0.6 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020713) |
| Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.7.10) Gecko/20050716 Firefox/1.0.6 |
| NaverBot-1.0 (NHN Corp. / +82-2-3011-1989 / nhnbot@naver.com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.7) Gecko/20050414 Firefox/1.0.3 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.5) Gecko/20050302 Firefox/0.9.6 |
| Snappy/1.1 ( http://www.urltrends.com/ ) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20021125) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.10) Gecko/20050716 Firefox/1.0.6 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20020507) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; pt-BR; rv:1.7.8) Gecko/20050511 Firefox/1.0.4 |
| Clushbot/3.59-Hector (+http://www.clush.com/bot.html) |
| Clushbot/3.60-Peleus (+http://www.clush.com/bot.html) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20020501) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.7.7) Gecko/20050414 Firefox/1.0.3 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020708) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-us) AppleWebKit/412 (KHTML, like Gecko) Safari/412 |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020101) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020601) |
| Clushbot/3.62-Laomedon (+http://www.clush.com/bot.html) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; pt-PT; rv:1.7.10) Gecko/20050717 Firefox/1.0.6 |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020519) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20020813) |
| e-SocietyRobot(http://www.yama.info.waseda.ac.jp/~yamana/es/) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020626) |
| Mozilla/5.0 (Windows; U; Win98; de-DE; rv:1.7.10) Gecko/20050717 Firefox/1.0.6 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20021002) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.5) Gecko/20050729 Netscape/8.0.3.3 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20020818) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20021219) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.10) Gecko/20050715 Firefox/1.0.6 SUSE/1.0.6-4.1 |
| IlTrovatore-Setaccio/1.2 (It search engine; http://www.iltrovatore.it/bot.html; bot@iltrovatore.it) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; de-DE; rv:1.7.5) Gecko/20041122 Firefox/1.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8b4) Gecko/20050908 Firefox/1.4 |
| SBIder/0.7 (SBIder; http://www.sitesell.com/sbider.html; http://support.sitesell.com/contact-support.html) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/312.5 (KHTML, like Gecko) Safari/312.3 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20020825) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20021101) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; da-DK; rv:1.7.10) Gecko/20050717 Firefox/1.0.6 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20021218) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20020908) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020804) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20020506) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20020314) |
| FAST MetaWeb Crawler (helpdesk at fastsearch dot com) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/412.7 (KHTML, like Gecko) Safari/412.5 |
| Findexa Crawler (http://www.findexa.no/gulesider/article26548.ece) |
| IP2LocationBot/1.0 http://www.ip2location.com |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20020622) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020122) |
| NutchCVS/0.7 (Nutch; http://lucene.apache.org/nutch/bot.html; nutch-agent@lucene.apache.org) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20020527) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020814) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020319) |
| yoono/1.0 web-crawler/1.0 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20021118) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.12) Gecko/20050915 Firefox/1.0.7 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.10) Gecko/20050920 Firefox/1.0.6 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr-FR; rv:1.7.9) Gecko/20050713 Firefox/1.0.5 |
| PeerFactor 404 crawler |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020412) |
| MJ12bot/v1.0.2 (http://majestic12.co.uk/bot.php?+) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.10) Gecko/20050909 Fedora/1.0.6-1.2.fc4 Firefox/1.0.6 |
| InternetArchive/0.8-dev (Nutch; http://lucene.apache.org/nutch/bot.html; nutch-agent@lucene.apache.org) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020727) |
| LocalcomBot/1.2 (+http://www.local.com/bot.htm) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20020125) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; pt-PT; rv:1.7.12) Gecko/20050919 Firefox/1.0.7 |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20021112) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de-DE; rv:1.7.12) Gecko/20050919 Firefox/1.0.7 |
| Mozilla/5.0 (Windows; U; Win 9x 4.90; fr-FR; rv:1.7.7) Gecko/20050414 Firefox/1.0.3 |
| SBIder/0.8-dev (SBIder; http://www.sitesell.com/sbider.html; http://support.sitesell.com/contact-support.html) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20021009) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020805) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20021219) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20021021) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.12) Gecko/20050922 Firefox/1.0.7 (Ubuntu package 1.0.7) |
| updated/0.2-dev (updated; http://www.updated.com; crawler@updated.com) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020910) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; it-IT; rv:1.7.12) Gecko/20050919 Firefox/1.0.7 |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020121) |
| AnswerBus (http://www.answerbus.com/) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.7.12) Gecko/20050919 Firefox/1.0.7 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20020611) |
| Computer_and_Automation_Research_Institute_Crawler crawler@ilab.sztaki.hu |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20020204) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.12) Gecko/20050915 Firefox/1.0.7 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.5) Gecko/20041210 Firefox/1.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.7.12) Gecko/20050919 Firefox/1.0.7 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20020827) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020907) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20021109) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; nl-NL; rv:1.7.10) Gecko/20050717 Firefox/1.0.6 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20020520) |
| NutchOSU-VLIB/0.7 (Nutch; http://lucene.apache.org/nutch/bot.html; nutch-agent@lucene.apache.org) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20020613) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20020107) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.12) Gecko/20050915 |
| Mozilla/5.0 (Windows; U; Win 9x 4.90; en-US; rv:1.7.10) Gecko/20050716 Firefox/1.0.6 |
| Sensis Web Crawler (search_commentsatsensisdotcomdotau) |
| MJ12bot/v1.0.3 (http://majestic12.co.uk/bot.php?+) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20020521) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN; rv:1.7.12) Gecko/20050919 Firefox/1.0.7 |
| Java/1.5.0_05 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-AR; rv:1.7.12) Gecko/20050919 Firefox/1.0.7 |
| !Susie (http://www.sync2it.com/susie) |
| Mnogosearch-3.1.21 |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20021024) |
| Mozilla/5.0 (Windows; U; Win 9x 4.90; fr-FR; rv:1.7.10) Gecko/20050717 Firefox/1.0.6 |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020716) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020114) |
| Mozilla/5.0 (Windows; U; Win98; de-DE; rv:1.7.8) Gecko/20050511 Firefox/1.0.4 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20020501) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020428) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.9) Gecko/20050711 Firefox/1.0.5 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; ja-jp) AppleWebKit/412.7 (KHTML, like Gecko) Safari/412.5 |
| NutchCVS/0.7.1 (Nutch; http://lucene.apache.org/nutch/bot.html; nutch-agent@lucene.apache.org) |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020814) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.10) Gecko/20050822 Firefox/1.0.6 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/412.6 (KHTML, like Gecko) Safari/412.2 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020628) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020827) |
| sohu agent |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8b5) Gecko/20051006 Firefox/1.4.1 |
| Metaspinner/0.01 (Metaspinner; http://www.meta-spinner.de/; support@meta-spinner.de/) |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20021027) |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020619) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020512) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20021219) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8b5) Gecko/20051006 Firefox/1.4.1 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.12) Gecko/20050915 Firefox/1.0.7 (ax) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020824) |
| OmniExplorer_Bot/4.32 (+http://www.omni-explorer.com) WorldIndexer |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020515) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.7.10) Gecko/20050717 Firefox/1.0.6 |
| IP2PhraseBot/1.0 http://www.ip2phrase.com |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020607) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20021107) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20020412) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8b5) Gecko/20051006 Firefox/1.4.1 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20020422) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020303) |
| DNSGroup/0.1 (DNS Group Crawler; http://www.dnsgroup.com/; crawler@dnsgroup.com) |
| ichiro/2.0 (ichiro@nttr.co.jp) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20020124) |
| MJ12bot/v1.0.5 (http://majestic12.co.uk/bot.php?+) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; no-no) AppleWebKit/312.1 (KHTML, like Gecko) Safari/312 |
| Java/1.5.0 |
| Mozilla/5.0 (X11; U; Linux i686; es-ES; rv:1.7.12) Gecko/20050922 Firefox/1.0.7 (Debian package 1.0.7-1) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20021106) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20020217) |
| IlTrovatore-Setaccio/1.2 (Italy search engine; http://www.iltrovatore.it/bot.html; bot@iltrovatore.it) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8b5) Gecko/20051021 Flock/0.4 Firefox/1.0+ |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20021119) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/416.11 (KHTML, like Gecko) Safari/416.12 |
| RufusBot (Rufus Web Miner; http://64.124.122.252/feedback.html) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020518) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.5) Gecko/20051012 Netscape/8.0.4 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020128) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20020917) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020102) |
| Slurp/0.8-dev (Nutch; http://lucene.apache.org/nutch/bot.html; nutch-agent@lucene.apache.org) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20020605) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020606) |
| PHP version tracker (http://www.nexen.net/phpversion/bot.php) |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020728) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/416.11 (KHTML, like Gecko) Shiira/1.1 Safari/125 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.12) Gecko/20050922 Fedora/1.0.7-1.1.fc4 Firefox/1.0.7 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20021111) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020301) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20020414) |
| dcs |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20020328) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20021212) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20021009) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020301) |
| RB2B-bot v0.1 (Using Fast Enterprise Crawler 6 search@reedbusiness.com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.11) Gecko/20050728 |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020116) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020315) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; pt-BR; rv:1.7.10) Gecko/20050717 Firefox/1.0.6 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; ko-kr) AppleWebKit/416.11 (KHTML, like Gecko) Safari/416.12 |
| Mozilla/5.0 (compatible; Yahoo! Slurp China; http://misc.yahoo.com.cn/help.html) |
| Mozilla/5.0 (Windows; U; Win98; en-GB; rv:1.7.12) Gecko/20050919 Firefox/1.0.7 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20020808) |
| ejupiter.com |
| Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.7.12) Gecko/20050915 Firefox/1.0.7 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20020825) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20021011) |
| NASA Search 1.0 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020103) |
| ccubee/3.2 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; fr) AppleWebKit/416.11 (KHTML, like Gecko) Safari/416.12 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.8) Gecko/20051107 Firefox/1.5 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.12) Gecko/20050922 Firefox/1.0.7 (Debian package 1.0.7-1) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; nb-NO; rv:1.7.6) Gecko/20050226 Firefox/1.0.1 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/312.5.1 (KHTML, like Gecko) Safari/312.3.1 |
| Mozilla/5.0 (Windows; U; Win 9x 4.90; en-US; rv:1.7.12) Gecko/20050915 Firefox/1.0.7 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020327) |
| LWP::Simple/5.803 |
| Wget/1.10.2 |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; pl-PL; rv:1.7.12) Gecko/20050919 Firefox/1.0.7 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020323) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020801) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20021219) |
| HTMLParser/1.6 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20021021) |
| Mozilla/5.0 (X11; U; Linux i686; de-DE; rv:1.7.12) Gecko/20050920 Firefox/1.0.7 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020714) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020817) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20020212) |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020705) |
| KFSW-Bot (Version: 1.01, powered by KFSW, www.kfsw.de) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20021116) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; fr-FR; rv:1.7.12) Gecko/20050919 Firefox/1.0.7 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20020205) |
| OmniExplorer_Bot/5.01 (+http://www.omni-explorer.com) WorldIndexer |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.12) Gecko/20051011 Firefox/1.0.7 |
| Java/1.4.2_10 |
| NetResearchServer/4.0(loopimprovements.com/robot.html) |
| Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.7.12) Gecko/20050915 |
| ConveraCrawler/0.9d (+http://www.authoritativeweb.com/crawl) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.2) Gecko/20030208 Netscape/7.02 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020925) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8) Gecko/20051111 Firefox/1.5 |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020118) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8) Gecko/20051111 Firefox/1.5 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020517) |
| genieBot (http://64.5.245.11/faq/faq.html) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-AR; rv:1.7.10) Gecko/20050717 Firefox/1.0.6 |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020325) |
| findlinks/1.0.9 (+http://wortschatz.uni-leipzig.de/findlinks/) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8) Gecko/20051111 Firefox/1.5 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020404) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20020416) |
| Thumbnail.CZ robot 1.1 (http://thumbnail.cz/why-no-robots-txt.html) |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020302) |
| FAST Enteprise Crawler/6 (www dot fastsearch dot com) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.12) Gecko/20050920 Firefox/1.0.7 SUSE/1.0.7-0.1 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20020824) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20021026) |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020821) |
| findlinks/1.1-a5 (+http://wortschatz.uni-leipzig.de/findlinks/) |
| xirq/0.1-beta (xirq; http://www.xirq.com; xirq@xirq.com) |
| Mozilla/5.0 (compatible; Theophrastus/1.2; +http://users.cs.cf.ac.uk/N.A.Smith/theophrastus.php) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr-FR; rv:1.7.12) Gecko/20050919 Firefox/1.0.7 |
| Mozilla/5.0 (Windows; U; Win98; fr; rv:1.7.3) Gecko/20040910 |
| Randy |
| MJ12bot/v1.0.6 (http://majestic12.co.uk/bot.php?+) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20021201) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020526) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20021019) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fi; rv:1.8) Gecko/20051111 Firefox/1.5 |
| OmniExplorer_Bot/5.18 (+http://www.omni-explorer.com) WorldIndexer |
| Mozilla/5.0 (compatible; Konqueror/3.3; Linux) (KHTML, like Gecko) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020317) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20020616) |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020726) |
| onCHECK Linkchecker von www.scientec.de fuer www.onsinn.de |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; ro-RO; rv:1.7.10) Gecko/20050717 Firefox/1.0.6 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; ru-RU; rv:1.7.12) Gecko/20050919 Firefox/1.0.7 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020103) |
| findlinks/1.1-a7 (+http://wortschatz.uni-leipzig.de/findlinks/) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-us) AppleWebKit/416.12 (KHTML, like Gecko) Safari/416.13 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; hu-HU; rv:1.7.12) Gecko/20050919 Firefox/1.0.7 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20020308) |
| Java/1.6.0-rc |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020326) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20020909) |
| LWP::Simple/5.65 |
| Mozilla/5.0 (X11; U; Linux i686; de; rv:1.8) Gecko/20051111 Firefox/1.5 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020117) |
| NavissoBot/1.7 (+http://navisso.com/) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020705) |
| silk/1.0 (+http://www.slider.com/silk.htm)/3.7 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20020712) |
| Java/1.4.2_03 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.8) Gecko/20051111 Firefox/1.5 |
| msnbot/0.9 (+http://search.msn.com/msnbot.htm) |
| MJ12bot/v1.0.7 (http://majestic12.co.uk/bot.php?+) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020201) |
| Poirot |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020821) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20021218) |
| Mozilla/5.0 (Windows; U; Win 9x 4.90; fr; rv:1.8) Gecko/20051111 Firefox/1.5 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.8) Gecko/20051111 Firefox/1.5 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020722) |
| Vortex/2.2 (+http://marty.anstey.ca/robots/vortex/) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/416.12 (KHTML, like Gecko) Safari/416.13 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020113) |
| zibber-v0.1(www.zibb.com/crawler/) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20020528) |
| curl/7.14.1 (i386-portbld-freebsd4.7) libcurl/7.14.1 OpenSSL/0.9.8 zlib/1.1.3 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020404) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20021014) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/312.5.2 (KHTML, like Gecko) Safari/312.3.3 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20021106) |
| http://www.almaden.ibm.com/cs/crawler [fc14] |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020324) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; ro; rv:1.8) Gecko/20051111 Firefox/1.5 |
| plinki/0.1 (you got plinked! (thats a good thing..); http://www.plinki.com; crawl@plinki.com) |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020505) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/417.9 (KHTML, like Gecko) Safari/417.8 |
| noxtrumbot/1.0 (crawler@noxtrum.com) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20020515) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020802) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.12) Gecko/20050922 Fedora/1.0.7-1.1.fc3 Firefox/1.0.7 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; lt-LT; rv:1.8) Gecko/20051111 Firefox/1.5 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20021017) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8) Gecko/20051111 Firefox/1.5 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.5) Gecko/20041107 Firefox/1.0 StumbleUpon/1.9995 |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20021020) |
| Mozilla/5.0 (Windows; U; Win 9x 4.90; en-US; rv:1.8) Gecko/20051111 Firefox/1.5 |
| Test_Robot_1.1/Virem |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.5) Gecko/20041220 K-Meleon/0.9 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20020827) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.8) Gecko/20050927 Debian/1.7.8-1sarge3 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020208) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020303) |
| http://www.monogol.de |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-us) AppleWebKit/417.9 (KHTML, like Gecko) Safari/417.8 |
| ccubee/4.0 |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020406) |
| Mozilla/5.0 (compatible; Konqueror/3.4; Linux 2.6.12-10-686; X11) KHTML/3.4.3 (like Gecko) (Kubuntu package 4:3.4.3-0ubuntu1) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20021220) |
| Mozilla/5.0 (compatible; Scrubby/2.2; +http://www.scrubtheweb.com/) |
| Mozilla/5.0 (Windows;) NimbleCrawler 1.12 obeys UserAgent NimbleCrawler For problems contact: crawler@healthline.com |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; nl; rv:1.8) Gecko/20051111 Firefox/1.5 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/85.8.5 (KHTML, like Gecko) Safari/85.8.1 |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020912) |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020608) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020902) |
| htdig/3.1.6 (unconfigured@htdig.searchengine.maintainer) |
| Szukacz/1.5 (robot; www.szukacz.pl/html/jak_dziala_robot.html; info@szukacz.pl) |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20021019) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.4.1) Gecko/20051204 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-us) AppleWebKit/312.8 (KHTML, like Gecko) Safari/312.5 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20020511) |
| Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1 |
| Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.7.8) Gecko/20050511 Firefox/1.0.4 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8) Gecko/20051107 Firefox/1.5 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.7.2) Gecko/20040804 Netscape/7.2 |
| IE Plagin12:19:09 02/09/06 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20021007) |
| carleson/1.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.5) Gecko/20060127 Netscape/8.1 |
| HooWWWer/2.1.3 (debugging run) (+http://cosco.hiit.fi/search/hoowwwer/ | mailto:crawler-infohiit.fi) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.1) Gecko/20060124 Firefox/1.5.0.1 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1 |
| Exabot/2.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; sv-SE; rv:1.7.12) Gecko/20050919 Firefox/1.0.7 |
| NaverBot-1.0 (NHN Corp. / +82-31-784-1989 / nhnbot@naver.com) |
| ichiro/2.0 (http://help.goo.ne.jp/door/crawler.html) |
| curl/7.13.1 (i386-portbld-freebsd4.10) libcurl/7.13.1 OpenSSL/0.9.7d zlib/1.1.4 |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; fi; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20020305) |
| Mozilla/5.0 (Windows;) NimbleCrawler 1.13 obeys UserAgent NimbleCrawler For problems contact: crawler@healthline.com |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; tr; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; sv-SE; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1 |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020420) |
| voyager/1.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; pl-PL; rv:1.7.12) Gecko/20050915 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; tr-TR; rv:1.7.12) Gecko/20050919 Firefox/1.0.7 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20021106) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; rv:1.7.3) Gecko/20040913 Firefox/0.10 |
| NextopiaBOT (+http://www.nextopia.com) distributed crawler client beta v1.1 |
| Mozilla/5.0 (compatible; Inktomi) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20020817) |
| Snoopy v1.01 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20021224) |
| csci_b659/0.13 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; hu; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8) Gecko/20051111 Firefox/1.5 |
| sproose/0.1-alpha (sproose crawler; http://www.sproose.com/bot.html; crawler@sproose.com) |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020928) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20020228) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.7.12) Gecko/20050915 Firefox/1.0.7 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020207) |
| MVAClient |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020211) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020728) |
| HTTP/1.0 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020219) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; it; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020927) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020421) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; fr; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-AR; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20021016) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1 |
| SygolBot http://www.sygol.com |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020228) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20021012) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20021104) |
| Shim-Crawler(Mozilla-compatible; http://www.logos.ic.i.u-tokyo.ac.jp/crawler/; crawl@logos.ic.i.u-tokyo.ac.jp) |
| Mozilla/5.0 (compatible; heritrix/1.6.0 +http://innovationblog.com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; ro; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1 |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020905) |
| Wget/1.9+cvs-dev |
| NutchCVS/0.7.1 (Nutch running at UW; http://www.nutch.org/docs/en/bot.html; sycrawl@cs.washington.edu) |
| miniRank/1.6 (Website ranking; www.minirank.com; robot) |
| Java/1.5.0_06 |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020818) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-GB; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/312.8 (KHTML, like Gecko) Safari/312.6 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/417.9 (KHTML, like Gecko) Safari/417.9.2 |
| NextGenSearchBot 1 (for information visit http://www.zoominfo.com/NextGenSearchBot) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020410) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1 |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en) AppleWebKit/417.9 (KHTML, like Gecko) Safari/417.9.2 |
| exactseek.com |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20020125) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020818) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.12) Gecko/20060202 CentOS/1.0.7-1.4.3.centos4 Firefox/1.0.7 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20020609) |
| Syntryx ANT Scout Chassis Pheromone; Mozilla/4.0 compatible crawler |
| Mozilla/5.0 (X11; U; Linux i686; zh-CN; rv:1.7.5) Gecko/20041215 Firefox/1.0 Red Hat/1.0-12.EL4 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.10) Gecko/20050925 Firefox/1.0.4 (Debian package 1.0.4-2sarge5) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20021016) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; nl; rv:1.8.0.2) Gecko/20060308 Firefox/1.5.0.2 |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; es-AR; rv:1.8.0.2) Gecko/20060308 Firefox/1.5.0.2 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; tr; rv:1.8.0.2) Gecko/20060308 Firefox/1.5.0.2 |
| ccubee/3.5 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8.0.2) Gecko/20060308 Firefox/1.5.0.2 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020515) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8.0.2) Gecko/20060308 Firefox/1.5.0.2 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.2) Gecko/20060308 Firefox/1.5.0.2 |
| noyona_0_1 |
| Mozilla/5.0 (compatible; Konqueror/3.5) KHTML/3.5.2 (like Gecko) |
| Accoona-AI-Agent/1.1.1 (crawler at accoona dot com) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020825) |
| NutchCVS/0.7.2 (Nutch; http://lucene.apache.org/nutch/bot.html; nutch-agent@lucene.apache.org) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.8.0.2) Gecko/20060308 Firefox/1.5.0.2 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020427) |
| NPBot/3 (NPBot; http://www.nameprotect.com; npbot@nameprotect.com) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-us) AppleWebKit/418 (KHTML, like Gecko) Safari/417.9.2 |
| EasyDL/3.04 http://keywen.com/Encyclopedia/Bot |
| Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.8b2) Gecko/20050326 Firefox/1.0+ (MOOX M3) |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020812) |
| Mozilla/5.0 (compatible; AnsearchBot/1.0; +http://www.ansearch.com.au/) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.0.2) Gecko/20060308 Firefox/1.5.0.2 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; de-de) AppleWebKit/418 (KHTML, like Gecko) Safari/417.9.2 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20020211) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; hu; rv:1.8.0.2) Gecko/20060308 Firefox/1.5.0.2 |
| IlTrovatore/1.2 (IlTrovatore; http://www.iltrovatore.it/bot.html; bot@iltrovatore.it) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20021128) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8.0.3) Gecko/20060426 Firefox/1.5.0.3 |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8.0.3) Gecko/20060426 Firefox/1.5.0.3 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.3) Gecko/20060426 Firefox/1.5.0.3 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.3) Gecko/20060426 (CK-IBM) Firefox/1.5.0.3 |
| PlantyNet_WebRobot_V1.9 dhkang@plantynet.com |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020317) |
| RAMPyBot - www.giveRAMP.com/0.1 (RAMPyBot - www.giveRAMP.com; http://www.giveramp.com/bot.html; support@giveRAMP.com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8.0.3) Gecko/20060426 Firefox/1.5.0.3 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/418 (KHTML, like Gecko) Safari/417.9.2 |
| NutchCVS/0.8-dev (Nutch running at UW; http://www.nutch.org/docs/en/bot.html; sycrawl@cs.washington.edu) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.13) Gecko/20060411 Firefox/1.0.8 SUSE/1.0.8-0.2 |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020806) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; nl; rv:1.8.0.3) Gecko/20060426 Firefox/1.5.0.3 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; pl; rv:1.8.0.3) Gecko/20060426 Firefox/1.5.0.3 |
| Octora Beta - www.octora.com |
| Crawllybot/0.1/0.1 (Crawllybot/0.1; http://www.crawlly.com; crawler@crawlly.com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; pt-BR; rv:1.8.0.3) Gecko/20060426 Firefox/1.5.0.3 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20021007) |
| Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.8.0.3) Gecko/20060426 Firefox/1.5.0.3 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20020721) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20020410) |
| Crawllybot/0.1 (Crawllybot; +http://www.crawlly.com; crawler@crawlly.com) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020723) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20020817) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.8.0.3) Gecko/20060426 Firefox/1.5.0.3 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20021220) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.3) Gecko/20060426 Firefox/1.5.0.3 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.0.3) Gecko/20060426 Firefox/1.5.0.3 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/418 (KHTML, like Gecko) Safari/417.9.3 |
| Mozilla/5.0 (compatible; AnsearchBot/1.0; +http://www.ansearch.com/) |
| Skywalker/0.1 (Skywalker; anonymous; anonymous) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020710) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.0.2) Gecko/20060308 Firefox/1.5.0.2 |
| Wget/1.10.1 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020420) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.10) Gecko/20060505 Firefox/1.0.4 (Debian package 1.0.4-2sarge7) |
| curl/7.13.2 (i386-pc-linux-gnu) libcurl/7.13.2 OpenSSL/0.9.7e zlib/1.2.2 libidn/0.5.13 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; cs; rv:1.8.0.3) Gecko/20060426 Firefox/1.5.0.3 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20021025) |
| WinkBot/0.06 (Wink.com search engine web crawler; http://www.wink.com/Wink:WinkBot; winkbot@wink.com) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020917) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; fr-fr) AppleWebKit/312.8 (KHTML, like Gecko) Safari/312.6 |
| Snapbot/1.0 |
| Mozilla/5.0 (X11; U; Linux i686; it; rv:1.8.0.3) Gecko/20060426 Firefox/1.5.0.3 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020611) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.13) Gecko/20060414 |
| Jakarta Commons-HttpClient/3.0 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20021211) |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020126) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.4) Gecko/20060508 Firefox/1.5.0.4 |
| psycheclone |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN; rv:1.8.0.3) Gecko/20060426 Firefox/1.5.0.3 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8.0.4) Gecko/20060508 Firefox/1.5.0.4 |
| Mozilla/2.0 (compatible; Ask Jeeves/Teoma; +http://about.ask.com/en/docs/about/webmasters.shtml) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20021226) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8.0.3) Gecko/20060427 Camino/1.0.1 |
| Mozilla/5.0 (Windows; U; Windows NT 5.2; cs; rv:1.8.0.4) Gecko/20060508 Firefox/1.5.0.4 |
| Java/1.6.0-beta |
| mozilla/firefox |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.0.4) Gecko/20060508 Firefox/1.5.0.4 |
| Example |
| updated/0.1-alpha (updated crawler; http://www.updated.com; crawler@updated.com) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020225) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; pt-BR; rv:1.8.0.4) Gecko/20060508 Firefox/1.5.0.4 |
| Mozilla/5.0 (compatible; DNS-Digger/1.0; +http://www.dnsdigger.com) |
| Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.8.0.4) Gecko/20060508 Firefox/1.5.0.4 |
| Anonymous/0.0 (Anonymous; http://www.anonymous.com; noreply@anonymous.com) |
| Microsoft Data Access Internet Publishing Provider DAV 1.1 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020216) |
| GT::WWW/1.026 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20020927) |
| Mozilla/5.0 (Windows;) NimbleCrawler 2.0.0 obeys UserAgent NimbleCrawler For problems contact: crawler@healthline.com |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.8.0.4) Gecko/20060508 Firefox/1.5.0.4 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020424) |
| Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.8.0.3) Gecko/20060426 Firefox/1.5.0.3 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20021128) |
| sproose/0.1 (sproose bot; http://www.sproose.com/bot.html; crawler@sproose.com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.8.0.4) Gecko/20060508 Firefox/1.5.0.4 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; ro-RO; rv:1.7.12) Gecko/20050919 Firefox/1.0.7 |
| Gigabot |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.0.4) Gecko/20060508 Firefox/1.5.0.4 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20021202) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20020406) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8.0.4) Gecko/20060508 Firefox/1.5.0.4 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20020311) |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020723) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; pl; rv:1.8.0.4) Gecko/20060508 Firefox/1.5.0.4 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20020807) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020120) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; nb-NO; rv:1.8.0.4) Gecko/20060508 Firefox/1.5.0.4 |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20021110) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20020906) |
| Mozilla/5.0 (Windows NT 5.1; U; en) |
| Links4US-Crawler, (+http://links4us.com/) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020507) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8.0.4) Gecko/20060508 Firefox/1.5.0.4 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; nl; rv:1.8.0.4) Gecko/20060508 Firefox 1.5.3 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20020712) |
| Crawler/1.0+http://elibron.com |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; it-it) AppleWebKit/418.8 (KHTML, like Gecko) Safari/419.3 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/418.8 (KHTML, like Gecko) Safari/419.3 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20020701) |
| West Wind Internet Protocols 4.35 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.12) Gecko/20060208 Firefox/1.0.7 |
| IP2MapBot/2.0 http://www.ip2map.com |
| NG-Search/0.90 (NG-SearchBot; http://www.ng-search.com; ) |
| LocalcomBot/1.2.2 (+http://www.local.com/bot.htm) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20021125) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020628) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; tr; rv:1.8.0.4) Gecko/20060508 Firefox/1.5.0.4 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.13) Gecko/20060418 Fedora/1.0.8-1.1.fc4 Firefox/1.0.8 |
| Microsoft Internet Explorer IE/6.0 Windows |
| Mozilla/5.0 (Windows; U; Win98; en-GB; rv:1.8.0.4) Gecko/20060508 Firefox/1.5.0.4 |
| Mozilla/5.0 (compatible; robtexbot/1.0; http://www.robtex.com/ ) |
| Mozilla/3.0 (compatible; TALWinHttpClient) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020823) |
| Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.8.0.4) Gecko/20060705 Firefox/1.5.0.4 |
| Lynx/2.8.5rel.1 libwww-FM/2.14 SSL-MM/1.4.1 OpenSSL/0.9.8a |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-AR; rv:1.8.0.4) Gecko/20060508 Firefox/1.5.0.4 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20020602) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20020213) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.4) Gecko/20060614 Fedora/1.5.0.4-1.2.fc5 Firefox/1.5.0.4 pango-text |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.5) Gecko/20060719 Firefox/1.5.0.5 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.5) Gecko/20060719 Firefox/1.5.0.5 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20020926) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020613) |
| 192.comAgent |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.0.5) Gecko/20060719 Firefox/1.5.0.5 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; nl; rv:1.8.0.5) Gecko/20060719 Firefox/1.5.0.5 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.8.0.6) Gecko/20060728 Firefox/1.5.0.6 |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; es-ES; rv:1.8.0.5) Gecko/20060719 Firefox/1.5.0.5 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20020422) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; nl; rv:1.8.0.6) Gecko/20060728 Firefox/1.5.0.6 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.6) Gecko/20060728 Firefox/1.5.0.6 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20021118) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.0.6) Gecko/20060728 Firefox/1.5.0.6 |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; es-ES; rv:1.8.0.6) Gecko/20060728 Firefox/1.5.0.6 |
| Mozilla/5.0 (Windows; U; Win 9x 4.90; el; rv:1.8.0.6) Gecko/20060728 Firefox/1.5.0.6 |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020120) |
| asked/Nutch-0.8 (web crawler; http://asked.jp; epicurus at gmail dot com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; tr; rv:1.8.0.6) Gecko/20060728 Firefox/1.5.0.6 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1b1) Gecko/20060710 Firefox/2.0b1 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20020707) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20020104) |
| MJ12bot/v1.0.8 (http://majestic12.co.uk/bot.php?+) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8.0.6) Gecko/20060728 Firefox/1.5.0.6 |
| msnbot-media/1.0 (+http://search.msn.com/msnbot.htm) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20020222) |
| Mozilla/5.0 (Windows;) NimbleCrawler 2.0.1 obeys UserAgent NimbleCrawler For problems contact: crawler@healthline.com |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20021209) |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020921) |
| Exabot/3.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; zh-CN; rv:1.8.0.6) Gecko/20060728 Firefox/1.5.0.6 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; ru; rv:1.8.0.6) Gecko/20060728 Firefox/1.5.0.6 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020905) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20021009) |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X; es) AppleWebKit/418.8 (KHTML, like Gecko) Safari/419.3 |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020320) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.0.6) Gecko/20060728 Firefox/1.5.0.6 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20020217) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20020604) |
| Cazoodle/Nutch-0.9-dev (Cazoodle Nutch Crawler; http://www.cazoodle.com; mqbot@cazoodle.com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; de; rv:1.8.0.6) Gecko/20060728 Firefox/1.5.0.6 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; hu; rv:1.8.0.6) Gecko/20060728 Firefox/1.5.0.6 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020319) |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en) AppleWebKit/417.9 (KHTML, like Gecko) Safari/417.8 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; el; rv:1.8.0.6) Gecko/20060728 Firefox/1.5.0.6 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.8.0.6) Gecko/20060728 Firefox/1.5.0.6 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8.0.6) Gecko/20060728 Firefox/1.5.0.6 |
| ozelot/2.7.3 (Search engine indexer; www.flying-cat.de/ozelot; ozelot@flying-cat.de) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20020906) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20021119) |
| larbin_2.6.3 larbin2.6.3@unspecified.mail |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20021011) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-us) AppleWebKit/418.8 (KHTML, like Gecko) Safari/419.3 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020722) |
| onCHECK-Robot, www.onsearch.de |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; cs; rv:1.8.0.7) Gecko/20060909 Firefox/1.5.0.7 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.10) Gecko/20060410 Firefox/1.0.8 Mandriva/1.0.6-16.5.20060mdk (2006.0) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020309) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.7) Gecko/20060909 Firefox/1.5.0.7 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020219) |
| miniRank/2.0 (miniRank; http://minirank.com/; website ranking engine) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8.0.4) Gecko/20060613 Camino/1.0.2 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20020925) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020312) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.3a) Gecko/20021207 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-AR; rv:1.8.0.7) Gecko/20060909 Firefox/1.5.0.7 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020407) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020313) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020622) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20020206) |
| HooWWWer/2.3.0 (debugging run) (+http://cosco.hiit.fi/search/hoowwwer/ | mailto:crawler-infohiit.fi) |
| ilial/Nutch-0.9-dev |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20021122) |
| www.adressendeutschland.de |
| iVia Page Fetcher (http://ivia.ucr.edu/useragents.shtml) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20021122) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.0.7) Gecko/20060909 Firefox/1.5.0.7 |
| i1searchbot/2.0 (i1search web crawler; http://www.i1search.com; crawler@i1search.com) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8.0.7) Gecko/20060909 Firefox/1.5.0.7 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.3) Gecko/20040913 |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8.0.7) Gecko/20060909 Firefox/1.5.0.7 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20021014) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1) Gecko/20060918 Firefox/2.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; sv-SE; rv:1.8.0.7) Gecko/20060909 Firefox/1.5.0.7 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020424) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; es-AR; rv:1.8.0.7) Gecko/20060909 Firefox/1.5.0.7 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20020115) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; nl; rv:1.8.0.7) Gecko/20060909 Firefox/1.5.0.7 |
| NG-Search/0.9.8 (NG-SearchBot; http://www.ng-search.com) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020121) |
| holmes/3.9 (OnetSzukaj/5.0; +http://szukaj.onet.pl) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN; rv:1.7.6) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; pl; rv:1.8.0.7) Gecko/20060909 Firefox/1.5.0.7 |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20021201) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20021008) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020315) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.5) Gecko/20060912 Netscape/8.1.2 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1) Gecko/20061003 Firefox/2.0 |
| MQBOT/Nutch-0.9-dev (MQBOT Nutch Crawler; http://falcon.cs.uiuc.edu; mqbot@cs.uiuc.edu) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020416) |
| Mozilla/5.0 (X11; U; Linux i686; es-AR; rv:1.8.0.7) Gecko/20060830 Firefox/1.5.0.7 (Debian-1.5.dfsg+1.5.0.7-1) |
| crawler43.ejupiter.com |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20020405) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020712) |
| Python-urllib/2.4 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8.0.7) Gecko/20060909 Firefox/1.5.0.7 |
| NutchEC2Test/Nutch-0.9-dev (Testing Nutch on Amazon EC2.; http://lucene.apache.org/nutch/bot.html; ec2test at lucene.com) |
| silk/1.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.10) Gecko/20050716 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.8.0.7) Gecko/20060909 Firefox/1.5.0.7 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/418.9 (KHTML, like Gecko) Safari/419.3 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20020428) |
| CobWeb/1.0 (auto classifier) |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020511) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8.0.7) Gecko/20060911 Camino/1.0.3 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.5) Gecko/20041215 Firefox/1.0 Red Hat/1.0-12.EL4 |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1) Gecko/20061010 Firefox/2.0 |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020422) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20020125) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.0.7) Gecko/20060909 Firefox/1.5.0.7 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20020721) |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020917) |
| MQBOT/Nutch-0.9-dev (MQBOT Crawler; http://falcon.cs.uiuc.edu; mqbot@cs.uiuc.edu) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1) Gecko/20061010 Firefox/2.0 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20020912) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8.1) Gecko/20061010 Firefox/2.0 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20020604) |
| RootCertificate Robot/Nutch-0.8.1 (RCR is robot for rootCertificate.com; http://www.rootcertificate.com/rcr; info at rootcertificate dot com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.8.1) Gecko/20061010 Firefox/2.0 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.10) Gecko/20050921 Firefox/1.0.7 Mandriva/1.0.6-15mdk (2006.0) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.8.0.8) Gecko/20061025 Firefox/1.5.0.8 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20020115) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.0.8) Gecko/20061025 Firefox/1.5.0.8 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20021024) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.8) Gecko/20061025 Firefox/1.5.0.8 |
| Mozilla/5.0 (compatible; heritrix/1.8.0 +http://www.accelobot.com) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020114) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020301) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20020314) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020628) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20020121) |
| Mozilla/5.0 (X11; U; Linux i686; nb-NO; rv:1.8.1) Gecko/20061023 SUSE/2.0-44.1 Firefox/2.0 |
| Combine/3 http://combine.it.lth.se/ |
| CazoodleBot/Nutch-0.9-dev (CazoodleBot Crawler; http://www.cazoodle.com; mqbot@cazoodle.com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1) Gecko/20061010 Firefox/2.0 |
| Yandex/1.01.001 (compatible; Win16; I) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20020709) |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20021019) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20021203) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20021013) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8.1) Gecko/20061010 Firefox/2.0 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; es-ES; rv:1.8.0.8) Gecko/20061025 Firefox/1.5.0.8 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020822) |
| TurnitinBot/2.1 (http://www.turnitin.com/robot/crawlerinfo.html) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1) Gecko/20061010 Firefox/2.0 VRE Toolbar 1.4 |
| Java/1.5.0_09 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020918) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20020120) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020621) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/418.9.1 (KHTML, like Gecko) Safari/419.3 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020621) |
| Mozilla/5.0 (X11; U; Linux i686; sv-SE; rv:1.8.1) Gecko/20060601 Firefox/2.0 (Ubuntu-edgy) |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.0.8) Gecko/20061025 Firefox/1.5.0.8 |
| Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.8.1) Gecko/20061010 Firefox/2.0 |
| Jigsaw/2.2.5 W3C_CSS_Validator_JFouffa/2.0 |
| Cynthia 1.0 |
| W3C_Validator/1.432.2.22 |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8.0.8) Gecko/20061025 Firefox/1.5.0.8 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020524) |
| SBIder/SBIder-0.8.2-dev (http://www.sitesell.com/sbider.html) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.1) Gecko/20061010 Firefox/2.0 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20020523) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.7) Gecko/20060830 Firefox/1.5.0.7 (Debian-1.5.dfsg+1.5.0.7-2) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020202) |
| Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.8.1) Gecko/20061010 Firefox/2.0 |
| sproose/1.0beta (sproose bot; http://www.sproose.com/bot.html; crawler@sproose.com) |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20021015) |
| curl/7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8a zlib/1.2.3 libidn/0.6.2 |
| Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.8.0.7) Gecko/20060913 Fedora/1.5.0.7-1.fc5 Firefox/1.5.0.7 pango-text |
| PEAR HTTP_Request class ( http://pear.php.net/ ) |
| Pingdom GIGRIB v1.1 (http://www.pingdom.com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN; rv:1.8.0.9) Gecko/20061206 Firefox/1.5.0.9 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1 |
| Lynx/2.8.5rel.5 libwww-FM/2.14 SSL-MM/1.4.1 OpenSSL/0.9.7g-fips |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20021223) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.9) Gecko/20061206 Firefox/1.5.0.9 |
| Mozilla/5.0 (compatible; BecomeBot/3.0; +http://www.become.com/site_owners.html) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1 |
| Mozilla/5.0 (compatible; Herold; +http://www.herold.at) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; bg; rv:1.8.0.9) Gecko/20061206 Firefox/1.5.0.9 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; bg; rv:1.8.0.6) Gecko/20060728 Firefox/1.5.0.6 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20021202) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020705) |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020317) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8.0.9) Gecko/20061206 Firefox/1.5.0.9 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.0.9) Gecko/20061206 Firefox/1.5.0.9 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1 |
| web2express.org/Nutch-0.9-dev (leveled playing field; http://web2express.org/; info at web2express.org) |
| eApolloBot/1.0 (eApollo search engine robot; http://www.eapollo.com; eapollo at global-opto dot com) |
| Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.8.0.9) Gecko/20061206 Firefox/1.5.0.9 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020303) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; nl; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1 |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020219) |
| panscient.com |
| Mozilla/5.0 (compatible; Webbot/0.1; http://www.webbot.ru/bot.html) |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en) AppleWebKit/418.9.1 (KHTML, like Gecko) Safari/419.3 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20020721) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20021024) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1 |
| kitty/Nutch-0.8.1 (kitty; http://lucene.apache.org/nutch/; sz603@uow.edu.au) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20020604) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1 |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020906) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20020608) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.7) Gecko/20060831 Firefox/1.5.0.7 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20020725) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20020203) |
| NetWhatCrawler/0.06-dev (NetWhatCrawler from NetWhat.com; http://www.netwhat.com; support@netwhat.com) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20020606) |
| nutch0.9/Nutch-0.9-dev |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fi; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; fr) AppleWebKit/418.9.1 (KHTML, like Gecko) Safari/419.3 |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8.0.6) Gecko/20060728 Firefox/1.5.0.6 |
| Mozilla/5.0 (compatible; Exabot/3.0; +http://www.exabot.com/go/robot) |
| Mozilla/5.0 (X11; U; Linux i686; pt-BR; rv:1.8.0.7) Gecko/20060830 Firefox/1.5.0.7 (Debian-1.5.dfsg+1.5.0.7-2) |
| flyindex.net 1.0/http://www.flyindex.net |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20020702) |
| LinkWalker/2.0 |
| Nutch0.8/Nutch-0.9-dev |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020826) |
| caizhi.cc/Nutch-0.8.1 (CZ; http://www.caizhi.cc; yangtao@caizhi.cc) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020907) |
| caizhi/infomation/Nutch-0.8.1 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020415) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020113) |
| Mozilla/5.0 (compatible; DNS-Digger/1.1; +http://www.dnsdigger.com) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20020617) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN; rv:1.8.1b1) Gecko/20060817 BonEcho/2.0b1 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20021122) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; pt-BR; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1 |
| Mozilla/5.0 (X11; U; Linux i686; fr; rv:1.8.1.1) Gecko/20060601 Firefox/2.0.0.1 (Ubuntu-edgy) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020427) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20020822) |
| Pingdom GIGRIB (http://www.pingdom.com) |
| ConveraCrawler/0.9e (+http://www.authoritativeweb.com/crawl) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.1) Gecko/20061205 Iceweasel/2.0.0.1 (Debian-2.0.0.1+dfsg-2) |
| Mozilla/5.0 (compatible; DNS-Digger-Explorer/1.0; +http://www.dnsdigger.com) |
| Java/1.4.2_12 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20020226) |
| nutchCVS/Nutch-0.8.1 (nutch; http://lucene.apache.org/nutch/bot.html; nutch-agent@lucene.apache.org) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; nl; rv:1.8.0.9) Gecko/20061206 Firefox/1.5.0.9 |
| Mozilla/5.0 (X11; U; Linux i686; fi-FI; rv:1.8.0.9) Gecko/20070102 Ubuntu/dapper-security Firefox/1.5.0.9 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1 |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1 |
| semanticdiscovery/2.0(http://www.semanticdiscovery.com/robot.html) |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020917) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8.0.9) Gecko/20061206 Firefox/1.5.0.9 |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20021128) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020506) |
| KBeeBot/0.5 Robot |
| complex_network_group/Nutch-0.9-dev (discovering the structure of the world-wide-web; http://cantor.ee.ucla.edu/~networks/crawl; nimakhaj@gmail.com) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20021208) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; sv-SE; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; pl; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; pt-BR; rv:1.8.0.7) Gecko/20060909 Firefox/1.5.0.7 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020220) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020411) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020210) |
| dragonfly(ebingbong#playstarmusic.com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; pl; rv:1.8.0.9) Gecko/20061206 Firefox/1.5.0.9 |
| DataFountains/Dmoz Downloader (http://ivia.ucr.edu/useragents.shtml) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1 VRE Toolbar 1.42 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.8.0.9) Gecko/20061206 Firefox/1.5.0.9 |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-GB; rv:1.8.1.2) Gecko/20070219 Firefox/2.0.0.2 |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020104) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.1.2) Gecko/20070219 Firefox/2.0.0.2 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; nb-NO; rv:1.8.0.10) Gecko/20070216 Firefox/1.5.0.10 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.10) Gecko/20070216 Firefox/1.5.0.10 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020105) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; hu; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.2) Gecko/20070219 Firefox/2.0.0.2 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.2) Gecko/20070219 Firefox/2.0.0.2 |
| Wells Search II |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20020618) |
| Wget/1.10.2 (Red Hat modified) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8.1.2) Gecko/20070219 Firefox/2.0.0.2 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020514) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20021013) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.0.10) Gecko/20070216 Firefox/1.5.0.10 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020311) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.8.1.2) Gecko/20070219 Firefox/2.0.0.2 |
| (null) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20020922) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20020419) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; ru; rv:1.8.0.10) Gecko/20070216 Firefox/1.5.0.10 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20020327) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.0.10) Gecko/20070216 Firefox/1.5.0.10 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; sv-SE; rv:1.8.0.10) Gecko/20070216 Firefox/1.5.0.10 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.8.0.10) Gecko/20070216 Firefox/1.5.0.10 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20020319) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.8.0.10) Gecko/20070216 Firefox/1.5.0.10 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; hu; rv:1.8.0.10) Gecko/20070216 Firefox/1.5.0.10 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20020626) |
| Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.8.1.2) Gecko/20070219 Firefox/2.0.0.2 |
| Mozilla/5.0 (X11; U; Linux i686; de; rv:1.8.1.1) Gecko/20061221 Firefox/2.0.0.1 (Swiftfox) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9a1) Gecko/20061204 GranParadiso/3.0a1 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020903) |
| MJ12bot/v1.2.0 (http://majestic12.co.uk/bot.php?+) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8.0.10) Gecko/20070216 Firefox/1.5.0.10 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; pl; rv:1.8.0.10) Gecko/20070216 Firefox/1.5.0.10 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020703) |
| Blaiz-Bee/2.00.8315 (BE Internet Search Engine http://www.rawgrunt.com) |
| favorstarbot/1.0 (+http://favorstar.com/bot.html) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20021202) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20021218) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8.0.10) Gecko/20070216 Firefox/1.5.0.10 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; tr; rv:1.8.1.2) Gecko/20070219 Firefox/2.0.0.2 |
| DataFountains/DMOZ Feature Vector Corpus Creator (http://ivia.ucr.edu/useragents.shtml) |
| Mirago-Test-Robot (http://www.miragorobot.com) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20020816) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.2) Gecko/20060601 Firefox/2.0.0.2 (Ubuntu-edgy) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.11) Gecko/20070312 Firefox/1.5.0.11 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.3) Gecko/20070309 Firefox/2.0.0.3 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8.1.3) Gecko/20070309 Firefox/2.0.0.3 |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020922) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8.1.3) Gecko/20070309 Firefox/2.0.0.3 |
| Xenu Link Sleuth 1.2i |
| WWW-Mechanize/1.22 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20021002) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20021206) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20070321 Firefox/2.0.0.3 (Swiftfox) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20021214) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20020601) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.3) Gecko/20070309 Firefox/2.0.0.2 Alexa Toolbar |
| boitho.com-dc/0.83 ( http://www.boitho.com/dcbot.html ) |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020712) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20020117) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; tr; rv:1.8.0.11) Gecko/20070312 Firefox/1.5.0.11 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20020120) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; fr) AppleWebKit/418.9 (KHTML, like Gecko) Safari/419.3 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020403) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; it; rv:1.8.1.3) Gecko/20070309 Firefox/2.0.0.3 |
| boitho.com-dc/0.85 ( http://www.boitho.com/dcbot.html ) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.1) VoilaBot BETA 1.2 (http://www.voila.com/) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.8.1.3) Gecko/20070309 Firefox/2.0.0.3 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20021223) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020708) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.3) Gecko/20070309 Firefox/2.0.0.3 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020404) |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020624) |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020627) |
| metatagsdir/0.7 (+http://metatagsdir.com/directory/) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.8.1.3) Gecko/20070309 Firefox/2.0.0.3 |
| VisBot/2.0 (Visvo.com Crawler; http://www.visvo.com/bot.html; bot@visvo.com) |
| LocalcomBot/1.3.0 (+http://www.local.com/bot.htm) |
| curl/7.15.4 (i486-pc-linux-gnu) libcurl/7.15.4 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.3 |
| tellbaby/Nutch-0.9 (www.tellbaby.com) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20021216) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020407) |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020609) |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X; sv-se) AppleWebKit/419 (KHTML, like Gecko) Safari/419.3 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-AR; rv:1.8.1.3) Gecko/20070309 Firefox/2.0.0.3 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; pt-BR; rv:1.8.1.3) Gecko/20070309 Firefox/2.0.0.3 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.8.0.11) Gecko/20070312 Firefox/1.5.0.11 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020812) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20020620) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20020404) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.0.11) Gecko/20070312 Firefox/1.5.0.11 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.0.11) Gecko/20070312 Firefox/1.5.0.11 |
| Twiceler-0.9 http://www.cuill.com/twiceler/robot.html |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.3) Gecko/20070309 Firefox/2.0.0.3 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20060601 Firefox/2.0.0.3 (Ubuntu-edgy) |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20021114) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020710) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.2) Gecko/20070222 SeaMonkey/1.1.1 |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20021017) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20020823) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8.0.11) Gecko/20070312 Firefox/1.5.0.11 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20021207) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020502) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20020925) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.8.0.11) Gecko/20070312 Firefox/1.5.0.11 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8.1.3) Gecko/20070309 Firefox/2.0.0.3 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20020725) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.1.3) Gecko/20070309 Firefox/2.0.0.3 |
| Mozilla/5.0 (compatible; heritrix/1.12.0 +http://www.accelobot.com) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20020723) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020122) |
| Snapbot/1.0 (+http://www.snap.com) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.7) Gecko/20061011 Fedora/1.5.0.7-7.fc6 Firefox/1.5.0.7 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20021216) |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20021125) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020117) |
| Mozilla/5.0 (compatible; http://www.livedir.net) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; nl; rv:1.8.1.3) Gecko/20070309 Firefox/2.0.0.3 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020727) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020719) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1) Gecko/20061023 SUSE/2.0-30 Firefox/2.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.2; pt-BR; rv:1.8.1.3) Gecko/20070309 Firefox/2.0.0.3 |
| HPL/Nutch-0.9 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20021117) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20021116) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.8.1.3) Gecko/20070309 Firefox/2.0.0.3 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20021103) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20021115) |
| Acorn/Nutch-0.9 (Non-Profit Search Engine; acorn.isara.org; acorn at isara dot org) |
| Python-urllib/2.5 |
| iSEEKbot/iSEEKbot-0.9-dev (http://beta.iseek.com/iseekbot.html; bot at iseek dot com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1) Gecko/20061003 Firefox/2.0 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.1) Gecko/20070202 Firefox/2.0.0.1 |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020725) |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020117) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; sv-SE; rv:1.8.1.3) Gecko/20070309 Firefox/2.0.0.3 |
| Nutch/Nutch-0.9 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020822) |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en) AppleWebKit/419 (KHTML, like Gecko) Safari/419.3 |
| curl/7.15.5 (i486-pc-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8c zlib/1.2.3 libidn/0.6.5 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20021213) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; sk; rv:1.8.1.3) Gecko/20070309 Firefox/2.0.0.3 |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20021126) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20020602) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; pl; rv:1.8.1.3) Gecko/20070309 Firefox/2.0.0.3 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020514) |
| User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8.1.3) Gecko/20070309 Firefox/2.0.0.3 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.11) Gecko/20070327 Ubuntu/dapper-security Firefox/1.5.0.11 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20020820) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; cs; rv:1.8.1.2) Gecko/20070219 Firefox/2.0.0.2 |
| Mail.Ru/1.0 |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020103) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; da; rv:1.8.1.3) Gecko/20070309 Firefox/2.0.0.3 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020411) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; ru; rv:1.8.1.3) Gecko/20070309 Firefox/2.0.0.3 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020403) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20020911) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020609) |
| curl/7.10.6 (i386-redhat-linux-gnu) libcurl/7.10.6 OpenSSL/0.9.7a ipv6 zlib/1.1.4 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020616) |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.4) Gecko/20070515 Firefox/2.0.0.4 |
| Java/1.5.0_11 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.12) Gecko/20070508 Firefox/1.5.0.12 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.4) Gecko/20070515 Firefox/2.0.0.4 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; tr; rv:1.8.1.4) Gecko/20070515 Firefox/2.0.0.4 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; el; rv:1.8.1.4) Gecko/20070515 Firefox/2.0.0.4 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020108) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.11) Gecko/20070502 Firefox/1.5.0.11 Flock/0.7.13.1 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20020311) |
| ilial/Nutch-0.9 (Ilial, Inc. is a Los Angeles based Internet startup company. For more information please visit http://www.ilial.com/crawler; http://www.ilial.com/crawler; crawl@ilial.com) |
| Nutch/Nutch-0.8.1 (Nutch; Nutch; Nutch) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; ro-RO; rv:1.7.6) Gecko/20050318 Firefox/1.0.2 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.8.1.4) Gecko/20070515 Firefox/2.0.0.4 |
| Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.8.1.1) Gecko/20061228 Firefox/2.0.0.1 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.0.12) Gecko/20070508 Firefox/1.5.0.12 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.5) Gecko/20060728 CentOS/1.5.0.5-0.el4.1.centos4 Firefox/1.5.0.5 pango-text |
| Mozilla/5.0 (compatible; MileNSbot/0.1beta; http://www.milens.net/robotstxt.html) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.2) Gecko/20060425 Firefox/1.5.0.2 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8.0.12) Gecko/20070508 Firefox/1.5.0.12 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.8.0.12) Gecko/20070508 Firefox/1.5.0.12 |
| ilial/Nutch-0.9 (Ilial, Inc. is a Los Angeles based Internet startup company.; http://www.ilial.com/crawler; crawl@ilial.com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-AR; rv:1.8.1.4) Gecko/20070515 Firefox/2.0.0.4 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.8.1.4) Gecko/20070515 Firefox/2.0.0.4 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20021123) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020522) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.4) Gecko/20070515 Firefox/2.0.0.4 |
| Mozilla/5.0 (X11; U; Linux i686; de; rv:1.8.1.3) Gecko/20070413 Firefox/2.0.0.3 |
| Mediapartners-Google/2.1 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20020828) |
| MQBOT/Nutch-0.9-dev (MQBOT Nutch Crawler; http://vwbot.cs.uiuc.edu; mqbot@cs.uiuc.edu) |
| Bloodhound/Nutch-0.9 (Testing Crawler for Research - obeys robots.txt and robots meta tags ; http://balihoo.com/index.aspx; robot at balihoo dot com) |
| Mozilla/5.0 (Twiceler-0.9 http://www.cuill.com/twiceler/robot.html) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20020824) |
| Mozilla/5.0 (compatible; Ask Jeeves/Teoma; +http://about.ask.com/en/docs/about/webmasters.shtml) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.0.12) Gecko/20070508 Firefox/1.5.0.12 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20021014) |
| Mozilla/5.0 (X11; U; Linux i686; es-ES; rv:1.8.1.4) Gecko/20070508 Iceweasel/2.0.0.4 (Debian-2.0.0.4-0etch1) |
| Mozilla/5.0 (compatible; heritrix/1.12.1 +http://newtestbabes.com) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.8.1.4) Gecko/20070515 Firefox/2.0.0.4 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.4) Gecko/20070515 Firefox/2.0.0.4 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20020217) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20021007) |
| WebAlta Crawler/1.3.33 (http://www.webalta.net/ru/about_webmaster.html) (Windows; U; Windows NT 5.1; ru-RU) |
| WDG_Validator/1.6.4 |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-GB; rv:1.8.1.4) Gecko/20070515 Firefox/2.0.0.4 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.1.4) Gecko/20070515 Firefox/2.0.0.4 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.8.1.4) Gecko/20070515 Firefox/2.0.0.4 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; nl; rv:1.8.1.4) Gecko/20070515 Firefox/2.0.0.4 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8.1.4) Gecko/20070515 Firefox/2.0.0.4 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20020328) |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en) AppleWebKit/522.11 (KHTML, like Gecko) Version/3.0.2 Safari/522.12 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20020504) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; ca; rv:1.8.1.4) Gecko/20070515 Firefox/2.0.0.4 |
| Mozilla/5.0 (compatible; Konqueror/3.5; Linux) KHTML/3.5.5 (like Gecko) (Debian) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; +http://brandnewday.com) Gecko/20070508 Firefox/1.5.0.12 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; +http://process1.com) Gecko/20070508 Firefox/1.5.0.12 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20020222) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8.1.4) Gecko/20070515 Firefox/2.0.0.4 |
| Gaisbot/3.0+(robot06@gais.cs.ccu.edu.tw;+http://gais.cs.ccu.edu.tw/robot.php) |
| Mozilla/5.0 (X11; U; Linux i686; fr; rv:1.8.1.4) Gecko/20070515 Firefox/2.0.0.4 |
| WebAlta Crawler/1.3.34 (http://www.webalta.net/ru/about_webmaster.html) (Windows; U; Windows NT 5.1; ru-RU) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.7) Gecko/20060802 Mandriva/1.5.0.7-1mdv2007.0 (2007.0) Firefox/1.5.0.7 |
| Snoopy v0.94 |
| RedBot/redbot-1.0 (Rediff.com Crawler; redbot at rediff dot com) |
| IE7.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; pl; rv:1.8.1.4) Gecko/20070515 Firefox/2.0.0.4 |
| Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.8.1.4) Gecko/20070515 Firefox/2.0.0.4 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; pt-BR; rv:1.8.1.4) Gecko/20070515 Firefox/2.0.0.4 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20020110) |
| Sirketcebot/v.01 (http://www.sirketce.com/bot.html) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; ru; rv:1.8.1.4) Gecko/20070515 Firefox/2.0.0.4 |
| bot/1.0 (bot; http://; bot@bot.bot) |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020405) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; pl; rv:1.8.1) Gecko/20061010 Firefox/2.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; el; rv:1.8.1.4) Gecko/20070515 Firefox/2.0.0.4 VRE Toolbar 1.42 |
| nokia6610I/1.0 (4.10) Profile/MIDP-1.0 Configuration/CLDC-1.0 (FAST WAP Proxy/1.0) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020516) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.4) Gecko/20070515 Firefox/2.0.0.4;MEGAUPLOAD 1.0 |
| GurujiBot/1.0 (+http://www.guruji.com/en/WebmasterFAQ.html) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1 |
| tags2dir.com/0.8 (+http://tags2dir.com/directory/) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20020724) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.5) Gecko/20070713 Firefox/2.0.0.5 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.5) Gecko/20070713 Firefox/2.0.0.5 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020512) |
| NextGenSearchBot 1 (for information visit http://about.zoominfo.com/About/NextGenSearchBot.aspx) |
| Mozilla/5.0 (compatible; Konqueror/3.5; Linux) KHTML/3.5.7 (like Gecko) |
| Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.8.1.5) Gecko/20070713 Firefox/2.0.0.5 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/522.11 (KHTML, like Gecko) Version/3.0.2 Safari/522.12 |
| Yeti/0.01 (nhn/1noon, yetibot@naver.com, check robots.txt daily and follow it) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.5) Gecko/20070713 Firefox/2.0.0.5 Creative ZENcast v2.00.07 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.5) Gecko/20061201 Firefox/2.0.0.5 (Ubuntu-feisty) |
| Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.7.10) Gecko/20050724 Firefox/1.0.6 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020403) |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020901) |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020108) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; fr; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20021020) |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20021124) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6 |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020824) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6 |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020120) |
| Mozilla/5.0 (X11; U; Linux i686; es-ES; rv:1.8.0.10) Gecko/20070226 Fedora/1.5.0.10-1.fc6 Firefox/1.5.0.10 pango-text |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.5pre) Gecko/20070712 Firefox/2.0.0.4 Navigator/9.0b2 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020918) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020702) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020416) |
| TestCrawler/Nutch-0.9 (Testing Crawler for Research ; http://chitchit.org/TestCrawler.html; amitjain at spro dot net) |
| Nokia6680/1.0 ((4.04.07) SymbianOS/8.0 Series60/2.6 Profile/MIDP-2.0 Configuration/CLDC-1.1 (for mobile crawler) ) |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20021010) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; es-ES; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6 |
| Mozilla/5.0 (compatible; heritrix/1.12.1 +http://www.page-store.com) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20020428) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020305) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6 |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020309) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; lt; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-AR; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6 |
| multicrawler (+http://sw.deri.org/2006/04/multicrawler/robots.html) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20020521) |
| LWP::Simple/5.805 |
| Pluggd/Nutch-0.9 (Pluggd automated crawler; http://www.pluggd.com; support at pluggd dot com) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020721) |
| Scrubby/3.0 (+http://www.scrubtheweb.com/help/technology.html) |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020327) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6 |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X; ja-jp) AppleWebKit/419.3 (KHTML, like Gecko) Safari/419.3 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; cs; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; hu; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; tr; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6 |
| Mozilla/5.0 (compatible; heritrix/1.12.1 +http://www.page-store.com [email:paul@page-store.com]) |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20021217) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020816) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/419.3 (KHTML, like Gecko) Safari/419.3 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020515) |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020203) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; es-ES; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6;MEGAUPLOAD 1.0 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6 |
| yacybot (i386 Linux 2.6.9-023stab044.4-smp; java 1.6.0_02; Europe/en) http://yacy.net/yacy/bot.html |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.8.1.7) Gecko/20070914 Firefox/2.0.0.7 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.8.1.7) Gecko/20070914 Firefox/2.0.0.4;MEGAUPLOAD 1.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8) Gecko/20051111 Firefox/1.5 |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X; it-it) AppleWebKit/522.11.1 (KHTML, like Gecko) Version/3.0.3 Safari/522.12.1 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.7) Gecko/20070914 Firefox/2.0.0.7 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20021008) |
| Mozilla/5.0 (X11; U; Linux i686; pl; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.8.1.7) Gecko/20070914 Firefox/2.0.0.7 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20021107) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20020110) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8.1.7) Gecko/20070914 Firefox/2.0.0.7 |
| Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.8.1.7) Gecko/20070914 Firefox/2.0.0.7 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20021225) |
| nrsbot/5.0(loopimprovements.com/robot.html) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8.1.7) Gecko/20070914 Firefox/2.0.0.7 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.7) Gecko/20070914 Firefox/2.0.0.7 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; fr; rv:1.8.1.7) Gecko/20070914 Firefox/2.0.0.7 |
| Nokia6682/2.0 (3.01.1) SymbianOS/8.0 Series60/2.6 Profile/MIDP-2.0 configuration/CLDC-1.1 UP.Link/6.3.0.0.0 (compatible;YahooSeeker/M1A1-R2D2; http://help.yahoo.com/help/us/ysearch/crawling/crawling-01.html) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20021117) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8.1.7) Gecko/20070914 Firefox/2.0.0.7 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; ru; rv:1.8.0.12) Gecko/20070508 Firefox/1.5.0.12 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.4) Gecko/20070509 SeaMonkey/1.1.2 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en) AppleWebKit/522.15.5 (KHTML, like Gecko) Version/3.0.3 Safari/522.15.5 |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en) AppleWebKit/522.11.1 (KHTML, like Gecko) Version/3.0.3 Safari/522.12.1 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.7) Gecko/20070914 (CK-Orange) Firefox/2.0.0.7 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20020515) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; sv-SE; rv:1.8.1.7) Gecko/20070914 Firefox/2.0.0.7 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.8.1.7) Gecko/20070914 Firefox/2.0.0.7 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.12; ips-agent) Gecko/20050922 Fedora/1.0.7-1.1.fc4 Firefox/1.0.7 |
| owsBot/0.2 (owsBot; www.oneworldstreet.com; owsBot) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20021107) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020817) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; tr; rv:1.8.1.7) Gecko/20070914 Firefox/2.0.0.7 |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; de; rv:1.8.1.7) Gecko/20070914 Firefox/2.0.0.7 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.6) Gecko/20050223 Firefox/1.0.1 |
| Blazer |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; pl; rv:1.8.1.7) Gecko/20070914 Firefox/2.0.0.7 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.6) Gecko/20061201 Firefox/2.0.0.6 (Ubuntu-feisty) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20020608) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20020313) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; hu; rv:1.8.1.7) Gecko/20070914 Firefox/2.0.0.7 |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X; sv-SE; rv:1.8.1.7) Gecko/20070914 Firefox/2.0.0.7 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20020713) |
| PCbot/3.0 |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.7) Gecko/20070914 Firefox/2.0.0.7 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; bg; rv:1.8.1.7) Gecko/20070914 Firefox/2.0.0.7 |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; hu; rv:1.8.1.7) Gecko/20070914 Firefox/2.0.0.7 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/312.8.1 (KHTML, like Gecko) Safari/312.6 |
| TMCrawler |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20021205) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20020302) |
| Mozilla/5.0 (compatible; OsO; http://oso.octopodus.com/abot.html) |
| Balihoo/Nutch-1.0-dev (Crawler for Balihoo.com search engine - obeys robots.txt and robots meta tags ; http://balihoo.com/index.aspx; robot at balihoo dot com) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020926) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.5pre) Gecko/20070604 Firefox/2.0.0.4 Navigator/9.0b1 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.8) Gecko/20071008 Firefox/2.0.0.8 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20020313) |
| Lexxe/Robot |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; tr; rv:1.8.1.8) Gecko/20071008 Firefox/2.0.0.8 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8.1.8) Gecko/20071008 Firefox/2.0.0.8 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020608) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.8) Gecko/20071008 Firefox/2.0.0.8 |
| Robotgenius crawler/Nutch-1.0-dev (http://robotgenius.net; misc at robotgenius dot net) |
| Mozilla/5.0 (X11; U; Linux i686; ro; rv:1.8.1.6) Gecko/20071008 Ubuntu/7.10 (gutsy) Firefox/2.0.0.6 |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8.1.8) Gecko/20071008 Firefox/2.0.0.8 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; lt; rv:1.8.1.8) Gecko/20071008 Firefox/2.0.0.8 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20020707) |
| ccubee/2008 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.8.1.8) Gecko/20071008 Firefox/2.0.0.8 |
| Mozilla/5.0 (Windows;) NimbleCrawler 2.0.2 obeys UserAgent NimbleCrawler For problems contact: crawler@healthline.com |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en) AppleWebKit/419.3 (KHTML, like Gecko) Safari/419.3 |
| Mozilla/5.0 (compatible; MJ12bot/v1.2.1; http://www.majestic12.co.uk/bot.php?+) |
| netseer/Nutch-0.9 (NetSeer, Inc. is a Los Angeles based Internet startup company.; http://www.ilial.com/crawler; crawl@ilial.com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.8.1.8) Gecko/20071008 Firefox/2.0.0.8 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020420) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.8pre) Gecko/20071019 Firefox/2.0.0.8 Navigator/9.0.0.1 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020423) |
| http://www.submission.it link_checker/1.1 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.8.1.9) Gecko/20071025 Firefox/2.0.0.9 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.8.1.9) Gecko/20071025 Firefox/2.0.0.9 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020122) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.9) Gecko/20071025 Firefox/2.0.0.9 |
| Lynx/2.8.4rel.1 libwww-FM/2.14 SSL-MM/1.4.1 OpenSSL/0.9.7a |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-AR; rv:1.8.1.9) Gecko/20071025 Firefox/2.0.0.9 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; nl; rv:1.8.1.9) Gecko/20071025 Firefox/2.0.0.9 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20021015) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20021203) |
| Teemer (NetSeer, Inc. is a Los Angeles based Internet startup company.; http://www.netseer.com/crawler.html; crawler@netseer.com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; cs; rv:1.8.1.9) Gecko/20071025 Firefox/2.0.0.9 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.1.9) Gecko/20071025 Firefox/2.0.0.9 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.8.1.9) Gecko/20071025 Firefox/2.0.0.9 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.9) Gecko/20071025 Firefox/2.0.0.9 |
| Mozilla/5.0 (compatible; LiteFinder/1.0; +http://www.litefinder.net/about.html) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20021114) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.9) Gecko/20071025 Firefox/2.0.0.9 VRE Toolbar 1.42 |
| boitho.com-dc/0.82 ( http://www.boitho.com/dcbot.html ) |
| wikiwix-bot-3.0 |
| dmoz_tool3 |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020903) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en; rv:1.8.1.8) Gecko/20071010 Camino/1.5.2 |
| TheRarestParser (http://therarestwords.com/) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.8.1.9) Gecko/20071025 Firefox/2.0.0.9 |
| Mozilla/5.0 (X11; U; Linux i686; es-ES; rv:1.8.1.6) Gecko/20070730 SUSE/2.0.0.6-25 Firefox/2.0.0.6 |
| Mozilla/5.0 (X11; U; Linux i686; de; rv:1.8.1) Gecko/20061023 SUSE/2.0-30 Firefox/2.0 |
| voyager-hc/1.0 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/522.11.1 (KHTML, like Gecko) Version/3.0.3 Safari/522.12.1 |
| Steeler/3.3 (http://www.tkl.iis.u-tokyo.ac.jp/~crawler/) |
| sait/Nutch-0.9 (SAIT Research; http://www.samsung.com) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.8) Gecko/20071022 Ubuntu/7.10 (gutsy) Firefox/2.0.0.8 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020217) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20020111) |
| Nutch crawler/Nutch-0.9 (picapage.com; admin@picapage.com) |
| Jakarta Commons-HttpClient/3.1-rc1 |
| nutch/Nutch-0.9 (nutch) |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.9) Gecko/20071025 Firefox/2.0.0.9 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020823) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; fr; rv:1.8.1.4) Gecko/20070515 Firefox/2.0.0.4 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; pt-BR; rv:1.8.1.9) Gecko/20071025 Firefox/2.0.0.9 |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en) AppleWebKit/523.12 (KHTML, like Gecko) Version/3.0.4 Safari/523.12 |
| Mozilla/5.0 (compatible; Yahoo! Slurp/3.0; http://help.yahoo.com/help/us/ysearch/slurp) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; tr; rv:1.8.1.9) Gecko/20071025 Firefox/2.0.0.9 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; pl; rv:1.8.1.9) Gecko/20071025 Firefox/2.0.0.9 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20020307) |
| nutchsearch/Nutch-0.9 (Nutch Search 1.0; herceg_novi at yahoo dot com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.8.1.9) Gecko/20071025 Firefox/2.0.0.4;MEGAUPLOAD 1.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.10) Gecko/20071115 Firefox/2.0.0.10 |
| ninetowns |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-us) AppleWebKit/523.10.3 (KHTML, like Gecko) Version/3.0.4 Safari/523.10 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8.1.9) Gecko/20071025 Firefox/2.0.0.9 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-AR; rv:1.8.1.10) Gecko/20071115 Firefox/2.0.0.10 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.1.8) Gecko/20071008 Firefox/2.0.0.8 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc4; i686 Linux; 20020909) |
| Mozilla/5.0 (compatible; Google Keyword Tool; +http://adwords.google.com/select/KeywordToolExternal) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.10) Gecko/20071115 Firefox/2.0.0.10 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8.1.10) Gecko/20071115 Firefox/2.0.0.10 |
| MSRBOT (http://research.microsoft.com/research/sv/msrbot/ |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.8.1.10) Gecko/20071115 Firefox/2.0.0.10 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11 |
| nutch/Nutch-1.0-dev (nutch) |
| Mozilla/5.0 (X11; U; Linux i686; en; rv:1.8.1.8) Gecko/20061201 Epiphany/2.18 Firefox/2.0.0.8 (Ubuntu-feisty) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; sv-SE; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20021107) |
| Harry |
| Lovel as 1.0 ( +http://www.everatom.com) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; fr; rv:1.8.1.9) Gecko/20071025 Firefox/2.0.0.9 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11 |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11 |
| Java/1.6.0_02 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc5; i686 Linux; 20020628) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20020105) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; hu; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11 |
| Mozilla/5.0 (compatible; Charlotte/1.0b; http://www.searchme.com/support/) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; pl; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11 |
| Java/1.6.0_01 |
| Grub/2.0 (Grub.org crawler; http://www.grub.org/; bot@grub.org) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20020213) |
| Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11 |
| void/1.0 (void) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11 |
| boitho.com-dc/0.86 ( http://www.boitho.com/dcbot.html ) |
| GeonaBot/1.2; http://www.geona.com/ |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020702) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020125) |
| TestAgentName/Nutch-0.8.1 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; tr; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11 |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11 |
| Mozilla/5.0 (compatible; Gigamega.bot/1.0; +http://www.gigamega.net/bot.html) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-AR; rv:1.8.1.8) Gecko/20071008 Dealio Toolbar 3.1 Firefox/2.0.0.8 |
| Mozilla/5.0 (compatible; heritrix/1.12.0 +http://seekda.com) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20020802) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; fr; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/523.12 (KHTML, like Gecko) Version/3.0.4 Safari/523.12 |
| CazoodleBot/CazoodleBot-0.1 (CazoodleBot Crawler; http://www.cazoodle.com/cazoodlebot; cazoodlebot@cazoodle.com) |
| disco/Nutch-1.0-dev (experimental crawler; www.discoveryengine.com; disco-crawl@discoveryengine.com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11;MEGAUPLOAD 1.0 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; de; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11 |
| Mozilla/5.0 (compatible; Mytagsbot/1.0; +http://www.mylinkmaps.com/pages/bot) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.11) Gecko/20071204 Ubuntu/7.10 (gutsy) Firefox/2.0.0.11 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9b2) Gecko/2007121120 Firefox/3.0b2 |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:0.9.4) Gecko/20011128 Netscape6/6.2.1 |
| TheRarestParser/0.2a (http://therarestwords.com/) |
| Mozilla/5.0 (compatible; MojeekBot/2.0; http://www.mojeek.com/bot.html) |
| Mozilla/5.0 (compatible; Bazbot/1.0; +http://www.bazsites.com/ |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20021224) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9b2) Gecko/2007121120 Firefox/3.0b2 |
| User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.8.1.2) Gecko/20060911 SUSE/2.0.0.2-2.13 Firefox/2.0.0.2 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; da; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11 |
| WebAlta Crawler/2.0 (http://www.webalta.net/ru/about_webmaster.html) (Windows; U; Windows NT 5.1; ru-RU) |
| Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11 |
| Mozilla/5.0 (X11; U; Linux i686; pt-BR; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11 |
| New 2008 year YaMotoR lover v 1.0 ( +http://www.everatom.com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.11) Gecko/20071127 Firefox/2.0 MEGAUPLOAD 1.0 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020402) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.6) Gecko/20070601 Firefox/2.0.0.6 |
| Blanco |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020416) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0 x64; en-US; rv:1.9a8pre) Gecko/2007090213 Minefield/3.0a8pre |
| W3C_Validator/1.575 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8.1.9; .NET CLR 2.0.50727) Gecko/20071025 Firefox/2.0.0.9 |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.0.12) Gecko/20070508 Firefox/1.5.0.12 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20021008) |
| Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.8.0.12) Gecko/20070508 Firefox/1.5.0.12 |
| Mozilla/5.0 (compatible; heritrix/2.0.0-RC1 +http://www.aol.com) |
| Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.8.1.2) Gecko/20060911 SUSE/2.0.0.2-2.13 Firefox/2.0.0.2 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc1; i686 Linux; 20020320) |
| Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.8.1.11) Gecko/20080109 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; it; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11 |
| Mozilla/5.0 (compatible; heritrix/1.12.1 +http://www.page-store.com) [email:paul@page-store.com] |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; sl; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; ru; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11 |
| Mozilla/5.0 (X11; U; Linux x86_64; es-AR; rv:1.8.1.11) Gecko/20071204 Ubuntu/7.10 (gutsy) Firefox/2.0.0.11 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.11) Firefox/2.0.0.11 |
| Nokia6680/1.0 ((4.04.07) SymbianOS/8.0 Series60/2.6 Profile/MIDP-2.0 Configuration/CLDC-1.1 (botmobi find.mobi/bot.html find@mtld.mobi) ) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; ru; rv:1.8.1.9) Gecko/20071025 Firefox/2.0.0.9 |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X; fr; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12 |
| Mozilla/5.0 (Windows; U; Windows NT 5.2; pl; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20021102) |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-GB; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-AR; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.6) Gecko/20070730 SUSE/2.0.0.6-25 Firefox/2.0.0.6 |
| WinHTTP Robot/1.0 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.10) Gecko/20071115 Iceweasel/2.0.0.10 (Debian-2.0.0.10-0etch1) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; pt-BR; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.8) Gecko/20061025 Firefox/1.5.0.8 VRE Toolbar 1.4 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.8) Gecko/20071030 Fedora/2.0.0.8-2.fc8 Firefox/2.0.0.8 |
| agodar |
| nnn/ttt (n) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; hu; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12;MEGAUPLOAD 1.0 |
| Mozilla/5.0 (compatible; Konqueror/3.0; i686 Linux; 20020827) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.11;MEGAUPLOAD 1.0 |
| Mozilla/5.0 (compatible; pmoz.info ODP link checker; +http://pmoz.info/doc/botinfo.htm) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.8.1.12; .NET CLR 2.0.50727) Gecko/20080201 Firefox/2.0.0.12 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; de; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12 |
| Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.8.1.12) Gecko/20080207 Ubuntu/7.10 (gutsy) Firefox/2.0.0.12 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; nl; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.12) Gecko/20080201 Firefox/2.0;MEGAUPLOAD 1.0 |
| msnbot/1.1 (+http://search.msn.com/msnbot.htm) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.12) Gecko/20080201 Firefox/2.0 MEGAUPLOAD 1.0 |
| ISC Systems iRc Search 2.1 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20020603) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.11) Gecko/20060601 Firefox/2.0.0.11 (Ubuntu-edgy) |
| Mozilla/0.6 Beta (Windows) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.4) Gecko/20060602 Firefox/1.5.0.4 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; tr; rv:1.8.1.10) Gecko/20071115 Firefox/2.0.0.10 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12 |
| iVia Downloader |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X; es-ES; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.12) Gecko/20080207 Ubuntu/7.10 (gutsy) Firefox/2.0.0.12 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020710) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20021019) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.8.1.12) Gecko/20080304 Firefox/2.0.0.12 Flock/1.1 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.12) Gecko/20080201 Dealio Toolbar 3.1 Firefox/2.0.0.12 |
| Snoopy v1.2.3 |
| ShablastBot 1.0 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; hu; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; fr; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12 |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; it; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12 Creative ZENcast v1.02.12 |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12 |
| localSystem/Nutch-0.9 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9b4) Gecko/2008030714 Firefox/3.0b4 |
| 1 april. International humor day |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12 Creative ZENcast v2.00.13 |
| Sigram/Nutch-1.0-dev (Test agent for Nutch development; http://www.sigram.com/bot.html; bot at sigram dot com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; lt; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_5_2; en-us) AppleWebKit/525.13 (KHTML, like Gecko) Version/3.1 Safari/525.13 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20021204) |
| nrsbot/5.02 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; ar; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11 |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_2; en-us) AppleWebKit/525.13 (KHTML, like Gecko) Version/3.1 Safari/525.13 |
| CCBot/1.0 (+http://www.commoncrawl.org/bot.html) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr-FR; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8.1.13) Gecko/20080311 Firefox/2.0.0.13 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20021011) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; sk-SK; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11 |
| Mozilla/5.0 (compatible; Exabot Test/3.0; +http://www.exabot.com/go/robot) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.8.1.13) Gecko/20080311 Firefox/2.0.0.13 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.13) Gecko/20080311 Firefox/2.0.0.13 |
| Mozilla/5.0 (compatible; heritrix/1.12.1 +http://www.pubsub.com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.8.1.13) Gecko/20080311 Firefox/2.0.0.13 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.1.13) Gecko/20080311 Firefox/2.0.0.13 Creative ZENcast v2.00.13 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.1.13) Gecko/20080311 Firefox/2.0.0.13 |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020314) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.8.1.13) Gecko/20080311 Firefox/2.0.0.11;MEGAUPLOAD 1.0 |
| Yeti/1.0 (+http://help.naver.com/robots/) |
| Mozilla/5.0 (compatible; ScoutJet; +http://www.scoutjet.com/) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.13) Gecko/20080311 Firefox/2.0.0.13 Creative ZENcast v1.02.08 |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.13) Gecko/20080311 Firefox/2.0.0.13 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9b5) Gecko/2008032620 Firefox/3.0b5 |
| nrsbot/5.0(loopip.com/robot.html) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20020916) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.13; .NET CLR 3.0; ffco7) Gecko/20080311 Firefox/2.0.0.13 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; de; rv:1.8.1.13) Gecko/20080311 Firefox/2.0.0.13 |
| JOES_DEV_PHP |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_2; fr-fr) AppleWebKit/525.13 (KHTML, like Gecko) Version/3.1 Safari/525.13 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.7) Gecko/20040531 Firefox/0.8.0+ |
| Rankivabot/3.2 (www.rankiva.com; 3.2; vzmxikn) |
| Java/1.6.0_05 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.13) Gecko/20080311 Firefox/2.0.0.13 |
| Mozilla/5.0 (compatible; Charlotte/1.1; http://www.searchme.com/support/) |
| Mozilla/5.0 (compatible; Konqueror/3.5; Linux) KHTML/3.5.5 (like Gecko) (Exabot-Thumbnails) |
| SeznamBot/2.0 (+http://fulltext.seznam.cz/) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.8.1.9) Gecko/20071104 Firefox/2.0.0.9 Flock/0.9.1.4 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.1) VoilaBot BETA 1.2 (support.voilabot@orange-ftgroup.com) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_4_11; en) AppleWebKit/525.13 (KHTML, like Gecko) Version/3.1 Safari/525.13 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.14) Gecko/20080404 Firefox/2.0.0.7;MEGAUPLOAD 1.0 |
| BrightCrawler (http://www.brightcloud.com/brightcrawler.asp) |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_4_11; en) AppleWebKit/525.13 (KHTML, like Gecko) Version/3.1 Safari/525.13 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.8.1.14) Gecko/20080404 Firefox/2.0.0.14 |
| Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20021021) |
| REAP-crawler Nutch/Nutch-1.0-dev (Reap Project; http://reap.cs.cmu.edu/REAP-crawler/; Reap Project) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.8.1.14) Gecko/20080404 Firefox/2.0.0.14 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.14) Gecko/20080404 Firefox/2.0.0.14 |
| Mozilla/5.0 (compatible; heritrix/2.0.0 +http://www.mariamosolova.com) |
| Mozilla/5.0 (X11; U; Linux i686; fr; rv:1.8.1.9) YooTron 1.2 www.yoolink.fr |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.14) Gecko/20080404 Firefox/2.0.0.14 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.14) Gecko/20080404 Firefox/2.0.0.11;MEGAUPLOAD 1.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:2.0a1pre) Gecko/2008031802 Minefield/4.0a1pre |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20020228) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; hu; rv:1.8.1.14) Gecko/20080404 Firefox/2.0.0.14 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9b5) Gecko/2008041514 Firefox/3.0b5 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.8.1.14) Gecko/20080404 Firefox/2.0.0.14 |
| EmailSearch |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8.1.14) Gecko/20080404 Firefox/2.0.0.14 |
| Python-urllib/2.1 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.7pre) Gecko/20070815 Firefox/2.0.0.6 Navigator/9.0b3 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.1.14) Gecko/20080404 Firefox/2.0.0.11;MEGAUPLOAD 1.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.14) Gecko/20080404 FFMB3 Firefox/2.0.0.14 |
| Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.8.1.11) Gecko/20080109 (Charlotte/0.9t; http://www.searchme.com/support/) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; it; rv:1.8.1.14) Gecko/20080404 Firefox/2.0.0.14 |
| CROBOT |
| Internet Explorer 6.0 |
| Eurobot/Nutch-1.0-dev (1.0) |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20021212) |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_2; es-es) AppleWebKit/525.13 (KHTML, like Gecko) Version/3.1 Safari/525.13 |
| Mozilla/5.0 (compatible; Firefox mutant; +http://ws.daum.net/aboutkr.html) Daumoa/2.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.8.1.14) Gecko/20080404 Firefox/2.0.0.14 |
| TulipChain/6.03 (http://ostermiller.org/tulipchain/) Java/1.6.0_05 (http://java.sun.com/) Windows_XP/5.1 RPT-HTTPClient/0.3-3 |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.14) Gecko/20080404 Firefox/2.0.0.14 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc2; i686 Linux; 20021121) |
| TheRarestParser/0.2a |
| wauuu engine/Wauuu 0.1 (wauuu engine; http://www.wauuu.com; wauuu@wauuu.com) |
| Yobot/1.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.14) Gecko/20080404 Firefox/2.0.0.12;MEGAUPLOAD 1.0 |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc5; i686 Linux; 20020916) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc4; i686 Linux; 20020804) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8.1.14) Gecko/20080404 Firefox/2.0.0.14 |
| Xenu Link Sleuth 1.2j |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; pt-BR; rv:1.8.1.14) Gecko/20080404 Firefox/2.0.0.14 |
| MSR-ISRCCrawler |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; fr; rv:1.8.1.14) Gecko/20080404 Firefox/2.0.0.14 |
| MyBrowser |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc3; i686 Linux; 20020927) |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; it; rv:1.8.1.14) Gecko/20080404 Firefox/2.0.0.14 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; ja; rv:1.8.1.7) Gecko/20070914 Firefox/2.0.0.7/Mozilla/5.0 (MaSagool; http://sagool.jp/) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; pl; rv:1.8.1.14) Gecko/20080404 Firefox/2.0.0.14 |
| exooba/exooba crawler (exooba; exooba) |
| Python-urllib/1.17 |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc6; i686 Linux; 20021203) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9) Gecko/2008051206 Firefox/3.0 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; es-ES; rv:1.8.1.14) Gecko/20080404 Firefox/2.0.0.14 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; fr; rv:1.8.1.11; .NET CLR 2.0.50727) Gecko/20071127 Firefox/2.0.0.11 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.14) Gecko/20080404 Firefox/2.0.0.14;MEGAUPLOAD 1.0 |
| exooba crawler/exooba crawler (crawler for exooba.com; http://www.exooba.com/; info at exooba dot com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; ro; rv:1.8.1.14) Gecko/20080404 Firefox/2.0.0.14 |
| crawl test/Nutch-1.0-dev |
| Mozilla/5.0 (compatible; Konqueror/3.0-rc1; i686 Linux; 20020228) |
| Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20020427) |
| Mozilla/5.0 (X11; U; Linux i686; de-DE; rv:1.2.1) Gecko/20021204 |
| swish-e http://swish-e.org/ |
| Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.8.1.14) Gecko/20080404 Firefox/2.0.0.14 |
| Mozilla/5.0+(Macintosh;+U;+Intel+Mac+OS+X;+en)+AppleWebKit/419.2+(KHTML,+like+Gecko)+Safari/419.3 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.8.1.5) Gecko/20070713 Firefox/2.0.0.5 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; tr; rv:1.8.1.14) Gecko/20080404 Firefox/2.0.0.14 |
| QEAVis agent/Nutch-0.9 (http://nlp.uned.es/qeavis/) |
| FAST ESP Document Retriever/CVS HEAD 2007-10-18 23:53:34 UTC |
| CyberPatrol SiteCat Webbot (http://www.cyberpatrol.com/cyberpatrolcrawler.asp) |
| msnbot-media/1.1 (+http://search.msn.com/msnbot.htm) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; ru; rv:1.8.1.14) Gecko/20080404 Firefox/2.0.0.14 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.8.1.14; ns) Gecko/20080530 Firefox/2.0.0.14 Flock/1.2.1 |
| Jakarta Commons-HttpClient/3.1 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.13) Gecko/20080313 SeaMonkey/1.1.9 |
| Nokia6300/2.0 (05.50) Profile/MIDP-2.0 Configuration/CLDC-1.1 (botmobi find.mobi/bot.html abuse@mtld.mobi) |
| Mozilla/5.0 (Twiceler-0.9 http://www.cuill.com/twiceler/robot.html) One-time, weeklong image crawl |
| Nokia6300/2.0 (05.50) Profile/MIDP-2.0 Configuration/CLDC-1.1 (botmobi http://find.mobi/bot.html abuse@mtld.mobi) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9) Gecko/2008052906 Firefox/3.0 |
| exooba crawler/0.5.1 (crawler for exooba.com; http://www.exooba.com/; info at exooba dot com) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.14) Gecko/20080416 Fedora/2.0.0.14-1.fc8 Firefox/2.0.0.14 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; ru; rv:1.9) Gecko/2008052906 Firefox/3.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.15) Gecko/20080623 Firefox/2.0.0.15 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_4_11; en) AppleWebKit/525.13 (KHTML, like Gecko) Shiira Safari/125 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9) Gecko/2008052906 Firefox/3.0 |
| Mozilla/5.0 (000000000; 0; 00000 000 00 0; 00000; 00000000000) DDDDDDDDDDDDDD DDDDDDDDDDDDDDDD |
| OOZBOT/0.17 (--; http://www.setooz.com/oozbot.html; pvvpr at iiit dot ac dot in) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; tr; rv:1.8.1.15) Gecko/20080623 Firefox/2.0.0.15 |
| Mozilla/5.0 (compatible; http://www.whoisde.de/2.1; +http://www.whoisde.de) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9) Gecko/2008052906 Firefox/3.0 |
| Basalt v090b |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; lt; rv:1.9) Gecko/2008052906 Firefox/3.0 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; fr; rv:1.9) Gecko/2008052906 Firefox/3.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9) Gecko/2008052906 Firefox/3.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.9) Gecko/2008052906 Firefox/3.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; tr; rv:1.9) Gecko/2008052906 Firefox/3.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; nl; rv:1.9) Gecko/2008052906 Firefox/3.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; pl; rv:1.8.1.15) Gecko/20080623 Firefox/2.0.0.15 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.8.1.15) Gecko/20080623 Firefox/2.0.0.15 |
| Mozilla/5.0 (X11; U; Linux i686; de-DE; rv:1.6) Gecko/20040207 Firefox/0.8 |
| Onetime Program |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.15) Gecko/20080623 Firefox/2.0.0.15 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8.1.15) Gecko/20080623 Firefox/2.0.0.15 |
| CentiverseBot - investigator/3.0 (http://www.centiverse-project.net) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.5; en-US; rv:1.9) Gecko/2008061004 Firefox/3.0 |
| Mozilla/5.0 (Windows; U; Win98; sv-SE; rv:1.8.1.16) Gecko/20080702 Firefox/2.0.0.16 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.15) Gecko/20080703 Firefox/2.0.0.15 Flock/1.2.3 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.16) Gecko/20080702 Firefox/2.0.0.16 |
| Mozilla/5.0 (X11; U; Linux i686; en-US) (+http://www.smsarea.net) Gecko/20070515 Firefox/2.0.0.5 |
| TheRarestParser/0.4b (http://rarestblog.com/therarestparser/) |
| Mozilla/5.0 (X11; U; Linux x86_64; de; rv:1.8.1.1) Gecko/20061223 BonEcho/2.0.0. |
| rulinki.ru project |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.1 |
| DotBot/1.0.1 (http://www.dotnetdotcom.org/#info, crawler@dotnetdotcom.org) |
| SBIder/Nutch-1.0-dev (http://www.sitesell.com/sbider.html) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.8.1.16) Gecko/20080702 Firefox/2.0.0.16 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; ro; rv:1.8.1.16) Gecko/20080702 Firefox/2.0.0.16 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.1 |
| Mozilla/5.0 (X11; U; Linux i686; rv:1.7.3) Gecko/20041123 Firefox/0.10.1 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; nl; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.1 |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.0.1) Gecko/2008070206 Firefox/3.0.1 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; ru; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11 WebMoney Advisor |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.1 |
| Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.8.1.4) Gecko/20080721 BonEcho/2.0.0.4 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_4_11; en) AppleWebKit/525.18 (KHTML, like Gecko) Version/3.1.2 Safari/525.22 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.1 |
| Yandex/1.01.001 (compatible; Win16; H) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; tr; rv:1.8.1.16) Gecko/20080702 Firefox/2.0.0.16 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.1 |
| Onetime ShopFinder Program |
| eCatch/3.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; sv-SE; rv:1.8.1.15) Gecko/20080623 Firefox/2.0.0.15 |
| Mozilla/5.0 (compatible; WebDataCentreBot/1.0; +http://WebDataCentre.com/) |
| DNSMine 0.9 (http://www.DNSmine.com) |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.4; en-US; rv:1.9.0.1) Gecko/2008070206 Firefox/3.0.1 |
| trial ll@specify.this |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_4; en-us) AppleWebKit/525.18 (KHTML, like Gecko) Version/3.1.2 Safari/525.20.1 |
| Mozilla/5.0 (compatible; heritrix/1.15.1-200807172326 +http://www.accelobot.com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.9.0.1) Gecko/2008070208 Firefox/2.0.0.11 MEGAUPLOAD 2.0 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; de; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.1 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; fr; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.1 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9.0.1) Gecko/2008070206 Firefox/3.0.1 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; es-ES; rv:1.8.1.16) Gecko/20080702 Firefox/2.0.0.16 |
| DotBot/1.0.1 (http://www.dotnetdotcom.org/, crawler@dotnetdotcom.org) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; de; rv:1.8.1.16) Gecko/20080702 Firefox/2.0.0.16 |
| intumit/Nutch-1.0-dev |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.16) Gecko/20080702 Firefox/2.0.0.16 WebMoney Advisor |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.12) Gecko/20080219 Firefox/2.0.0.12 Navigator/9.0.0.6 |
| Mozilla/5.0 (compatible; heritrix/1.12.1b +http://netarkivet.dk/website/info.html) |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X; es) AppleWebKit/418.9.1 (KHTML, like Gecko) Safari/419.3 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.11) Gecko/20071127 Firefox 2.0.0.11 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/525.13 (KHTML, like Gecko) Chrome/0.2.149.27 Safari/525.13 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.1 (.NET CLR 3.5.30729) |
| Yeti/1.0 (NHN Corp.; http://help.naver.com/robots/) |
| SonyEricssonK750i/R1N Browser/SEMC-Browser/4.2 Profile/MIDP-2.0 Configuration/CLDC-1.1/Nutch-1.0-dev |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.16) Gecko/20080702 Firefox/2.0.0.16 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/525.13 (KHTML, like Gecko) Chrome/0.2.149.27 Safari/525.13 |
| BusinessSeek.biz_bot http://www.BusinessSeek.biz |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-GB; rv:1.8.0.11) Gecko/20070312 Firefox/1.5.0.11 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.1 |
| Yanga WorldSearch Bot v1.1/beta (http://www.yanga.co.uk/) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.1 |
| KnowItAll/Nutch-0.9 (Nutch-UW-Crawler; http://cs.washington.edu/homes/mjc/crawler.html; uwcrawler08@gmail.com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/525.13 (KHTML, like Gecko) Chrome/0.2.149.29 Safari/525.13 |
| LocalBot/1.1 (+http://www.localbot.co.uk/) |
| Mozilla/5.0 (compatible; discobot/1.0; +http://discoveryengine.com/discobot.html) |
| robotgenius/Nutch-1.0-dev |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; pt-BR; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.1 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; tr; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.1 |
| ia_archiver (+http://www.alexa.com/site/help/webmasters; crawler@alexa.com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.16; nse) Gecko/20080826 Firefox/2.0.0.16 Flock/1.2.5 |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X; fr-fr) AppleWebKit/523.10.3 (KHTML, like Gecko) Version/3.0.4 Safari/523.10 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.1 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/525.13 (KHTML, like Gecko) Chrome/0.2.149.30 Safari/525.13 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; sv-SE; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.1 |
| Mozilla/5.0 (compatible; Konqueror/3.5; Linux 2.6.22-grml) KHTML/3.5.9 (like Gecko) (Debian package 4:3.5.9.dfsg.1-4) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.1 Creative ZENcast v2.00.13 |
| Mozilla/5.0 (iPod; U; CPU iPhone OS 2_0 like Mac OS X; en-us) AppleWebKit/525.18.1 (KHTML, like Gecko) Version/3.1.1 Mobile/5A347 Safari/525.20 |
| Java/1.6.0_04 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3 |
| Mozilla/5.0 (Twiceler-0.9 http://www.cuil.com/twiceler/robot.html) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; cs; rv:1.9.0.2) Gecko/2008091620 Firefox/3.0.2 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; pt-BR; rv:1.8.1.17) Gecko/20080829 Firefox/2.0.0.17 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/525.13 (KHTML, like Gecko) Chrome/0.2.149.30 Safari/525.13 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3 |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-GB; rv:1.9.0.3) Gecko/2008092414 Firefox/3.0.3 |
| Mozilla/5.0 (X11; U; Linux i686; en-GB; rv:1.8.1.16) Gecko/20080715 Ubuntu/7.10 (gutsy) Firefox/2.0.0.16 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.1) Gecko/2008072403 Mandriva/3.0.1-1mdv2008.1 (2008.1) Firefox/3.0.1 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; fr; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3 |
| Java/1.5.0_16 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.17) Gecko/20080829 Firefox/2.0.0.17 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.8.1.16) Gecko/20080702 Firefox/2.0.0.16 |
| online link validator (http://www.dead-links.com/) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3 FlyPaper/RC1 |
| kalooga/KaloogaBot (Kalooga; http://www.kalooga.com/info.html?page=crawler; crawler@kalooga.com) |
| DomainCrawler/1.0 (info@domaincrawler.com; http://www.domaincrawler.com/domains/view/notionlogic.com) |
| Mozilla/5.0 (compatible; DotBot/1.1; http://www.dotnetdotcom.org/, crawler@dotnetdotcom.org) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8.1.17) Gecko/20080829 Firefox/2.0.0.17 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.17) Gecko/20080829 Firefox/2.0.0.17 |
| Mozilla/5.0 (compatible; DBLBot/1.0; +http://www.dontbuylists.com/) |
| Mozilla/5.0 (compatible; SuchbaerBot/0.4; +http://bot.suchbaer.de/info.html) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.1 RPT-HTTPClient/0.3-3 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; sk; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3,gzip(gfe) (via translate.google.com) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-GB; rv:1.9) Gecko/2008061004 Firefox/3.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; de; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; pt-PT; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; pl; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3 |
| WinInet Progress Sample |
| Java/1.6.0_07 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; ro; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3 |
| Mozilla/5.0 (X11; U; Linux i686; en-GB; rv:1.9.0.3) Gecko/2008092510 Ubuntu/8.04 (hardy) Firefox/3.0.3 |
| voyager/2.0 (http://www.kosmix.com/html/crawler.html) |
| Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.3) Gecko/2008092510 Ubuntu/8.04 (hardy) Firefox/3.0.3 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/0.3.154.9 Safari/525.19 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; es-ES; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3 |
| Mozilla/5.0 (compatible; MJ12bot/v1.2.3; http://www.majestic12.co.uk/bot.php?+) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.8.1.17) Gecko/20080829 Firefox/2.0.0.17 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; tr; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3 |
| Isidorus/2.0 (Isidorus; http://www.isidorus.com; crawler@isidorus.com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.3) Gecko/2008092417 Firefox/2.0.0.17;MEGAUPLOAD 1.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; hu; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3 |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3 |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_4_11; en) AppleWebKit/525.18 (KHTML, like Gecko) Version/3.1.2 Safari/525.22 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.3) Gecko/2008092417 YFF3 Firefox/3.0.3 |
| Mozilla/5.0 (compatible; SuchbaerBot/0.4.3a; +http://bot.suchbaer.de/info.html) |
| Site-Perf.com performance testing bot |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; lt; rv:1.8.1.18) Gecko/20081029 Firefox/2.0.0.18 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.18) Gecko/20081112 Fedora/2.0.0.18-1.fc8 Firefox/2.0.0.18 |
| Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.8.0.1) Gecko/20060124 Firefox/1.5.0.1 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.4) Gecko/2008102920 Firefox/3.0.4 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; fr; rv:1.9.0.4) Gecko/2008102920 Firefox/3.0.4 |
| Mozilla/5.0 (X11; U; Linux i686;pl-PL; rv:1.7.10) Gecko/20050717 Firefox/1.0.6 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.18) Gecko/20081029 Firefox/2.0.0.18 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.4) Gecko/2008102920 YFF3 Firefox/3.0.4 |
| NetID.com Bot 1.0 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.8.1.18) Gecko/20081029 Firefox/2.0.0.18 |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_5; en-us) AppleWebKit/525.18 (KHTML, like Gecko) Version/3.1.2 Safari/525.20.1 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.4) Gecko/2008102920 Firefox/3.0.4 |
| DefaultCrawlTest/0.6 (Ram Crawl Test; devarajaswami at yahoo dot com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.9.0.4) Gecko/2008102920 Firefox/3.0.4 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9.0.4) Gecko/2008102920 Firefox/3.0.4 |
| Swami1/Swami-1.1 (Swami Vertical Site Fetcher; devarajaswami at yahoo dot com) |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_5; en-us) AppleWebKit/525.26.2 (KHTML, like Gecko) Version/3.2 Safari/525.26.12 |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; fr; rv:1.9.0.4) Gecko/2008102920 Firefox/3.0.4 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; de; rv:1.9.0.4) Gecko/2008102920 Firefox/2.0.0.13 (de) (TL-FF) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fi; rv:1.9.0.4) Gecko/2008102920 Firefox/3.0.4 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.12) Gecko/20080304 Firefox/2.0.0.12 Flock/1.1 |
| Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.4) Gecko/2008111319 Ubuntu/8.10 (intrepid) Firefox/3.0.4 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.4) Gecko/2008102920 Firefox/3.0.4;MEGAUPLOAD 1.0 |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-GB; rv:1.9.0.4) Gecko/2008102920 Firefox/3.0.4 |
| BobCrawl/Nutch-0.9 (Test/Development crawler; http://notavalable.com; notavailable@notavailable.com) |
| Mozilla/5.0 (X11; U; Linux i686; es-AR; rv:1.9.0.4) Gecko/2008111318 Ubuntu/8.10 (intrepid) Firefox/3.0.4 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/0.4.154.29 Safari/525.19 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.4) Gecko/2008111317 Ubuntu/8.04 (hardy) Firefox/3.0.4 |
| CazoodleBot/0.0.2 (http://www.cazoodle.com/contact.php; cbot@cazoodle.com) |
| Mozilla/5.0 (compatible; NetcraftSurveyAgent/1.0; +info@netcraft.com) |
| Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.8.1.18) Gecko/20081203 Firefox/2.0.0.18 |
| Horny Sex Search/Nutch-0.9 (HornySexSearch.com Crawler; http://www.hornysexsearch.com; Contact HornySexSearch.com) |
| Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.4) Gecko/2008111217 Fedora/3.0.4-1.fc10 Firefox/3.0.4 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.2) Gecko/2008092418 CentOS/3.0.2-3.el5.centos Firefox/3.0.2 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/0.4.154.29 Safari/525.19 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.9.0.4) Gecko/2008102920 Firefox/3.0.4 |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; it; rv:1.9.0.4) Gecko/2008102920 Firefox/3.0.4 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.4) Gecko/2008111217 Fedora/3.0.4-1.fc9 Firefox/3.0.4 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.5) Gecko/2008120122 Firefox/3.0.5 |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_6; en-us) AppleWebKit/528.8+ (KHTML, like Gecko) Version/3.2.1 Safari/525.27.1 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.0.5) Gecko/2008120122 Firefox/3.0.4;MEGAUPLOAD 1.0 |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_4_11; en) AppleWebKit/525.27.1 (KHTML, like Gecko) Version/3.2.1 Safari/525.27.1 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; el; rv:1.9.0.5) Gecko/2008120122 Firefox/3.0.5 |
| Dillo/0.8.6-i18n-misc |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.0.5) Gecko/2008120121 Firefox/3.0.5 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.9.0.5) Gecko/2008120122 Firefox/3.0.5 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/1.0.154.36 Safari/525.19 |
| Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.9.0.5) Gecko/2008120122 Firefox/3.0.5 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; pl; rv:1.9.0.5) Gecko/2008120122 Firefox/3.0.5 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9.0.5) Gecko/2008120122 Firefox/3.0.5 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.5) Gecko/2008120122 Firefox/3.0.5 |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_6; en-us) AppleWebKit/525.27.1 (KHTML, like Gecko) Version/3.2.1 Safari/525.27.1 |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; it; rv:1.9.0.5) Gecko/2008120122 Firefox/3.0.5 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.9.0.5) Gecko/2008120122 Firefox/3.0.5 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8.1.20) Gecko/20081217 Firefox/2.0.0.20 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/1.0.154.43 Safari/525.19 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-AR; rv:1.9.0.5) Gecko/2008120122 Firefox/3.0.4;MEGAUPLOAD 1.0 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; nl; rv:1.9.0.5) Gecko/2008120122 Firefox/3.0.5 |
| IBrowse/2.2 (AmigaOS 3.5) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.0.5) Gecko/2008120122 Firefox/3.0.5 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.5) Gecko/2008120122 Firefox/2.0.0.9;MEGAUPLOAD 1.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.0.5) Gecko/2008120122 Firefox/3.0.5 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/1.0.154.43 Safari/525.19 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.9.0.5) Gecko/2008120122 Firefox/3.0.5 |
| FollowSite Bot ( http://www.followsite.com/bot.html ) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; lv; rv:1.9.0.5) Gecko/2008120122 Firefox/3.0.5 |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_5; en-us) AppleWebKit/525.18.1 (KHTML, like Gecko) Version/3.1.2 Safari/525.20.1 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.1.20) Gecko/20081217 Firefox/2.0.0.20 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; hu; rv:1.9.0.5) Gecko/2008120122 Firefox/3.0.5 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; id; rv:1.9.0.5) Gecko/2008120122 Firefox/3.0.5 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; pl; rv:1.9.0.5) Gecko/2008120122 Firefox/3.0.5 Ubiquity/0.1.5 Creative ZENcast v1.02.10 |
| CatchBot/1.0; +http://www.catchbot.com |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; pt-PT; rv:1.9.0.5) Gecko/2008120122 Firefox/3.0.5 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; ja; rv:1.9.0.5) Gecko/2008120122 Firefox/3.0.5 |
| Mozilla/5.0 (compatible; heritrix/1.14.1 +http://www.toofaan.com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12 |
| Mozilla/5.0 (compatible; heritrix/1.14.2 yptrino +http://www.buddybuzz.net/yptrino) |
| curl/7.12.2 (i686-pc-linux-gnu) libcurl/7.12.2 OpenSSL/0.9.6e zlib/1.1.4 |
| Mozilla/5.0 (X11; U; Linux i686; nl; rv:1.9.0.5) Gecko/2008121621 Ubuntu/8.04 (hardy) Firefox/3.0.5 |
| Jambot/0.1.1 (Jambot; http://www.jambot.com/blog; crawler@jambot.com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; tr; rv:1.9) Gecko/2008051206 Firefox/3.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.5) Gecko/2008120122 AppenDict/3.1 Firefox/3.0.5 |
| robotgenius |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.20) Gecko/20081217 Firefox/2.0.0.20 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/1.0.154.46 Safari/525.19 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.9.0.6) Gecko/2009011913 Firefox/3.0.6 |
| Mozilla/5.0 (compatible; X11; U; Linux i686 (x86_64); en-US; +http://gnomit.com/) Gecko/2008092416 Gnomit/1.0 |
| digiSelf crawler/Nutch-1.0-dev (http://agent.digiself.com ; MyEmail) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.6) Gecko/2009011913 Firefox/3.0.6 FirePHP/0.2.1 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.0.6) Gecko/2009011913 Firefox/3.0.6 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/1.0.154.48 Safari/525.19 |
| LucidMedia ClickSense/4.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.6) Gecko/2009011913 Firefox/3.0.6 |
| Shelob (shelob@gmx.net) |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; it; rv:1.9.0.6) Gecko/2009011912 Firefox/3.0.6 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.0.6) Gecko/2009011913 Firefox/3.0.6 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; ru; rv:1.9.0.6) Gecko/2009011913 AdCentriaIM/1.7 Firefox/3.0.6 |
| SapphireWebCrawler/Nutch-1.0-dev (Sapphire Web Crawler using Nutch; http://boston.lti.cs.cmu.edu/crawler/; mhoy@cs.cmu.edu) |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_6; fr-fr) AppleWebKit/525.27.1 (KHTML, like Gecko) Version/3.2.1 Safari/525.27.1 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8) Gecko/20051111 Firefox/3.0.5, Ant.com Toolbar 1.2 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.6) Gecko/2009011913 Firefox/3.0.6 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.5) Gecko/2008120122 Firefox/3.0.5 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.6) Gecko/2009011913 Firefox/3.0.6 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.6) Gecko/2009011913 Firefox/3.0.3 |
| Mozilla/5.0 (compatible; kmky-not-a-bot/0.2; http://www.kilomonkey.com/notabot.txt ) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; it; rv:1.9.0.6) Gecko/2009011913 Firefox/3.0.6 |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; nl; rv:1.9.0.6) Gecko/2009011912 Firefox/3.0.6 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; pl; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.1 |
| Wget/1.11.4 |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.9.0.6) Gecko/2009011913 Firefox/3.0.6 |
| Mozilla/5.0 (X11; U; Linux i686; pl-PL; rv:1.9.0.6) Gecko/2009020911 Ubuntu/8.10 (intrepid) Firefox/3.0.6 |
| Crawler for Sika Solutions (http://www.sika-sol.co.uk/) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; pt-BR; rv:1.9.0.6) Gecko/2009011913 Firefox/3.0.6 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.6) Gecko/2009020911 Ubuntu/8.04 (hardy) Firefox/3.0.6 |
| Mozilla/5.0 (compatible; MJ12bot/v1.2.4; http://www.majestic12.co.uk/bot.php?+) |
| curl/7.19.3 (i586-pc-mingw32msvc) libcurl/7.19.3 zlib/1.2.3 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/1.0.154.48 Safari/525.19 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de-DE) AppleWebKit/525.28 (KHTML, like Gecko) Version/3.2.2 Safari/525.28.1 |
| Nokia6680/1.0 (4.04.07) SymbianOS/8.0 Series60/2.6 Profile/MIDP-2.0 Configuration/CLDC-1.1 (botmobi find.mobi/bot.html find@mtld.mobi) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.0.4) Gecko/2008102920 Firefox/3.0.4 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9.0.6) Gecko/2009011912 Firefox/3.0.6 |
| yacybot (i386 Linux 2.6.24-23-server; java 1.6.0_0; Europe/de) http://yacy.net/bot.html |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.9.0.6) Gecko/2009011913 Firefox/3.0.6 GTB5 |
| YowedoBot/Yowedo 1.0 (Search Engine crawler for yowedo.com; http://yowedo.com/en/partners.html; crawler@yowedo.com) |
| Java/1.6.0_03 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.7) Gecko/2009021910 Firefox/3.0.7 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.6) Gecko/2009011913 Firefox/3.0.6 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.0.7) Gecko/2009021910 Firefox/3.0.7 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.7) Gecko/2009030422 Ubuntu/8.04 (hardy) Firefox/3.0.7 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en; rv:1.9.0.6) Gecko/2009011913 Firefox/3.0.6 |
| Mozilla/5.0 (compatible; WebmasterCoffee/0.7; +http://webmastercoffee.com/about) |
| W3C_Validator/1.606 |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.9.0.7) Gecko/2009021910 Firefox/3.0.7 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.6) Gecko/2009011913 Firefox/3.0.6 (.NET CLR 3.5.30729) RPT-HTTPClient/0.3-3 |
| Xenu Link Sleuth 1.3 |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.0.6) Gecko/2009011912 Firefox/3.0.6 |
| Mozilla/5.0 (compatible; FirstSearchBot/1.0; +http://www.first-search.com/notionlogic.com.htm) |
| Mozilla/5.0 (Windows; U; Windows NT 5.2; ru; rv:1.9.0.7) Gecko/2009021910 MRA 5.3 (build 02563) Firefox/3.0.7 |
| Mozilla/5.0 (compatible; Charlotte/1.0t; http://www.searchme.com/support/) |
| Mozilla/5.0 (compatible; heritrix/1.12.0 +http://www.webcorp.org.uk/) |
| Mozilla/5.0 (compatible; Goooglebot/2.1; +http://www.google.com/bot.html) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.20) Gecko/20081217 Firefox/2.0.0.7;MEGAUPLOAD 1.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.7) Gecko/2009021910 Firefox/3.0.7 GTB5 |
| linkdexbot/Nutch-1.0-dev (http://www.linkdex.com/; crawl at linkdex dot com) |
| SeznamBot/2.0 (+http://fulltext.sblog.cz/robot/) |
| Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/1.0.154.48 Safari/525.19 |
| http://hilfe.acont.de/bot.html ACONTBOT |
| msnbot/2.0b |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-us) AppleWebKit/85 (KHTML, like Gecko) Safari/85 |
| Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.0.7) Gecko/2009021910 Firefox/3.0.7 GTB5 |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; it; rv:1.9.0.7) Gecko/2009021906 Firefox/3.0.7 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; pt-BR; rv:1.9.0.7) Gecko/2009021910 Firefox/3.0.7 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.9.0.7) Gecko/2009021910 Firefox/3.0.7 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.8) Gecko/2009032609 Firefox/3.0.8 |
| SimilarPages/Nutch-1.0-dev (SimilarPages Nutch Crawler; http://www.similarpages.com; info@similarpages.com) |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.0.8) Gecko/2009032608 Firefox/3.0.8 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.1 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b3) Gecko/20090305 Firefox/3.1b3 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9.0.8) Gecko/2009032609 Firefox/3.0.8 |
| Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.9) Gecko Firefox/3.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.8) Gecko/2009032609 Firefox/3.0.8 GTB5 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/1.0.154.53 Safari/525.19 |
| Mozilla/5.0 (X11; U; Linux i686; en-GB; rv:1.7.6) Gecko/20050405 Epiphany/1.6.1 (Ubuntu) (Ubuntu package 1.0.2) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9.0.8) Gecko/2009032609 Firefox/3.0.8 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/1.0.154.53 Safari/525.19 |
| com_viewer larbin2.6.3@unspecified.mail |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.8) Gecko/2009032609 Firefox/3.0.8 (.NET CLR 3.5.30729) FirePHP/0.2.4 |
| PHP/5.2.9 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; nl; rv:1.9.0.8) Gecko/2009032609 Firefox/3.0.8 |
| fr_com_viewer larbin2.6.3@unspecified.mail |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; lv; rv:1.9.0.8) Gecko/2009032609 Firefox/3.0.8 (.NET CLR 3.5.30729) |
| Mozilla/5.0 GurujiBot/1.0 (+http://www.guruji.com/en/WebmasterFAQ.html) |
| Mozilla/5.001 (windows; U; NT4.0; en-us) |
| cutop.com |
| Mozilla/5.0 (compatible; Butterfly/1.0; +http://labs.topsy.com/butterfly.html) Gecko/2009032608 Firefox/3.0.8 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.8) Gecko/2009032609 Firefox/3.0.8 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.9.0.8) Gecko/2009032609 Firefox/3.0.8 (.NET CLR 3.5.30729) |
| RiceComputerArchitecture Crawler/Nutch-1.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.0.8) Gecko/2009032609 Firefox/3.0.8 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; nl; rv:1.9.0.8) Gecko/2009032609 Firefox/3.0.8 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.8) Gecko/2009032711 Ubuntu/8.10 (intrepid) Firefox/3.0.8 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; it; rv:1.9.0.8) Gecko/2009032609 Firefox/3.0.8 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.0.8) Gecko/2009032609 Firefox/3.0.8 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.8) Gecko/2009032609 Firefox/3.0.8 (.NET CLR 3.5.30729) FirePHP/0.2.4 |
| Mediapartners-Google |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.8) Gecko/2009032609 Firefox/3.0.8 FirePHP/0.2.4 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.4) Gecko/2008112016 Firefox/3.0.4 Flock/2.0.2 |
| Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.0.8) Gecko/2009032609 Firefox/3.0.8 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.0.8) Gecko/2009032609 Firefox/3.0.8 (.NET CLR 3.5.30729) |
| FAST ESP Document Retriever/CVS HEAD 2008-01-02 20:19:11 UTC |
| ; |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.20) Gecko/20081217 Firefox/2.0.0.20 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; ru; rv:1.8.1.20) Gecko/20081217 Firefox/2.0.0.20 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.9.0.9) Gecko/2009040821 Firefox/3.0.8, Ant.com Toolbar 1.3 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; id; rv:1.9.0.8) Gecko/2009032609 Firefox/3.0.8 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; pt-BR; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.9) Gecko/2009040821 Firefox/2.0.0.14;MEGAUPLOAD 1.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.9) Gecko/2009040821 Firefox/3.0.9 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.0.9) Gecko/2009040821 Firefox/3.0.9 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.9) Gecko/2009040821 Firefox/3.0.9 |
| GingerCrawler/1.0 (Language Assistant for Dyslexics; www.gingersoftware.com/crawler_agent.htm; support at ginger software dot com) |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.0.10) Gecko/2009042315 Firefox/3.0.10 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; ru; rv:1.9.0.10) Gecko/2009042316 Firefox/3.0.10 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.9.0.4) Gecko/2008102920 Firefox/3.0.4 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.9) Gecko/2009040821 YFF3 Firefox/3.0.9 (.NET CLR 3.5.30729) |
| msnbot/2.0b (+http://search.msn.com/msnbot.htm) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; sv-SE; rv:1.9.0.10) Gecko/2009042316 Firefox/3.0.10 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.10) Gecko/2009042316 Firefox/3.0.10 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.10) Gecko/2009042316 Firefox/3.0.10 (.NET CLR 3.5.30729) FirePHP/0.2.4 |
| larbin-m larbin@magellan.cc |
| ms research robot |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.0.10) Gecko/2009042316 Firefox/3.0.10 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.9.0.10) Gecko/2009042316 Firefox/3.0.10 |
| renlifangbot/1.0 (+http://search.msn.com/msnbot.htm) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9) Gecko |
| copyright sheriff (+http://www.copyrightsheriff.com/) |
| Loserbot/1.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/530.5 (KHTML, like Gecko) Chrome/2.0.172.23 Safari/530.5 |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-GB; rv:1.9.0.8) Gecko/2009032609 Firefox/3.0.8 |
| com_web_viewer larbin2.6.3@unspecified.mail |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/1.0.154.65 Safari/525.19 |
| google_four_web larbin2.6.3@unspecified.mail |
| 08_800w_web1_ak larbin2.6.3@unspecified.mail |
| nutch.us/Nutch-1.0 (www.nutch.us; crawler@nutch.us) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; ru; rv:1.8.1.18) Gecko/20081029 Firefox/2.0.0.18 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.7) Gecko/20071018 BonEcho/2.0.0.7 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.9.0.10) Gecko/2009042316 Firefox/3.0.10 |
| robotgenius (http://robotgenius.net) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.10) Gecko/2009042316 Firefox/3.0.10 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; es-AR; rv:1.9.0.10) Gecko/2009042316 Firefox/3.0.10 (.NET CLR 3.5.30729),gzip(gfe) (via translate.google.com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; sr; rv:1.9.0.10) Gecko/2009042316 Firefox/3.0.10 GTB5 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; es-AR; rv:1.9.0.10) Gecko/2009042316 Firefox/3.0.10 |
| YandexSomething/1.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.0.10) Gecko/2009042316 Firefox/3.0.10 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.10) Gecko/2009042708 Fedora/3.0.10-1.fc10 Firefox/3.0.10 |
| Jakarta Commons-HttpClient/3.0.1 |
| Mozilla/5.0 (Windows; U; Windows NT 6.1; fr; rv:1.9.0.10) Gecko/2009042316 Firefox/3.0.10 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/1.0.154.65 Safari/525.19 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; fr; rv:1.9.0.10) Gecko/2009042316 Firefox/3.0.10 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; id; rv:1.9.0.4) Gecko/2008102920 Firefox/3.0.4 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.10) Gecko/2009042316 Firefox/3.0.10 GTB5 |
| uw_cse_xwc/Nutch-0.9 (University of Washington Computer Science extensible web crawler project |
| weblab/Nutch-0.9 (weblab testing) |
| Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.9.0.10) Gecko/2009042316 Firefox/3.0.10 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; pt-BR; rv:1.9.0.10) Gecko/2009042316 Firefox/3.0.10 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; it; rv:1.9.0.10) Gecko/2009042315 Firefox/3.0.10 |
| Java/1.6.0_10 |
| Mozilla/5.0 (compatible; GurujiBot/1.0; +http://www.guruji.com/en/WebmasterFAQ.html) |
| Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.9.0.10) Gecko/2009042316 Firefox/3.0.10 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.10) Gecko/2009042316 Firefox/3.0.10 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; pl; rv:1.9.0.10) Gecko/2009042316 Firefox/3.0.10 |
| Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.0.10) Gecko/2009042316 Firefox/3.0.10 |
| web_bh larbin2.6.3@unspecified.mail |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; de; rv:1.9.0.10) Gecko/2009042316 Firefox/3.0.10 |
| nutch-solr-integration/Nutch-1.0 |
| Mozilla/5.0 (compatible; heritrix/2.0.2 +http://seekda.com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.0.10) Gecko/2009042316 Firefox/3.0.10 |
| MSIndianWebcrawl |
| User-Agent=Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.1 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.20) Gecko/20081217 Firefox/2.0.0.20 GTB5 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9.0.11) Gecko/2009060215 Firefox/3.0.11 (.NET CLR 3.5.30729) |
| SHC/Nutch-1.0 (SemanticHacker Crawler; http://www.semantichacker.com/crawler-info; abuse@semantichacker.com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.11) Gecko/2009060215 Firefox/3.0.11 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/530.5 (KHTML, like Gecko) Chrome/2.0.172.31 Safari/530.5 |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_7; en-us) AppleWebKit/525.28.3 (KHTML, like Gecko) Version/3.2.3 Safari/525.28.3 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; tr; rv:1.9.0.10) Gecko/2009042316 Firefox/3.0.10 (.NET CLR 1.1.4322) |
| Mozilla/5.0 (X11; U; Linux i686; ru; rv:1.8.1.19) Gecko/20081202 Firefox/2.0.0.4 (Debian-2.0.0.19-0etch1) GTB5 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.11) Gecko/2009060215 Firefox/3.0.11 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.8.1.19) Gecko/20081204 SeaMonkey/1.1.14 |
| Nokia-N73/1.0 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9.0.10) Gecko/2009042316 Firefox/3.0.10 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/530.5 (KHTML, like Gecko) Chrome/2.0.172.31 Safari/530.5 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1) Gecko/20090616 Firefox/3.5 GTB5 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.11) Gecko/2009060215 Firefox/3.0.11 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.0.11) Gecko/2009060215 Firefox/3.0.11 |
| NutchCVS/Nutch-1.0 (http://lucene.apache.org/nutch/bot.html; nutch-agent@lucene.apache.org) |
| SaladSpoon/ShopSalad 1.0 (Search Engine crawler for ShopSalad.com; http://shopsalad.com/en/partners.html; crawler@shopsalad.com) |
| Mozilla/5.0 (compatible; discobot/1.1; +http://discoveryengine.com/discobot.html) |
| deva/deva-1.0 (Deva Fetcher; devarajaswami at yahoo dot com) |
| Java/1.6.0_14 |
| Mozilla/5.0 (Windows; U; Windows NT 6.1; fr; rv:1.9.0.11) Gecko/2009060215 Firefox/3.0.11 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.11) Gecko/2009060215 Firefox/3.0.11 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.9.0.11) Gecko/2009060215 Firefox/3.0.11 |
| Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.11) Gecko/2009060309 Ubuntu/8.04 (hardy) Firefox/3.0.11 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/530.5 (KHTML, like Gecko) Chrome/2.0.172.33 Safari/530.5 |
| Mozilla/5.0 (X11; U; Linux x86_64; uk-UA; rv:1.9.0.11) Gecko/2009061208 Iceweasel/3.0.7 (Debian-3.0.7-1mlrepo1) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.12) Gecko/20070718 Fedora/1.5.0.12-4.fc6 Firefox/1.5.0.12 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.10) Gecko/2009042316 Firefox/3.0.10 |
| .NET Framework Test Client |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.11) Gecko/2009060215 Firefox/3.0.6;MEGAUPLOAD 1.0 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/530.5 (KHTML, like Gecko) Chrome/2.0.172.33 Safari/530.5 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.10) Gecko/2009042316 Firefox/3.0.10 FBSMTWB |
| smart.apnoti.com Robot/v1.34 (http://smart.apnoti.com/index/aboutApnotiWebCrawler) |
| curl |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.0.11) Gecko/2009060215 Firefox/3.0.11 GTB5 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8) Gecko/20060321 Firefox/2.0a1 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8b4) Gecko/20050908 Firefox/1.4 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.11) Gecko/2009060215 Firefox/3.0.4 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_4_11; en) AppleWebKit/525.27.1 (KHTML, like Gecko) Version/3.2.1 Safari/525.27.1 |
| http://nesearch.theboggle.com visited and indexed you! |
| Mozilla/5.0 (Windows; N; Windows NT 5.2; en-US) AppleWebKit/523.15 (KHTML, like Gecko, Safari/419.3) Qt/4.4.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3, Ant.com Toolbar 1.3 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.1pre) Gecko/20090706 Ubuntu/9.04 (jaunty) Shiretoko/3.5.1pre AutoPager/0.5.2.2 (http://www.teesoft.info/) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.1) Gecko/20090624 Firefox/3.5 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.9.1) Gecko/20090624 Firefox/3.5 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.1) Gecko/20090624 Firefox/3.5 (.NET CLR 3.5.30729) FirePHP/0.3 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.8.1.20) Gecko/20081217 Firefox/2.0.0.20 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; pl; rv:1.9.0.11) Gecko/2009060215 Firefox/3.0.11 (.NET CLR 3.5.30729) |
| 50.nu/0.01 ( +http://50.nu/bot.html ) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; cs; rv:1.9.0.11) Gecko/2009060215 Firefox/3.0.11 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; pt-BR; rv:1.9.0.11) Gecko/2009060215 Firefox/3.0.11 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; es-AR; rv:1.9.0.11) Gecko/2009060215 Firefox/3.0.11 Creative ZENcast v2.01.01 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1) Gecko/20090624 Firefox/3.5 |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; it; rv:1.9.1) Gecko/20090624 Firefox/3.5 |
| page_viewer larbin2.6.3@unspecified.mail |
| Wget/1.11.4 (Red Hat modified) |
| Mozilla/5.0 (X11; U; Linux i686; zh-TW; rv:1.8.1.20) Gecko/20081217 Firefox/2.0.0.20 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.1) Gecko/20090715 Firefox/3.5.1 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; pt-BR; rv:1.9.0.11) Gecko/2009060215 Firefox/3.0.11 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; fr; rv:1.9.1.1) Gecko/20090715 Firefox/3.5.1 |
| OOZBOT/0.20 ( http://www.setooz.com/oozbot.html ; agentname at setooz dot_com ) |
| Mozilla/5.0 (compatible; Scarlett/ 1.0; +http://www.ellerdale.com/crawler.html) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.12) Gecko/2009070611 Firefox/3.0.12 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; nl; rv:1.9.0.12) Gecko/2009070611 Firefox/3.0.12 (.NET CLR 3.5.30729) |
| Java/1.5.0_19 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.12) Gecko/2009070611 Firefox/3.0.12 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.8.1.15) Gecko/20080623 Firefox/2.0.0.15 VRE Toolbar 1.42 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9.0.3) Gecko/2008092414 Firefox/3.0.3 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_5_7; en-us) AppleWebKit/530.19.2 (KHTML, like Gecko) Version/4.0.2 Safari/530.19 |
| Mozilla/5.0 (X11; U; Linux i686; es-ES; rv:1.9.0.12) Gecko/2009070811 Ubuntu/9.04 (jaunty) Firefox/3.0.12 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/530.5 (KHTML, like Gecko) Chrome/2.0.172.37 Safari/530.5 |
| gh-index-bot/Nutch-1.0 (GH Web Search.; lucene.apache.org; gh_email at someplace dot com) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.12) Gecko/2009070611 Firefox/3.0.12 GTB5 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; lv; rv:1.9.0.12) Gecko/2009070611 Firefox/3.0.12 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.12) Gecko/2009070611 Firefox/3.0.12 GTB5 |
| Mozilla/5.0 (iPhone; U; CPU like Mac OS X; en) AppleWebKit/420+ (KHTML, like Gecko) Version/3.0 Mobile/1A543a Safari/419.3 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; pl; rv:1.9.0.12) Gecko/2009070611 Firefox/3.0.12 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; ro; rv:1.9.0.12) Gecko/2009070611 Firefox/3.0.12 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; lt; rv:1.9.0.12) Gecko/2009070611 Firefox/3.0.12 |
| li_viewer larbin2.6.3@unspecified.mail |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; zh-CN; rv:1.9.0.10) Gecko/2009042316 Firefox/3.0.10 (.NET CLR 3.5.30729) QQDownload/1.7 |
| SaladSpoon/ShopSalad 1.0 (Search Engine crawler for ShopSalad.com; http://shopsalad.com/en/partners.html; crawler AT shopsalad.com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.0.12) Gecko/2009070611 Firefox/3.0.12 |
| PHPCrawl |
| Mozilla/5.0 (X11; U; Linux i686; en-GB; rv:1.9.0.12) Gecko/2009070812 Ubuntu/8.04 (hardy) Firefox/3.0.12 |
| Sogou+web+robot+(+http://www.sogou.com/docs/help/webmasters.htm#07) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.1) Gecko/20090715 Firefox/3.5.1 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (compatible; heritrix/1.14.3 +http://www.accelobot.com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (X11; U; Linux i686; fr; rv:1.9.0.12) Gecko/2009070811 Ubuntu/9.04 (jaunty) Firefox/3.0.12 |
| GT::WWW/1.030 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9.0.12) Gecko/2009070611 Firefox/3.0.12 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; es-AR; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (compatible; heritrix/1.14.3 +http://www.toofaan.com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; cs; rv:1.9.0.10) Gecko/2009042316 Firefox/3.0.10 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/530.5 (KHTML, like Gecko) Chrome/2.0.172.39 Safari/530.5 |
| Nutch/Nutch-1.0-dev (A Nutch-based crawler.; http://lucene.apache.org/nutch/bot.html; nutch-agent AT lucene.apache.org) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/530.5 (KHTML, like Gecko) Chrome/2.0.172.39 Safari/530.5 |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.9.0.13) Gecko/2009073022 Firefox/3.0.13 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 |
| CrawlerTest/Nutch-1.0-dev |
| Mozilla/5.0 (Windows; U; Windows NT 6.1; hr; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.2; ru; rv:1.9.1.1) Gecko/20090715 Firefox/3.5.1 |
| baypup/1.1 (Baypup; http://www.baypup.com/; jason@baypup.com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.1) Gecko/20090715 Firefox/3.5.1 GTB5 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; ro; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 (.NET CLR 3.5.30729) |
| findfiles.net/0.96 (Robot;test_robot@gmx-topmail.de) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.13) Gecko/2009073022 Firefox/3.0.13 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.13) Gecko/2009073022 Firefox/3.0.13 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.13) Gecko/2009073022 Firefox/3.0.13 (.NET CLR 3.5.30729) |
| smart.apnoti.com Robot/v1.34 (http://smart.apnoti.com/en/aboutApnotiWebCrawler.html) |
| Mozilla/5.0 (compatible; Tagoobot/3.0; +http://www.tagoo.ru) |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; sv-SE; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 FirePHP/0.3 |
| Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.11) Gecko/2009061118 Fedora/3.0.11-1.fc10 Firefox/3.0.11 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 (.NET CLR 3.5.30729) |
| Mozilla/5.0 X11; U; Linux i686; en-US; rv:1.9.0.5 Gecko/20080528 Epiphany/2.22 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (compatible; heritrix/${pom.version} +http://www.adsafemedia.com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.5) Gecko/20060731 Firefox/1.5.0.5 Flock/0.7.4.1 |
| nutch.biz/Nutch-1.0 (nutch.biz; crawler@nutch.biz) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.13) Gecko/2009073022 Firefox/3.0.13 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en; rv:1.9.0.13) Gecko/2009073022 Firefox/3.5.2 (.NET CLR 3.5.30729) SurveyBot/2.3 (DomainTools) |
| Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.202.0 Safari/532.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.10) Gecko/2009042316 Firefox/3.0.13 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 6.1; nl; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 (.NET CLR 3.5.30729) |
| WyoBot 1.0 |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.0.13) Gecko/2009073021 Firefox/3.0.13 |
| http://www.amagit.com/ |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_4_11; en) AppleWebKit/531.9 (KHTML, like Gecko) Version/4.0.3 Safari/531.9 |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; fr; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/530.19.2 (KHTML, like Gecko) Version/4.0.2 Safari/530.19.1 |
| Mozilla/5.0 (compatible; Plukkie/1.1; http://www.botje.com/plukkie.htm) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/530.9 (KHTML, like Gecko) Iron/2.0.178.0 Safari/530.9 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.6; taptu-downloader) Gecko/20070725 Firefox/2.0.0.6 |
| Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.2) Gecko/20090803 Ubuntu/9.04 (jaunty) Shiretoko/3.5.2 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:x.x.x) Gecko/20041107 Firefox/x.x |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; nl; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 GTB5 |
| al_viewer larbin2.6.3@unspecified.mail |
| Mozilla/5.0 (compatible; XLNT/0.9a; +http://www.xlnt24.com/) |
| Mozilla/5.0 (OS/2; U; Warp 4.5; en-US; rv:1.8.1.14) Gecko/20080507 Firefox/2.0.0.14 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; pt-PT; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; ru; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.1 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; pt-BR; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/530.5 (KHTML, like Gecko) Chrome/2.0.172.43 Safari/530.5 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 |
| Mozilla/5.0 X11 U Linux x86_64 en-US rv:1.9.0.11 Gecko/2009060200 SUSE/3.0.11-0.1.1 Firefox/3.0.11 |
| SBIder/SBIder-1.0 (http://www.sitesell.com/sbider.html) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8.1.20) Gecko/20081217 Firefox/2.0.0.20 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 |
| WebAlta Crawler/1.3.23 (http://www.webalta.net/ru/about_webmaster.html) (Windows; U; Windows NT 5.1; ru-RU) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.9.0.14) Gecko/2009082707 Firefox/3.0.14 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (compatible;+ParchBot/1.0;++http://www.parchmenthill.com/search.htm) |
| Huasai/1.0 |
| Mozilla/5.0 (compatible; MJ12bot/v1.2.5; http://www.majestic12.co.uk/bot.php?+) |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; fr; rv:1.9.0.14) Gecko/2009082707 Firefox/3.0.14 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_4_11; en) AppleWebKit/525.18 (KHTML, like Gecko) Version/3.1.1 Safari/525.18 |
| SOFTWING_TEAR_AGENT_1_0 |
| Nokia6682/2.0 (3.01.1) SymbianOS/8.0 Series60/2.6 Profile/MIDP-2.0 configuration/CLDC-1.1 UP.Link/6.3.0.0.0 (compatible;YahooSeeker/M1A1-R2D2; http://help.yahoo.com/help/us/ysearch/crawling/crawling-01.html) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7) Gecko/20040613 Firefox/0.8.0+ |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_1; en-us) AppleWebKit/531.9 (KHTML, like Gecko) Version/4.0.3 Safari/531.9 |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_8; fi-fi) AppleWebKit/531.9 (KHTML, like Gecko) Version/4.0.3 Safari/531.9 |
| Mozilla/5.0 (Windows; U; Windows NT 5.2; de; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; hu; rv:1.9.0.13) Gecko/2009073022 Firefox/3.0.13 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; ru; rv:1.9.1) Gecko/20090624 Firefox/3.5 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.4) Gecko/2008111318 Ubuntu/8.10 (intrepid) Firefox/3.0.6 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; el; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.212.0 Safari/532.0 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.14) Gecko/2009082707 Firefox/3.0.14 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (X11; U; Linux; i686; en-US; rv:1.6) Gecko Epiphany/1.2.5 |
| Mozilla/5.0 (X11; U; Linux i686; pl-PL; rv:1.9.0.2) Gecko/20121223 Ubuntu/9.25 (jaunty) Firefox/3.8 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; ru; rv:1.9.1.3) Gecko/20090824 MRA 5.5 (build 02842) Firefox/3.5.3,gzip(gfe) (via translate.google.com) |
| tudelft-research/Nutch-1.0 |
| sphsearch.org/Nutch-1.0 (sphsearch.org; crawler@sphsearch.org) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 (.NET CLR 3.5.30729) |
| htmlSQL/0.5 |
| temaseek.com/Nutch-1.0 (temaseek.com; crawler@temaseek.com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.13) Gecko/2009073022 Firefox/3.0.13 GTB6 (.NET CLR 3.5.30729) |
| Isara/Isara-1.0 (A non-profit search engine for the benefit of charity.; http://www.isara.org; search@isara.org) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; es-AR; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_8; ca-es) AppleWebKit/531.9 (KHTML, like Gecko) Version/4.0.3 Safari/531.9 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; tr; rv:1.9.0.14) Gecko/2009082707 Firefox/3.0.14 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; fr-FR) AppleWebKit/531.9 (KHTML, like Gecko) Version/4.0.3 Safari/531.9.1 |
| LegalX Deep Analyzer ( http://www.legalx.net ) http://www.legalx.net |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.195.25 Safari/532.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.2; it; rv:1.8.1.20) Gecko/20081217 Firefox/2.0.0.20 |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_4_11; es) AppleWebKit/531.9 (KHTML, like Gecko) Version/4.0.3 Safari/531.9 |
| netEstate NE Crawler |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.14) Gecko/2009082707 Firefox/3.0.14 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.195.27 Safari/532.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.14) Gecko/2009082707 Firefox/3.0.14 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 GTB6 (.NET CLR 3.5.30729) |
| page_test larbin2.6.3@unspecified.mail |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.9.0.14) Gecko/2009082707 Firefox/3.0.14 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 (.NET CLR 3.5.30729) FirePHP/0.3 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 GTB5 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; fr; rv:1.9.0.14) Gecko/2009082707 Firefox/3.0.14 |
| Mozilla/5.0 (Windows; U; Windows NT 6.1; fr; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.8.1.20) Gecko/20081217 Firefox/2.0.0.20 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; cs; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 (.NET CLR 3.5.30729) |
| Mozzila/4.0 |
| Python-urllib/2.6 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.8.1.20) Gecko/20081217 Firefox/2.0.0.20 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; pt-BR; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.14) Gecko/2009082707 Firefox/3.0.14 GTB5 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.4) Gecko/20091016 Firefox/3.5.4 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; fr; rv:1.9.0.15) Gecko/2009101601 Firefox/3.0.15 (.NET CLR 3.5.30729) |
| intrepid/0.1 |
| Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.3) Gecko/20090926 Iceweasel/3.5.3 (Debian-3.5.3-1) |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_8; en-us) AppleWebKit/531.9 (KHTML, like Gecko) Version/4.0.3 Safari/531.9 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; tr; rv:1.9.0.15) Gecko/2009101601 Firefox/3.0.15 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; es-ES; rv:1.8.1.20) Gecko/20081217 Firefox/2.0.0.20 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.1.4) Gecko/20091016 Firefox/3.5.4 GTB5 (.NET CLR 3.5.30729) |
| Allrati/1.1 (+) |
| Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.4) Gecko/20091016 Firefox/3.5.4 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.3; ips-agent) Gecko/20090824 Fedora/1.0.7-1.1.fc4 Firefox/3.5.3 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.0.15) Gecko/2009101601 Firefox/3.0.15 |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1.4) Gecko/20091016 Firefox/3.5.4 GTB5 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.15) Gecko/2009101601 Firefox/3.0.15 GTB5 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.1.4) Gecko/20091016 Firefox/3.5.4 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.4) Gecko/20091028 Ubuntu/9.10 (karmic) Firefox/3.5.4 |
| (Free; Private Label Search Engine; +WWW.LISTINGISFREE.COM) |
| LWP::Simple/5.814 |
| Mozilla/5.0 (X11; U; Linux; C -) AppleWebKit/527+ (KHTML, like Gecko, Safari/419.3) Qt/4.5.1 |
| Mozilla/5.0 (compatible; MJ12bot/v1.3.1; http://www.majestic12.co.uk/bot.php?+) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; tr; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.15) Gecko/2009101601 Firefox/3.0.15 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (compatible; DNS-Digger-Explorer/1.02; +http://www.dnsdigger.com) |
| Mozilla/5.0 (compatible; proximic; +http://www.proximic.com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; ro; rv:1.9.0.10) Gecko/2009042316 Firefox/3.0.10 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; it; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5 GTB6 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en) AppleWebKit (KHTML, like Gecko) |
| LegalX Web Analysis Agent - http://www.legalx.net |
| SheenBot/SheenBot-1.0.0 (Sheen web crawler) |
| Nokia3230 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.195.33 Safari/532.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.15) Gecko/2009101601 Firefox/3.0.15 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9.0.15) Gecko/2009101600 Firefox/3.0.15 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; sv-SE; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1) Gecko/20090624 Firefox/3.5 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5 GTB5 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.7) Gecko/2009021910 Firefox/3.0.7 (.NET CLR 3.5.30729) |
| YLC_Test/1.0 |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5 |
| SheenBot/SheenBot-1.0.1 (Sheen web crawler) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; de; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5 |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5 GTB5 |
| User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5 GTB6 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.0.15) Gecko/2009101601 Firefox/3.0.15 (.NET CLR 3.5.30729) |
| baypup/colbert (Baypup; http://sf.baypup.com/webmasters; jason@baypup.com) |
| Mozilla/5.0 (compatible; Nmap Scripting Engine; http://nmap.org/book/nse.html) |
| Linguee Bot (http://www.linguee.com/bot) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5 |
| baypup/colbert (Baypup; http://www.baypup.com/webmasters; jason@baypup.com) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.15) Gecko/2009101601 Firefox/3.0.15 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/532.5 (KHTML, like Gecko) Chrome/4.0.249.30 Safari/532.5 |
| Mozilla/5.0 (compatible; aiHitBot-DS/1.0; +http://www.aihit.com/) |
| Linguee Bot (http://www.linguee.com/bot; bot@linguee.com) |
| KIT webcrawler/0.2.4 |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.0.15) Gecko/2009101600 Firefox/3.0.15 |
| Ruby |
| Zscho.de Crawler/Nutch-1.0-Zscho.de-semantic_patch (Zscho.de Crawler, collecting for machine learning; http://zscho.de/) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.195.38 Safari/532.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.16) Gecko/2009120208 Firefox/3.0.16 |
| SheenBot/SheenBot-1.0.2 (Sheen web crawler) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.195.33 Safari/532.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.6) Gecko/20091201 Firefox/3.5.6 |
| Mozilla/5.0 (Linux; favicon Internet survey research; bmenrigh@ucsd.edu) |
| Xenu Link Sleuth 1.3.5 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.16) Gecko/2009120208 Firefox/3.0.16 GTB6 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (X11; U; Linux x86_64; en-GB; rv:1.9.1.5) Gecko/20091109 Ubuntu/9.10 (karmic) Firefox/3.5.5 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.16) Gecko/2009120208 Firefox/3.0.16 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 6.1; en-GB; rv:1.9.1.6) Gecko/20091201 Firefox/3.5.6 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1.6) Gecko/20091201 Firefox/3.5.6 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (compatible; Purebot/1.1; +http://www.puritysearch.net/) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; da; rv:1.9.1.6) Gecko/20091201 Firefox/3.5.6 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.195.38 Safari/532.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.5) Gecko/20091107 Firefox/3.5.5 CometBird/3.5.5 |
| Mozilla/5.0 (compatible; HealthAnalysisAgent/1.0; http://www.healthdump.com) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1) Gecko/20090624 Firefox/3.5 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.6) Gecko/20091201 Firefox/3.5.6 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (compatible; LegalAnalysisAgent/2.0; http://www.legalx.net) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; ro; rv:1.9.1.6) Gecko/20091201 Firefox/3.5.6 |
| Mozilla/5.0 (compatible; aiHitBot/1.0; +http://www.aihit.com/) |
| Mozilla/5.0 (compatible; ParchedAnalysisAgent/8.0; http://www.parchednomore.com) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.195.38 Safari/532.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 GTB6 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (X11; U; Linux; i686; en-US; rv:1.6) Gecko Galeon/1.3.14 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.1.6) Gecko/20091201 Firefox/3.5.6 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; tr; rv:1.9.1.6) Gecko/20091201 Firefox/3.5.6 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 6.1; fr; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2b5) Gecko/20091204 Firefox/3.6b5 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.1) Gecko/2008071616 CentOS/3.0.1-1.el5.centos Firefox/3.0.1 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; ro; rv:1.9.0.17) Gecko/2009122116 Firefox/3.0.17 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; pl; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 |
| Java/1.6.0_17 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 GTB6 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; da; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; it; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 |
| Mozilla/5.0 (compatible; EagleContentAnalysis/1.0; http://www.healtheagle.com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.5 (KHTML, like Gecko) Chrome/4.0.249.64 Safari/532.5 |
| Who.is Bot |
| AdultsVisit.us/Nutch-1.0 (www.AdultsVisit.us; crawler@AdultsVisit.us) |
| Java/1.6.0_06 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.17) Gecko/2009122116 Firefox/3.0.17 |
| SheenBot/SheenBot-1.0.3 (Sheen web crawler) |
| Mozilla/5.0 (X11; U; Linux i686; rv:1.8) Gecko/20060109 Firefox/1.5 |
| Mozilla/5.0 (Windows; U; Windows NT 6.1; it; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.6) Gecko/20091201 SUSE/3.5.6-1.1.1 Firefox/3.5.6 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/530.17 (KHTML, like Gecko) Version/4.0 Safari/530.17 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 FBSMTWB |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.5 (KHTML, like Gecko) Chrome/4.0.249.78 Safari/532.5 |
| cmu/Nutch-1.0 (university; www.cmu.edu; cmu@cmu.edu) |
| Mozilla/5.0 (Windows; U; Windows NT 6.1; en-GB; rv:1.9) Gecko/2008052906 Firefox/3.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.6) Gecko/20060728 Firefox/1.6 |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 |
| Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 GTB6 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; pt-BR; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.17) Gecko/2009122116 Firefox/3.0.1;MEGAUPLOAD 1.0 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_2; en-us) AppleWebKit/531.21.8 (KHTML, like Gecko) Version/4.0.4 Safari/531.21.10 |
| Mozilla/5.0 (Windows; U; Windows NT 6.1; en-GB; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; th; rv:1.9.2) Gecko/20100115 Firefox/3.6 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; hu; rv:1.9.2) Gecko/20100115 Firefox/3.6 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/532.5 (KHTML, like Gecko) Chrome/4.0.249.78 Safari/532.5 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; sv-SE; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 (.NET CLR 3.5.30729) |
| lwp-request/2.07 |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_8; en-us) AppleWebKit/531.9 (KHTML, like Gecko) Version/4.0.3 Safari/531.9/Nutch-1.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 GTB6 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.9.2) Gecko/20100115 Firefox/3.6 |
| Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.9.2) Gecko/20100115 Firefox/3.6 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 GTBDFff GTB7.0 |
| CMS Survey |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.7) Gecko/20100106 Ubuntu/9.10 (karmic) Firefox/3.5.7 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; nl; rv:1.9.2) Gecko/20100115 Firefox/3.6 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; cs; rv:1.9.2) Gecko/20100115 Firefox/3.6 GTB6 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.5 (KHTML, like Gecko) Chrome/4.0.249.89 Safari/532.5 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; tr; rv:1.9.2) Gecko/20100115 Firefox/3.6 |
| Mozilla/5.0 (compatible; spbot/2.0; +http://www.seoprofiler.com/bot/ ) |
| AboutUsBot |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.2 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; lt; rv:1.9.2) Gecko/20100115 Firefox/3.6 |
| Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/532.5 (KHTML, like Gecko) Chrome/4.0.249.89 Safari/532.5 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6 GTBDFff GTB7.0 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.1.8) Gecko/20100202 Firefox/3.5.8 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1.8) Gecko/20100202 Firefox/3.5.8 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/533.1 (KHTML, like Gecko) Chrome/5.0.335.1 Safari/533.1 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/532.5 (KHTML, like Gecko) Chrome/4.1.249.1021 Safari/532.5 |
| Test-Fetcher-0.1/Nutch-0.9 (Awesomeness) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.9.1.4) Gecko/20091016 Firefox/3.5.4 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5 GTB6 QQDownload/1.7 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.13) Gecko/2009080717 Firefox/3.0.13 |
| Mozilla/5.0 (iPhone; U; CPU iPhone OS 3_0 like Mac OS X; en) AppleWebKit/528.18 (KHTML, like Gecko) Version/4.0 Mobile/7A341 Safari/528.16 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; nb-NO; rv:1.9.2) Gecko/20100115 Firefox/3.6 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2) Gecko/20100301 Ubuntu/9.10 (karmic) Firefox/3.6 GTB6 |
| Mozilla/5.0 (X11; U; Linux i686; pl; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5 FirePHP/0.4 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.8) Gecko/20100214 Ubuntu/9.10 (karmic) Firefox/3.5.8 |
| Mozilla/5.0 (compatible; spbot/2.0.1; +http://www.seoprofiler.com/bot/ ) |
| Qryos |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.1.8) Gecko/20100202 Firefox/3.5.8 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.9.2) Gecko/20100115 Firefox/3.6 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1.8) Gecko/20100202 Firefox/3.5.8 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 (.NET CLR 3.5.30729) FirePHP/0.4 |
| Mozilla/5.0 (Windows; U; Windows NT 6.1; de; rv:1.9.2) Gecko/20100115 Firefox/3.6 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 6.1; es-AR; rv:1.9.2) Gecko/20100115 Firefox/3.6 |
| Mozilla/5.0 [Internet Explorer] |
| Java/1.6.0_15 |
| NutchCrawler/Nutch-0.9 (ExperimentalCrawler; http://lucene.apache.org/nutch/; info at email dot com) |
| Java/1.6.0_18 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.9.1.8) Gecko/20100202 Firefox/3.5.8 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2) Gecko/20100121 Remi/fc12 Firefox/3.6 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.9.0.4) Gecko/2008102920 Firefox/3.0.4 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6 GTB6 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5 GTB6 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.8) Gecko/20100202 Firefox/3.5.8 |
| Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6 GTBDFff GTB7.0 (.NET CLR 3.5.30729) |
| moeenbot |
| Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/532.5 (KHTML, like Gecko) Chrome/4.1.249.1036 Safari/532.5 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6 GTB6 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.5 (KHTML, like Gecko) Chrome/4.1.249.1042 Safari/532.5 |
| Java/1.6.0_16 |
| SheenBot/SheenBot-1.0.4 (Sheen web crawler) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/532.5 (KHTML, like Gecko) Chrome/4.1.249.1042 Safari/532.5 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.5 (KHTML, like Gecko) Chrome/4.1.249.1036 Safari/532.5 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; sl; rv:1.9.2.2) Gecko/20100316 Firefox/3.6.2 |
| Mozilla/5.0 (compatible; spbot/2.0.2; +http://www.seoprofiler.com/bot/ ) |
| curl/7.18.2 (i486-pc-linux-gnu) libcurl/7.18.2 OpenSSL/0.9.8g zlib/1.2.3.3 libidn/1.10 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; es-ES; rv:1.9.1.8) Gecko/20100202 Firefox/3.5.8 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.1.7) Gecko/20091221 Ant.com Toolbar 2.0 Firefox/3.5.7 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.2) Gecko/20100316 Firefox/3.6.2 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.9.2.2) Gecko/20100316 Firefox/3.6.2 |
| Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.2) Gecko/20100316 Firefox/3.6.2 |
| Mozilla/5.0 (compatible; aiHitBot/1.0-DS; +http://www.aihit.com/) |
| Mozilla/5.0 (X11; U; Linux i686; pl-PL; rv:1.9.0.2) Gecko/20121223 Ubuntu/9.25 (jaunty) Firefox/3.5.8 |
| Mozilla/5.0 (X11; U; Linux i686; it; rv:1.9.1.8) Gecko/20100214 Ubuntu/9.10 (karmic) Firefox/3.5.8 |
| Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 |
| BlackBerry9000/4.6.0.167 Profile/MIDP-2.0 Configuration/CLDC-1.1 VendorID/102 ips-agent |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_3; en-us) AppleWebKit/531.22.7 (KHTML, like Gecko) Version/4.0.5 Safari/531.22.7 |
| Zscho.de/Nutch-0.9-semantic_patch |
| msnbot/2.0b (+http://search.msn.com/msnbot.htm)._ |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 GTB7.0 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-MX; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.3) Gecko/20091020 Ubuntu/9.10 (karmic) Firefox/3.5.3 |
| Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9) Gecko/2008052906 Firefox/3.0 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.2.3) Gecko/20100401 YFF35 Firefox/3.6.3 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 (.NET CLR 1.1.4322) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.5 (KHTML, like Gecko) Chrome/4.1.249.1045 Safari/532.5 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 |
| Mozilla/5.0 (compatible; BuiltWith/0.3; +http://builtwith.com/bot.html) |
| Mozilla/5.0 (X11; U; Linux x86_64; pt-BR; rv:1.9.1.8) Gecko/20100214 Ubuntu/9.10 (karmic) Firefox/3.5.8 |
| Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/532.5 (KHTML, like Gecko) Chrome/4.1.249.1045 Safari/532.5 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 GTB6 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 6.1; fr; rv:1.9.2.2) Gecko/20100324 Namoroka/3.6.2 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1.1) Gecko/20090715 Firefox/3.5.1 (.NET CLR 3.5.30729) |
| GeoHasher/Nutch-1.0 (GeoHasher Web Search Engine; geohasher.gotdns.org; geo_hasher at yahoo * com) |
| WWW-Mechanize/1.60 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.5 (KHTML, like Gecko) Chrome/4.1.249.1059 Safari/532.5 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 |
| Nokia3108/1.0 (03.01) Profile/MIDP-1.0 Configuration/CLDC-1.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.17 Safari/533.4 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 GTB7.0 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; hu; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 GTB7.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 GTB7.0 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (compatible; MJ12bot/v1.3.3; http://www.majestic12.co.uk/bot.php?+) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 ( .NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.9.1.4) Gecko/20091016 Firefox/3.5.4 (.NET CLR 3.5.30729) |
| php-openid/2.1.1 (php/5.2.5.hiphop) curl/7.19.4 |
| Java/1.6.0_20 |
| Mozilla/5.0 (Windows; U; Windows NT 6.1; ro; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.9) Gecko/2009040821 Firefox/3.0.9 FBSMTWB |
| Wget/1.11.4 Red Hat modified |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 (.NET CLR 3.5.30729) |
| Yandex.Server/2009.5 |
| research-scan-bot/Nutch-1.0 |
| Mozilla/5.0 (Windows; U; Windows NT 6.1; fr; rv:1.9.1.9) Gecko/20100315 Firefox/3.5.9 |
| Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/532.5 (KHTML, like Gecko) Chrome/4.1.249.1064 Safari/532.5 |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_8; en-us) AppleWebKit/531.22.7 (KHTML, like Gecko) Version/4.0.5 Safari/531.22.7 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; hu; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 ( .NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 ( ) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; ru; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 |
| Mozilla/5.0 (compatible; discobot/1.1; +http://discoveryengine.com/discobot.html |
| Java/1.6.0_12 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.9) Gecko/20100401 Ubuntu/9.10 (karmic) Firefox/3.5.9 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 GTB7.0 ( .NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.9) Gecko/20100315 Firefox/3.5.9 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.5 (KHTML, like Gecko) Chrome/4.1.249.1064 Safari/532.5 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 |
| url_test larbin2.6.3@unspecified.mail |
| Equilibrium |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; pl; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 FireTorrent/2.0.1 (.NET CLR 3.5.30729) ZarafaCheck/1.1.1.20080624.110 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9.1.9) Gecko/20100315 Firefox/3.5.9 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; cs; rv:1.9.2.2) Gecko/20100316 Firefox/3.6.2 ( .NET CLR 3.5.30729) |
| PenTest.sg/Nutch-1.1 (www.PenTest.sg; crawler@PenTest.sg) |
| Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.55 Safari/533.4 |
| Mozilla/5.0 (compatible; spbot/2.0.3; +http://www.seoprofiler.com/bot/ ) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; it; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; hu; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 ( .NET CLR 3.5.30729) |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.2) Gecko/20060911 SUSE/2.0.0.2-2.13 Firefox/2.0.0.2 |
| Mozilla/5.0 (Windows; U; Windows NT 6.1; ru-RU; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (compatible; spbot/2.0.4; +http://www.seoprofiler.com/bot ) |
| Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6 WebMoney Advisor |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.55 Safari/533.4 |
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.9) Gecko/20100401 Ubuntu/9.10 (karmic) Ant.com Toolbar 2.0.1 Firefox/3.5.9 |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 |
| Mozilla/5.0 (compatible;picmole/1.0 +http://www.picmole.com) |
| Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9.0.5) Gecko/2008120121 (BT-flames) Firefox/3.0.5 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.10) Gecko/2009042316 Firefox/3.0.10 (.NET CLR 3.5.30729) FBSMTWB |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; hu; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; sv-SE; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 |
| IS Alpha/Nutch-1.0 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.70 Safari/533.4 |
| Mozilla/5.0 (Windows; U; Windows NT 6.1; fr; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.19) Gecko/2010031422 Firefox/3.0.19 ( .NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; lt; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 GTB7.0 ( ) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; tr; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.70 Safari/533.4 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3,gzip(gfe) (via translate.google.com) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1.9) Gecko/20100315 Firefox/3.5.9 (.NET CLR 3.5.30729) |
| acquia-crawler (detected bad behaviour? please tell us at it@acquia.com) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.4) Gecko/20100611 Firefox/3.6.4 |
| Mozilla/5.0 (Windows; U; Windows NT 6.1; pt-BR; rv:1.9.2.4) Gecko/20100611 Firefox/3.6.4 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.2) Gecko/20100115 Firefox/3.6 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN; rv:1.9.1) Gecko/20090624 Firefox/3.5 GTB5 |
| Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.86 Safari/533.4 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.9.1.10) Gecko/20100504 Firefox/3.5.10 |
| Mozilla/5.0 (compatible; YandexBot/3.0; MirrorDetector; +http://yandex.com/bots) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; sv-SE; rv:1.9.2.6) Gecko/20100625 Firefox/3.6.6 |
| Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots) |
| Jayde crawler |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.6) Gecko/20100625 Firefox/3.6.6 |
| Wget/1.12 (linux-gnu) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.19) Gecko/2010031422 Firefox/3.0.19 (.NET CLR 3.5.30729) |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_2; en-us) AppleWebKit/531.9 (KHTML, like Gecko) Version/4.0.3 Safari/531.9 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.99 Safari/533.4 |
| Mozilla/5.0 (compatible; Search17Bot/1.1; http://www.search17.com/bot.php) |
| Mozilla/5.0 (Windows; U; Windows NT 6.1; de; rv:1.9.2.6) Gecko/20100625 Firefox/3.6.6 |
| MLBot (www.metadatalabs.com/mlbot) |
| Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.99 Safari/533.4 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.6) Gecko/20100625 Firefox/3.6.6 ( .NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.2.6) Gecko/20100625 Firefox/3.6.6 ( .NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1.10) Gecko/20100504 Firefox/3.5.10 ( .NET CLR 3.5.30729; .NET4.0C) |
| swkong bot/1.0 (+http://www.swkong.com/addurl) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; nl; rv:1.9.2.6) Gecko/20100625 Firefox/3.6.6 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; lt; rv:1.9.2.6) Gecko/20100625 Firefox/3.6.6 ( .NET CLR 3.5.30729) |
| Flight Deck Bot (experimental) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9.0.17) Gecko/2009122116 Firefox/3.0.17 GTB7.1 (.NET CLR 3.5.30729) |
| Mozilla/5.0 ( compatible; SETOOZBOT/0.30 ; http://www.setooz.com/bot.html ) |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; pt-BR; rv:1.9.2.6) Gecko/20100625 Firefox/3.6.6 ( .NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.9.2.6) Gecko/20100625 Firefox/3.6.6 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9.2.6) Gecko/20100625 Firefox/3.6.6 GTB6 (.NET CLR 3.5.30729) |
| nrsbot/6.0(loopip.com/robot.html) |
| Mozilla/5.0 (X11; U; Linux i686; de; rv:1.9.1.10) Gecko/20100622 Fedora/3.5.10-1.fc12 Firefox/3.5.10 |
| Nudelbot 0.8 Beta (+http://www.nudelbot.com/bot.html) |
| SexShop.Sg/Nutch-1.1 (www.SexShop.Sg; crawler@SexShop.Sg) |
| Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.6) Gecko/20100625 Firefox/3.6.6 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.9.2.7) Gecko/20100713 Firefox/3.6.7 ( .NET CLR 3.5.30729; .NET4.0E) |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; tr; rv:1.9.2.7) Gecko/20100713 Firefox/3.6.7 GTB7.0 ( .NET CLR 3.5.30729) |
| Mozilla/5.0 (Windows; U; Windows NT 6.1; cs; rv:1.9.2.7) Gecko/20100713 Firefox/3.6.7 |
| Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.9.2.7) Gecko/20100713 Firefox/3.6.7 |
| Mozilla/5.0 (Windows; U; Windows NT 5.1; tr; rv:1.9.2.8) Gecko/20100722 Firefox/3.6.8 ( .NET CLR 3.5.30729) |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.2.8) Gecko/20100722 Firefox/3.6.8 |
| Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.125 Safari/533.4 |
| Flight Deck Bot 1.3 beta (http://www.flightdeckreports.com/bot) |
| Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_8; en-us) AppleWebKit/533.16 (KHTML, like Gecko) Version/5.0 Safari/533.16 |