WebCrawler's Knowledge Blog

Open-Search-Server, experiences? (open source web crawler) — Question

webmining — Sat, 01 Aug 2009 15:14:03 +0000

Was wondering what other’s experiences have been using the open source web crawler, Open Search Server.

http://www.open-search-server.com/

Does anyone know of any documentation that exists that details the GUI and what all the configurations, variables, and query parameters mean?

Also, has anyone used the xml api and had success getting data out of the index?

Web Craweler Design — Article

webmining — Sat, 01 Aug 2009 09:52:30 +0000

Good article on web crawler design.

http://thinkpink.com/bp/WebCrawler/WWW94.html

View Hidden Website Directories — Tip

webmining — Sat, 01 Aug 2009 09:33:07 +0000

Here’s a tip on how- to View Hidden Website Directories

Writing a Web Crawler in Java — Article

webmining — Sat, 01 Aug 2009 09:21:05 +0000

Here’s a great article on how web crawlers work, their benefits, and the different reasons for writing one.

http://java.sun.com/developer/technicalArticles/ThirdParty/WebCrawler/

Introduction to Web Crawlers — Video Lecture Series

webmining — Fri, 31 Jul 2009 23:52:36 +0000

Video Lecture Series
Introduction to Web Crawler & Human Based Search Engines

Web Crawler knowledge hounds unite!

webmining — Fri, 31 Jul 2009 22:45:11 +0000

Hello World!
This site is for anyone that desires cutting edge knowledge on the latest trends, technologies, and research regarding web crawlers, spiders, ants, bots, and worms.

Have fun!!!

Hello world!

webmining — Fri, 31 Jul 2009 22:34:10 +0000

Welcome to WordPress.com. This is my first post.