We have collection of more than 1 Million open source products ranging from Enterprise product to small libraries in all platforms. We aggregate information from all open source repositories. Search and find the best for your needs. Check out projects section.



Arachnode.net

    •        29561

An open source .NET web crawler written in C# using SQL 2005/2008. Arachnode.net is a complete and comprehensive .NET web crawler for downloading, indexing and storing Internet content including e-mail addresses, files, hyperlinks, images, and Web pages. Its features include

  • .NET architecture
  • Configurable Rules and Actions
  • Lucene.NET Integration
  • SQL Server 2008 and full-text indexing
  • .DOC/.PDF/.PPT/.XLS Indexing
  • HTML to XML and XHTML
  • Multi-threading and Throttling
  • Respectful Crawling
  • Analysis Services
  • SQL Server 2008 and SSIS
  • EXIF data extraction

http://arachnode.net/
Tags
Implementation
License
Platform

   




Related Projects

Open Search Server

  •    C++

Open Search Server is both a modern crawler and search engine and a suite of high-powered full text search algorithms. Built using the best open source technologies like lucene, zkoss, tomcat, poi, tagsoup. Open Search Server is a stable, high-performance piece of software.

Norconex HTTP Collector - Enterprise Web Crawler

  •    Java

Norconex HTTP Collector is a full-featured web crawler (or spider) that can manipulate and store collected data into a repositoriy of your choice (e.g. a search engine). It very flexible, powerful, easy to extend, and portable.

Nutch - Highly extensible, highly scalable Web crawler

  •    Java

Nutch is open source web-search software. It builds on Lucene Java, adding web-specifics, such as a crawler, a link-graph database, parsers for HTML and other document formats, etc.

ASPseek

  •    C++

ASPseek is an Internet search engine software developed by SWsoft.ASPseek consists of an indexing robot, a search daemon, and a CGI search frontend. It can index as many as a few million URLs and search for words and phrases, use wildcards, and do a Boolean search. Search results can be limited to time period given, site or Web space (set of sites) and sorted by relevance (PageRank is used) or date.

Heritrix

  •    Java

Heritrix is the Internet Archive's open-source, extensible, web-scale, archival-quality web crawler project. Heritrix is designed to respect the robots.txt exclusion directives and META robots tags, and collect material at a measured, adaptive pace unlikely to disrupt normal website activity.


mnoGoSearch

  •    C

mnoGoSearch for UNIX consists of a command line indexer and a search program which can be run under Apache Web Server, or any other HTTP server supporting CGI interface. mnoGoSearch for Unix is distributed in sources and can be compiled with a number of databases, depending on user's choice. It is known to work on a wide variety of the modern Unix operating systems including Linux, FreeBSD, Mac OSX, Solaris and others.

Grub

  •    CSharp

Grub Next Generation is distributed web crawling system (clients/servers) which helps to build and maintain index of the Web. It is client-server architecture where client crawls the web and updates the server. The peer-to-peer grubclient software crawls during computer idle time.

Tantivy - Full-text search engine library inspired by Lucene and written in Rust

  •    Rust

Tantivy is a full text search engine library written in rust. It is closer to Lucene than to Elastic Search and Solr in the sense it is not an off-the-shelf search engine server, but rather a crate that can be used to build such a search engine.

Elasticlunr.js - Lightweight full-text search engine in Javascript for browser search and offline search

  •    Javascript

Elasticlunr.js is a lightweight full-text search engine in Javascript for browser search and offline search. Elasticlunr.js is developed based on Lunr.js, but more flexible than lunr.js. Elasticlunr.js provides Query-Time boosting and field search. Elasticlunr.js is a bit like Solr, but much smaller and not as bright, but also provide flexible configuration and query-time boosting.

Manticore Search - High performance full-text search engine with SQL and JSON support

  •    C++

Manticore Search is an open source high performance full-text search oriented engine. It is a fork of Sphinx Search. Manticore Search is written in C++. It means speed and low resource consumption, it means you don’t have to worry about a garbage collector that suddenly makes a trouble.

Lucene - A high-performance, full-featured text search engine library

  •    Java

Apache Lucene is a high-performance, full-featured text search engine library written entirely in Java. It is a technology suitable for nearly any application that requires full-text search, especially cross-platform.

Solr - Blazing-fast, open source enterprise search platform

  •    Java

Solr is the popular, blazing fast open source enterprise search platform from the Apache Lucene project. Its major features include powerful full-text search, hit highlighting, faceted search, dynamic clustering, database integration, and rich document (e.g., Word, PDF) handling. Solr is highly scalable, providing distributed search and index replication, and it powers the search and navigation features of many of the world's largest internet sites.

Strus - Full text Search Engine in C++

  •    C++

The open source project strus provides a collection of C++ (C++98) libraries and command line tools for building a full-text search engine. The strus search engine can be build using any key value store database that provides an upper bound seek function for the stored key/value pairs. Currently there exists an implementation based on the LevelDB library.

MeiliSearch - Ultra relevant, instant and typo-tolerant full-text search API

  •    Rust

MeiliSearch is a powerful, fast, open-source, easy to use, and deploy search engine. The search and indexation are fully customizable and handles features like typo-tolerance, filters, and synonyms. It features include Search as-you-type experience, Full text search, Supports Synonym, Highly customizable, RESTfull API and lot more.

Sphinix - Search server

  •    C++

Sphinix is free open-source SQL full-text search engine. How do you implement full-text search for that 10+ million row table, keep up with the load, and stay relevant? Sphinx is good at those kinds of riddles.

MG4J - Managing Gigabytes for Java

  •    Java

MG4J (Managing Gigabytes for Java) is a free full-text search engine for large document collections written in Java. MG4J is a highly customisable, high-performance, full-fledged search engine providing state-of-the-art features (such as BM25/BM25F scoring) and new research algorithms. The main points of MG4J are Powerful indexing, Multi-index interval semantics, Virtual fields, Clustering and lot more.

ItemsJS - Full text search engine in Javascript

  •    Javascript

Full text, faceted, dependency free search engine in javascript. Created to perform fast search on small json dataset (up to 1000 elements).

Lemur - Search Engine

  •    Java

The Lemur Project develops search engines, browser toolbars, text analysis tools, and data resources that support research and development of information retrieval and text mining software. The project is best known for its Indri search engine, Lemur Toolbar, and ClueWeb09 dataset.

Lux - XML Search engine

  •    Java

Lux is an open source XML search engine using Lucene /Solr and Saxon XQuery/XSLT processor. Lux provides XML-aware indexing, an XQuery 1.0 optimizer that rewrites queries to use the indexes, and a function library for interacting with Lucene via XQuery. These capabilities are tightly integrated with Solr, and leverage its application framework in order to deliver a REST service, application server, and supporting tools.

Xapian - Search Engine Library

  •    C++

Xapian is an Open Source Search Engine Library. It is written in C++, with bindings to allow use from Perl, Python, PHP, Java, Tcl, C# and Ruby. Xapian is a highly adaptable toolkit which allows developers to easily add advanced indexing and search facilities to their own applications. It supports the Probabilistic Information Retrieval model and also supports a rich set of boolean query operators.

Social Icons

Subscribe to our mailing list





Most Viewed Product

JForum
Asgbookphp - PHP Guestbook Script
Phorum - PHP based forum software
Pligg - Social Publishing CMS
Usebb - UseBB forum software in PHP 4 and 5.3.
Crawltrack - Tracks the visits of Crawler
MyBB - professional,efficient discussion board
Microlinkr - Tiny url generator
Webalizer - fast web server log file analysis
Simple Machines Forum - Elegant, Effective and Powerful

Recently Viewed Product

simplewall - Simple tool to configure Windows Filtering Platform (WFP) which can configure network activity on your computer
vue-schedule-calendar - 日程调度日历。
Leaflet.Editable - Make geometries editable in Leaflet.
tokens - Directory of ERC20 token images. Upload yours to get displayed in the Trust Wallet
awesome-network-manager - Network Manager interface for Awesome WM
segyio - Fast Python library for SEGY files.
Croppie - A Javascript Image Cropper
yinyue - MV and MP3 downloader for NetEase cloud music (https://music.163.com)
XcodeTheme - My Xcode theme - Sundell's Colors
react-native-localize - 🌍 A toolbox for your React Native app localization (formerly react-native-languages)

We have large collection of open source products. Follow the tags from Tag Cloud >>


Open source products are scattered around the web. Please provide information about the open source projects you own / you use. Add Projects.






2018 © findbestopensource.com. Made in India. All trademarks and copyrights are held by respective owners. Advertise  |  About us  |  Privacy Policy  |  Terms of Use  |