dhtspider - Bittorrent dht network spider

  •        36

Bittorrent dht network infohash spider, for engiy.com[a bittorrent resource search engine]

https://github.com/alanyang/dhtspider#readme

Dependencies:

bencode : ^0.9.0

Tags
Implementation
License
Platform

   




Related Projects

dht - BitTorrent DHT Protocol && DHT Spider.

  •    Go

See the video on the Youtube.It contains two modes, the standard mode and the crawling mode. The standard mode follows the BEPs, and you can use it as a standard dht server. The crawling mode aims to crawl as more metadata info as possiple. It doesn't follow the standard BEPs protocol. With the crawling mode, you can build another BTDigg.

magnetico - Autonomous (self-hosted) BitTorrent DHT search engine suite.

  •    Python

Autonomous (self-hosted) BitTorrent DHT search engine suite. Both programs, combined together, allows anyone with a decent Internet connection to access the vast amount of torrents waiting to be discovered within the BitTorrent DHT space, without relying on any central entity.

bootstrap-dht - DHT bootstrap server

  •    C++

The DHT bootstrap server can be used as an introducer to the bittorrent DHT network. Like the ones running at router.utorrent.com and router.bittorrent.com. BitTorrent clients can use this server to join the DHT, assuming some number of clients are agreeing on using the same server.

bittorrent-dht - Simple, robust, BitTorrent DHT implementation

  •    Javascript

Simple, robust, BitTorrent DHT implementation


peer-tweet - Decentralized feeds using BitTorrent's DHT

  •    Javascript

BitTorrent's DHT is probably one of the most resilient and censorship-resistant networks on the internet. PeerTweet uses this network to allow users to broadcast tweets to anyone who is listening. When you start PeerTweet, it generates a hash @33cwte8iwWn7uhtj9MKCs4q5Ax7B which is similar to your Twitter username (ex. @lmatteis). The difference is that you have entire control over what can be posted because only you own the private key associated with such address. Furthermore, thanks to the DHT, what you post cannot be stopped by any government or institution. Once you find other PeerTweet addresses you trust (and are not spam), you can follow them. This configures your client to store this user's tweets and broadcasts them to the DHT every once in a while to keep their feed alive. This cooperation of following accounts, allows for feeds to stay alive in the DHT network. The PeerTweet protocol also publishes your actions such as I just followed @919c.. or I just liked @9139.. and I just retweeted @5789... This allows the possibility for new users to find other addresses they can trust; if I trust the user @6749.. and they're following @9801.., then perhaps I can mark @9801.. as not spam. This idea of publicly tweeting about your actions also allows for powerful future crawling analysis of this social graph.

dhtcrawler2 - dhtcrawler is a DHT crawler written in erlang

  •    Shell

dhtcrawler is a DHT crawler written in erlang. It can join a DHT network and crawl many P2P torrents. The program save all torrent info into database and provide an http interface to search a torrent by a keyword. dhtcrawler2 is an extended version to dhtcrawler. It has improved a lot on crawling speed, and much more stable.

dht - Kademlia/Mainline DHT node in Go.

  •    Go

This is a golang Kademlia/Bittorrent DHT library that implements BEP 5. It's typically used by a torrent client such as Taipei-Torrent, but it could also be used by a standalone DHT routers, or for other more creative purposes.

Norconex HTTP Collector - A Web Crawler in Java

  •    Java

Norconex HTTP Collector is a web spider, or crawler that aims to make Enterprise Search integrators and developers's life easier. It is Portable, Extensible, reusable, Robots.txt support, Obtain and manipulate document metadata, Resumable upon failure and lot more.

Bt - Java BitTorrent Done Right!

  •    Java

Bt is a lightweight framework for P2P-lovers and enthusiastic BitTorrent researchers, perfect choice for light enterprise and home usage and experimentation. It offers good performance, reliability and is highly customizable. With Bt you can create a production-grade BitTorrent client in a matter of minutes. Bt is still in its' early days, but is actively developed and designed with stability and maintainability in mind.

DHTCrawler - python 编写的DHT Crawler 网络爬虫,抓取磁力链接

  •    Python

python 编写的DHT Crawler 网络爬虫,抓取磁力链接

node-crawler - Web Crawler/Spider for NodeJS + server-side jQuery ;-)

  •    Javascript

Web Crawler/Spider for NodeJS + server-side jQuery ;-)

Norconex HTTP Collector - Enterprise Web Crawler

  •    Java

Norconex HTTP Collector is a full-featured web crawler (or spider) that can manipulate and store collected data into a repositoriy of your choice (e.g. a search engine). It very flexible, powerful, easy to extend, and portable.

go_spider - [爬虫框架 (golang)] An awesome Go concurrent Crawler(spider) framework

  •    Go

A crawler of vertical communities achieved by GOLANG. Latest stable Release: Version 1.2 (Sep 23, 2014).

colly - Elegant Scraper and Crawler Framework for Golang

  •    Go

Colly provides a clean interface to write any kind of crawler/scraper/spider. With Colly you can easily extract structured data from websites, which can be used for a wide range of applications, like data mining, data processing or archiving.

kadoh - The Kademlia DHT in Javascript for Node.js and Browsers

  •    Javascript

KadOH is a framework to build P2P applications for browsers and node.js. By implementing the basis of the Kademlia DHT, KadOH lets you build distributed web applications for mobile and desktop devices. With its flexible and extensible design, you can easily adapt KadOH to fit your needs. KadOH is available under the MIT License. See the wiki for more informations ! Also take look at our report and be aware that this document may be outdated.

torrent - Full-featured BitTorrent client package and utilities

  •    Go

This repository implements BitTorrent-related packages and command-line utilities in Go. The emphasis is on use as a library from other projects. It's been used 24/7 in production by a downstream, private service since late 2014.There is support for protocol encryption, DHT, PEX, uTP, and various extensions. See the package documentation for a more complete list. There are several data storage backends provided: blob, file, and mmap, and you can write your own, such as to store data on S3, or in a database. You can use the provided binaries in ./cmd, or use package torrent as a library for your own applications.

torsniff - torsniff - a sniffer that sniffs torrents from BitTorrent network

  •    Go

torsniff is a torrent sniffer, it sniffs torrents that people are using to download movies, music, docs, games and so on from BitTorrent network. A torrent has valuable information, so you can use torsniff to build your own torrent database(e.g: The Pirate Bay), or to do data mining and analyzing.