awesome-web-scraper - A collection of awesome web scaper, crawler.

  •        2

A collection of awesome web scaper, crawler. Please, read the Contribution Guidelines before submitting your suggestion.

https://github.com/duyetdev/awesome-web-scraper

Tags
Implementation
License
Platform

   




Related Projects

awesome-no-login-web-apps - 🚀 Awesome (free) web apps that work without login

  •    Shell

Curated list of awesome web apps that work without requesting you to create an account. To save the world from creating user accounts and installing software applications for every damn thing.

awesome-webgl - A curated list of awesome WebGL libraries, resources and much more

  •    

This is a curated list of awesome WebGL libraries, resources and much more. WebGL (Web Graphics Library) is a JavaScript API for rendering interactive 3D computer graphics and 2D graphics within any compatible web browser without the use of plug-ins. WebGL is integrated completely into all the web standards of the browser allowing GPU accelerated usage of physics and image processing and effects as part of the web page canvas.

crawler - A high performance web crawler in Elixir.

  •    Elixir

A high performance web crawler in Elixir, with worker pooling and rate limiting via OPQ. Below is a very high level architecture diagram demonstrating how Crawler works.

awesome-web-security - 🐶 A curated list of Web Security materials and resources.

  •    

🐶 Curated list of Web Security materials and resources.Needless to say, most of websites on-line are suffered from various type of bugs, which might eventually lead to vulnerabilities. Why would this happen so often? Many factors can be involved, including misconfiguration, shortage of engineers' security skills, and etc. Therefore, here is the curated list of Web Security materials and resources for learning the cutting edge penetrating techniques.


awesome-opensource-apps - :house::information_source: Curated list of awesome open source crafted web & mobile applications - Learn, Fork, Contribute & Most Importantly Enjoy!

  •    

An awesome curated list of open source crafted web & mobile applications - Learn, Fork, Contribute & Most Importantly Enjoy!. You want to develop an app, write tests for a feature or implement a feature and you don't know how to go about it, there might just be one app/repository here with the solution to your problem.

awesome-pwa - Awesome list of progressive web apps! (PR welcomed ;))

  •    

Inspired by the awesome list thing. A Progressive Web App uses modern web capabilities to deliver an app-like user experience. They evolve from pages in browser tabs to immersive, top-level apps, leveraging the web's low friction.

awesome-go-storage - A curated list of awesome Go storage projects and libraries

  •    

A curated list of awesome Go storage projects and libraries. Inspired by awesome-go. Storage Servers implemented in Go.

awesome-spinners - 💫 A curated collection of dazzling web spinners

  •    Javascript

A curated collection of dazzling web spinners. Inspired by the awesome list thing.

awesome-django - Repository mirror of GitLab: https://gitlab.com/rosarior/awesome-django

  •    Makefile

A curated list of awesome Django apps, projects and resources. A: An awesome package is one that is mature (not recently released), is well maintained, has a good amount of users, has good documentation, follows the best practices, and which latest release is less than 1 year old. Awesome Django packages and projects are the ones that inspire and serve as examples.

awesome-angular - :page_facing_up: A curated list of awesome Angular resources by @TipeIO

  •    HTML

Awesome list of Angular seed repos, starters, boilerplates, examples, tutorials, components, modules, videos, and anything else in the Angular ecosystem. View as github page.Angular is a development platform for building mobile and desktop web applications.

awesome-honeypots - an awesome list of honeypot resources

  •    Python

A curated list of awesome honeypots, plus related components and much more, divided into categories such as Web, services, and others, with a focus on free and open source projects. There is no pre-established order of items in each category, the order is for contribution. If you want to contribute, please read the guide.

Goutte - Goutte, a simple PHP Web Scraper

  •    PHP

Goutte is a screen scraping and web crawling library for PHP. Goutte provides a nice API to crawl websites and extract data from the HTML/XML responses.

awesome-selfhosted - This is a list of Free Software network services and web applications which can be hosted locally

  •    Javascript

Selfhosting is the process of locally hosting and managing applications instead of renting from SaaS providers. This is a list of Free Software network services and web applications which can be hosted locally. Non-Free software is listed on the Non-Free page.

awesome-jupyter - A curated list of awesome Jupyter projects, libraries and resources

  •    

A curated list of awesome Jupyter projects, libraries and resources. Jupyter is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and narrative text. Your contributions are always welcome! Please take a look at the contribution guidelines first.

awesome - A collection of awesome Craft CMS plugins, articles, resources and shiny things.

  •    

Carefully curated list of awesome Craft CMS articles, resources and shiny things. Craft CMS is a focused content management system for developers, designers, and web professionals that blends flexibility, power, and ease of use for clients. Built to be as exceptionably scalable and as flexible as possible, without compromising on the ease of use for content authors.

typography - A collection of web typography resources

  •    

A curated collection of awesome web typography articles.