ferret - Declarative web scraping

•    Go

---

ferret is a web scraping system aiming to simplify data extraction from the web for such things like UI testing, machine learning and analytics. Having its own declarative language, ferret abstracts away technical details and complexity of the underlying technologies, helping to focus on the data itself. It's extremely portable, extensible and fast. The following example demonstrates the use of dynamic pages. First of all, we load the main Google Search page, type search criteria into an input box and then click a search button. The click action triggers a redirect, so we wait till its end. Once the page gets loaded, we iterate over all elements in search results and assign the output to a variable. The final for loop filters out empty elements that might be because of inaccurate use of selectors.

query-language data-mining scraping scraping-websites dsl cdp crawling scraper crawler chrome

lldpd - implementation of IEEE 802.1ab (LLDP)

•    C

---

LLDP (Link Layer Discovery Protocol) is an industry standard protocol designed to supplant proprietary Link-Layer protocols such as Extreme's EDP (Extreme Discovery Protocol) and CDP (Cisco Discovery Protocol). The goal of LLDP is to provide an inter-vendor compatible mechanism to deliver Link-Layer notifications to adjacent network devices. lldpd implements both reception and sending. It also implements an SNMP subagent for net-snmp to get local and remote LLDP information. The LLDP-MIB is partially implemented but the most useful tables are here. lldpd also partially implements LLDP-MED.

cdp network discovery lldp lldpd

cdp-js - Libraries/SDK modules for multi-platform application development

•    TypeScript

---

This monorepo contains cdp sdk modules.Folder and file structure of this repository is the following list.

cdp typescript

cordova-plugin-cdp-nativebridge

•    Javascript

---

The libraries provides easier way to implement connection between JavaScript and Native code than the cordova original framework.cordova-plugin-cdp-nativebridge: cordova plugin module which achieves generic Native Bridge function.

cdp cordova native bridge ecosystem:cordova cordova-android cordova-ios

huis-ui-creator

•    Javascript

---

cdp typescript backbone jquery-mobile

foxdriver - Foxdriver is a Node library which provides a high-level API to control Firefox over the Remote Debugging Protocol

•    Javascript

---

Foxdriver is a Node library which provides a high-level API to control Firefox over the Remote Debugging Protocol. The Firefox Remote Debugging Protocol consists of multiple actors that provide different methods. The Foxdriver API allows you to launch a Firefox instance and connects to the protocol interface automatically. From there you can access the methods of all actors.

firefox remote debugging debug protocol cdp devtools

node-websteps - End-to-end web testing on top of Chrome Debugging Protocol

•    Javascript

---

Pure JavaScript except for Chrome itself. No Java-based Selenium or platform-specific binary chromedriver required. API described by the test.

testing end-to-end acceptance integration chrome chrome-debugging-protocol cdp

dai.js

•    Javascript

---

Dai.js is a JavaScript library that makes it easy to build applications on top of MakerDAO's Dai Stablecoin System. You can use Maker's contracts to open Collateralized Debt Positions, withdraw loans in Dai, trade tokens on OasisDEX, and more. The library features a pluggable, service-based architecture, which allows users maximal control when integrating the Maker functionality into existing infrastructures. It also includes convenient configuration presets for out-of-the-box usability, a powerful smart contract state inspector, and support for both front-end and back-end applications.

makerdao mkr dai cdp eth margin

cdp - Package cdp provides type-safe bindings for the Chrome DevTools Protocol (CDP), written in the Go programming language

•    Go

---

Package cdp provides type-safe bindings for the Chrome Debugging Protocol (CDP), written in the Go programming language. The bindings are generated (by cdpgen) from the latest tip-of-tree (tot) protocol definitions and are mainly intended for use with Google Chrome or Chromium, however, they can be used with any debug target (Node.js, Edge, Safari, etc.) that implement the protocol. This package can be used for any kind of browser automation, scripting or debugging via the Chrome Debugging Protocol.

devtools-protocol chrome-devtools-protocol chrome-debugging-protocol cdp rpc debugging

cdp-cli - Command line tools for generating start point of multi-platform application development (Details: see cdp-js repository)

•    HTML

---

cdp command line interface for creating project's boilerplate. Folder and file structure of this repository is the following list.

cdp cli cordova electron typescript

cdp-lib - Application template/libraries for cdp-js (Details: see cdp-js repository)

•    Javascript

---

Contains npm modules used primarily by cdp boilerplate generator. Folder and file structure of this repository is the following list.

cdp typescript cordova electron

cdp4j - cdp4j - Chrome DevTools Protocol for Java

•    Java

---

cdp4j - Chrome DevTools Protocol for Java

chromium chrome test-automation chrome-debugger-protocol chrome-devtools chrome-developer-protocol automation cdp web web-automation chrome-devtools-protocol selenium selenium-webdriver webdriver crawling crawling-framework chrome-headless

dataflowkit - Extract structured data from web sites. Web sites scraping.

•    Go

---

Dataflow kit ("DFK") is a Web Scraping framework for Gophers. It extracts data from web pages, following the specified CSS Selectors. You can use it in many ways for data mining, data processing or archiving.

golang-library extract-data chrome-fetcher scraping-websites crawling scraper scraping cdp headless

cri - Type safe go bindings to interact with chrome remote interface.

•    Go

---

Package cri provides type-safe bindings for devtools protocol. It can be used with Chrome or any other target that implements the interface. Protocol is generated by cmd/generate.sh. Script fetches latest version of protocol and generates types and domain (accessibility, domdebugger, performance etc.) packages. Master branch reflects tip of tree.

cdp devtools-protocol headless-chrome