sitepod - A Sitemap Generator written in PHP

•    PHP

---

Sitepod; A Sitemap Generator written in PHP. Sitepod is build using the Fat-Free Framework, a powerful yet easy-to-use PHP micro-framework designed to build dynamic and robust web applications. This will install Sitepod into a directory structure vendor/nasa/sitepod/ relative to wherever the command was executed.

sitemap-generator gplv3 search-engine sitemap sitemaps sitemap-xml sitemap-builder sitemap-files sitemap-url sitemap-php

sitemapper - parses sitemaps for Node.JS

•    Javascript

---

Parse through a sitemaps xml to get all the urls for your crawler.

sitemap sitemap-xml parse xml robots.txt sitemaps crawlers webcrawler

express-sitemap-xml - Serve sitemap.xml from a list of URLs in Express

•    Javascript

---

Create an Express middleware that serves sitemap.xml from a list of URLs. This package automatically handles sitemaps with more than 50,000 URLs. In these cases, multiple sitemap files will be generated along with a "sitemap index" to comply with the sitemap spec and requirements from search engines like Google.

express nodejs sitemap sitemap-generator sitemap-xml middleware google serve-sitemap serve-sitemap.xml site-map site-map-xml sitemap.xml sitemaps xml

gopher-parse-sitemap - A high effective golang library for parsing big-sized sitemaps and avoiding high memory usage

•    Go

---

A high effective golang library for parsing big-sized sitemaps and avoiding high memory usage. The sitemap parser was written on golang without external dependencies. See https://www.sitemaps.org/ for more information about the sitemap format. Time by time needs to parse really huge sitemaps. If you just unmarshal the whole file to an array of structures it produces high memory usage and the application can crash due to OOM (out of memory error).

sitemap parser parse scraping sitemap-xml scraping-websites sitemap-files sitemaps sitemap-url sitemap-index parsing-big-sitemaps