Anonymity in Sequence Data

•    

---

BF-2PkA is an algorithm for anonymization of sequence data. It transforms a dataset in a k-anonymous dataset. Moreover, the sequential pattern mining results are preserved very well.

databunker - Secure SDK/vault for personal records/PII built to comply with GDPR

•    Go

---

Project demo is available at: https://demo.databunker.org/. Please add a ⭐star if you like our project. We live in a world where the privacy of our information is nonexistent. The EU has been working to remediate this fallacy with GDPR, and the US (California) follows with a first sparrow called CCPA.

security privacy encryption database vault application-server compliance passportjs tokenization gdpr data-protection legaltech anonymization pii data-anonymization secure-storage privacy-by-design user-consent piidata ccpa

miller - Miller is like awk, sed, cut, join, and sort for name-indexed data such as CSV, TSV, and tabular JSON

•    C

---

Miller is like awk, sed, cut, join, and sort for name-indexed data such as CSV, TSV, and tabular JSON. With Miller, you get to use named fields without needing to count positional indices, using familiar formats such as CSV, TSV, JSON, and positionally-indexed.

data-processing data-cleaning csv csv-files csv-format csv-reader streaming-data streaming-algorithms tsv json json-data data-reduction data-regression statistics statistical-analysis devops devops-tools tabular-data command-line command-line-tools

CleverCSV - CleverCSV is a Python package for handling messy CSV files

•    Python

---

CleverCSV provides a drop-in replacement for the Python csv package with improved dialect detection for messy CSV files. It also provides a handy command line tool that can standardize a messy file or generate Python code to import it. Click here to go to the introduction with more details about CleverCSV. If you're in a hurry, below is a quick overview of how to get started with the CleverCSV Python package and the command line interface.

csv-converter data-science data-mining csv csv-files python-library python3 datascience csv-format csv-reading csv-parser csv-reader csv-export csv-import csv-parsing

dasel - Query, update and convert data structures from the command line

•    Go

---

Dasel (short for data-selector) allows you to query and modify data structures using selector strings. Comparable to jq / yq, but supports JSON, YAML, TOML, XML and CSV with zero runtime dependencies.

config cli yaml toml parser json query portable xml configuration update selector data-structures xml-parser devops-tools yaml-processor json-processing

Elasticsearch-Exporter - A small script to export data from one Elasticsearch cluster into another.

•    Javascript

---

A command line script to import/export data from ElasticSearch to various other storage systems. This is a brand new implementation with lots of bugs and way too little time to test everything for one lonely developer, so please consider this beta at best and provide feedback, bug reports and maybe even patches.

export import elastichsearch data-pump file transfer tools cli command-line database backup restore archive synchronize dump replicate mongodb mysql csv hbase datastore bigquery

csv-parser - Streaming csv parser inspired by binary-csv that aims to be faster than everyone else

•    Javascript

---

csv-parser can convert CSV into JSON at at rate of around 90,000 rows per second (perf varies with data, try bench.js with your data).The data emitted is a normalized JSON object. Each header is used as the property name of the object.

csv parser fast json

esri2open - this repo is an ESRI toolbox and tool(s) that export ESRI Feature Classes to open data formats, CSV, JSON, and GeoJSON

•    Python

---

This repo is an ESRI toolbox and tool(s) that exports ESRI Feature Classes to open data formats, CSV, JSON, SQLite, and GeoJSON. Much of the data in government coffers is contained in spatial databases. A large percentage of government spatial data is created and managed using ESRI software. While the common interchange format, the ESRI Shapefile, is easily exported and imported by many other softwares, this data file format (the Shapefile) is not intrinsically part of the www ecology. Moreover, ESRI software does not provide an export of its generic 'feature class' (shapefile, file geodatabase, and personal geodatabase) to the most common open data file formats, CSV, JSON, and/or GeoJSON. Finally while open source tools easily transform ESRI shapefiles to open data, most government geospatial infrastructures only have ESRI tools. Lacking this basic export feature presented here, means the lion's share of government spatial data users cannot export their data to the most common open data formats.

WCF Data Service Format Extensions for CSV, TXT

•    

---

This project add support for Legacy formats like CSV, TXT (CSV Export) to the data service output and allow $format=txt query. By default WCF Data Services support Atom and JSON responses however legacy systems do not understand ATOM or JSON but they understand CSV, TXT f...

wcf wcf-data-services

node-csvtojson - Blazing fast and Comprehensive CSV Parser for Node.JS / Browser / Command Line.

•    TypeScript

---

Here is a free online csv to json convert service utilizing latest csvtojson module. csvtojson has released version 2.0.0.

csv csvtojson csvparser nodejs csv-data csv-row csv-columns csv-stream parse csv-parser parse-csv json csv-to-json csv-convert tojson convert-csv-to-json csv-json

FSharp.Data - F# Data: Library for Data Access

•    HTML

---

The F# Data library (FSharp.Data.dll) implements everything you need to access data in your F# applications and scripts. It implements F# type providers for working with structured file formats (CSV, HTML, JSON and XML) and for accessing the WorldBank data. It also includes helpers for parsing CSV, HTML and JSON files and for sending HTTP requests.We're open to contributions from anyone. If you want to help out but don't know where to start, you can take one of the Up-For-Grabs issues, or help to improve the documentation.

fsharp html csv json xml worldbank http typeprovider data

active_importer - Define importers that load tabular data from spreadsheets or CSV files into any ActiveRecord-like ORM

•    Ruby

---

Define importers that load tabular data from spreadsheets or CSV files into any ActiveRecord-like ORM. Define classes that you instruct on how to import data into data models.

tabular-data spreadsheet csv-files activerecord orm data-import importer

readr - Read flat files (csv, tsv, fwf) into R

•    R

---

The goal of readr is to provide a fast and friendly way to read rectangular data (like csv, tsv, and fwf). It is designed to flexibly parse many types of data found in the wild, while still cleanly failing when data unexpectedly changes. If you are new to readr, the best place to start is the data import chapter in R for data science. To accurately read a rectangular dataset with readr you combine two pieces: a function that parses the overall file, and a column specification. The column specification describes how each column should be converted from a character vector to the most appropriate data type, and in most cases it's not necessary because readr will guess it for you automatically.

r csv fwf parsing

open-data-maker - make it easy to turn a lot of potentially large csv files into easily accessible open data

•    Ruby

---

The goal of this project is to make it easy to turn a lot of potentially large csv files into open data via an API and the ability for people to download smaller csv files with a subset of the data. Preliminary research suggests that open data users (journalists and others) actually know how to work with spreadsheets really well, but a lot of the data sets that we have in government are huge.

Obsei - Low code AI powered automation tool

•    Python

---

Obsei is a low code AI powered automation tool. It can be used in various business flows like social listening, AI based alerting, brand image analysis, comparative study and more. It consist of Observer, Analyzer and Informer. Observer observes the platform like Twitter, Facebook, App Stores, Google reviews, Amazon reviews, News, Website etc and feed that information. Analyzer performs text analysis like classification, sentiment, translation, PII on the analyzed data. Informer sends it to ticketing system, data store, dataframe etc for further action and analysis.

nlp workflow natural-language-processing sentiment-analysis text-classification customer-support text-analysis artificial-intelligence text-analytics social-network-analysis workflow-automation low-code anonymization issue-tracking-system process-automation customer-engagement lowcode business-process-automation social-listening

spreadsheet_architect - Spreadsheet Architect is a library that allows you to create XLSX, ODS, or CSV spreadsheets super easily from ActiveRecord relations, plain Ruby objects, or tabular data

•    Ruby

---

Spreadsheet Architect is a library that allows you to create XLSX, ODS, or CSV spreadsheets super easily from ActiveRecord relations, plain Ruby objects, or tabular data. When NOT using the :data option, ie. on an AR Relation or using the :instances option, Spreadsheet Architect requires an instance method defined on the class to generate the data. It looks for the spreadsheet_columns method on the class. If you are using on an ActiveRecord model and that method is not defined, it would fallback to the models column_names method (not recommended). If using the :data option this is ignored.

spreadsheet xlsx ods csv export activerecord rails

csv - CSV data manipulation made easy in PHP

•    PHP

---

Csv is a simple library to ease CSV parsing, writing and filtering in PHP. The goal of the library is to be powerful while remaining lightweight, by utilizing PHP native classes whenever possible.Full documentation can be found at csv.thephpleague.com.

csv-document csv-converter csv-filter csv-query csv

SCSM CSV Connector

•    

---

CSV Connector allows you to specify a data file and mapping location and a scheuled interval in minutes. At each scheduled interval Service Manager will import the data in the CSV data file into SCSM to update or insert data.

Tails - Live operating system which preserves privacy and anonymity

•    C

---

Tails is a live system that aims to preserve your privacy and anonymity. It helps you to use the Internet anonymously and circumvent censorship almost anywhere you go and on any computer but leaving no trace unless you ask it to explicitly. Tails relies on the Tor anonymity network to protect your privacy online. It encrypt and sign your emails and documents using the de facto standard OpenPGP either from Tails email client, text editor or file browser.

operating-system live-operating-system linux-distribution secure-operating-system privacy

postgres-copy - Simple PostgreSQL's COPY command support in ActiveRecord models

•    Ruby

---

This Gem will enable your AR models to use the PostgreSQL COPY command to import/export data in CSV format. If you need to tranfer data between a PostgreSQL database and CSV files, the PostgreSQL native CSV parser will give you a greater performance than using the ruby CSV+INSERT commands. I have not found time to make accurate benchmarks, but in the use scenario where I have developed the gem I have had a four-fold performance gain. This gem was written having the Rails framework in mind, I think it could work only with active-record, but I will assume in this README that you are using Rails.