leven - Measure the difference between two strings with the fastest JS implementation of the Levenshtein distance algorithm

•    Javascript

---

Measure the difference between two strings using the fastest JS implementation of the Levenshtein distance algorithm

leven levenshtein distance algorithm algo string difference diff fast fuzzy similar similarity compare comparison edit text match matching

jieba-php - "結巴"中文分詞：做最好的 PHP 中文分詞、中文斷詞組件。 / "Jieba" (Chinese for "to stutter") Chinese text segmentation: built to be the best PHP Chinese word segmentation module

•    PHP

---

"Jieba" (Chinese for "to stutter") Chinese text segmentation: built to be the best PHP Chinese word segmentation module. Scroll down for English documentation.

nlp natural-language-processing chinese-text-segmentation machine-learning

java-string-similarity - Implementation of various string similarity and distance algorithms: Levenshtein, Jaro-winkler, n-Gram, Q-Gram, Jaccard index, Longest Common Subsequence edit distance, cosine similarity

•    Java

---

Implementation of various string similarity and distance algorithms: Levenshtein, Jaro-winkler, n-Gram, Q-Gram, Jaccard index, Longest Common Subsequence edit distance, cosine similarity...

levenshtein-distance cosine-similarity string-distance damerau-levenshtein distance distance-measure jaro-winkler similarity-measures shingles algorithm jvm

fuzzysearch - :pig: Tiny and fast fuzzy search in Go

•    Go

---

Inspired by bevacqua/fuzzysearch, a fuzzy matching library written in JavaScript. But contains some extras like ranking using Levenshtein distance (see RankMatch()) and finding matches in a list of words (see Find()). Fuzzy searching allows for flexibly matching a string with partial input, useful for filtering data very quickly based on lightweight user input.

fuzzy-search algorithm

fuzzysearch - :pig: Tiny and fast fuzzy search in Go

•    Go

---

Inspired by bevacqua/fuzzysearch, a fuzzy matching library written in JavaScript. But contains some extras like ranking using Levenshtein distance (see RankMatch()) and finding matches in a list of words (see Find()). Fuzzy searching allows for flexibly matching a string with partial input, useful for filtering data very quickly based on lightweight user input.

fuzzy-search algorithm

lac - Baidu's open-source lexical analysis tool for Chinese, including word segmentation, part-of-speech tagging & named entity recognition

•    C++

---

Baidu's open-source lexical analysis tool for Chinese, including word segmentation, part-of-speech tagging & named entity recognition.

lexical-analysis word-segmentation part-of-speech-tagger named-entity-recognition chinese-word-segmentation chinese-nlp

fuzzywuzzy - Fuzzy String Matching in Python

•    Python

---

Fuzzy string matching like a boss. It uses Levenshtein Distance to calculate the differences between sequences in a simple-to-use package.

gse - Go efficient text segmentation; support english, chinese, japanese and other. Go 语言高性能分词

•    Go

---

Go efficient text segmentation; support english, chinese, japanese and other. Dictionary with double array trie (Double-Array Trie) to achieve, Sender algorithm is the shortest path based on word frequency plus dynamic programming.

segment nlp gse chinese english japanese trie

jcseg - Jcseg是基于mmseg算法的一个轻量级中文分词器，同时集成了关键字提取，关键短语提取，关键句子提取和文章自动摘要等功能，并且提供了一个基于Jetty的web服务器，方便各大语言直接http调用，同时提供了最新版本的lucene, solr, elasticsearch的分词接口！

•    Java

---

A lightweight open source chinese tokenizer with keywords, key sentences, summary extracts support, offered the latest lucene,solr,elasticsearch API.

jcseg mmseg chinese-word-segmentation natural-language-processing pos-tagging nlp nlp-keywords-extraction lucene-analyzer lucene-tokenizer solr-plugin elasticsearch-analyzer chinese-text-segmentation chinese-nlp keywords-extraction jcseg-analyzer

textdistance - Compute distance between sequences

•    Python

---

TextDistance -- python library for comparing distance between two or more sequences by many algorithms. Work in progress. Now all algorithms compare two strings as array of bits.

distance algorithm textdistance hamming-distance levenshtein-distance damerau-levenshtein damerau-levenshtein-distance algorithms distance-calculation jellyfish

tntsearch - A fully featured full text search engine written in PHP

•    PHP

---

We created also some demo pages that show tolerant retrieval with n-grams in action. The package has bunch of helper functions like jaro-winkler and cosine similarity for distance calculations. It supports stemming for English, Croatian, Arabic, Italian, Russian, Portuguese and Ukrainian. If the built in stemmers aren't enough, the engine lets you easily plugin any compatible snowball stemmer. Some forks of the package even support Chinese. Unlike many other engines, the index can be easily updated without doing a reindex or using deltas.

fulltext search search-engine full-text-search fuzzy-search geo-search fuzzy fuzzy-matching tntsearch laravel-scout

jiebaR - Chinese text segmentation with R. R语言中文分词 （文档已更新 🎉 ：https://qinwenfeng.com/jiebaR/ )

•    C++

---

This is a package for Chinese text segmentation, keyword extraction and speech tagging.

cppjieba jieba chinese-text-segmentation lexical-analysis chinese nlp

jieba - 结巴中文分词

•    Python

---

"Jieba" (Chinese for "to stutter") Chinese text segmentation: built to be the best Python Chinese word segmentation module.

natural-language-processing nlp

Chinese-Word-Vectors - 100+ Chinese Word Vectors 上百种预训练中文词向量

•    Python

---

This project provides 100+ Chinese Word Vectors (embeddings) trained with different representations (dense and sparse), context features (word, ngram, character, and more), and corpora. One can easily obtain pre-trained vectors with different properties and use them for downstream tasks. Moreover, we provide a Chinese analogical reasoning dataset CA8 and an evaluation toolkit for users to evaluate the quality of their word vectors.

chinese chinese-word-segmentation embeddings word-embeddings vectors-trained embedding

rmmseg-cpp - an re-implementation of rmmseg (Chinese word segmentation library for Ruby) in C++

•    Ruby

---

an re-implementation of rmmseg (Chinese word segmentation library for Ruby) in C++

jellyfish - 🎐 a python library for doing approximate and phonetic matching of strings.

•    Python

---

Jellyfish is a python library for doing approximate and phonetic matching of strings. Written by James Turk <james.p.turk@gmail.com> and Michael Stephens.

levenshtein soundex hamming metaphone jaro-winkler fuzzy-search

Arab Techies

•    Javascript

---

A collection of open source libraries and tools that provide solutions for common problems in processing Arabic text, especially in web applications. text normalization, phrase segmentation, text indexing, stop word lists, common spelling mistakes.

talisman - A straightforward & modular NLP, machine learning & fuzzy matching library for JavaScript

•    Javascript

---

The library's full documentation can be found here. Be sure to lint & pass the unit tests before submitting your pull request.

natural-language-processing machine-learning fuzzy-matching clustering record-linkage bayes bloom-filter canberra caverphone chebyshev cologne cosine classifier daitch-mokotoff dice fingerprint fuzzy hamming k-means jaccard jaro lancaster levenshtein lig metaphone mra ngrams nlp nysiis perceptron phonetic porter punkt schinke sorensen soundex stats tfidf tokenizer tversky vectorizer winkler

HanLP - 自然语言处理 中文分词 词性标注 命名实体识别 依存句法分析 关键词提取 新词发现 短语提取 自动摘要 文本分类 拼音简繁

•    Java

---

汉语言处理包

nlp natural-language-processing hanlp crf hmm trie textrank doublearraytrie neural-network chinese-word-segmentation text-mining pos-tagging dependency-parser text-classification word2vec perceptron named-entity-recognition text-clustering

fast-levenshtein - Efficient Javascript implementation of Levenshtein algorithm with locale-specific collator support

•    Javascript

---

An efficient Javascript implementation of the Levenshtein algorithm with locale-specific collator support. If you are not using any module loader system then the API will then be accessible via the window.Levenshtein object.

levenshtein distance string