textdistance - Compute distance between sequences

•    Python

---

TextDistance -- python library for comparing distance between two or more sequences by many algorithms. Work in progress. Now all algorithms compare two strings as array of bits.

distance algorithm textdistance hamming-distance levenshtein-distance damerau-levenshtein damerau-levenshtein-distance algorithms distance-calculation jellyfish

consimilo - A Clojure library for querying large data-sets on similarity

•    Clojure

---

consimilo is a library that utilizes locality sensitive hashing (implemented as lsh-forest) and minhashing, to support top-k similar item queries. Finding similar items across expansive data-sets is a common problem that presents itself in many real world applications (e.g. finding articles from the same source, plagiarism detection, collaborative filtering, context filtering, document similarity, etc...). Searching a corpus for top-k similar items quickly grows to an unwieldy complexity at relatively small corpus sizes (n choose 2). LSH reduces the search space by "hashing" items in such a way that collisions occur as a result of similarity. Once the items are hashed and indexed the lsh-forest supports a top-k most similar items query of ~O(log n). There is an accuracy trade-off that comes with the enormous increase in query speed. More information can be found in chapter 3 of Mining Massive Datasets. You can continue to add to this forest by passing it as the first argument to add-all-to-forest. The forest data structure is stored in an atom, so the existing forest is modified in place.

minhash-lsh-algorithm minhash lsh lsh-forest data-sketching data-sketches similarity similarity-search jaccard-similarity cosine-distance hamming-distance plagiarism-detection recommender-system collaborative-filtering document-similarity

text-metrics - Calculate various string metrics efficiently in Haskell

•    Haskell

---

The library provides efficient implementations of various strings metric algorithms. It works with strict Text values. edit-distance allows to specify costs for every operation when calculating Levenshtein distance (insertion, deletion, substitution, and transposition). This is rarely needed though in real-world applications, IMO.

string-metrics haskell levenshtein-distance hamming-distance jaro-distance jaro-winkler-distance jaccard-similarity

spark-stringmetric - Spark functions to run popular phonetic and string matching algorithms

•    Scala

---

Making similarity functions and phonetic algorithms readily available for fuzzy matching analyses in Spark. Update your build.sbt file to import the libraries.

cosine-distance spark fuzzy-score hamming-distance jaccard-similarity jaro-winkler double-metaphone nysiis refined-soundex

strutil - Golang metrics for calculating string similarity and other string utility functions

•    Go

---

strutil provides string metrics for calculating string similarity as well as other string utility functions. Full documentation can be found at: https://pkg.go.dev/github.com/adrg/strutil. The package defines the StringMetric interface, which is implemented by all the string metrics. The interface is used with the Similarity function, which calculates the similarity between the specified strings, using the provided string metric.

string smith-waterman levenshtein jaro-winkler string-metrics string-distance jaccard-similarity jaccard string-matching string-similarity hamming-distance jaro n-gram jaccard-index overlap-coefficient dice-coefficient smith-waterman-gotoh sorensen-dice n-gram-intersection strutil

StringComparison - String Comparision in C#.NET

•    CSharp

---

StringComparison is a library developed for reconciling naming conventions between different models of the electric grid. I have stripped off the power system specific code and put together what can effectively be used as a string extension for determining approximate equality between two strings. All of the algorithms used here have been pulled from online resources, translated into C#, and compiled into this library. I found several other similar open-source implementations around but nothing for .NET/C#. Adding the *.dll to your project will give you access to this extension and the individual extensions under the hood of the IsSimilarity() extension. While all of the algorithms are exposed and can be used and can provide their raw results, they have been conveniently combined in a way that they can selectively be used to judge the approximate equality of two strings. This is done through the IsSimilar extension and by setting the desired StringComparisonOptions and StringComparisonTolerance.

string comparison jaro-winkler levenshtein-distance longest-common-subsequence jaccard-distance hamming-distance jaro-distance longest-common-substring overlap-coefficient ratcliff-obershelp-similarity sorensen-dice-distance tanimoto-coefficient

ceja - PySpark phonetic and string matching algorithms

•    Python

---

Run pip install ceja to install the library. Import the functions with import ceja. After importing the code you can run functions like ceja.nysiis, ceja.jaro_winkler_similarity, etc.

pyspark jaro-winkler nysiis metaphone damerau-levenshtein hamming-distance porter-stemmer jaro-similarity match-rating-comparisons