OHC - Java large off heap cache

  •        423

Off-Heap concurrent hash map intended to store GBs of serialized data. It has optional per entry or default TTL/expireAt, Entry eviction and expiration without a separate thread, Capable of maintaining huge amounts of cache memory, Suitable for tiny/small entries with low overhead using the chunked implementation.

OHC provides two implementations for different cache entry characteristics: - The _linked_ implementation allocates off-heap memory for each entry individually and works best for medium and big entries. - The _chunked_ implementation allocates off-heap memory for each hash segment as a whole and is intended for small entries.

https://github.com/snazy/ohc
http://caffinitas.org/

Tags
Implementation
License
Platform

   




Related Projects

Agrona - Library to build high-performance applications in Java and C++

  •    Java

Agrona provides a library of data structures and utility methods that are a common need when building high-performance applications in Java and C++. It supports Buffers, Map, Sets, Cache, Queues and lot more.

LArray - Large off-heap arrays for Java/Scala

  •    Scala

A library for managing large off-heap arrays that can hold more than 2G (2^31) entries in Java and Scala. Notably LArray is disposable by calling LArray.free or you can let GC automatically release the memory. LArray also can be used to create an mmap (memory-mapped file) whose size is more than 2GB

MapDB - Embedded Database Engine

  •    Java

MapDB is an embedded database engine. It provides Maps and other collections backed by disk or memory storage. It offers excellent performance comparable to java collections, but is not limited by GC overhead. It is also a full database engine with storage backends, transactions, cache algorithms, expiration and many other options. MapDB provides concurrent Maps, Sets and Queues backed by disk storage or off-heap memory.

Chronicle Map - High performance, off-heap, key-value, in memory, persisted data store

  •    Java

Chronicle Map is a high performance, off-heap, key-value, in memory, persisted data store. It works like a standard java map yet it automatically distributes data between processes, these processes can be both on the same server or across your network. In other words its a low latency, huge data key value store, which can store terabytes of data locally to your process.

slice - Java library for efficiently working with heap and off-heap memory

  •    Java

Library for efficiently working with heap and off-heap memory


heap-exploitation - This book on heap exploitation is a guide to understanding the internals of glibc's heap and various attacks possible on the heap structure

  •    C

This short book is written for people who want to understand the internals of 'heap memory', particularly the implementation of glibc's 'malloc' and 'free' procedures, and also for security researchers who want to get started in the field of heap exploitation.The first section of the book covers and in-depth, yet concise, description about heap internals. The second section covers some of the most famous attacks. It is assumed that the reader is unfamiliar with this topic. For experienced readers, this text might be good for a quick revision.

Valgrind

  •    C

Valgrind is an award-winning instrumentation framework for building dynamic analysis tools. There are Valgrind tools that can automatically detect many memory management and threading bugs, and profile your programs in detail. You can also use Valgrind to build new tools.

HugeCollections - Huge Collections for Java using efficient off heap storage

  •    Java

Huge Collections for Java using efficient off heap storage

hammerspace - Off-heap large object storage

  •    Ruby

Hammerspace ... is a fan-envisioned extradimensional, instantly accessible storage area in fiction, which is used to explain how animated, comic, and game characters can produce objects out of thin air.This gem provides persistent, concurrently-accessible off-heap storage of strings with a familiar hash-like interface. It is optimized for bulk writes and random reads.

cqengine - Ultra-fast SQL-like queries on Java collections

  •    Java

CQEngine – Collection Query Engine – is a high-performance Java collection which can be searched with SQL-like queries, with extremely low latency. Supports on-heap persistence, off-heap persistence, disk persistence, and supports MVCC transaction isolation.

DirectMemory

  •    Java

DirectMemory is a cache implementation featuring off-heap memory storage (a-la BigMemory) to enable caching of large (or large numbers of) objects without degrading jvm performance. ATTENTION PLEASE: DIRECTMEMORY IS MOVING TO APACHE - check incubation status here:

Sophia - Advanced transactional MVCC key-value/row storage library

  •    C

Sophia is RAM-Disk hybrid storage. It is designed to provide best possible on-disk performance without degradation in time. It has guaranteed O(1) worst case complexity for read, write and range scan operations. It provides Full ACID compliancy, MVCC engine, Optimistic, non-blocking concurrency with N-writers and M-readers, Prefix search, Automatic key-expire, Implemented as small C-written library with zero dependencies and lot more.

bigcache - Efficient cache for gigabytes of data written in Go.

  •    Go

Fast, concurrent, evicting in-memory cache written to keep big number of entries without impact on performance. BigCache keeps entries on heap but omits GC for them. To achieve that operations on bytes arrays take place, therefore entries (de)serialization in front of the cache will be needed in most use cases.

heapster - production heap profiling for the JVM. compatible with google-perftools.

  •    C

Heapster provides an agent library to do heap profiling for JVM processes with output compatible with Google perftools. The goal of Heapster is to be able to do meaningful (sampled) heap profiling in a production setting.By default, Heapster samples every 512 kB, this can be changed with the environment variable HEAPSTER_SAMPLE_PERIOD (in bytes).

HeapInspector-for-iOS - Find memory issues & leaks in your iOS app without instruments

  •    Objective-C

HeapInspector is a debug tool that monitors the memory heap with backtrace recording in your iOS app. You can discover memory leaks, no longer used objects, abandoned memory and more issues directly on your device without ever starting Instruments. Since ARC has been introduced we don't need to manage the retain & release anymore. ARC is very powerful and makes Objective-C more stable. ARC decreased the number of crashes and improves the memory footprint. ARC is technically doing a powerful job. It knows when to retain, autorelease and release. But ARC doesn't think about the overall architecture how to design for low memory usage. You should be aware that you can still do a lot of things wrong with your memory (even with ARC). You can still get memory pressures or peaks with ARC.

SwiftPriorityQueue - A Generic Priority Queue in Pure Swift

  •    Swift

SwiftPriorityQueue is a pure Swift (no Cocoa) implementation of a generic priority queue data structure, appropriate for use on all platforms (macOS, iOS, Linux, etc.) where Swift is supported. It features a straightforward interface and can be used with any type that implements Comparable. It utilizes comparisons between elements rather than separate numeric priorities to determine order. Internally, SwiftPriorityQueue uses a classic binary heap, resulting in O(lg n) pushes and pops. It includes in-source documentation, an A* based example maze solving program (for macOS), and unit tests (pull requests are welcome for additional unit tests in particular).

Apache Mnemonic - Non-volatile hybrid memory storage oriented library

  •    Java

Apache Mnemonic is a non-volatile hybrid memory storage oriented library, it proposed a non-volatile/durable Java object model and durable computing service that bring several advantages to significantly improve the performance of massive real-time data processing/analytics. developers are able to use this library to design their cache-less and SerDe-less high performance applications.

heapster - production heap profiling for the JVM. compatible with google-perftools.

  •    C

production heap profiling for the JVM. compatible with google-perftools.

dbgmem

  •    C

DBGMEM is a memory debugger designed for large scale C and C++ programs on Linux. It helps you to find: memory leaks, heap memory corruption, passing of illegal argument to selected standard library functions, use of freed or uninitialized heap blocks