jemalloc - Scalable, Concurrent, Malloc Implementation

  •        502

jemalloc is a general purpose malloc(3) implementation that emphasizes fragmentation avoidance and scalable concurrency support. It is intended for use as the system-provided memory allocator, as in FreeBSD's libc library, as well as for linking into C/C++ applications. jemalloc provides many introspection, memory management, and tuning features beyond the standard allocator functionality.

http://jemalloc.net/
https://github.com/jemalloc/jemalloc

Tags
Implementation
License
Platform

   




Related Projects

memory-allocators - Custom memory allocators in C++ to improve the performance of dynamic memory allocation

  •    C++

When applications need more memory this can be allocated in the heap (rather than in the stack) in runtime. This memory is called 'dynamic memory' because it can't be known at compile time and its need changes during the execution. Our programs can ask for dynamic memory usin 'malloc'. Malloc returns an address to a position in memory where we can store our data. Once we're done with that data, we can call 'free' to free the memory and let others processes use it. For this project I've implemented different ways to manage by ourselves dynamic memory in C++.This means that instead of using native calls like 'malloc' or 'free' we're going to use a custom memory allocator that will do this for us but in a more efficient way. The goal, then, is to understand how the most common allocators work, what they offer and compare them to see which one performs better.

Mesh - A memory allocator that automatically reduces the memory footprint of C/C++ applications.

  •    C++

Mesh is a drop in replacement for malloc(3) that compacts the heap without rewriting application pointers. Mesh is described in an academic paper (arxiv PDF) that will appear at PLDI 2019.

ralloc - An alternative pure-Rust memory allocator.

  •    Rust

A fast & memory efficient userspace allocator. This allocator is used as the default Redox.

shadow - jemalloc heap exploitation framework

  •    Python

Apart from the tool's source code, this repository also includes documentation on setting up an Android userland debugging environment for utilizing shadow, a quick overview of Android's jemalloc structures using shadow, and some notes on how double, unaligned and arbitrary free() bugs behave on Android's jemalloc. When you issue a jemalloc-specific command for the first time, shadow parses all jemalloc metadata it knows about and saves them to a Python pickle file. Subsequent commands use this pickle file instead of parsing the metadata from memory again in order to be faster.

scalloc - A Fast, Multicore-Scalable, Low-Fragmentation Memory Allocator

  •    C++

scalloc provides general-purpose memory allocation involving many threads on many cores can be done with high performance, multicore scalability, and low memory consumption. The main ideas behind the design of scalloc are: uniform treatment of small and big objects through so-called virtual spans, efficiently and effectively reclaiming free memory through fast and scalable global data structures.


heap-exploitation - This book on heap exploitation is a guide to understanding the internals of glibc's heap and various attacks possible on the heap structure

  •    C

This short book is written for people who want to understand the internals of 'heap memory', particularly the implementation of glibc's 'malloc' and 'free' procedures, and also for security researchers who want to get started in the field of heap exploitation.The first section of the book covers and in-depth, yet concise, description about heap internals. The second section covers some of the most famous attacks. It is assumed that the reader is unfamiliar with this topic. For experienced readers, this text might be good for a quick revision.

memory - STL compatible C++ memory allocator library using a new RawAllocator concept that is similar to an Allocator but easier to use and write

  •    C++

The C++ STL allocator model has various flaws. For example, they are fixed to a certain type, because they are almost necessarily required to be templates. So you can't easily share a single allocator for multiple types. In addition, you can only get a copy from the containers and not the original allocator object. At least with C++11 they are allowed to be stateful and so can be made object not instance based. But still, the model has many flaws. Over the course of the years many solutions have been proposed. for example EASTL. This library is another. But instead of trying to change the STL, it works with the current implementation. See example/ for more.

HeapInspector-for-iOS - Find memory issues & leaks in your iOS app without instruments

  •    Objective-C

HeapInspector is a debug tool that monitors the memory heap with backtrace recording in your iOS app. You can discover memory leaks, no longer used objects, abandoned memory and more issues directly on your device without ever starting Instruments. Since ARC has been introduced we don't need to manage the retain & release anymore. ARC is very powerful and makes Objective-C more stable. ARC decreased the number of crashes and improves the memory footprint. ARC is technically doing a powerful job. It knows when to retain, autorelease and release. But ARC doesn't think about the overall architecture how to design for low memory usage. You should be aware that you can still do a lot of things wrong with your memory (even with ARC). You can still get memory pressures or peaks with ARC.

OHC - Java large off heap cache

  •    Java

Off-Heap concurrent hash map intended to store GBs of serialized data. It has optional per entry or default TTL/expireAt, Entry eviction and expiration without a separate thread, Capable of maintaining huge amounts of cache memory, Suitable for tiny/small entries with low overhead using the chunked implementation.

rpmalloc - Public domain cross platform lock free thread caching 32-byte aligned memory allocator implemented in C

  •    C

The code should be easily portable to any platform with atomic operations and an mmap-style virtual memory management API. The API used to map/unmap memory pages can be configured in runtime to a custom implementation and mapping granularity/size. This library is put in the public domain; you can redistribute it and/or modify it without any restrictions. Or, if you choose, you can use it under the MIT license.

LArray - Large off-heap arrays for Java/Scala

  •    Scala

A library for managing large off-heap arrays that can hold more than 2G (2^31) entries in Java and Scala. Notably LArray is disposable by calling LArray.free or you can let GC automatically release the memory. LArray also can be used to create an mmap (memory-mapped file) whose size is more than 2GB

Apache Mnemonic - Non-volatile hybrid memory storage oriented library

  •    Java

Apache Mnemonic is a non-volatile hybrid memory storage oriented library, it proposed a non-volatile/durable Java object model and durable computing service that bring several advantages to significantly improve the performance of massive real-time data processing/analytics. developers are able to use this library to design their cache-less and SerDe-less high performance applications.

slice - Java library for efficiently working with heap and off-heap memory

  •    Java

Library for efficiently working with heap and off-heap memory

oomd - A userspace out-of-memory killer

  •    C++

oomd is userspace Out-Of-Memory (OOM) killer for linux systems. Out of memory killing has historically happened inside kernel space. On a memory overcommitted linux system, malloc(2) and friends will never fail. However, if an application dereferences the returned pointer and the system has run out of physical memory, the linux kernel is forced take extreme measures, up to and including killing processes. This is typically a slow and painful process because the kernel spends an unbounded amount of time swapping in and out pages and evicting the page cache. Furthermore, configuring policy is not very flexible while being somewhat complicated.

Shared Memory Arena

  •    C

a shared memory allocator that provides a simple interface for named shared memory segments; shared memory between any processes; provides shared memory queues; does not use the same address between processes; fast structured arena file scales well

dbgmem

  •    C

DBGMEM is a memory debugger designed for large scale C and C++ programs on Linux. It helps you to find: memory leaks, heap memory corruption, passing of illegal argument to selected standard library functions, use of freed or uninitialized heap blocks

MapDB - Embedded Database Engine

  •    Java

MapDB is an embedded database engine. It provides Maps and other collections backed by disk or memory storage. It offers excellent performance comparable to java collections, but is not limited by GC overhead. It is also a full database engine with storage backends, transactions, cache algorithms, expiration and many other options. MapDB provides concurrent Maps, Sets and Queues backed by disk storage or off-heap memory.

DieHard - DieHard: An error-resistant memory allocator for Windows, Linux, and Mac OS X

  •    C

This distribution includes the source for three systems. DieHard was the direct inspiration for the Fault-Tolerant Heap incorporated in Windows 7, though it goes far beyond it in terms of reliability.

scala-offheap - Experimental type-safe off-heap memory for Scala.

  •    Scala

Garbage collection is the standard memory management paradigm on the JVM. In theory, it lets one completely forget about the hurdles of memory management and delegate all of it to the underlying runtime. In practice, GC often leads to scalability issues on large heaps and latency-sensitive workloads. The goal of this project is to expose a completely different memory management paradigm to the developers: explicitly annotated region-based memory. This paradigm gives more control over memory management without the need to micro-manage allocations.

Agrona - Library to build high-performance applications in Java and C++

  •    Java

Agrona provides a library of data structures and utility methods that are a common need when building high-performance applications in Java and C++. It supports Buffers, Map, Sets, Cache, Queues and lot more.