An Introduction to the UnQLite Embedded NoSQL Database Engine

  •        0

We aggregate and tag open source projects. We have collections of more than one million projects. Check out the projects section.

UnQLite is an embedded NoSQL database engine. It's a standard Key/Value store similar to the more popular Berkeley DB and a document-store database similar to MongoDB with a built-in scripting language called Jx9 that looks like Javascript. Unlike most other NoSQL databases, UnQLite does not have a separate server process. UnQLite reads and writes directly to ordinary disk files. A complete database with multiple collections is contained in a single disk file. The database file format is cross-platform, you can freely copy a database between 32-bit and 64-bit systems or between big-endian and little-endian architectures.

In this article, we shall introduce the concept behind UnQLite, it's architecture and a high level overview of its C/C++ API.

Web Site:

Stable Version: 1.1.8 (January 2018)

System requirements: Any OS with a modern C compiler

License: open source,



The UnQLite Architecture

UnQLite Architecture

Like most modern database engines, UnQLite is built-up on layers, The upper-layers hence the document-store and the Key/Value store layers are presented to the host application via a set of exported interfaces (i.e. The UnQLite API).

The principal task of a database engine is to store and retrieve records. UnQLite support both structured and raw database record storage. The Document-store layer is used to store JSON docs (i.e. Objects, Arrays, Strings, etc.) in the database and is powered by the Jx9 programming language while the Key/Value store layer is used to store raw records in the database.

Key/Value store layer.

UnQLite is a standard key/value store similar to BerkeleyDB, Tokyo Cabinet, LevelDB, etc. but, with a rich feature set including support for transactions (ACID), concurrent reader, etc. Under the KV store, both keys and values are treated as simple arrays of bytes, so content can be anything from ASCII strings, binary blob and even disk files. The KV store layer is presented to host applications via a set of interfaces, these includes: unqlite_kv_store(), unqlite_kv_append(), unqlite_kv_fetch_callback(), etc.

Document store layer.

The document store that is used to store JSON docs (i.e. Objects, Arrays, Strings, etc.) in the database is powered by the Jx9 programming language.

Jx9 is an embeddable scripting language also called extension language designed to support general procedural programming with data description facilities. Jx9 is a Turing-Complete, dynamically typed programming language based on JSON and implemented as a library in the UnQLite core.

Jx9 is built with a ton of features and has a clean and familiar syntax similar to C and Javascript. Being an extension language, Jx9 has no notion of a main program, it only works embedded in a host application. The host program (UnQLite in our case) can write and read Jx9 variables and can register C/C++ functions to be called by Jx9 code.

Pluggable Run-time Interchangeable Storage Engines.

UnQLite works with run-time interchangeable storage engines (i.e. Hash, B+Tree, R+Tree, LSM, etc.). The storage engine works with key/value pairs where both the key and the value are byte arrays of arbitrary length and with no restrictions on content. UnQLite come with two built-in KV storage engine: A Virtual Linear Hash (VLH) storage engine is used for persistent on-disk databases with O(1) lookup time and an in-memory hash-table or Red-black tree storage engine is used for in-memory databases. Future versions of UnQLite might add other built-in storage engines (i.e. LSM).

Transaction Manger/Pager module.

The underlying storage engine requests information from the disk in fixed-size chunks. The default chunk size is 4096 bytes but can vary between 512 and 65536 bytes. The pager module is responsible for reading, writing, and caching these chunks. The pager module also provides the rollback and atomic commit abstraction and takes care of locking of the database file. The storage engine requests particular pages from the page cache and notifies the pager module when it wants to modify pages or commit or rollback changes. The pager module handles all the messy details of making sure the requests are handled quickly, safely, and efficiently.

Virtual File System.

UnQLite is designed to run on multitude of platforms. In order to provide portability between these platforms (i.e. Between POSIX and Win32/64 operating systems), UnQLite uses an abstraction layer to interface with the operating system. Each supported operating system has its own implementation.

An instance of the unqlite_vfs object defines the interface between the UnQLite core and the underlying operating system. The "vfs" in the name of the object stands for "Virtual File System".

Introduction to the UnQLite C/C++ API

Early versions of UnQLite were very easy to learn since they only supported 12 C/C++ interfaces. But as the UnQLite engine has grown in capability, new C/C++ interfaces have been added so that now there are over 95 distinct APIs. This can be overwhelming to a new programmer. Fortunately, most of the C/C++ interfaces in UnQLite are very specialized and never need to be used. Despite having so many entry points, the core API is still relatively simple and easy to code to. This article aims to provide all of the background information needed to easily understand how the UnQLite database engine works.

The principal task of a database engine is to store and retrieve records. UnQLite support both structured and raw database record storage.

In order to accomplish this purpose, the developer needs to know about two objects:

  • The Database Engine Handle: unqlite
  • The UnQLite (Via Jx9) Virtual Machine Object: unqlite_vm

The database engine handle and optionally, the virtual machine object are controlled by a small set of C/C++ interface routines. The dozens of the C/C++ interface routines and two objects form the core functionality of UnQLite. The developer who understands them will have a good foundation for using UnQLite.

Typical Usage of the Core C/C++ Interfaces

An application that wants to use UnQLite will typically use the following interfaces with their optional components such as the transaction manager and the cursors interfaces. Note that an application may switch between the Key/Value store and the Document-store interfaces without any problem.


This routine opens a connection to an UnQLite database file and returns a database handle. This is often the first UnQLite API call that an application makes and is a prerequisite for most other UnQLite APIs. Many UnQLite interfaces require a pointer to the database handle as their first parameter and can be thought of as methods on the database handle. This routine is the constructor for the database handle.








Key/Value Store Interfaces

Under the Key/Value store, both keys and values are treated as simple arrays of bytes, so content can anything from ASCII strings, binary blob and even disk files. This set of interfaces allows clients to store and retrieve raw database records efficiently regardless of the underlying KV storage engine.








Document Store Interfaces

These set of interfaces works with the unqlite_vm object which is obtained after successful compilation of the target Jx9 script. Jx9 is the scripting language which power the document-store interface to UnQLite.

The Document-Store interface to UnQLite works as follows:

Obtain a new database handle via unqlite_open().

Compile your Jx9 script using one of the compile interfaces such as unqlite_compile() or unqlite_compile_file(). On successful compilation, the engine will automatically create an instance of this structure (unqlite_vm) and a pointer to this structure is made available to the caller.

When something goes wrong during compilation of the target Jx9 script due to a compile-time error, the caller must discard this pointer and fix its erroneous Jx9 code. Compile-time error logs can be extracted via a call to unqlite_config() with a configuration verb set to UNQLITE_CONFIG_JX9_ERR_LOG.

Optionally, configure the virtual machine using unqlite_vm_config().

Optionally, register one or more foreign functions or constants using the unqlite_create_function() or unqlite_create_constant() interfaces.

Execute the compiled Jx9 program by calling unqlite_vm_exec().

Optionally, extract the contents of one or more variables declared inside your Jx9 script using unqlite_vm_extract_variable().

Optionally, Reset the virtual machine using unqlite_vm_reset(), then go back to step 6. Do this zero or more times.

Destroy the virtual machine using unqlite_vm_release()





  and many more

Database Cursors

Cursors provide a mechanism by which you can iterate over the records in a database. Using cursors, you can seek, fetch, move, and delete database records.




Manual Transaction Manager

This set of interfaces allows the host application to manually start a write-transaction. Note that UnQLite is smart enough and will automatically start a write-transaction in the background when needed and so call to these routines is usually not necessary.


This routine is the destructor for the unqlite handle obtained by a prior successful call to unqlite_open().

Each database connection must be closed in order to avoid memory leaks and malformed database image.


This article only mentions the foundational UnQLite interfaces. The UnQLite library includes many other APIs implementing useful features that are not described here. A complete list of functions that form the UnQLite application programming interface is found at the C/C++ Interface Specification. Refer to that document for complete and authoritative information about all UnQLite interfaces.



We publish blog post about open source products. If you are interested in sharing knowledge about open source products, please visit write for us

Subscribe to our newsletter.

We will send mail once in a week about latest updates on open source tools and technologies. subscribe our newsletter

Related Articles

An introduction to MongoDB

  • mongodb database document-oriented-databse no-sql c++ data-mining

MongoDB is the most exciting SQL-free database currently available in the market. The new kid on the block, called MongoDB is a scalable, high-performance, open source, schema free and document oriented database that focuses on the ideas of NoSQL Approach. Written in C++, it has taken rapid strides since its emergence into the public sphere as a popular way to build your database applications.

Read More

Lucene / Solr as NoSQL database

  • lucene solr no-sql nosql document-store

Lucene and Solr are most popular and widely used search engine. It indexes the content and delivers the search result faster. It has all capabilities of NoSQL database. This article describes about its pros and cons.

Read More

Introduction to Apache Cassandra

  • cassandra database nosql

Apache Cassandra was designed by Facebook and was open-sourced in July 2008. It is regarded as perfect choice when the users demand scalability and high availability without any impact towards performance. Apache Cassandra is highly scalable, high-performance distributed database designed to handle large voluminous amounts of data across many commodity servers with no failure.

Read More

Connect to MongoDB and Perform CRUD using Java

  • java mongodb database programming

MongoDB is a popular and widely used open source NoSQL database. MongoDB is a distributed database at its core, so high availability, horizontal scaling, and geographic distribution is quite possible. It is licensed under Server Side Public License. Recently they moved to Server Side Public License, before that MongoDB was released under AGPL. This article will provide basic example to connect and work with MongoDB using Java.

Read More

Getting Started with Spring Batch

  • spring-batch spring-boot batch-processing

The best way to design a system for handling bulk workloads is to make it a batch system. If we are already using Spring, it will be easy to add a Spring Batch to the project. Spring batch provides a lot of boiler plate features required for batch processing like chunk based processing, transaction management and declarative input/output operations. It also provides job control for start, stop, restart, retry and skip processing also.

Read More

Desktop Apps using Electron JS with centralized data control

  • electronjs couchdb pouchdb desktop-app

When there is a requirement for having local storage for the desktop application context and data needs to be synchronized to central database, we can think of Electron with PouchDB having CouchDB stack. Electron can be used for cross-platform desktop apps with pouch db as local storage. It can sync those data to centralized database CouchDB seamlessly so any point desktop apps can recover or persist the data. In this article, we will go through of creation of desktop apps with ElectronJS, PouchDB and show the sync happens seamlessly with remote CouchDB.

Read More

Whats new in Lucene / Solr 4.0

  • lucene solr new-release

The release 4.0 is one of the important milestone for Lucene and Solr. It has lot of new features and performance important. Few important ones are highliggted in this article.

Read More

Caching using Ehcache Java Library

  • ehcache cache java map key-value

Ehcache from Terracotta is one of Java's most widely used Cache. It is concurrent and highly scalable. It has small footprint with SL4J as the only dependencies. It supports multiple strategies like Expiration policies, Eviction policies. It supports three storage tiers, heap, off-heap, disk storage. There are very few caching products supports multiple tier storage. If you want to scale, you cannot store all items in heap there should be support for off-heap and disk storage. Ehcache is licensed under Apache 2.0. In this article, we can see about basic usage of Ehcache.

Read More

Quick Start Programming Guide for redis using java client Jedis

  • redis jedis redis-client programming database java

Redis is an open source (BSD licensed), in-memory data structure store, used also as a database cache and message broker. It supports data structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperloglogs, geospatial indexes with radius queries and streams. This article explains about how to communicate with Redis using Java client Jedis.

Read More

Holistic usage guide for OpenSSL

  • openssl security certificate tools

OpenSSL is a general purpose cryptographty toolkit that provides an open source implementation of Transport Layer Security(TLS) and Secure Socket Layer(SSL) protocols. It is written in C,assembly and Perl language but wrappers are available in all languages. This article explains about OpenSSL commands.

Read More

Univention Corporate Server - An open source identity management system

  • ucs identity-management-system

Univention Corporate Server is an open source identity management system, an IT infrastructure and device management solution and an extensible platform with a store-like App Center that includes tested third party applications and further UCS components: This is what Univention combines in their main product Univention Corporate Server, a Debian GNU/Linux based enterprise distribution. This article provides you the overview of Univention Corporate Server, its feature and installation.

Read More

Cache using Hazelcast InMemory Data Grid

  • hazelcast cache key-value

Hazelcast is an open source In-Memory Data Grid (IMDG). It provides elastically scalable distributed In-Memory computing, widely recognized as the fastest and most scalable approach to application performance. Hazelcast makes distributed computing simple by offering distributed implementations of many developer-friendly interfaces from Java such as Map, Queue, ExecutorService, Lock and JCache.

Read More

Data Fetching and Form Building using NextJS

  • nextjs data-fetching form-handling

Next.js is one of the easy-to-learn frameworks for server-side pre-render pages for client-side web applications. In this blog, we will see how we can fetch data from API and make it pre-render pages. Also, let's see how forms work in Next.js and collect the data without maintaining the database.

Read More

React Reduce Vs Jotai

  • jottai react redux

While developing web applications, we built many components and having a data model across the application wide model will be difficult. We need common store management to update and get information across all the components. In React based Web UI, redux is the official centralized state management to be used which is internally wired as state changes to the component. There are other alternatives to redux which are light-weight and simple to use. One among them is Jotai, a minimalistic API for state management. In this blog we will take one of the redux examples given in the redux site and do the same in Jotai for deeper understanding.

Read More

Should web application store images in Database or File system?

  • database image-store filesystem

Web developers most frequent question, Should user images be stored in database or file system? Which is the best way. Both has some pros and cons.

Read More

Build Simple Ecommerce site using React and Jotai

  • ecommerce react jotai

Retail Ecommerce website can be created quickly with React. It can be created using React, Bootstrap, React DOM and Jotai. Data flow within the commerce site is done using Jotai in a light-weight manner. The main workflow of showing all the items in the gallery to user and user adding to the cart will be done as part of this blog.

Read More

Column database vs OLAP

  • business-intelligence olap column-database

OLAP (Online Analytical Processing), Reporting, Data mining related tasks are usually done by Business intelligence products. They do powerful Extraction, Transformation and Loading (ETL) the data and provides various reports. They use relational database as its back end. How could they generate better reports? Will column DB do a better job?

Read More

Angular Service Workers Usage Guide

  • angular service-worker offline-app

Web developers come across scenarios like web application completely breaks when workstation goes offline. Likewise to get into our application, every time we need to open a browser and then access it. Instead if it is in app, it will be easy to access for end-user. Push notifications similar to email client need to be done through web application. All these are addressed by a magic called service worker.

Read More

Light4j Cookbook - Rest API, CORS and RDBMS

  • light4j sql cors rest-api

Light 4j is a fast, lightweight and cloud-native microservices framework. In this article, we will see what and how hybrid framework works and integrate with RDMS databases like MySQL, also built in option of CORS handler for in-flight request.

Read More

How hashmap works in Java. My style of learning.

  • java hashmap opensource-learning

This is the most frequently asked questions in the interview. Googling will throw many links related to this topic. How to learn the implementation of hash map? My style of learning using open source learning technique.

Read More

We have large collection of open source products. Follow the tags from Tag Cloud >>

Open source products are scattered around the web. Please provide information about the open source projects you own / you use. Add Projects.