Electron Database - RxDB with different storage for SQLite, Filesystem and In-Memory

Electron (aka Electron.js) is a framework developed by github that is designed to create desktop applications with the Web technology stack consisting of HTML, CSS and JavaScript. Because the desktop application runs on the client's device, it is suitable to use a database that can store and query data locally. This allows you to create so-called local first apps that store data locally and even work when the user has no internet connection. While there are many options to store data in Electron, for complex realtime apps using RxDB is recommended because it is a database made for UI-based client-side application, not a server-side database.

Databases for Electron ​

An Electron runtime can be divided into two parts:

This is important to understand because choosing the right database depends on your use case and on which of these JavaScript runtimes you want to keep the data.

Server Side Databases in Electron.js ​

Because Electron runs on a desktop computer, you might think that it should be possible to use a common "server" database like MySQL, PostgreSQL or MongoDB. In theory, you could ship the correct database server binaries with your electron application and start a process on the client's device that exposes a port to the database that can be consumed by Electron. In practice, this is not a viable way to go because shipping the correct binaries and opening ports is way to complicated and troublesome. Instead you should use a database that can be bundled and run inside of Electron, either in the main or in the renderer process.

Localstorage / IndexedDB / WebSQL as alternatives to SQLite in Electron ​

Because Electron uses a common Chrome web browser in the renderer process, you can access the common Web Storage APIs like Localstorage , IndexedDB and WebSQL. This is easy to set up and storing small sets of data can be achieved in a short span of time.

But as soon as your application goes beyond a simple TODO-app, there are multiple obstacles that come in your way. One thing is the bad multi-tab support. If you have more than one renderer process, it becomes hard to manage database writes between them. Each browser tab could modify the database state while the others do not know of the changes and keep an outdated UI.

Another thing is performance. IndexedDB is slow , mostly because it has to go through layers of browser security and abstractions. Storing and querying a lot of data might become your performance bottleneck. Localstorage and WebSQL are even slower, by the way. Using these Web Storage APIs is generally only recommended when you know for sure that there will be always only one rendering process and performance is not that relevant. The main reason for that is the security- and abstraction layers that write- and read operations have to go through when using the browsers IndexedDB API. So instead of using IndexedDB in Electron in the renderer process, you should use something that runs in the "main" process in Node.js like the Filesystem RxStorage or the In Memory RxStorage .

RxDB ​

RxDB is a NoSQL database for JavaScript applications. It has many features that come in handy when RxDB is used with UI based applications like your Electron app. For example, it is able to subscribe to query results of single fields of documents. It has encryption and compression features and most important it has a battle tested replication protocol that can be used to do a realtime sync with your backend.

Because of the flexible storage layer of RxDB, there are many options on how to use it with Electron:

It is recommended to use the SQLite RxStorage because it has the best performance and is the easiest to set up. However it is part of the 👑 Premium Plugins which must be purchased, so to try out RxDB with Electron, you might want to use one of the other options. To start with RxDB, I would recommend using the Dexie.js RxStorage in the renderer processes. Because RxDB is able to broadcast the database state between browser tabs, having multiple renderer processes is not a problem like it would be when you use plain IndexedDB without RxDB. In production, you would always run the RxStorage in the main process with the RxStorage Electron IpcRenderer & IpcMain plugins.

First, you have to install all dependencies via npm install rxdb rxjs . Then you can assemble the RxStorage and create a database with it:

import { createRxDatabase } from 'rxdb';
import { getRxStorageDexie } from 'rxdb/plugins/storage-dexie';

// create database
const db = await createRxDatabase({
    name: 'exampledb',
    storage: getRxStorageDexie()
});

// create collections
const collections = await myRxDatabase.addCollections({
    humans: {
        /* ... */
    }
});

// insert document
await collections.humans.insert({id: 'foo', name: 'bar'});

// run a query
const result = await collections.humans.find({
    selector: {
        name: 'bar'
    }
}).exec();

// observe a query
await collections.humans.find({
    selector: {
        name: 'bar'
    }
}).$.subscribe(result => {/* ... */});

For better performance in the renderer tab, you can later switch to the IndexedDB RxStorage . But in production, it is recommended to use the SQLite RxStorage or the Filesystem RxStorage in the main process so that database operations do not block the rendering of the UI. To learn more about using RxDB with Electron, you might want to check out this example project .

SQLite in Electron.js without RxDB ​

SQLite is a SQL based relational database written in the C programming language that was crafted to be embedded inside of applications and stores data locally. Operations are written in the SQL query language similar to the PostgreSQL syntax.

Using SQLite in Electron is not possible in the renderer process , only in the main process . To communicate data operations between your main and your renderer processes, you have to use either @electron/remote (not recommended) or the ipcRenderer (recommended). So you start up SQLite in your main process and whenever you want to read or write data, you send the SQL queries to the main process and retrieve the result back as JSON data.

To install SQLite, use the SQLite3 package which is a native Node.js module. You also need the @electron/rebuild package to rebuild the SQLite module against the currently installed Electron version.

Install them with npm install sqlite3 @electron/rebuild . Then you can rebuild SQLite with ./node_modules/.bin/electron-rebuild -f -w sqlite3 In the JavaScript code of your main process you can now create a database:

const sqlite3 = require('sqlite3');
const db = new sqlite3.Database('/path/to/database/file.db');
// create a table and insert a row
db.serialize(() => {
  db.run("CREATE TABLE Users (name, lastName)");
  db.run("INSERT INTO Users VALUES (?, ?)", ['foo', 'bar']);
});

Also you have to set up the ipcRenderer so that message from the renderer process are handled:

ipcMain.handle('db-query', async (event, sqlQuery) => {
  return new Promise(res => {
      db.all(sqlQuery, (err, rows) => {
        res(rows);
      });
  });
});

In your renderer process, you can now call the ipcHandler and fetch data from SQLite:

const rows = await ipcRenderer.invoke('db-query', "SELECT * FROM Users");

The downside of SQLite (or SQL in general) is that it is lacking many features that are handful when using a database together with UI based applications. It is not possible to observe queries or document fields and there is no replication method to sync data with a server. This makes SQLite a good solution when you just want to store data on the client or process expensive SQL queries on the server, but it is not suitable for more complex operations like two-way replication, encryption, compression and so on. Also developer helpers like TypeScript type safety are totally out of reach.

Follow up ​