Creating Your Own NoSQL Database in Python

Databases are crucial for keeping, organizing, and fetching information.

There are these databases called NoSQL, which are loved for their adaptability and ability to grow. They've become quite popular as options to the traditional SQL databases.

If you're interested in learning more about how NoSQL databases work, or if you want to make your own solution for a particular need, building your own NoSQL database using Python is a fantastic project.

This article will walk you through the basic steps to create a simple, yet working, NoSQL database in Python.

If you prefer, you can follow along with the video version:

NoSQL Databases Explained

NoSQL databases are a type of database that can handle a variety of data models, unlike traditional relational databases (like SQL) that use tables with fixed schemas.

The term "NoSQL" stands for "Not Only SQL," indicating that these databases can work with data in ways that go beyond the capabilities of SQL.

Here are some key features of NoSQL databases:

• Flexible Data Model: NoSQL databases can store data in various formats, such as key-value pairs, documents (like JSON), column-family, or graph databases.
• Schema-less: Unlike relational databases, NoSQL databases don't require a predefined schema. This means you can store data without having to specify its structure beforehand, making it easier to evolve your data model over time.
• Horizontal Scalability: NoSQL databases are designed to scale out horizontally, which means you can add more servers to handle increased data and traffic. This makes them suitable for applications that need to handle large amounts of data or have high performance requirements.
• High Performance: NoSQL databases are built for speed and can handle large volumes of data quickly. They achieve this by using techniques like sharding, partitioning, and caching, which help distribute data and queries across multiple servers.
• Variety of Use Cases: NoSQL databases can be used in various scenarios, such as real-time analytics, content management, social networks, and IoT applications. Their flexibility and scalability make them a popular choice for modern applications with diverse data needs.

In this project, we'll be creating a document database, a type of NoSQL database that stores JSON data, where each record is a unique document with its own set of data.

Creating Your Own NoSQL Database in Python for JSON Documents

Ensure you have Python installed on your system. You can install it from the official Python website.

You’ll also need a text editor or an Integrated Development Environment (IDE) such as PyCharm or VSCode.

The Database Class

Let's start by creating the main Database class. You can create a file called database.py:

import json
import os
import uuid


class JSONNoSQLDatabase:
    # A simple NoSQL database using JSON files
    def __init__(self, filename='database.json'):
        # Initialize the database with a filename
        self.filename = filename
        # Load the database from the file
        if os.path.exists(self.filename):
            # If the file exists, load the database
            with open(self.filename, 'r') as file:
                self.store = json.load(file)
        else:
            # If the file does not exist, create an empty database
            self.store = {}

    # Save the database to the file
    def save(self):
        # Save the database to the file, overwriting the existing content with the updated content
        with open(self.filename, 'w') as file:
            json.dump(self.store, file, indent=4)

    # Insert a document into the database
    def insert(self, document):
        # Generate a unique document ID, insert the document into the database, and save the database
        doc_id = str(uuid.uuid4())
        self.store[doc_id] = document
        self.save()

        return doc_id

    # Update a document in the database
    def update(self, doc_id, document):
        # Check if the document ID exists, update the document, and save the database, or raise a KeyError
        if doc_id not in self.store:
            raise KeyError("Document ID does not exist.")
        self.store[doc_id] = document
        self.save()

    # Get a document from the database
    def get(self, doc_id):
        # Get the document with the specified document ID, or return None if the document ID does not exist
        return self.store.get(doc_id, None)

    # Delete a document from the database
    def delete(self, doc_id):
        # Check if the document ID exists, delete the document, and save the database, or raise a KeyError
        if doc_id in self.store:
            del self.store[doc_id]
            self.save()
        else:
            raise KeyError("Document ID does not exist.")

The class JSONNoSQLDatabase is created to handle the database operations. Here's a breakdown of the code:

• Import necessary libraries: json for working with JSON data, os for file operations, and uuid for generating unique IDs.
• Define the JSONNoSQLDatabase class with the following methods below.
• __init__: Initializes the database with a filename. If the file exists, it loads the database from the file; otherwise, it creates an empty database.
• save: Saves the database to the file, overwriting the existing content with the updated content.
• insert: Generates a unique document ID, inserts the document into the database, and saves the database. It returns the document ID.
• update: Checks if the document ID exists, updates the document, and saves the database. If the document ID doesn't exist, it raises a KeyError.
• get: Gets the document with the specified document ID. If the document ID doesn't exist, it returns None.
• delete: Checks if the document ID exists, deletes the document, and saves the database. If the document ID doesn't exist, it raises a KeyError.

This simple NoSQL database implementation allows you to store, retrieve, update, and delete JSON documents using unique IDs. The database is saved as a JSON file, making it easy to persist and work with the data.

Add Query Capabilities

Now that we have our database class and we can load, save, and manage documents, we need to be able to query the database to retrieve data.

Let's expand the database classes with query capabilities: