Mastering File System Monitoring with Watchdog in Python

I’ve found that file system monitoring is indispensable in modern applications.

Whether I'm setting up logging, live reloading during development, or enhancing security, keeping an eye on file changes ensures that my applications react immediately to critical events.

For my projects, I rely on Watchdog, a powerful Python library that simplifies file system monitoring across multiple platforms—Windows, macOS, and Linux.

Watchdog has an easy-to-use interface that lets me monitor files or directories without having to worry about the underlying platform-specific details.

This simplicity allows me to focus on building my applications rather than getting tangled up in complex file system event management.

SPONSORED By Python's Magic Methods - Beyond __init__ and __str__

This book offers an in-depth exploration of Python's magic methods, examining the mechanics and applications that make these features essential to Python's design.

Get the eBook

Getting Started with Watchdog

The first step I take is installing Watchdog using pip:

pip install watchdog

Watchdog works on all major operating systems. I always ensure my Python environment is running Python 3.x or later, which is required for the library.

Basic Setup

Once installed, I start by importing and initializing Watchdog in my Python script.

The process centers on two components:

• Observer: This continuously monitors the file system for events.
• FileSystemEventHandler: This lets me define custom callbacks to handle those events.

For a quick start, here’s an example that I use to monitor a directory:

from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler
import time

class MyHandler(FileSystemEventHandler):
    def on_modified(self, event):
        if not event.is_directory:
            print(f"Modified: {event.src_path}")

    def on_created(self, event):
        print(f"Created: {event.src_path}")

    def on_deleted(self, event):
        print(f"Deleted: {event.src_path}")

# Replace with the path I want to monitor
path = "/path/to/watch"
handler = MyHandler()
observer = Observer()
observer.schedule(handler, path=path, recursive=False)
observer.start()

try:
    while True:
        time.sleep(1)
except KeyboardInterrupt:
    observer.stop()
observer.join()

How this code works:

Custom Event Handler: I define MyHandler as a subclass of FileSystemEventHandler and override the methods:

• on_modified: to print a message when a file is modified.
• on_created: to log when a file or directory is created.
• on_deleted: to note when a file or directory is deleted.

Observer Setup: I specify the target path, create an Observer instance, and then schedule it with my event handler. By setting recursive=False, I monitor only the specified directory. The observer runs in an infinite loop until I gracefully stop it with a KeyboardInterrupt (Ctrl+C).

If you are interested in more Python file sync and handling articles, check out my other articles:

https://developer-service.blog/synchronizing-files-between-two-directories-using-python/

https://developer-service.blog/handling-large-files-and-optimizing-file-operations-in-python/

https://developer-service.blog/scripts-for-managing-files-and-directories-with-python/

Understanding Watchdog’s Core Components

Let me explain how Watchdog’s components work together to build a robust file monitoring framework:

This diagram visually outlines the key components:

• Watchdog Library sits at the top, connecting to the Observers.
• Observers pass events to Schedulers.
• Finally, Schedulers manage the process that delivers file system events to Event Handlers.

Observers

I use observers to continuously scan directories or files for changes—whether that’s modifications, creations, deletions, or moves.

They act as an intermediary between the file system and my event handlers, ensuring that events are captured in real time.

I instantiate an observer using the Observer() class, schedule it for a given path with schedule(), and then start it with start().

This kicks off the monitoring process until I decide to shut it down.

Event Handlers

The FileSystemEventHandler class comes with methods like:

• on_created: when a file or directory is added.
• on_deleted: when one is removed.
• on_modified: for any content changes.
• on_moved: for renaming or moving events.

I extend FileSystemEventHandler to override these methods so I can define exactly how my application should respond, such as logging events, triggering updates, or integrating with other systems.

Schedulers

Schedulers are responsible for defining how and when the observer checks for events:

With the schedule() method, I associate an event handler with a particular path.

If I set recursive=True, the observer will monitor the chosen directory and all its subdirectories; with recursive=False, it’ll monitor only the top-level directory.

Advanced Use Cases

Monitoring Multiple Directories

In larger projects where I need to track changes across several directories, I schedule each one with a single observer.

For example:

from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler
import time

class MyHandler(FileSystemEventHandler):
    def on_any_event(self, event):
        print(f"{event.event_type.capitalize()}: {event.src_path}")

def monitor_directories(directories):
    handler = MyHandler()
    observer = Observer()
    for directory in directories:
        observer.schedule(handler, path=directory, recursive=True)
    observer.start()
    try:
        while True:
            time.sleep(1)
    except KeyboardInterrupt:
        observer.stop()
    observer.join()

directories_to_monitor = ["/path/dir1", "/path/dir2"]
monitor_directories(directories_to_monitor)

Filtering and Logging

Sometimes, I only need to act on specific types of files.

By adjusting my event handler to check for file extensions, I can filter events (for example, handling only .txt files) and log them using Python’s logging module: