If you’ve ever relied on ESPHome devices inside Home Assistant, you might have run into the same frustration I did: sometimes the values just don’t update when you expect them to. Maybe it’s a sensor that reports sporadically, or a switch state that seems out of sync.

I recently started getting into home automation, and I’ve been loving the process of exploring both the software and the hardware side. For hardware, I chose to work with ESP32 boards running ESPHome, they’re cheap, powerful, and flexible enough for just about any DIY automation project.

Still, even with that setup, I wanted a way to see the raw updates directly from my ESP devices, without relying solely on Home Assistant. That’s how I ended up building a little Python logger, one that connects to my devices, listens for updates, and saves everything into daily CSV files. It’s been a game changer for troubleshooting and for keeping long-term historical logs.

Why not just use Home Assistant logs?

Home Assistant is amazing, but it isn’t really designed to give you a per-device raw data log. It’s fantastic at automation and dashboards, you can see the current state of your devices, set up history graphs, and even use the recorder integration to store data in a database.

But there are a few limitations I ran into:

  • The history database is optimized for Home Assistant dashboards, not for raw data analysis. It prunes older entries and doesn’t always give you every state change.
  • If a sensor only reports sporadically, it’s hard to tell whether that’s because the sensor didn’t send data or because Home Assistant just didn’t capture it.
  • Exporting data from Home Assistant can be clunky if all you want is a simple CSV file you can open in Excel, Pandas, or Grafana.

On top of that, I use ESPHome’s deep sleep mode on some of my ESP32 devices to save battery. That means the device only wakes up occasionally, sends its readings, and then goes back to sleep. While this is perfect for power efficiency, it makes it even harder to rely on Home Assistant’s database to know exactly when the updates happened and whether something was missed during the wake/sleep cycles.

I wanted something simpler: a “black box recorder” for my ESP devices that would capture every single update as it happens, with timestamps, and store it in a CSV file.

That’s where ESPHome’s native API comes in. With Python, you can connect directly to the device (the same way Home Assistant does behind the scenes) and log every update in real time. Instead of depending on what Home Assistant decides to store, you get a first-hand record straight from the source, including those infrequent updates from deep-sleeping devices.

Step 1: Talking to ESPHome from Python

ESPHome exposes a simple TCP API (default port 6053). It’s the same mechanism that Home Assistant uses under the hood when you add an ESPHome device through the integration. The nice part is that we don’t need to reinvent the wheel, there’s already a fantastic Python library called aioesphomeapi that handles authentication, device discovery, and streaming updates.

What this API gives us is direct access to raw sensor data, switches, binary sensors, and even services on the device. Instead of waiting for Home Assistant to decide how and when to store data, we can connect straight to the source and capture every update as it happens.

Here’s the minimal script I started with, just to print out values:

import asyncio
from aioesphomeapi import APIClient
from dotenv import load_dotenv
import os

# Load environment variables
load_dotenv()
API_HOST = os.getenv("API_HOST")
API_PASSWORD = os.getenv("API_PASSWORD")

async def main():
    # Connect to your ESPHome device by IP or hostname
    client = APIClient(API_HOST, 6053, password=API_PASSWORD, noise_psk=API_PASSWORD)
    await client.connect(login=True)

    # Define a callback function that will be triggered every time
    # the device sends a new state update (sensor reading, switch state, etc.)
    def state_callback(state):
        print("State update:", state)

    # Subscribe to all state updates
    client.subscribe_states(state_callback)

    # Keep the script running so we continue receiving updates
    while True:
        await asyncio.sleep(10)

asyncio.run(main())

When I ran this for the first time, it was a real “aha!” moment. Suddenly, I could watch every single update flowing in:

  • A temperature sensor reporting every 30 seconds
  • A binary sensor flipping between ON and OFF
  • Even deep-sleep devices popping online briefly, sending their data, and then disappearing again

Here is the output of one of my sensors:

State update: SensorState(key=3640870636, device_id=0, state=26.569412231445312, missing_state=False)
State update: SensorState(key=1365279611, device_id=0, state=1016.6199951171875, missing_state=False)
State update: SensorState(key=1277562889, device_id=0, state=63.120174407958984, missing_state=False)
State update: SensorState(key=426116527, device_id=0, state=156542.0, missing_state=False)
State update: SensorState(key=2104753181, device_id=0, state=84.01597595214844, missing_state=False)
State update: SensorState(key=1883882361, device_id=0, state=840.1597900390625, missing_state=False)
State update: SensorState(key=3764174908, device_id=0, state=1.2201377153396606, missing_state=False)
State update: SensorState(key=2195749573, device_id=0, state=2.855468988418579, missing_state=False)
State update: TextSensorState(key=1338075055, device_id=0, state='Stabilizing', missing_state=False)
State update: TextSensorState(key=688020976, device_id=0, state='Good', missing_state=False)

What’s powerful here is that this stream is unfiltered. It’s not averaged, not pruned, not optimized for dashboards. It’s the ground truth of what the device is actually sending. That’s exactly what I needed for troubleshooting and long-term logging.

And since this whole project was built in Python, I’ll share another resource you might find handy: my free 📘 Python One-Liner Cheat Sheet.

Sometimes, the best code is the shortest code, without sacrificing readability. That’s why I put together this downloadable PDF packed with Python tricks, idiomatic expressions, and productivity boosters.

Step 2: Making it Useful (Logging to CSV)

Printing to the console is fun, but it’s not very practical. If you really want to do something useful with the data, you need it in a format that’s easy to store, search, and analyze.

That’s why I decided to log everything into CSV files. CSV is one of those formats that never goes out of style:

  • You can double-click it and open it in Excel or Google Sheets
  • You can load it straight into Pandas with two lines of Python
  • It’s plain text, so you don’t need any special tools or databases to work with it
  • It’s simple enough that you can even inspect it with a text editor

But just dumping everything into a single file forever didn’t feel right. Over time, it would become massive and unmanageable. That’s why I also added daily rotation, each day gets its own file, named like esphome_2025-09-06.csv. This makes it easy to archive old data, analyze trends by day, or even compress older logs without touching the new ones.

So I wrote a reusable ESPHomeLogger class. It:

  • Connects to a device using the ESPHome native API
  • Subscribes to all updates (sensors, switches, binary sensors, etc.)
  • Writes each update into a CSV row with:
    • a timestamp (ISO 8601, e.g. 2025-09-06T14:23:01)
    • the entity ID (sensor.living_room_temp)
    • the friendly name (Living Room Temperature)
    • the raw state value (23.5)
  • Rotates automatically at midnight, starting a new file with headers

This way, I don’t just see updates in real time, but I also have a permanent historical record of every value my ESPHome device reported, neatly organized into daily slices.

Here’s the core of it: