Raspberry + Kobo E-Reader Bird Identification
I had an old Kobo e-reader for which I had no use anymore but which I also didn’t really want to throw away, so I looked for a way to give the device a new purpose. This project failed spectacularly: the already bruised hardware fell apart completely during my weeks of tinkering. But since I had a semi-finished project at the time, I ordered another second-hand e-reader from a local shop to complete the work. From an idealistic standpoint, this was unfortunately the opposite of what I wanted; instead of repurposing old hardware, I had to buy another one, questioning my moral integrity throughout the process. Nevertheless, I still think it’s a cute demonstration of what you can do with unused embedded hardware lying around.
The project works in tandem with a Raspberry Pi equipped with a small USB microphone. The Pi listens to ambient audio and uses the BirdNet machine-learning model to identify birds from their singing patterns. I also wrote a spartanic web scraper for wikipedia bird articles to store a small local database with bird data on the Pi. The result isn’t perfect (leaving out the fact that I mostly scraped German articles) — some of the scraped articles gave me a picture of bird eggs instead of the actual birds, but I already fixed a couple of those manually. When the Pi detects a bird with its ML model, it sends the data via its own WiFi access point to the Kobo e-reader, which displays the bird’s name, an image, and the first lines of the Wikipedia article on its screen.
Displaying this custom data works by rasterizing the data onto a virtual page and sending it directly to the framebuffer. This is similar to my other DRM projects, but the Kobo uses a quite old Linux kernel before DRM was part of it. Instead, it uses the old FrameBuffer API, which is accessible through /dev/fb0. The concept is kinda the same, but because the reader’s display is E-Ink, I had to look into the driver code to figure out its quriks; for example, sending data alone isn’t enough, as the display also expects an explicit refresh command.
The project is written mostly in Rust, with the web scraper in Python. When I started the project, the Rust library for the BirdNet model didn’t seem to work and crashed on my machine. I reported the problem to a developer on Discord, and after I picked up the project again a few weeks later, everything suddenly worked. Turns out, reporting bugs does actually help!
There’s still a lot that could be done, though. The main issue was some power-saving logic in the e-reader constantly cutting the WiFi connection to the Pi. I found some kernel parameters which looked like they have something to do with power management, but fiddling with them made little difference. In the end, I solved the problem by sending ping-pong packets every three seconds between the Pi and the Kobo to keep them connected. However, the reader’s battery will be depleted already after about six hours. Ideally, the reader could stay in low-power mode with WiFi always on, so it wakes up instantly when a bird is detected.
I also wonder how well a battery for the Pi would work to make the whole device portable. A watertight case and solar energy would make it far easier to put the Pi somewhere in the garden. Thanks to the WiFi solution, the display can still be placed more accessibly. My reader also has two hardware buttons that I guess could maybe be accessed via /dev/input, so it might be possible to add some interactive functionality as well.