Microscope camera kiosk
Touchscreen kiosk for the Raspberry Pi HQ camera (IMX477) used as a microscope: live preview from boot, raw frame recording, full-resolution stills, and automatic capture-SSD mounting. - app.py: GTK3 fullscreen UI over cage — preview, REC/STOP, PHOTO, 90-degree rotate button, tap-to-open exposure/gain/settings sliders, and an "SSD NOT MOUNTED" badge - camera.py: picamera2 wrapper with separate video (ISP preview + packed 12-bit raw) and still (4056x3040) configurations - recorder.py: writer thread turning the raw stream into one Bayer frame per file plus index.jsonl/session.json, with dropped/missed accounting - photos.py: still capture to DNG + JPEG, and a background worker that renders a viewable 16-bit sRGB TIFF from the 12-bit Bayer frame - storage.py: SSD mount state and the date/time/format session layout - tools/unpack_raw.py: desktop-side unpacker/demosaicer - system/ + install.sh: systemd units, udev rule, mount helper, sudoers Note: raw frames are written byte-for-byte as the sensor produced them, so they stay decodeable regardless of how the stills pipeline evolves.
This commit is contained in:
Microscope Camera
232
README.md
Normal file
232
README.md
Normal file
@@ -0,0 +1,232 @@
|
||||
# Microscope Camera
|
||||
|
||||
Touchscreen kiosk for the Raspberry Pi 4 + IMX477 (HQ) microscope camera.
|
||||
|
||||
* live preview on the 1024x600 HDMI touchscreen from boot
|
||||
* **REC / STOP** — raw 12-bit Bayer frame recording, 2028x1520, ~29-30 fps
|
||||
* **PHOTO** — full-resolution 4056x3040 still as DNG + 16-bit TIFF + JPEG
|
||||
* **⟳ rotate** (bottom-left) — turns the image 90° at a time, on screen *and*
|
||||
in the files it writes
|
||||
* **tap the status pill** (top-left) — opens sliders for exposure, gain,
|
||||
sharpness, contrast, saturation and brightness; tap it again to close
|
||||
* a red **SSD NOT MOUNTED** badge that appears only when the capture SSD is
|
||||
missing, and the SSD auto-mounts whenever it is plugged in
|
||||
* everything lands on the SSD as `date/time/format/`
|
||||
|
||||
## On-screen controls
|
||||
|
||||
| Control | Where | What it does |
|
||||
|---|---|---|
|
||||
| `● REC` / `■ STOP` | bottom-right | starts/stops raw frame capture into a new `date/time/raw` session |
|
||||
| `◉ PHOTO` | bottom-right | full-resolution still: DNG + 16-bit TIFF + JPEG in one `date/time` folder |
|
||||
| `⟳ n°` | bottom-left | rotates preview and saved output clockwise in 90° steps (remembered across reboots) |
|
||||
| status pill | top-left | tap to show/hide the camera settings sliders |
|
||||
| `⏏ MOUNT SSD` | above REC, only when needed | appears when the drive is present but not mounted |
|
||||
|
||||
### Rotating
|
||||
|
||||
The rotate button turns both the live preview and everything that gets saved:
|
||||
|
||||
* **JPEG / TIFF** pixels are physically rotated before they are written.
|
||||
* **DNG** keeps its sensor-native Bayer data (rotating a mosaic would change its
|
||||
colour-filter pattern) and records the rotation in the standard TIFF
|
||||
`Orientation` tag, which raw converters honour.
|
||||
* **Raw recordings** store frames sensor-native too and record the angle in
|
||||
`session.json` as `rotation`; `tools/unpack_raw.py` applies it on export, so
|
||||
raw frames you unpack on a desktop come out the way the screen showed them.
|
||||
|
||||
### Camera settings panel
|
||||
|
||||
Tapping the top-left status pill reveals sliders for the controls the sensor
|
||||
actually exposes (anything unsupported is greyed out):
|
||||
|
||||
* **Exposure** — logarithmic, from the sensor minimum (1 µs) up to 66 ms, so the
|
||||
useful low end is not crammed into the first pixel of travel.
|
||||
* **Gain** — analogue gain, 1.0x to 16.0x.
|
||||
* **Sharpness / Contrast / Saturation / Brightness** — ISP tuning.
|
||||
|
||||
Touching Exposure or Gain switches **Auto exposure** off automatically (AE would
|
||||
otherwise overwrite the slider every frame). The button at the top of the panel
|
||||
hands control back to AE and restores the frame-duration pin.
|
||||
|
||||
A manual exposure longer than `1/target_fps` temporarily widens
|
||||
`FrameDurationLimits`, because the recording pin would otherwise clamp it; the
|
||||
sensor cannot both expose for 50 ms and deliver 30 fps, so the frame rate drops
|
||||
for as long as the long exposure is set. Auto exposure restores the pin.
|
||||
|
||||
## Layout on the SSD
|
||||
|
||||
```
|
||||
/mnt/microscope/
|
||||
├── 2026-10-30/ <- date
|
||||
│ ├── 21-33-05/ <- time (start of the session)
|
||||
│ │ ├── raw/
|
||||
│ │ │ ├── frame_000001.raw 4.67 MB packed 12-bit Bayer
|
||||
│ │ │ └── ...
|
||||
│ │ ├── index.jsonl one line per frame: timestamp, exposure, gain, drops
|
||||
│ │ └── session.json totals: frames, bytes, fps, dropped, missed
|
||||
│ └── 21-40-12/ <- a photo session
|
||||
│ ├── raw/still_0001.dng 18 MB, untouched 12-bit Bayer
|
||||
│ ├── tiff/still_0001.tiff 74 MB, 16-bit sRGB (demosaiced, WB+colour matrix)
|
||||
│ ├── jpeg/still_0001.jpg ISP-processed 8-bit
|
||||
│ └── photo.json exposure, gain, white balance, timings
|
||||
└── _archive/ (old test recordings moved off the SD card)
|
||||
```
|
||||
|
||||
## What the record button actually writes
|
||||
|
||||
Frames are the sensor's raw stream, stored **verbatim**: MIPI-packed 12-bit
|
||||
Bayer, 2028x1520 (2x2 binned, full field of view), one file per frame. No
|
||||
debayer, no compression, no white balance — nothing is baked in. Per frame:
|
||||
`stride=3072`, `frame_bytes=4669440`, format `SBGGR12_CSI2P`.
|
||||
|
||||
Frame rate is governed by the SSD, not the sensor. Measured on this hardware:
|
||||
|
||||
| Target | Rate needed | Result |
|
||||
|---|---|---|
|
||||
| 40 fps (sensor mode max) | 247 MB/s | not possible |
|
||||
| **30 fps** | **140 MB/s** | **sustained: 29.0 fps, 0 dropped, 0 missed** |
|
||||
| 25 fps | 117 MB/s | comfortable |
|
||||
|
||||
The drive sustains ~156 MB/s (~145 MB/s after its 220 MB SLC burst), so 30 fps is
|
||||
close to the limit. Frame integrity is never fudged: `session.json` reports
|
||||
`dropped` (writer queue overflow) and `missed` (sensor frames that never reached
|
||||
disk, computed from `SensorTimestamp` gaps).
|
||||
|
||||
`record.limit_frame_duration` pins the sensor's frame duration to
|
||||
`1/target_fps`. Without it auto exposure can pick e.g. 66 ms in dim light and
|
||||
the recording silently runs at 15 fps. With it, AE compensates with gain
|
||||
instead. Set it `false` if you would rather have a bright, slow image.
|
||||
|
||||
## Reading raw frames on a desktop
|
||||
|
||||
```bash
|
||||
# whole session -> 16-bit .npy per frame (geometry read from session.json)
|
||||
python3 tools/unpack_raw.py /media/ssd/2026-10-30/21-33-05
|
||||
|
||||
# quick look: 8-bit PNG, contrast stretched
|
||||
python3 tools/unpack_raw.py --png frame_000001.raw
|
||||
|
||||
# 16-bit greyscale Bayer TIFF
|
||||
python3 tools/unpack_raw.py --tiff16 frame_000001.raw
|
||||
|
||||
# 16-bit RGB TIFF (bilinear demosaic of the 12-bit data)
|
||||
python3 tools/unpack_raw.py --rgb16 frame_000001.raw
|
||||
|
||||
# session metadata
|
||||
python3 tools/unpack_raw.py --info /media/ssd/2026-10-30/21-33-05
|
||||
```
|
||||
|
||||
Raw files are loadable without the tool as long as you know the geometry:
|
||||
|
||||
```python
|
||||
import numpy as np
|
||||
buf = np.fromfile("frame_000001.raw", dtype=np.uint8).reshape(1520, 3072)
|
||||
trip = buf[:, :2028 * 3 // 2].reshape(1520, 1014, 3).astype(np.uint16)
|
||||
b0, b1, b2 = trip[:, :, 0], trip[:, :, 1], trip[:, :, 2]
|
||||
even = (b0 << 4) | (b1 >> 4) # 12-bit unpacked, 0-4095
|
||||
odd = ((b1 & 0x0F) << 8) | b2
|
||||
image = np.empty((1520, 2028), np.uint16)
|
||||
image[:, 0::2], image[:, 1::2] = even, odd
|
||||
```
|
||||
|
||||
Note the byte order: in each 3-byte group byte0 holds bits 11..4 of the first
|
||||
pixel, byte1's **high** nibble holds its bits 3..0, and byte1's **low** nibble
|
||||
holds the top of the second pixel. Swapping those nibbles still yields values in
|
||||
0-4095, so it fails silently as colour-fringed mush — if your Python is
|
||||
producing that, this is why.
|
||||
|
||||
## The 16-bit TIFF on the photo button
|
||||
|
||||
The Pi's ISP only ever outputs 8-bit processed images, so a "16-bit TIFF of the
|
||||
ISP output" would just be 8-bit data in a 16-bit container. Instead the TIFF is
|
||||
built from the 12-bit Bayer frame through the same chain a raw converter uses:
|
||||
|
||||
unpack 12-bit MIPI -> black-level subtract -> white balance
|
||||
-> colour-correction matrix -> bilinear demosaic -> sRGB gamma -> 16-bit
|
||||
|
||||
so it opens as a normal, viewable 16-bit image rather than raw linear data. The
|
||||
untouched sensor data stays in the DNG beside it, so nothing is lost for
|
||||
quantitative work. The settings are in `config.json` under `photo`: set
|
||||
`tiff_apply_white_balance` / `tiff_apply_colour_matrix` / `tiff_apply_gamma` to
|
||||
`false` for a linear, un-white-balanced export instead. The camera's gains and
|
||||
matrix are recorded in `photo.json` either way.
|
||||
|
||||
## Layout of the software
|
||||
|
||||
```
|
||||
/opt/microscope-camera/
|
||||
├── app.py GTK3 kiosk UI: preview, buttons, badge, settings sliders
|
||||
├── camera.py picamera2 wrapper: video (raw+preview) and still configs
|
||||
├── recorder.py writer thread for raw frames + index.jsonl/session.json
|
||||
├── photos.py still capture (DNG/JPEG) and the 16-bit TIFF worker
|
||||
├── storage.py SSD mount state and the date/time/format session paths
|
||||
├── config.py defaults; config.json holds overrides
|
||||
├── config.json tunables (edit and restart the service to apply)
|
||||
├── install.sh reinstalls the app + system integration onto a fresh OS
|
||||
├── system/ copies of the installed systemd units, udev rule, helper
|
||||
└── tools/unpack_raw.py desktop-side unpacker / demosaicer
|
||||
```
|
||||
|
||||
The copies in `system/` are the source of truth for the machine-specific files;
|
||||
`install.sh` puts them back in place (see below).
|
||||
|
||||
System integration:
|
||||
|
||||
| Piece | Purpose |
|
||||
|---|---|
|
||||
| `/etc/systemd/system/microscope-camera.service` | runs `cage -s -- python3 app.py` fullscreen on tty1, `Restart=always` |
|
||||
| `/usr/local/sbin/microscope-ssd` | `mount` / `umount` / `status` helper (log: `/run/microscope-ssd.log`) |
|
||||
| `/etc/udev/rules.d/99-microscope-ssd.rules` | on plug-in, starts the mount service |
|
||||
| `/etc/systemd/system/microscope-ssd-mount.service` | oneshot mount unit triggered by udev |
|
||||
| `/etc/systemd/system/microscope-boot-check.service` | writes a boot self-test to `~/boot-check.log` + a screenshot |
|
||||
| `/etc/fstab` | `UUID=...` entry so the SSD also mounts at boot |
|
||||
| `/etc/sudoers.d/microscope-ssd` | lets the kiosk UI mount without a password |
|
||||
|
||||
Why the mount is split this way: udev workers run **without `CAP_SYS_ADMIN`**, so
|
||||
`mount` inside a `RUN+=` rule always fails with `permission denied`. The mount
|
||||
has to happen in a service. That service is deliberately oneshot *without*
|
||||
`RemainAfterExit`, otherwise it stays "active" after the first mount and systemd
|
||||
silently skips every later replug.
|
||||
|
||||
The SSD is mounted by three independent mechanisms, so a missing mount is always
|
||||
caught: the `fstab` entry mounts it at boot, the udev rule mounts it on hotplug,
|
||||
and the running app re-tries every few seconds as a backstop.
|
||||
|
||||
## Reinstalling
|
||||
|
||||
```bash
|
||||
sudo ./install.sh # packages, app, units, udev rule, fstab, enable
|
||||
```
|
||||
|
||||
`install.sh` is idempotent: it replaces the app and units, leaves an existing
|
||||
`config.json` alone, and only appends the fstab line if it is not already there.
|
||||
|
||||
## Operating
|
||||
|
||||
```bash
|
||||
systemctl status microscope-camera.service # is the UI running
|
||||
journalctl -u microscope-camera.service -f # live app log
|
||||
sudo -n /usr/local/sbin/microscope-ssd status # mounted | stale | unmounted | absent
|
||||
sudo systemctl restart microscope-camera # after editing config.json
|
||||
```
|
||||
|
||||
Touch buttons work directly. With a keyboard the same actions are on
|
||||
`r` (record), `p` (photo), `t` (rotate), `e` (settings panel) and `Esc`/`q` quit.
|
||||
|
||||
Recording refuses to start when the SSD is not mounted, so nothing silently
|
||||
fills the SD card. If the SSD is yanked mid-recording the writer notices within
|
||||
~0.5 s, stops the recording, and keeps whatever was already flushed.
|
||||
`stale` in the helper's status means a mount entry survived a yank; the next
|
||||
mount clears it automatically.
|
||||
|
||||
## Known limits
|
||||
|
||||
* **Light.** With the lens capped, AE wants 66 ms and the preview is black
|
||||
noise. Pin the frame duration (default) or add light for 30 fps.
|
||||
* The SSD's ~156 MB/s ceiling is the whole reason 40 fps raw is not possible at
|
||||
2028x1520. A faster drive would raise the ceiling.
|
||||
* Photos freeze the preview for ~2-3 s (sensor mode switch + settle + capture).
|
||||
The 74 MB TIFF is written by a background thread afterwards.
|
||||
* DNG comes from `pidng` (pure Python): ~0.6 s at 12 MP, fine for stills, far
|
||||
too slow for frames.
|
||||
Reference in New Issue
Block a user