senaite.astm 2.0: HL7 next to ASTM, and tools for commissioning

Many clinical and veterinary analysers do not write result files. They hold a network or serial connection to a host system and send each result as a message, in ASTM (LIS2-A2) or HL7 v2. For these instruments, file imports do not apply. senaite.astm is the middleware for them: a small Python 3 service that listens for the instrument, parses each message and forwards it to SENAITE.

Version 2.0 is on the 2.x branch of the repository. It is not published on PyPI yet; install it from GitHub.

How the pieces fit

senaite.astm accepts the connection, handles the protocol handshake, and parses the message into a JSON envelope with one bucket per record type: header (H), patient (P), order (O), result (R), comment (C) and a few more. It posts the envelope to the SENAITE JSON API push endpoint.

SENAITE core receives the envelope through the push consumer senaite.core.lis2a.import. The consumer reads the sender name from the header and hands the envelope to an importer registered under that name. The base importer finds the instrument in SENAITE by model or title and serial number, and finds the sample by the ID in the message.

The importer for the instrument decides what goes where: which result record belongs to which analysis keyword, what to do with flags and units, whether to attach the raw message. This part is written per instrument model in the project add-on. senaite.astm knows how to parse an instrument’s messages; it does not know how your lab has named its analysis services.

That split is the most important thing to understand before planning an integration. The middleware is ready for the instruments it supports. The mapping to your analysis services is project work, usually a small amount once real messages are available.

Instruments with a parsing schema

Version 2.0 ships message schemas for Abbott Afinion 2, bioMérieux mini VIDAS, Siemens DCA Vantage, Cepheid GeneXpert, Hitachi 7600, Horiba Pentra XLR, Horiba Yumizen H5xx, Roche cobas c111 and c311, Spotchem EL (SE-1520), Sysmex XN and Sysmex XP. Instruments outside the list often speak close enough to standard ASTM that a new schema is a short piece of work, but that has to be checked against the instrument’s interface specification.

What is new in 2.0

An HL7 v2 listener. senaite-hl7-server listens for HL7 v2 over MLLP, by default on port 2575, and acknowledges every message on the transport level. It maps the segments into the same buckets the ASTM parser produces: MSH to H, PID to P, OBR to O, OBX to R, NTE to C. Fields are keyed by their HL7 field number, so OBX-3 is R[i]["3"]. Segments outside this list are kept under metadata.unmapped_segments, and the original message text under metadata.hl7. An importer can therefore treat ASTM and HL7 instruments alike.

One limit to know: the HL7 server’s default consumer name, senaite.core.hl7.import, is not registered by SENAITE core. Pushing HL7 results into SENAITE needs a consumer and importer on the project side. Without --url, the HL7 server runs capture-only.

Capture-only mode. senaite-astm-server --capture-only -o DIR accepts connections and writes every session to disk, without contacting SENAITE. Combining it with --url is rejected, so a misconfigured service fails at start instead of silently dropping messages.

Captures in normal operation. With -o DIR set, every session is also written to disk while results are forwarded, with microsecond timestamps in the file names. If SENAITE is down and the push retries are exhausted, the capture is still there to replay.

A replay and conversion tool. senaite-astm-send sends captured files to SENAITE again. Useful options: --dry-run shows what would be sent, --validate-only checks that a capture parses, --substitute-sample-id OLD=NEW retargets a capture to a test sample (with --rebuild-checksums, because the edit breaks the frame checksum), and --scrub-phi replaces every patient field not on a short allowlist with <REDACTED> before a capture goes to a test system.

An inspection tool. senaite-astm-inspect reads captures without sending anything: instrument shows which schema matched, summary counts the records per file, diff compares two parsed captures.

A stats endpoint. --admin-port opens a read-only HTTP endpoint whose GET /stats returns JSON with uptime_seconds, active_sessions, frames_dispatched and last_dispatch_at. It binds to 127.0.0.1 by default, because it has no authentication.

Operational hardening. The push to SENAITE has a timeout (10 seconds to connect, 60 to read) and retries (-r, default 3, -d, default 5 seconds apart); 401 and 403 responses are not retried, since they need new credentials, not patience. On SIGTERM the server waits up to --shutdown-grace-seconds (default 30) for messages in flight.

Commissioning an instrument

This is the order we follow when a new analyser is connected.

1. Capture first. Run the server in capture-only mode on the instrument network and let the instrument send a normal day’s work, including a QC run and a repeat measurement.
2. Inspect. senaite-astm-inspect summary and instrument show whether the schema matches and what the messages contain: where the sample ID sits, which test codes the results carry, which flags appear.
3. Write the importer. Map the instrument’s test codes to the lab’s analysis keywords in the project add-on, using the real captures as test data.
4. Replay against a test system. senaite-astm-send with --substitute-sample-id and --scrub-phi pushes the captures into a test SENAITE with test samples, until every result lands where it should.
5. Go live with -o and --url both set, so every message is kept on disk as well as forwarded.

Running it

A minimal production start of the ASTM listener:

senaite-astm-server \
    -p 4010 \
    -o /var/lib/senaite-astm/captures \
    -u https://user:password@lims.example.com/senaite \
    --admin-port 4011 \
    --logfile /var/log/senaite-astm/server.log

The repository’s deployment notes contain a systemd unit and a supervisord section. Run the listener as its own user, firewall the listening port to the instrument network, and keep the credentials out of the unit file. Give the listener a SENAITE account of its own, so its imports are recognisable in the audit log.

For monitoring, poll /stats: uptime_seconds should grow between calls, and last_dispatch_at should not be older than the usual gap between messages during working hours. An instrument that stopped sending is noticed by the lab eventually; an alert notices it first.

When to talk to us

If you plan to connect analysers that speak ASTM or HL7, send us the instrument model and its host interface specification first. That tells us whether a schema exists, what the importer needs, and how much of the work is configuration. Get in touch.