Neurarc

A binary format for connectomes

Neural network data — connectivity, populations, model parameters — usually lives in HDF5. That works, but it’s heavy: a big dependency, and overhead you pay for on every open. ARC is a compact binary format designed as a lighter alternative for the cases where HDF5 is overkill.

The format

┌───────────────┬──────────────┬─────────────────┬──────────────┐
│  Magic (4 B)  │ Header Len   │  Header JSON    │  Binary Blob │
│               │  (8 B, LE)   │  (UTF-8)        │              │
└───────────────┴──────────────┴─────────────────┴──────────────┘

• 10 bytes per synapse in edge-list form; 6 bytes in CSR.
• Little-endian; supports u8, u32, f16, f32.
• Header as JSON, payload as binary — so metadata is inspectable without a format-specific reader, and the numeric data stays dense.

The measured tradeoff vs HDF5

I benchmarked ARC directly against HDF5 rather than asserting it’s better:

Claim	Status
ARC is smaller	True — 17% smaller
ARC opens faster	True — 106× faster with mmap
ARC reads faster	False — HDF5 wins on full data access
ARC is simpler	True — no HDF5 dependency
ARC writes faster	True — 1.9× faster

ARC’s niche: simpler, smaller, lower overhead for metadata and partial access. It is not a replacement for HDF5 at scale — the benchmark numbers make that clear, and the README says so.

Multi-language

The spec is implemented twice, with identical behavior:

# Python
pip install neurarc

# Rust
cargo add neurarc

The ARC spec

The full binary layout is specified in ARC_SPEC.md, versioned, so the format is a real standard rather than an implementation detail — the thing you’d hand to a team (or another species of brain) that needs to read the same data.