Tellstone logo Tellstone

Development Roadmap

Where We're Going

A contest of focus. Keep yours made of steel.

1
Core Engine & Persistence Done
2
Security & Transport In Progress
3
Data Durability & Recovery Planned
4
Intelligence Layer Planned
5
Ecosystem & Operations Planned
6
Observability & Developer Experience Planned

Phase 1 — Core Engine & Persistence

Done

High-performance in-memory key/value store with shared-nothing architecture, zero-copy binary protocol, Redis-compatible RESP2, and per-shard WAL persistence.

Sharded In-Memory Engine Docs →
  • N shards (default GOMAXPROCS) with per-shard RWMutex
  • FNV-1a key hash routing — no cross-shard coordination
  • Configurable via --num-shards
Dual Protocol Support Docs →
  • Zero-copy binary protocol (GET/SET/DEL)
  • Redis-compatible RESP2 (PING/GET/SET/DEL)
  • Same storage engine behind both protocols
TTL Eviction Docs →
  • Active timing wheel — O(1) per-tick eviction
  • Lazy eviction on read for expired keys
  • Configurable --evict-interval and --evict-slots
At-Rest Encryption Docs →
  • ChaCha20-Poly1305 encryption, off by default
  • Optional --encryption-key flag
Observability Docs →
  • Built-in Prometheus metrics exporter
  • OpenTelemetry (OTLP/gRPC) tracing
Per-Shard WAL Persistence Docs →
  • Append-only write-ahead log per shard
  • TTL-aware replay on startup
  • Tombstone deletes and crash-safe truncation
  • Zero-allocation writes — enabled with --enable-persistence

Phase 2 — Security & Transport

In Progress

Production readiness hinges on transport encryption and authentication.

2a — TLS / mTLS Transport
  • TLS 1.3 support for binary and RESP2 listeners
  • Mutual TLS for service-to-service auth in Kubernetes
  • Automatic certificate rotation via filesystem watcher or K8s Secret projection
  • STARTTLS upgrade path for RESP connections
2b — Authentication & Authorization
  • AUTH command for RESP protocol (password-based)
  • AUTH handshake for binary protocol (challenge-response)
  • API key system with per-key ACLs
  • OIDC / OAuth2 integration for SSO
  • ACL SETUSER / DELUSER / LIST for runtime access rules
  • Audit logging
2c — Encryption Key Management
  • Key rotation command — re-encrypt all values in-place
  • Key derivation from environment / Vault / KMS
  • Envelope encryption — master key decrypts per-shard DEKs

Phase 3 — Data Durability & Recovery

Planned

WAL compaction, portable snapshots, and off-site backup.

3a — WAL Compaction & Optimization
  • Background WAL compaction — merge and deduplicate records
  • WAL file rotation — split by size or time
  • Incremental WAL replay for faster startup
3b — Full-Database Snapshots
  • BGSAVE command — fork-free snapshot to compact binary format
  • Snapshot integrity via xxHash / SHA-256 checksums
  • LASTSAVE command
  • Point-in-time recovery — snapshot + WAL replay
  • Snapshot streaming — SAVE TO STDOUT
3c — Backup Integration
  • Pluggable backup backends: local FS, S3, GCS, Azure Blob
  • Encrypted backups with ChaCha20-Poly1305
  • Backup rotation policies
  • RESTORE command to rehydrate an empty instance
3d — Better Persistence Backend
  • Optional write-through mode via SQLite / BoltDB
  • LSM-tree-inspired tiered storage
  • MIGRATE command — move keys between shards

Phase 4 — Intelligence Layer

Planned

Vector search designed into the shared-nothing architecture from the start.

4a — Vector Index Foundation
  • New vector shard type — fixed-dimension float32 embeddings
  • VADD / VGET / VDEL / VCOUNT commands
4b — Approximate Nearest Neighbor Search
  • VSEARCH with KNN query support
  • HNSW index — high-recall, low-latency
  • IVF with product quantization — memory-efficient for large datasets
  • Configurable distance metrics: cosine, L2, inner product
4c — Vector Metadata Filtering
  • SQL-like WHERE clause on metadata
  • Filtered HNSW — pre-filter at graph traversal time
  • Composite indexes — similarity + metadata filter in one query
4d — AI Integration Points
  • Bulk ingestion pipeline — VIMPORT FROM CSV/JSON
  • Streaming ingestion from Kafka / NATS / gRPC
  • Integration hooks for LLM pipelines
  • Dimension auto-detection from first inserted vector

Phase 5 — Ecosystem & Operations

Planned

Client SDKs, protocol extensions, cluster mode, and write-through adapters.

5a — Client SDKs
  • Go client — official, high-performance
  • Python client with async support
  • Node.js client — TypeScript-first
  • Connection pooling, auto-reconnect, pub/sub in all SDKs
5b — Protocol Extensions
  • RESP3 support
  • Memcached protocol support (binary and text)
  • gRPC service definition for structured RPC
5c — Cluster Mode
  • Consistent hashing across nodes
  • Peer-to-peer replication (Raft or CRDTs)
  • CLUSTER INFO / CLUSTER NODES for topology discovery
  • Automatic failover and rebalancing
5d — Write-Through to External DBs
  • PostgreSQL, MariaDB, MSSQL write-through adapters
  • Configurable write policies: sync, async, batched
  • Read-through caching

Phase 6 — Observability & Developer Experience

Planned

Built-in dashboard, query layer, and operational tooling.

6a — Built-in Dashboard
  • Lightweight embedded web UI (single binary)
  • Real-time shard stats: keys, memory, hit rate, ops/sec
  • TTL histogram, eviction rate, WAL lag visualization
  • Connection inspector
6b — Query Layer
  • Wire SQL parser into the server
  • Full SQL-like queries with WHERE, LIKE, TTL filters
  • Slow query logging with configurable threshold
  • EXPLAIN query plan output
6c — Operational Tooling
  • INFO command — comprehensive server statistics
  • CONFIG SET / CONFIG GET — runtime config changes
  • SLOWLOG — track and query slow operations
  • MEMORY DOCTOR — usage analysis and suggestions
  • Graceful shard migration for rolling upgrades

Design Principles

These guide every decision on this roadmap:

Zero-allocation hot path

Performance is the core value prop; never regress the hot path.

Shared-nothing first

Avoid shared state; coordinate only when absolutely necessary.

Optional everything

Every feature is opt-in, disabled by default, zero cost when off.

Single binary

No external dependencies; embed what you can, link what you must.

Security by default

TLS and auth should be trivially deployable, not an afterthought.

Cloud-native

Kubernetes-native config, graceful shutdown, Prometheus metrics, OTLP tracing.