REM 1.0

Secure OTA Encryption Method

AES-256-CFB + HMAC-SHA256 + HKDF

Abstract

REM (Secure OTA Encryption Method) is a custom encryption protocol designed for secure over-the-air (OTA) firmware updates on resource-constrained devices such as the ESP32. It provides both confidentiality and integrity by combining AES-256-CFB encryption with HMAC-SHA256 authentication and HKDF-SHA256 key derivation. Unlike AEAD schemes that require buffering the full ciphertext for tag verification, REM supports streaming decryption and tag verification in chunks, making it suitable for large patches where device RAM is limited. This document specifies the REM 1.0 wire format, algorithms, security properties, and comparison with related methods.

Introduction and problem statement

Firmware updates delivered over the air must be confidential (so only the target device can read them) and authentic (so tampering is detected). Standard solutions such as AES-GCM or ChaCha20-Poly1305 provide both, but verifying their authentication tag typically requires the full ciphertext to be available, which on memory-limited embedded devices (e.g. ESP32) can be impractical for large patches. The project therefore needs an encryption method that (1) provides confidentiality and integrity, (2) allows decryption and verification in a streaming fashion with bounded RAM, and (3) uses separate keys for encryption and authentication. REM 1.0 is a protocol that meets these requirements by using AES-256-CFB for streaming encryption, HMAC-SHA256 for an integrity tag over nonce and ciphertext, and HKDF-SHA256 to derive distinct AES and HMAC keys from a single device key.

What is REM?

REM (Secure OTA Encryption Method) is this project’s encryption protocol for over-the-air firmware patches. It provides confidentiality and integrity: data is encrypted so only the device can read it, and a tag ensures that any change to the ciphertext is detected before use.

Confidentiality: AES-256-CFB encrypts the patch so only the device with the shared key can decrypt it.
Integrity: HMAC-SHA256 over nonce and ciphertext; the device verifies the tag before accepting data.
Key separation: One 32-byte device key is turned into two keys (AES and HMAC) via HKDF-SHA256 so encryption and authentication use different keys.
Streaming: Decryption and HMAC run in chunks on the ESP32, so large patches don’t require loading the whole file into RAM.

Methods and algorithms used

REM is a protocol that uses these standard algorithms (no custom ciphers or hashes):

Component	Algorithm	Role in REM
Symmetric encryption	AES-256-CFB	Encrypts/decrypts patch; 256-bit key, 16-byte nonce, segment size 128. Allows streaming on the device.
Message authentication	HMAC-SHA256	32-byte tag over (nonce ‖ ciphertext). Device verifies before using data.
Key derivation	HKDF-SHA256	Derives two 32-byte keys from device key: one for AES, one for HMAC. Salt: REM1-OTA-PATCH, context: REM1.
Hashing (inside HMAC)	SHA-256	Used inside HMAC for the integrity tag.
Signatures (with REM)	ECDSA P-256	Server signs plaintext patch; device verifies. SHA-256 hashes message before signing.
Randomness	CSPRNG	Server generates a random 16-byte nonce per encryption.

How REM works (step-by-step)

Server (encryption):

Start with the 32-byte device key (shared with the ESP32).
Run HKDF-SHA256 to get aes_key and hmac_key.
Generate a random 16-byte nonce.
Encrypt the plaintext patch with AES-256-CFB using aes_key and the nonce → ciphertext.
Compute tag = HMAC-SHA256(hmac_key, nonce ‖ ciphertext).
Output: REM1 (4 bytes) ‖ nonce (16) ‖ ciphertext ‖ tag (32).

Device (decryption):

Check magic bytes are REM1, then read the 16-byte nonce.
Derive the same aes_key and hmac_key from the device key via HKDF.
Read ciphertext in chunks; for each chunk: update HMAC, decrypt with AES-CFB, write plaintext (streaming).
Read the final 32 bytes as the stored tag; finish HMAC and compare. If mismatch, reject (integrity failure).
If match, accept the decrypted data (and in this project, verify ECDSA signature over the plaintext separately).

Wire format

Every REM 1.0 payload has this layout:

Field	Size	Description
Magic	4 bytes	`REM1` (ASCII)
Nonce	16 bytes	Random IV for AES-CFB
Ciphertext	N bytes	AES-256-CFB(nonce, plaintext)
Tag	32 bytes	HMAC-SHA256(hmac_key, nonce ‖ ciphertext)

Total length = 4 + 16 + N + 32 = 52 + plaintext length.

Parameter summary

Parameter	Value
Protocol version	REM 1.0 (magic `REM1`)
Device key size	256 bits (32 bytes)
AES key size	256 bits (32 bytes, derived)
HMAC key size	256 bits (32 bytes, derived)
Nonce (IV) size	128 bits (16 bytes)
AES block size	128 bits
CFB segment size	128 bits
Tag size	256 bits (32 bytes)
HKDF salt	`REM1-OTA-PATCH` (16 bytes, fixed)
HKDF context	`REM1` (4 bytes)

Key derivation

From the 32-byte device key K:

Salt: REM1-OTA-PATCH (fixed).
HKDF-SHA256 with context REM1 yields two 32-byte keys: aes_key, hmac_key.

AES key is used for AES-256-CFB; HMAC key is used for the tag. Signatures are unchanged: ECDSA P-256 over SHA-256 hash of the plaintext patch (server signs before encrypting).

Comparison with other methods

How REM 1.0 compares to other common encryption designs:

Method	Confidentiality	Integrity / authentication	Streaming decryption	Typical use
AES-256-CFB only	Yes	No	Yes	Legacy; no integrity check
REM 1.0 (this project)	Yes	Yes (HMAC)	Yes	OTA patches on ESP32
AES-256-GCM	Yes	Yes (AEAD)	No*	TLS, general AEAD
ChaCha20-Poly1305	Yes	Yes (AEAD)	No*	TLS, mobile/embedded

* Full ciphertext typically needed for tag verification; streaming is possible with extra design.

How REM is different from others

Vs plain AES-CFB: REM adds an HMAC tag and key derivation. AES-CFB alone gives confidentiality but no integrity; REM adds authenticate-then-use so tampering is detected.
Vs AES-GCM / ChaCha20-Poly1305: Those are AEAD (one primitive for encrypt+auth). REM combines AES-CFB (streaming) with a separate HMAC so the device can decrypt and verify in chunks without buffering the whole file—important for large OTA patches on memory-limited ESP32.
Vs “encrypt then MAC” with one key: REM derives two keys (AES and HMAC) from one device key via HKDF, so encryption and authentication use different keys (recommended practice).
Versioning: The REM1 magic lets the project introduce a future REM 2.0 without breaking devices that only accept REM1.

Why REM for this project?

Integrity: Plain AES-CFB does not detect tampering; REM adds an HMAC tag so modified ciphertext is rejected.
Key separation: HKDF gives distinct keys for encryption and MAC, which is good practice.
Streaming: ESP32 can decrypt and verify in chunks, so REM fits the same low-RAM OTA flow as before.
Versioning: The REM1 magic allows future protocol versions without breaking old devices.

Security properties and threat model

What REM protects against:

Eavesdropping: AES-256-CFB provides confidentiality; without the device key an attacker cannot recover plaintext from ciphertext.
Tampering: HMAC-SHA256 provides integrity; any change to nonce or ciphertext invalidates the tag and the device rejects the payload.
Key reuse across roles: Encryption and authentication use different keys derived via HKDF (key separation).

Assumptions and scope: The device key is shared securely and kept secret; REM does not protect against its compromise. The nonce must be unique per encryption (server uses CSPRNG). REM does not provide non-repudiation by itself; in this project ECDSA signatures over the plaintext provide server authenticity.

Implementation context

Server: Python (FastAPI); crypto_utils.rem_encrypt_patch(); PyCryptodome for HKDF, AES, HMAC-SHA256.

Device: ESP32 firmware (C++, mbedTLS); decryption and HMAC verification in chunks (e.g. 1024 bytes) so large patches do not require full file in RAM.

Application: REM encrypts delta (bsdiff) patches; server signs plaintext with ECDSA then encrypts with REM; device verifies tag, stream-decrypts, then verifies ECDSA on plaintext before applying the patch.

Limitations

Nonce reuse: Same nonce with same key weakens confidentiality; server must use a unique nonce per encryption (e.g. CSPRNG).
No built-in key rotation: Key rotation requires updating the device key on both server and device; REM does not define a key-rotation protocol.
Tag verification: Tag is verified after full ciphertext is processed; if verification fails, decrypted output must be discarded.

Conclusion

REM 1.0 is a custom encryption protocol that provides confidentiality and integrity for OTA firmware patches while supporting streaming decryption and tag verification on resource-constrained devices. It uses only standard algorithms (AES-256-CFB, HMAC-SHA256, HKDF-SHA256) and separates encryption and authentication keys via HKDF. The wire format is versioned with a magic header for future evolution. REM is implemented in this project for secure delta updates on the ESP32.

References

NIST SP 800-38A, Recommendation for Block Cipher Modes of Operation: Methods and Techniques (AES-CFB).
RFC 2104, HMAC: Keyed-Hashing for Message Authentication.
RFC 5869, HMAC-based Extract-and-Expand Key Derivation Function (HKDF).
NIST FIPS 186-4 and related standards for ECDSA (P-256) and SHA-256 used for signatures in this project.

Full protocol specification: REM_SPEC.md.