Compression

To use this module, include the following line in your project file:

require engine.format.compress_core // or require engine.core

Decompress gzip, zlib, and raw-deflate byte streams.

Many binary formats wrap their payload in a compressed stream - for example Tiled can store its tile-layer data as base64 + gzip/zlib. decompress inflates such a stream into raw bytes. It is decompress-only: there is no compress side.

Usage example:

var packed : array<uint8>
// ... fill `packed` with a gzip/zlib/deflate stream, e.g. base64_decode of a Tiled data layer ...
var raw : array<uint8>
if (decompress(packed, CompressFormat.Auto, raw)) {
    // `raw` now holds the inflated bytes
}

decompress(src, format, out) inflates src into out and returns true on success. It returns false (leaving out empty) on a malformed or truncated stream, or on a valid stream that inflates past the size cap - a bad or oversized input is a normal caller-checked outcome, not an error. src and out must be different arrays (out is cleared before src is read, so passing the same array returns false).

decompress(src, format) is a convenience overload that returns the inflated bytes directly - an empty array means failure:

let raw <- decompress(packed, CompressFormat.Auto)

A stream that legitimately inflates to zero bytes is indistinguishable from failure there, so use the bool overload when that distinction matters.

The inflated output is capped at 16 MiB. A stream that would expand past that (for example a decompression bomb - deflate expands up to ~1000:1) returns false rather than exhausting memory, so it is safe to run on untrusted data. Only the first stream is decoded: a multi-member gzip yields just its first member and any bytes trailing a complete stream are ignored.

Formats

The CompressFormat argument selects how the stream header is interpreted:

  • Auto - detect zstd, zlib or gzip from the header (the usual choice)

  • Gzip - a gzip stream (RFC 1952, the .gz wrapper)

  • Zlib - a zlib stream (RFC 1950)

  • Deflate - a raw deflate stream with no header (RFC 1951)

  • Zstd - a Zstandard frame (RFC 8878)

Decompressing Tiled tile data

A Tiled layer exported as base64 + gzip/zlib/zstd combines two steps - first base64-decode the text, then decompress the bytes:

require engine.format.compress_core
require public_base64                     // base64_decode lives here

var packed : array<uint8>
base64_decode(layerDataString, packed)   // text -> compressed bytes
var tileBytes : array<uint8>
if (decompress(packed, CompressFormat.Auto, tileBytes)) {
    // tileBytes are the little-endian uint32 global tile ids
}

Enumerations

CompressFormat

Which compressed stream header decompress should expect.

Values:

  • Auto = 0 - Detect zstd, zlib or gzip automatically from the header.

  • Gzip = 1 - A gzip stream (RFC 1952, the .gz wrapper).

  • Zlib = 2 - A zlib stream (RFC 1950).

  • Deflate = 3 - A raw deflate stream with no header (RFC 1951).

  • Zstd = 4 - Zstandard frame (RFC 8878)

Functions

decompress(src: array<uint8>; format: CompressFormat): array<uint8>

Inflates a gzip, zlib, raw-deflate or zstd byte stream and returns the raw bytes.

Convenience overload: an empty result means failure (malformed or truncated stream, or output past the 16 MiB cap). A stream that legitimately inflates to zero bytes looks the same - use the decompress(src, format, out) : bool overload when that matters.

Arguments:

  • src : array<uint8> - the compressed bytes

  • format : CompressFormat - which stream header src carries (Auto/Gzip/Zlib/Deflate/Zstd)

Returns:

  • array<uint8> - the inflated bytes; empty on failure

Usage example:

let raw <- decompress(packed, CompressFormat.Auto)
if (length(raw) != 0) {
    // `raw` holds the inflated bytes
}
decompress(src: array<uint8>; format: CompressFormat; out: array<uint8>): bool

Inflates a gzip, zlib, raw-deflate or zstd byte stream into raw bytes.

format selects how the stream header is read: Auto detects zstd, zlib or gzip, Gzip / Zlib / Deflate / Zstd force a specific wrapper (Deflate is a headerless raw stream). On success the inflated bytes are written to out and the function returns true. It returns false and leaves out empty on a malformed or truncated stream, or on a valid stream whose output would exceed the 16 MiB cap - all normal caller-checked outcomes, not errors, so no message is logged. src and out must be different arrays. Only the first stream is decoded (trailing bytes and extra gzip members are ignored).

Arguments:

  • src : array<uint8> implicit - the compressed bytes

  • format : CompressFormat - which stream header src carries (Auto/Gzip/Zlib/Deflate/Zstd)

  • out : array<uint8> implicit - receives the inflated bytes (emptied on failure); must not alias src

Returns:

  • bool - true if the first stream inflated cleanly within the 16 MiB cap

Usage example:

var packed : array<uint8>
base64_decode(dataString, packed)         // e.g. a Tiled base64 tile-data layer
var raw : array<uint8>
if (decompress(packed, CompressFormat.Auto, raw)) {
    // `raw` holds the inflated bytes
}