How LZW Compression Technique Works: Compression Explained

1. Introduction and Core Concepts

The lzw compression technique is one of the most influential lossless data compression algorithms in computer science history. Developed by Abraham Lempel, Jacob Ziv, and Terry Welch in 1984 as an improved version of the LZ78 algorithm, LZW has become a foundational method for reducing file sizes without losing any information. Unlike lossy techniques that discard data, the lzw compression technique guarantees perfect reconstruction of the original input, making it ideal for text, executables, and images where every bit matters.

At its core, the lzw compression technique is a dictionary-based compression method. It operates on a simple but powerful principle: frequently occurring patterns and substrings in data should be replaced with shorter codes. Instead of transmitting or storing repetitive sequences repeatedly, the algorithm builds a dynamic dictionary of strings encountered during processing and substitutes them with fixed-length or variable-length integer codes. This adaptive approach means no prior knowledge of the input statistics is required, unlike Huffman coding which needs frequency analysis beforehand.

What makes the lzw compression technique particularly elegant is its universality and simplicity. It is classified as an adaptive, lossless, dictionary coder that requires only a single pass over the data. Both the compressor and decompressor start with the same initial dictionary — typically all 256 possible single-byte characters — and build identical dictionaries synchronously as they process the data. This eliminates the need to transmit the dictionary itself, which is a significant overhead saving. The decoder can reconstruct the dictionary solely from the stream of codes it receives, a property that makes LZW exceptionally efficient for streaming and file-based applications.

Why LZW Matters Today

Despite being nearly four decades old, understanding the lzw compression technique remains essential for every software engineer and computer scientist. It forms the basis for ubiquitous file formats such as GIF, TIFF, and PDF, and was historically the standard for the Unix compress utility. Its influence extends to modern algorithms like DEFLATE (used in ZIP, GZIP, and PNG), which combines LZ77 with Huffman coding but inherits the same dictionary philosophy pioneered by LZW.

From an educational standpoint, LZW beautifully illustrates key concepts in data structures, including hash tables, trie-like dictionary management, and amortized analysis. It also demonstrates the critical trade-off between compression ratio, speed, and memory usage — a recurring theme in systems design. For interviews and practical engineering, knowing how LZW works helps you reason about when to apply dictionary compression versus entropy coding, and how to handle real-world constraints like fixed memory and variable-length codes.

2. How It Works

The mechanism of the lzw compression technique can be understood as a continuous process of pattern discovery and code substitution. The algorithm maintains a dictionary that maps strings to integer codes. Initially, this dictionary contains all possible single-character strings. For standard 8-bit data, this means entries 0-255 correspond to bytes 0x00 to 0xFF. The first available code for a new multi-character string is 256, and the dictionary grows from there as new patterns are discovered.

Compression proceeds sequentially through the input. The algorithm maintains a current pattern buffer, often called w, which represents the longest string seen so far that already exists in the dictionary. It then reads the next character c from the input and checks if the concatenated string w + c is present in the dictionary. If it is, then w is extended to w + c and the process continues without outputting anything. This greedy extension is crucial — it ensures we always match the longest possible dictionary entry. If w + c is not in the dictionary, the algorithm outputs the code for w, adds w + c to the dictionary with the next available code, and resets w to c to start a new match.

Decompression is the inverse process and is remarkably clever because it reconstructs the dictionary without any additional information. The decompressor starts with the same initial 256-entry dictionary. It reads the first code, translates it to a string, and outputs it. For each subsequent code k, it looks up the string entry corresponding to k. If k exists in the dictionary, entry is found directly. There is a special edge case where k equals the next code to be added — this occurs when the compressor encounters a pattern like cScSc where cSc is in the dictionary but cScSc is not. In this case, entry is constructed as w + w[0], where w is the previous decoded string. After outputting entry, the decompressor adds w + entry[0] to its dictionary, perfectly mirroring the compressor's behavior.

The Dictionary as a Learning Model

You can think of the LZW dictionary as a model that learns the language of the input file as it goes. Early in the file, compression is poor because the dictionary is small and most codes represent single characters. As more text is processed, longer phrases like " the ", " and ", or "compression" get added and subsequently replaced by single 12-bit codes, yielding significant savings. This adaptivity is why LZW performs exceptionally well on data with high repetition, such as source code, log files, and simple raster images, but may offer little benefit on already random or encrypted data.

A critical implementation detail is code width management. Early Unix implementations used a fixed 12-bit code (allowing 4096 dictionary entries). Modern variants start with 9-bit codes (512 entries) and grow to 12 or even 16 bits as the dictionary fills. When the dictionary becomes full, the algorithm must handle it — either by freezing the dictionary, flushing it and starting over, or using a least-recently-used eviction policy. The choice directly impacts compression ratio and is a key differentiator among LZW variants.

3. Implementation

Implementing the lzw compression technique in Python provides excellent insight into its dictionary management and streaming nature. The core data structure is a hash map (Python dict) for O(1) average-time lookups. While a trie would be more memory-efficient for large dictionaries, a hash table is simplest and fastest for typical implementation sizes up to 4096 entries.

Below is a clean, canonical implementation of the compression phase. It initializes the dictionary with all 256 extended ASCII characters, iterates through the input string, and emits integer codes. Note how the dictionary grows dynamically and how the longest match is greedily extended.

# LZW Compression - Core Algorithm
def lzw_compress(uncompressed: str, max_table_size: int = 4096) -> list[int]:
    """
    Compress a string using the lzw compression technique.
    Returns a list of integer codes.
    """
    # Initialize dictionary with all single-byte strings
    dict_size = 256
    dictionary = {chr(i): i for i in range(dict_size)}

    w = ""  # Current longest prefix found in dictionary
    result = []

    for c in uncompressed:
        wc = w + c
        if wc in dictionary:
            # Extend the pattern - longest match continues
            w = wc
        else:
            # Output code for w, add new pattern to dictionary
            result.append(dictionary[w])
            # Add wc to dictionary if space remains
            if dict_size < max_table_size:
                dictionary[wc] = dict_size
                dict_size += 1
            w = c

    # Output code for remaining w
    if w:
        result.append(dictionary[w])

    return result

Decompression must handle the special case where the received code is not yet in the dictionary. This happens only when the pattern is of the form w + w[0] and is a direct consequence of the compressor's one-code delay in adding new entries. A correct decompressor must explicitly check for this condition.

# LZW Decompression - Mirror of Compression
def lzw_decompress(compressed: list[int], max_table_size: int = 4096) -> str:
    """
    Decompress a list of integer codes to a string.
    Handles the special KwKwK edge case.
    """
    if not compressed:
        return ""

    dict_size = 256
    dictionary = {i: chr(i) for i in range(dict_size)}

    # First code is always a single character
    w = chr(compressed[0])
    result = [w]

    for k in compressed[1:]:
        if k in dictionary:
            entry = dictionary[k]
        elif k == dict_size:
            # Special case: KwKwK pattern (e.g., "AAA")
            # Entry must be w + w[0]
            entry = w + w[0]
        else:
            raise ValueError(f"Bad compressed code: {k}")

        result.append(entry)

        # Add new entry to dictionary: w + first char of entry
        if dict_size < max_table_size:
            dictionary[dict_size] = w + entry[0]
            dict_size += 1

        w = entry

    return "".join(result)

For production use, you need to manage variable-width codes and dictionary overflow. The following class encapsulates a more robust version that tracks code width and implements a dictionary reset strategy when the table fills, which prevents degraded compression on long inputs.

# Production-style LZW with Variable Code Width and Reset
class LZWCodec:
    def __init__(self, max_bits: int = 12):
        self.max_bits = max_bits
        self.max_table_size = 1 << max_bits  # 2**max_bits, e.g., 4096
        self.clear_code = None  # Optional: reserve code for reset

    def compress_with_stats(self, data: str):
        dict_size = 256
        dictionary = {chr(i): i for i in range(dict_size)}
        w = ""
        codes = []
        code_width = 9  # Start with 9 bits (512 entries)
        next_width_threshold = 1 << code_width

        for c in data:
            wc = w + c
            if wc in dictionary:
                w = wc
            else:
                codes.append((dictionary[w], code_width))
                if dict_size < self.max_table_size:
                    dictionary[wc] = dict_size
                    dict_size += 1
                    # Increase code width when threshold crossed
                    if dict_size >= next_width_threshold and code_width < self.max_bits:
                        code_width += 1
                        next_width_threshold <<= 1
                else:
                    # Dictionary full: reset (GIF-style) or freeze
                    dictionary = {chr(i): i for i in range(256)}
                    dict_size = 256
                    code_width = 9
                    next_width_threshold = 1 << code_width
                w = c

        if w:
            codes.append((dictionary[w], code_width))
        return codes

Finally, to store compressed data efficiently, you must pack integer codes into a byte stream. Since codes are 9-12 bits, they do not align to byte boundaries. The helper below demonstrates bit-packing, which is essential for achieving real file-size reduction rather than just producing a list of larger integers.

# Bit-packing LZW codes into bytes (for file storage)
def pack_codes(codes_with_width):
    """Pack variable-width codes into a byte array."""
    bit_buffer = 0
    bits_in_buffer = 0
    output = bytearray()

    for code, width in codes_with_width:
        bit_buffer = (bit_buffer << width) | code
        bits_in_buffer += width
        # Flush full bytes
        while bits_in_buffer >= 8:
            bits_in_buffer -= 8
            byte = (bit_buffer >> bits_in_buffer) & 0xFF
            output.append(byte)
            bit_buffer &= (1 << bits_in_buffer) - 1 if bits_in_buffer > 0 else 0

    # Flush remaining bits with padding
    if bits_in_buffer > 0:
        byte = (bit_buffer << (8 - bits_in_buffer)) & 0xFF
        output.append(byte)

    return bytes(output)

# Example usage
# original = "TOBEORNOTTOBEORTOBEORNOT"
# codec = LZWCodec(max_bits=12)
# packed = pack_codes(codec.compress_with_stats(original))
# print(f"Original: {len(original)} bytes, Packed: {len(packed)} bytes")

4. Step-by-Step Example

Walking through a concrete example is the best way to internalize the lzw compression technique. Let us compress the classic string ABABABA using an initial dictionary containing A=0, B=1 for simplicity (in practice it would be 0-255). We will use a small dictionary that can grow indefinitely for this illustration. The algorithm processes the input character by character, maintaining w and emitting codes only when a new pattern is discovered.

Initialization: Dictionary = { "A":0, "B":1 }, Next Code = 2, w = "", Output = [] . The input pointer is at the first character.

Step 1: Read 'A'. wc = "" + "A" = "A" which is in dictionary. So w = "A". No output.

Step 2: Read 'B'. wc = "A" + "B" = "AB" not in dictionary. Emit code for w ("A" = 0). Add "AB" as code 2. Set w = "B". Output = [0]. Dictionary = {A:0, B:1, AB:2}.

Step 3: Read 'A'. wc = "B" + "A" = "BA" not in dictionary. Emit code for "B" = 1. Add "BA" as code 3. w = "A". Output = [0,1].

Step 4: Read 'B'. wc = "A" + "B" = "AB" which IS in dictionary (code 2). So w = "AB". No output.

Step 5: Read 'A'. wc = "AB" + "A" = "ABA" not in dictionary. Emit code for "AB" = 2. Add "ABA" as code 4. w = "A". Output = [0,1,2].

Step 6: Read 'B'. wc = "A" + "B" = "AB" in dictionary. w = "AB". No output.

Step 7: Read 'A'. wc = "AB" + "A" = "ABA" in dictionary. w = "ABA". End of input. Emit code for "ABA" = 4. Final Output = [0,1,2,4]. The string of length 7 has been compressed to 4 codes. If codes are 2 bits each initially, this is a saving, and the dictionary now contains longer phrases that would be highly beneficial if the input continued with repeating "ABAB..." patterns.

Decompression Walkthrough

Decompressing [0,1,2,4] mirrors this exactly. Start with Dictionary {0:"A",1:"B"}, read 0 -> "A" output, w="A". Read 1 -> entry "B", output "B", add w+entry[0]="AB" as 2, w="B". Read 2 -> entry "AB", output "AB", add "B"+"A"="BA" as 3, w="AB". Read 4 -> 4 == dict_size (4), so special case: entry = w + w[0] = "AB" + "A" = "ABA", output "ABA", add "AB"+"A"="ABA" as 4, w="ABA". Result "A"+"B"+"AB"+"ABA" = "ABABABA" — perfect reconstruction.

This example highlights the greedy longest-match property and the one-step delay in dictionary updates. The compressor only adds a new string after it has emitted the code for its prefix, which is why the decompressor can reconstruct the same addition using the previous decoded string and the first character of the current entry.

5. Complexity Analysis

Analyzing the lzw compression technique requires examining both time and space behavior for compression and decompression. Unlike sorting algorithms with clear best and worst cases based on input order, LZW's performance depends heavily on input redundancy and dictionary size limits. The operations are dominated by dictionary lookups and insertions, which are typically implemented with hash tables.

For compression, each input character is processed exactly once. For each character, we perform one hash table lookup for w + c and at most one insertion and one output operation. With a hash table, the average time for lookup and insertion is O(1), leading to an overall O(n) average time complexity where n is the input length. However, in the worst case of hash collisions (if a poor hash function is used or an adversary crafts input), lookups could degrade to O(m) where m is dictionary size, leading to O(n * m) worst-case time. Using Python's robust dict or a balanced trie avoids this in practice, so worst-case is often quoted as O(n) with good hashing or O(n log m) with tree-based dictionaries.

Space complexity is determined by the dictionary size. In the unbounded version, the dictionary can grow to O(n) distinct substrings in the worst case (e.g., an input like "ABCDEF..." with no repetition, each new wc is unique, so dictionary size grows linearly with n). Each entry stores a string whose total length across all entries can be O(n^2) if stored naively, but with trie or prefix-code representation it is O(n). In practice, implementations cap the dictionary at a fixed size such as 4096 (12-bit) or 65536 (16-bit) entries, making space O(1) bounded — typically a few hundred kilobytes. Decompression has identical time and space characteristics, as it performs the same number of dictionary operations in lockstep.

Best-case compression occurs with highly repetitive input like "AAAAA...A" or "ABABAB...". Here, the dictionary quickly learns long patterns, so the number of output codes is O(sqrt(n)) or even O(log n) in terms of distinct phrases, and each lookup succeeds in extending w, minimizing dictionary insertions. Average-case on natural text or source code yields moderate compression ratios of 2:1 to 3:1 with linear time. Worst-case compression ratio occurs on random or already-compressed data where no pattern repeats; LZW will actually expand the data because each input character becomes a code larger than 8 bits (e.g., 12-bit codes for 8-bit input = 50% expansion) and the dictionary fills with useless entries.

6. Variations and Optimizations

The original lzw compression technique has spawned numerous variations aimed at improving compression ratio, speed, or adaptability to different data types. The most common optimization is variable-width code management. Instead of fixing codes at 12 bits from the start, encoders begin with 9-bit codes (512 entries) and incrementally increase to 10, 11, and 12 bits as the dictionary grows past 512, 1024, and 2048 entries. This saves significant space early in the file when the dictionary is small and most codes are < 512. GIF, for example, starts with code sizes based on the initial color table size and grows up to 12 bits.

Dictionary management when full is another major variation point. Three strategies dominate: Freeze (stop adding entries and use the existing dictionary as static), Reset/Clear (flush the dictionary and restart from the 256 initial entries, often signaled by a special CLEAR code), and LRU Eviction (replace least-recently-used entries). Unix compress with LZW monitors compression ratio and resets when it degrades, while GIF simply continues with a full dictionary without reset, which can hurt ratio on heterogeneous files. Modern implementations often use a hybrid: monitor hit rate and reset when the dictionary becomes stale.

Other notable variants include LZC (LZW with variable code size used in Unix compress), LZMW and LZT which add more aggressive phrase concatenation (e.g., adding w + entry instead of w + entry[0]), and LZW with predictive or entropy coding of the output codes (e.g., applying Huffman coding to LZW codes). For specialized data, enhancements like delta filtering before LZW (for images) or using a larger initial dictionary (for text with common digrams preloaded) can boost ratios by 10-20%. Hardware implementations often use a content-addressable memory (CAM) or double-hashing to achieve O(1) lookups at gigabit speeds, trading memory for speed.

Practical Optimization Tips

For software implementations, use a hash table with open addressing or Python's dict for speed, but consider storing dictionary entries as (prefix_code, suffix_char) tuples rather than full strings to save memory — this is essentially a trie encoded as arrays. Use the "dictionary as arrays" technique: maintain prefix = [-1]*MAX_SIZE and suffix = [0]*MAX_SIZE and reconstruct strings by traversing backwards, which is both memory-efficient and cache-friendly. Also, buffer I/O and bit-packing carefully; naive bit-by-bit output can dominate runtime. Pre-allocating the output list and using local variable binding (e.g., dictionary_get = dictionary.get) can yield 20-30% speedups in Python.

7. Real-World Applications

The lzw compression technique achieved widespread adoption because it is fast, requires no prior statistics, and is relatively simple to implement in hardware and software. Its most famous application is the Graphics Interchange Format (GIF), introduced by CompuServe in 1987. GIF uses LZW to compress 8-bit indexed color image data. Each image's color palette (up to 256 colors) forms the initial dictionary, and LZW excels at compressing the large runs of identical pixels and repeating patterns typical in logos, icons, and simple animations. The patent on LZW (held by Unisys) famously led to the development of PNG as a patent-free alternative using DEFLATE, but GIF remains ubiquitous.

Beyond GIF, LZW is the core of the TIFF (Tagged Image File Format) compression option, where it is used for lossless storage of scanned documents and high-resolution images. The Unix compress utility and its file extension .Z are pure LZW implementations that were standard for decades for compressing text files, source code, and log archives. In the PDF specification, LZWDecode is one of the standard filters for compressing streams, often used for monochrome images and text. Even modern systems like the V.42bis modem standard for data compression over telephone lines use an LZW variant to transparently compress data in transit.

In embedded and hardware contexts, LZW's fixed-memory, single-pass nature makes it attractive for printers, fax machines, and storage controllers where RAM is limited and streaming is required. For example, many PostScript printers implement LZW decompression in firmware to handle compressed image data. While web servers today prefer GZIP/DEFLATE (LZ77 + Huffman) and Brotli for HTTP compression due to better ratios, understanding LZW helps engineers appreciate why dictionary compression is still relevant for columnar databases, search indexes, and in-memory caches where fast decompression and random access to compressed blocks are needed.

8. Comparison with Alternatives

Choosing a compression algorithm requires balancing ratio, speed, memory, and patent or complexity constraints. The lzw compression technique belongs to the LZ family (LZ77, LZ78, LZW, Lempel-Ziv variants) but competes with entropy coders like Huffman and arithmetic coding, as well as hybrid schemes like DEFLATE and modern codecs like Brotli and Zstandard.

LZ77, used in DEFLATE, GZIP, and ZIP, uses a sliding window and encodes matches as (distance, length) pairs rather than a growing dictionary. It generally achieves better compression than pure LZW on text because it can reference recent data with variable-length matches and is often combined with Huffman coding for the literals and lengths. Huffman coding alone is optimal for symbol-by-symbol entropy coding but cannot exploit long-range repetition like dictionary methods. LZW's strength is its simplicity and fast decompression — no Huffman tree to build or bit-level entropy decoding, just table lookups.

DEFLATE, which powers most web compression, combines LZ77 with Huffman coding and typically outperforms LZW by 20-40% on text and mixed content, which is why it superseded LZW for HTTP. However, DEFLATE is more complex to implement and slower to decode. Newer algorithms like Brotli and Zstandard use larger dictionaries, context modeling, and more sophisticated entropy coding to achieve even higher ratios, but they require more memory and CPU. For applications needing extremely fast decompression or hardware simplicity, LZW or its variant LZ4 (a very fast LZ77 derivative) may still be preferred over higher-ratio but slower codecs.

9. Common Mistakes and Best Practices

Implementing the lzw compression technique correctly is deceptively tricky, and several pitfalls can lead to subtle bugs, poor compression, or data corruption. The most frequent mistake is mishandling the special decompression case where the code equals the next dictionary index. Beginners often treat this as an error, causing decompression to fail on inputs with repeating patterns like "AAA" or "ABABABA". The correct handling — synthesizing w + w[0] — must be explicitly coded and tested with such edge cases.

Another common error is failing to synchronize code width between compressor and decompressor. If the compressor increases from 9 to 10 bits at code 512 but the decompressor does not, all subsequent codes will be misaligned and decoded as garbage. Both sides must increment width at exactly the same dictionary size threshold, and this must account for any reserved codes like CLEAR or END-OF-INFO in GIF. Similarly, not handling dictionary overflow consistently leads to divergence; if one side resets and the other freezes, decompression will be incorrect after the table fills.

Performance pitfalls include using string concatenation for dictionary keys in a naive way that creates many temporary strings, and not bounding the dictionary size, which can cause memory exhaustion on large or adversarial inputs (e.g., a file with all unique substrings). In Python, building the dictionary with full strings is fine for small inputs but for multi-megabyte files, the memory overhead becomes significant. Best practice is to cap the table (e.g., at 4096 or 8192 entries) and implement a clear strategy — either freeze or reset — and to store entries as (prefix_code, char) pairs if memory is critical.

Best Practices Checklist

• Always test round-trip correctness: decompress(compress(data)) == data for edge cases including empty string, single character, repetitive patterns ("AAAAA"), and random data.
• Use variable-width codes starting at 9 bits and grow deterministically; document the thresholds and handle the final flush of bit buffers with proper padding.
• Bound the dictionary and define overflow behavior explicitly. For GIF compatibility, support CLEAR code 256 and END code 257 handling.
• Profile bit-packing and I/O — they often dominate runtime more than dictionary lookups. Use bytearray and bitwise operations, not string-based bit manipulation.
• Consider whether LZW is the right tool: for highly random or already compressed data, LZW may expand the file. Check compression ratio and fall back to storing uncompressed if expansion is detected, as done in TIFF and GIF encoders.

10. Interview Questions

Interviewers frequently use the lzw compression technique to assess understanding of data structures, algorithm design, and systems trade-offs. The following questions cover conceptual understanding, implementation details, and applied reasoning.

Q1: Why does LZW not need to transmit its dictionary to the decompressor?

Answer: Both compressor and decompressor start with the identical initial dictionary (typically all 256 single-byte values). The compressor adds a new entry w + c to the dictionary immediately after emitting the code for w when w + c is not found. The decompressor, upon decoding w and then the next entry, can deduce the same w + c string because it knows w (previous decoded string) and the first character of the next decoded entry c = entry[0]. Thus it can reconstruct the exact same sequence of dictionary additions in lockstep without any side channel. This synchronous adaptive property is what makes LZW so efficient — the dictionary is implicit in the code stream.

Q2: Explain the special case during decompression where the code is not yet in the dictionary. When does it occur?

Answer: This occurs when the input contains a pattern of the form cScSc where cS is already in the dictionary but cSc is not, and the compressor emits the code for cSc immediately after adding it. Concretely, for input "AAA" with dictionary {A:0}, compression emits [0] for "A", adds "AA" as 1, then sees "AA" in dictionary, reads next "A" to get "AAA" not in dictionary, emits code 1 for "AA", and adds "AAA" as 2. The decompressor receives [0,1] — after decoding 0 as "A" and adding "AA", the next code 1 is found, but if the sequence were [0,2] where 2 is the next code to be added, the decompressor would see a code equal to dict_size. The entry must be w + w[0]. For example, compressing "ABABABA" can trigger this. The fix is: if k == dict_size, entry = w + w[0].

Q3: What is the time and space complexity of LZW compression, and how does bounding the dictionary change it?

Answer: With a hash table dictionary, compression is O(n) average time where n is input length, because each character causes O(1) lookup and at most one insertion. Worst-case with poor hashing is O(n*m) where m is dictionary size, but with good hashing it remains O(n). Space is O(m) where m is dictionary entries; unbounded it can be O(n) in the worst case (all substrings unique). Bounding the dictionary to a fixed size like 4096 (12-bit codes) makes space O(1) — constant bounded memory — and keeps time O(n) in practice. The trade-off is compression ratio: a bounded dictionary may become stale on long heterogeneous files, so strategies like reset or freeze are used to maintain ratio while keeping memory constant.

Q4: How does LZW compare to Huffman coding? When would you choose one over the other?

Answer: LZW is a dictionary coder that replaces repeating multi-character patterns with codes, exploiting long-range repetition. Huffman is an entropy coder that assigns shorter bit codes to frequent single symbols based on frequency distribution, achieving optimal prefix coding for known symbol probabilities. LZW is adaptive and single-pass, needs no prior statistics, and excels on data with repeated phrases (text, simple images). Huffman needs either a prior scan to build frequencies or adaptive updates, and only compresses based on symbol frequency, not patterns. In practice, they are complementary: DEFLATE combines LZ77 (dictionary) with Huffman (entropy) to get both benefits. Choose LZW/Huffman alone when simplicity and speed matter (e.g., GIF, embedded); choose hybrid like DEFLATE/Brotli when maximum ratio is needed for web or archival.

Q5: How would you handle compressing a 10 GB log file with LZW in limited RAM?

Answer: You cannot use an unbounded dictionary — it would exhaust memory. Use a bounded dictionary (e.g., 12-bit = 4096 entries) with a reset strategy: monitor compression ratio or dictionary hit rate, and when the table fills or ratio degrades, emit a CLEAR code and reinitialize the dictionary to 256 entries. Process the file in streaming fashion, reading and compressing in chunks (e.g., 64KB blocks) to keep memory constant. Use variable-width codes to avoid wasting bits early, and bit-pack output to bytes incrementally rather than buffering all codes. For even better handling of large heterogeneous logs, consider block-wise reset or using a sliding-window variant like LZ77 which naturally adapts to local repetition without a global dictionary. Also detect incompressible blocks and store them uncompressed to avoid expansion.