Module encoding::types [] [src]

Interface to the character encoding.

Raw incremental interface

Methods which name starts with raw_ constitute the raw incremental interface, the lowest-available API for encoders and decoders. This interface divides the entire input to four parts:

The following figure illustrates an example of successive raw_feed calls:

1st raw_feed   :2nd raw_feed   :3rd raw_feed
----------+----:---------------:--+--+---------
          |    :               :  |  |
----------+----:---------------:--+--+---------
processed  unprocessed             |  remaining
                              problematic

Since these parts can span the multiple input sequences to raw_feed, raw_feed returns two offsets (one optional) with that the caller can track the problematic sequence. The first offset (the first usize in the tuple) points to the first unprocessed bytes, or is zero when unprocessed bytes have started before the current call. (The first unprocessed byte can also be at offset 0, which doesn't make a difference for the caller.) The second offset (upto field in the CodecError struct), if any, points to the first remaining bytes.

If the caller needs to recover the error via the problematic sequence, then the caller starts to save the unprocessed bytes when the first offset < the input length, appends any new unprocessed bytes while the first offset is zero, and discards unprocessed bytes when first offset becomes non-zero while saving new unprocessed bytes when the first offset < the input length. Then the caller checks for the error condition and can use the saved unprocessed bytes for error recovery. Alternatively, if the caller only wants to replace the problematic sequence with a fixed string (like U+FFFD), then it can just discard the first sequence and can emit the fixed string on an error. It still has to feed the input bytes starting at the second offset again.

Structs

CodecError

Error information from either encoder or decoder.

Enums

DecoderTrap

Trap, which handles decoder errors.

EncoderTrap

Traits

ByteWriter

Byte writer used by encoders. In most cases this will be an owned vector of u8.

Encoding

Character encoding.

RawDecoder

Decoder converting a byte sequence into a Unicode string. This is a lower level interface, and normally Encoding::decode should be used instead.

RawEncoder

Encoder converting a Unicode string into a byte sequence. This is a lower level interface, and normally Encoding::encode should be used instead.

StringWriter

String writer used by decoders. In most cases this will be an owned string.

Functions

decode

Determine the encoding by looking for a Byte Order Mark (BOM) and decoded a single string in memory. Return the result and the used encoding.

Type Definitions

DecoderTrapFunc

A type of the bare function in DecoderTrap values.

EncoderTrapFunc

A type of the bare function in EncoderTrap values.

EncodingRef

A trait object using dynamic dispatch which is a sendable reference to the encoding, for code where the encoding is not known at compile-time.