docarray.array.mixins.io.binary module#

class docarray.array.mixins.io.binary.LazyRequestReader(r)[source]#

Bases: object

class docarray.array.mixins.io.binary.BinaryIOMixin[source]#

Bases: object

Save/load an array to a binary file.

classmethod load_binary(file, protocol='pickle-array', compress=None, _show_progress=False, streaming=False, *args, **kwargs)[source]#

Load array elements from a compressed binary file.

Parameters:

  • file (Union[str, BinaryIO, bytes, Path]) – File or filename or serialized bytes where the data is stored.

  • protocol (str) – protocol to use

  • compress (Optional[str]) – compress algorithm to use

  • _show_progress (bool) – show progress bar, only works when protocol is pickle or protobuf

  • streaming (bool) – if True returns a generator over Document objects.

In case protocol is pickle the Documents are streamed from disk to save memory usage :rtype: Union[DocumentArray, Generator[Document, None, None]] :return: a DocumentArray object

Note

If file is str it can specify protocol and compress as file extensions. This functionality assumes file=file_name.$protocol.$compress where $protocol and $compress refer to a string interpolation of the respective protocol and compress methods. For example if file=my_docarray.protobuf.lz4 then the binary data will be loaded assuming protocol=protobuf and compress=lz4.

classmethod from_bytes(data, protocol='pickle-array', compress=None, _show_progress=False, *args, **kwargs)[source]#
Return type:

T

save_binary(file, protocol='pickle-array', compress=None)[source]#

Save array elements into a binary file.

Parameters:

  • file (Union[str, BinaryIO]) – File or filename to which the data is saved.

  • protocol (str) – protocol to use

  • compress (Optional[str]) –

    compress algorithm to use

    Note

    If file is str it can specify protocol and compress as file extensions. This functionality assumes file=file_name.$protocol.$compress where $protocol and $compress refer to a string interpolation of the respective protocol and compress methods. For example if file=my_docarray.protobuf.lz4 then the binary data will be created using protocol=protobuf and compress=lz4.

Comparing to save_json(), it is faster and the file is smaller, but not human-readable.

Note

To get a binary presentation in memory, use bytes(...).

Return type:

None

to_bytes(protocol='pickle-array', compress=None, _file_ctx=None, _show_progress=False)[source]#

Serialize itself into bytes.

For more Pythonic code, please use bytes(...).

Parameters:

  • _file_ctx (Optional[BinaryIO]) – File or filename or serialized bytes where the data is stored.

  • protocol (str) – protocol to use

  • compress (Optional[str]) – compress algorithm to use

  • _show_progress (bool) – show progress bar, only works when protocol is pickle or protobuf

Return type:

bytes

Returns:

the binary serialization in bytes

to_protobuf(ndarray_type=None)[source]#

Convert DocumentArray into a Protobuf message.

Parameters:

ndarray_type (Optional[str]) – can be list or numpy, if set it will force all ndarray-like object from all Documents to List or numpy.ndarray.

Return type:

DocumentArrayProto

Returns:

the protobuf message

classmethod from_protobuf(pb_msg)[source]#
Return type:

T

classmethod from_base64(data, protocol='pickle-array', compress=None, _show_progress=False, *args, **kwargs)[source]#
Return type:

T

to_base64(protocol='pickle-array', compress=None, _show_progress=False)[source]#
Return type:

str