Some readers may ask why there is a need for the zLidar format when the LiDAR community already has other compressed file formats, such as zLAS and LAZ? Importantly, neither of these compressed LiDAR file formats have an open specification, of the type that we see with LAS, e57, Shapefile, GeoTIFF, or other standard data formats used in geomatics.

The zLAS format (*.zlas), sometimes called the Esri optimized LAS file, is a proprietary format developed by Esri. While Esri maintain an open-source library (licensed under the Apache License, Version 2.0) that is available to 3^rd party developers to add support for reading and writing the data format in their own software, the zLAS format itself is closed-source. Furthermore, the zLAS encoding/decoding library is only available on MS Windows operating system and is therefore not a cross-platform solution.

The LAZ file format is a compelling file format for compressed LiDAR data and has been widely adopted by the geomatics community. It provides a LiDAR-specific compression scheme that often results in impressive compression ratios; LAZ files are significantly smaller than the corresponding LAS files. The LAZ file format is the LiDAR data format that is used within the popular LasTools software, developed by rapidlasso. Like Esri's zLas format, an open-source LAZ encoding/decoding library (LASZip) is available and has been widely adopted by geomatics software applications, including many open-source GIS. Unlike the zLAS format, it is theoretically possible to determine how LAZ files are formatted by interrogating this library's source code.

The general structure of the LAZ file format is partially described in a short conference proceedings (Isenburg, 2013); however, this paper is evidently out of date compared with the current LAZ scheme. For example, the Isenburg (2013) paper makes reference to several future changes to the data format that may or may not have been adopted. Furthermore, this paper does not provide sufficient detail to allow the reader to develop an independent LAZ encoder/decoder because of the brief nature of conference proceedings. In 2017, efforts were made to create a file specification for the LAZ file format, and an 'in progress' draft document does exist. However, it appears that the errorts to complete the specification have been abandoned as it has not been updated since it was created in 2017. Without a completed file specification, the LAZ file format is effectively defined by an open-source reference implementation rather than an open specification. That is, there is currently no document that completely describes the internal structure of a LAZ file (other than the source code), like there is with the LAS and e57 data formats. Interrogating the source code to determine the file structure is also very challenging given the sophisticated and complex nature of the data format and a codebase of this scale. The complexity of the LAZ format reflects its primary objective, which is to create the highest possible lossless compression rate for LiDAR data, for which the LAZ format has done a commendable job. While challenging, it is not impossible to create an independent LAZ encoder/decoder, as evidenced by the las-perf project. However, again, efforts to create an independent encoder/decoder for the format will always be confounded by the possibility of ongoing changes to the original reference implementation, LASZip. This makes maintaining alternative LAZ encoders/decoders very challenging and creates opportunities for fragmentation.

Using a reference implementation to define a file format carries with it the potential for data incompatibility, a situation where the main reader/writer software no longer supports some legacy data when the implementation adopts backwards incompatible changes. That is, if the maintainers of the LAZ format alter the file structure (or rather the encoder/decoder implementation) in a backwards-incompatible manner, all existing data stored using the previous format would become incompatible with the only existing reader/writer of the format. While this is unlikely to occur, it is a general vulnerability associated with the use of a reference implementation rather than an open specification. Software implementations are constantly evolving, and therefore, so too are implementation-defined file formats.

An open specification for a data format, however, serves as a contract with the user and developer communities regarding the long-term stability and viability of a particular data format. With data formats defined by open specifications, developers must ensure that their 3^rd party readers/writers are compatible with the specification, which adheres to strict versioning rules.

With the above rationale established, we present the zLidar data format in this document. The primary objectives of the zLidar data format are:

1. To provide a simple and flexible means for storing LiDAR data in a losslessly compressed format that significantly reduces data storage needs compared with LAS data.
2. To provide a compressed LiDAR data format that is defined by an open specification that allows for development of multiple independent encoder/decoder libraries that are accessible to the user and developer communities.

The main benefit of this approach is viable long-term storage of valuable LiDAR data resources in a manner that is less vulnerable to the ephemeral nature of individual software projects and their developers.

Weitz, Lindsay (Esri). 2015. Esri Optimized LAS (zLAS) I/O Runtime Library is Now Available. Online resource viewed 08/06/2020.

Isenburg, M., 2013. LASzip: lossless compression of LiDAR data. Photogrammetric engineering and remote sensing, 79(2), pp.209-217.