module/zstd/README.md


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65


    Contents⊕
  
  
  ZSTD-On-ZFS Library
  Manual
  
  Introduction
  Updating ZSTD
  Altering ZSTD and
  breaking changes
  
  
ZSTD-On-ZFS Library Manual
Introduction
This subtree contains the ZSTD library used in ZFS. It is heavily
cut-down by dropping any unneeded files, and combined into a single
file, but otherwise is intentionally unmodified. Please do not alter the
file containing the zstd library, besides upgrading to a newer ZSTD
release.
Tree structure:

zfs_zstd.c is the actual zzstd kernel
module.
lib/ contains the the unmodified, "amalgamated"
version of the Zstandard library, generated from our
template file
zstd-in.c is our template file for generating the
library
include/: This directory contains supplemental includes
for platform compatibility, which are not expected to be used by ZFS
elsewhere in the future. Thus we keep them private to ZSTD.

Updating ZSTD
To update ZSTD the following steps need to be taken:

Grab the latest release of ZSTD.
Update module/zstd/zstd-in.c if required. (see
zstd/contrib/single_file_libs/zstd-in.c in the zstd
repository)
Generate the "single-file-library" and put it to
module/zstd/lib/.
Copy the following files to module/zstd/lib/:

zstd/lib/zstd.h
zstd/lib/common/zstd_errors.h


This can be done using a few shell commands from inside the zfs
repo:
cd PATH/TO/ZFS

url="https://github.com/facebook/zstd"
release="$(curl -s "${url}"/releases/latest | grep -oP '(?<=v)[\d\.]+')"
zstd="/tmp/zstd-${release}/"

wget -O /tmp/zstd.tar.gz \
    "${url}/releases/download/v${release}/zstd-${release}.tar.gz"
tar -C /tmp -xzf /tmp/zstd.tar.gz

cp ${zstd}/lib/zstd.h module/zstd/lib/
cp ${zstd}/lib/zstd_errors.h module/zstd/lib/
${zstd}/contrib/single_file_libs/combine.sh \
    -r ${zstd}/lib -o module/zstd/lib/zstd.c module/zstd/zstd-in.c
Note: if the zstd library for zfs is updated to a newer version, the
macro list in include/zstd_compat_wrapper.h usually needs to be updated.
this can be done with some hand crafting of the output of the following
script: nm zstd.o | awk '{print "#define "$3 " zfs_" $3}' >
macrotable
Altering ZSTD and breaking
changes
If ZSTD made changes that break compatibility or you need to make
breaking changes to the way we handle ZSTD, it is required to maintain
backwards compatibility.
We already save the ZSTD version number within the block header to be
used to add future compatibility checks and/or fixes. However, currently
it is not actually used in such a way.