From c9c0d073da561bcbefbdf09c87fc75b227415619 Mon Sep 17 00:00:00 2001 From: Brian Behlendorf Date: Thu, 26 Aug 2010 11:22:58 -0700 Subject: Add build system Add autoconf style build infrastructure to the ZFS tree. This includes autogen.sh, configure.ac, m4 macros, some scripts/*, and makefiles for all the core ZFS components. --- ChangeLog | 576 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 576 insertions(+) create mode 100644 ChangeLog (limited to 'ChangeLog') diff --git a/ChangeLog b/ChangeLog new file mode 100644 index 000000000..e89461c44 --- /dev/null +++ b/ChangeLog @@ -0,0 +1,576 @@ +2010-08-13 Brian Behlendorf + + * : Tag zfs-0.5.0 + + * : The ChangeLog is being retired. Please use the git commit + logs for a full records of changes: 'git log --no-merges' + +2010-05-21 Brian Behlendorf + + * : Tag zfs-0.4.9 - Use 'git log --no-merges' for full change log. + + * : Build system improvements: + - Added support for the 'make -s' silent build option. + - Allow zfs_config.h to be included by dependent packages. + - Minor spec file updates. + - Minor build system message updates. + + * : Topic branch cleanup. Several old branches were removed and + numerous hunks which were accidentaly commited to incorrect topic + branches in the past were relocated to the correct topic branch. + + * *.c, *.h, *.sh, AUTHORS, COPYING, COPYRIGHT, DISCLAIMER, README: + Added standard header to source files which includes the copyright, + license, and author information. Additionally, updated the listed + top level files to the latest versions. + + * cmd/zpool/zpool_vdev.c: Check all partitions with check_file() + even when no libblkid is found. This oversight would result in + ZFS not detecting existing filesystems on partitions. + + * module/zfs/fm.c, module/zfs/zfs_fm, libzfs/libzfs_pool.c: + Added zevents which are similar to Solaris FMA support. The + existing FMA and sysevent call points in ZFS were unified in + to a single event type and used to create a user space visible + event notification system under Linux. The new 'zpool events' + command can be used to show all recent ZFS events. + + * module/zfs/spa.c, module/zfs/zil.c: Suppress large memory + allocation warnings for two particular kmem_alloc()'s. For + now we can live with them as is but long term a way should be + found to perform small allocations or use the vmem based slab. + + * module/zfs/zvol.c: Recreate volume and snapshot /dev links during + module load. Links in /dev for volumes/snapshots were only being + created at volume/snapshot creation time. Those links are now also + created dynamically at module load time based on the spa config. + + * module/zfs/zvol.c, module/zfs/include/sys/blkdev.h: The handler + zvol_request() should use the unlocked version of blk_end_request() + to avoid a deadlock in the ZVOL. + +2010-03-11 Brian Behlendorf + + * : Tag zfs-0.4.8 - Use 'git log --no-merges' for full change log. + + * : Build system improvements: + - Remove Module.markers and Module.symver{s} in clean target. + - Improved kernel source detection when none specified. + - Fix RPM definitions for the unknown distro/installation. + - Check for spl in ../spl if not found in install path. + - Include all headers regardless of depth in packages. + - Allow recursive configure/make. + + * lib/libzpool/kernel.c: Fixed several zdb bugs when opening a pool + - zdb wasn't getting the correct device size when the vdev is a + block device. In Solaris, fstat64() returns the device size but + in Linux an ioctl() is needed. + - make sure that we don't try to open a block device in write mode + from userspace. This shouldn't happen, because zdb opens devices + in read-only mode, and ztest only uses files. + + * lib/libspl/include/umem.h: Add umem_alloc_aligned() and honor + cache_align field for umem cache. Under linux we open block devices + with O_DIRECT which means we must provide aligned memory buffers. + + * lib/libzpool/kernel.c: Fix some incorrect error handling. In + vn_open(), if fstat64() returned an error, the real errno was being + obscured by calling close(). + + * scripts/*: Fix scripts to work when invoked from other directories. + + * module/zfs/arc.c: Fix struct ht_lock padding in arc.c. + + * lib/libefi/include/sys/uuid.h: Fix duplicate uuid_t typedef. + + * module/zfs/txg.c: Use CPU percentages for number of commit cb + threads. This doesn't change number of threads in the kernel, but it + reduces number of threads in ztest (important due to 32-bit address + limitations). + + * cmd/ztest/ztest.c: Clean up emulation of kernel threads in + userspace. Updated to use pthread thread specific data rather than + keeping a global list. This also fixes at least one easily + reproducible crash in ztest + + * META, config/kernel.m4: Add configure check for kernel build + options which are incompatible with the license. If your building + against a kernel deemed incompatible configure will fail and + suggest how you should rebuild your kernel. + + * config/kernel-fmode-t.m4: Linux 2.6.28 compat, add a check for the + fmode_t type. This typedef first appears in 2.6.28 kernels as part + of some block device operation reworking. + + * module/zfs/dmu_send.c: No inline to keep dmu_recv_stream() stack + frame less than 1024 bytes. Recent builds against 2.6.31 flagged + dmu_recv_stream() as stack heavy. Further analysis of this function + should be performed to further reduce its stack usage. + + * scripts/common.sh: Split the udev rule from a specific configuration + by providing a generic 60-zpool.rules file which uses a small helper + util 'zpool_id' to parse a configuration file by default located in + /etc/zfs/zdev.conf. The helper script maps a by-path udev name to a + more friendly name of for large configurations. + + Additionally, when running zpool-create.sh in-tree it will no longer + use udev because we would have to copy certain helper scripts in to + the installed system. To avoid this the config file in simply + parsed and symlinks are created in your working tree. The script + will use udev if it as run as part of an installed zfs-test package. + + * module/zfs/zvol.c: Use check_disk_change() instead of + revalidate_disk(). For 2.6.27 kernels are earlier revalidate_disk() + was not available. However, check_disk_change() has been available + for far longer and will properly inform the kernel of the volume + change for both older and newer kernels. + + * module/zfs/dmu.c: Fixed incorrect ASSERT3S() added by ZVOL. + + * module/zfs/vdev_raidz.c, module/zfs/zvol.c: Minor fixes for 32-bit. + + * scripts/zfs-update.sh, man/man8/*: Added man pages based on the + latest documentation and modified zfs-update.sh script to update them. + + * .gitignore: Updated .gitignore rules to exclude build products. + +2009-11-24 Brian Behlendorf + + * : Tag zfs-0.4.7 - Use 'git log --no-merges' for full change log. + + * module/zcommon/include/sys/fs/zfs.h, module/zfs/include/sys/blkdev.h, + module/zfs/include/sys/dmu.h, module/zfs/dmu.c, + module/zfs/include/sys/zvol.h, module/zfs/zvol.c, + module/zfs/zfs_ioctl.c: + Added the ZVOL block device, with the addition of the ZVOL real ZFS + based block devices are available and can be compared head to head + with Linux's MD and LVM block drivers. The Linux ZVOL has not yet + had any performance work done but from a user perspective it should + be functionally complete and behave like any other Linux block device. + The ZVOL has so far been tested using zconfig.sh on the following + x86_64 based platforms: FC11, CHAOS4, RHEL5, RHEL6, and SLES11. + However, more testing is required to ensure everything is working + as designed. + + * scripts/udev-rules/99-zpool.rules.promise, + scripts/zpool-config/promise-raid0-1x16.sh, + scripts/zpool-config/promise-raid10-8x2.sh, + scripts/zpool-config/promise-raidz-2x8.sh, + scripts/zpool-config/promise-raidz2-2x8.sh: + Additional test configurations for a small 16 drive JBOD. + + * module/zfs/arc.c: Linux 2.6.31 compat, mutexes can now exceed 64 + bytes depending on the the kernel build options. To account for + this increase the pad size to 256 bytes. + + * module/zfs/vdev_disk.c: Linux 2.6.31 compat, to get the hard + sector size use bdev_logical_block_size() this function replaces + bdev_hardsect_size(). + + * module/zfs/zfs_rlock.c: Prevent gcc uninit compiler warning in + zfs_range_unlock_reader(). + + * Makefile.am: Ensure *.order and *.markers build products are + removed by distclean rule. + +2009-11-02 Brian Behlendorf + + * : Tag zfs-0.4.6 - Use 'git log --no-merges' for full change log. + + * Rebased to ZFS b121 from OpenSolaris. + + * module/zfs/vdev_disk.c: Finally a feature complete implementation: + - Handle dynamic bio merge_bdev limitations when constructing the + bio set associated with a dio. This previously prevented us from + layering cleanly on the md and dm virtual devices. + - Removed hard coded 512 byte sector size. + - Correctly determine the device size when using a partition. + - Hold and extra dio reference when submitting bio's using + bio_submit() to prevent a completion race. + + * lib/libefi/*: Added fully function libefi library from Solaris. + This allows us to properly create and access GPT style partition + tables which are used when a whole device is added to a zpool. + + * cmd/zpool/zpool_vdev.c: Fully integrated zpool with Linux package + libblkid. This allows zpool to identify existing devices of + various types to prevent devices from accidentally being used. When + given a whole device with a GPT partition table all partitions will + be checked for existing filesystems. At the moment MBR style + partition tables cannot be check and the force option must be used. + + * cmd/zpool/zpool_vdev.c: Solaris devid support has been removed in + favor of Linux's udev. This means that a zpool device will always be + opened using the path provided at configuration time. This may + initially seem limiting but it has certain advantages: + - When creating a zpool where the physical location of the device + is NOT import simply create the pool using the /dev/disk/by-id paths. + This will ensure that regardless of physical location the device + will be properly located. + - When creating a zpool where the physical location of the device + is important use the /dev/disk/by-path paths. This will ensure that + devices are never accidentally detected and used in an incorrect + location which would compromise the redundancy of the system. + - Ever better you can create use your own udev rules file to setup + any mapping and naming convention you desire. One example of a + custom rule is to map physical device locations using grid with + numbers are letter for coordinates. Each letter might represent + a specific bus/channel and each number a specific device. For large + configurations this provides an easy way to identify devices. + + * module/zpios/zpios.c: Update to use kobject_set_name() for + increased portability. + + * modules/*/*: Update module init/exit access points to use + spl_module_{init,exit}() macro API. This ensures the cwd is + immediately set to '/' and may be leveraged latter for any + additional module setup/cleanup which is required. + + * cmd/ztest/ztest.c: Check ftrucate() return code to prevent + warnings when --fortify-source options is used in rpm builds. + + * config/Rules.am: Set DEBUG/NDEBUG globally when building user + space components. + + * scripts/zconfig.sh: Initial hook for running additional sanity + tests are part of 'make check'. Currently, there are only two + tests which do some basic configuration checking but they should + be extended as much as possible to prevent regressions. Tests + should also all be written so they run entirely in-tree. + + * scripts/zpios-sanity.sh: Initial hook for validating real IO + using all block devices and all raid configurations. Supported + device types include scsi, ide, md, dm, ram, loop, and file. + Supported raid types include raid0, raid10, raidz, and raidz2. + + * scripts/zpool-config/*: Update dragon and x4550 configs to use + custom udev rules file with <1-N> naming convention. Add + configs for md, dm, and ram block devices to verify functionality. + + * zfs-test.spec.in: Added zfs-test package which extends the existing + in-tree test infrastructure such that it can be run as part of an + installed package. This simplifies the testing of tagged releases. + + * zfs-modules.spec.in: Various spec file tweaks for the supported + distros: RHEL5, RHEL6, SLES10, SLES11, Chaos4, Fedora 11. + +2009-08-04 Brian Behlendorf + + * : Tag zfs-0.4.5 - Use 'git log --no-merges' for full change log. + + * FC11 and SLES11 support: This includes all compatibility changes + to support 2.6.29 based kernels and the required build system + improvements. + * lib/libspl/asm-generic/atomic.c: Generic user space atomic support. + * module/zpios/zpios.c: Register a basic compat ioctl handler for + 32-bit user vs 64-bit kernel compatibility. This is the default + build environment for all 64-bit SLES systems. Additionally + replace the use of 'struct timespec' which uses longs internally + and is therefore different sizes on 32-bit vs 64-bit objects with + 'struct zpios_timespec_t'.a + * config/kernel.m4: Check arch/default path when detecting kernel + objects under SLES. We still preferentially use arch/arch if + available but if that fails it is acceptable to use default. + * config/kernel.m4: Remove LINUXINCLUDE from autoconf wrapper. + This breaks the 2.6.28+ kernels build system, all kernel build systems + at least post 2.6.16 will set this properly so we should not. + * lib/libspl/include/assert.h: Add ASSERTV macro to simplify removing + variables (the V in ASSERTV) which are only used in ASSERT(). + Also revert all previously modified ASSERT()s to their original + definitions and use the ASSERTV macro to handle unused variables. + * module/zpios/zpios.c: Use spl device interfaces for portability. + * scripts/common.sh: The losetup -f option is unavailable for + SLES10 use the unused_loop_device() common function. Additionally + LOSETUP is now used to portably reference the losetup binary. + * module/zfs/vdev_disk.c: Unused destroy_dirty_buffers arg removed. + * module/zfs/vdev_disk.c: BIO_RW_SYNC renamed to BIO_RW_SYNCIO. + * module/zfs/vdev_disk.c: open/close_bdev_excl() renamed to + open/close_bdev_exclusive(). + * module/zfs/vdev_disk.c: Empty write barriers are supported as of + linux 2.6.24 and are now used to implement DKIOCFLUSHWRITECACHE. + * module/zfs/vdev_disk.c: The bi_end_io API changes make partial + IO's impossible handle this case cleanly. + * module/zfs/vdev_disk.c: BIO_RW_FAILFAST replaced with + BIO_RW_FAILFAST_{DEV|_TRANSPORT|_DRIVER}, use the legacy + BIO_RW_FAILFAST flag if it exists for now. + * module/zfs/dmu_objset.c, module/zfs/spa_history.c: Excessively + large stack frames (>2048) were detected in dmu_objset_snapshot() + and spa_history_log() and were reduced by allocating from the heap. + + * Rebased to ZFS b117 from OpenSolaris: + * module/zfs/dmu.c: Update zerocopy patch to be consistent with + new flags arg passed to dmu_read(). + * module/zpios/zpios.c: Update zpios to be aware of the new flags + argument available in dmu_read(). Additionally add a zpios command + line flag to set the DMU_READ_NO_PREFETCH flag for a test. + * module/zfs/include/sys/zfs_context.h: Wrap new sysevent includes + until sysevent or something like it is implemented. + * module/zfs/zfs_ioctl.c: Wrap all the ACL interfaces with HAVE_ZPL. + They are integrated with the vfs layer which is not yet supported. + * module/zcommon/zfs_prop.c: Export new quota related symbols. + * lib/libzfs/libzfs_util.c: Increase buffer size for nvlist which + is needed for large configurations. + + * Bug fixes: + * lib/libspl/include/sys/zfs_debug.h: Removed duplicate file. + * lib/libspl/include/sys/isa_defs.h: Fixed missing macro definitions + for little endian and big endian. + * cmd/zpios/zpios_main.c: Pretty-up the zpios 'make check' output. + * module/zfs/vdev_disk.c: Honor spa_mode() when opening block devs. + * module/zfs/vdev_disk.c: Initial error handling added to + vdev_disk_io_done() to revalidate the media on EIO. + * lib/libzfs/libzfs_util.c: Fix an accidentally introduced formatting + issue in zfs_nicenum() output. + +2009-07-02 Brian Behlendorf + + * : Tag zfs-0.4.4 - Use 'git log --no-merges' for full change log. + + * : Generic distro friendly build system / packaging improvements + for RPM based distros including CHAOS, RHEL, Fedora, and SLES. + + These changes bring the zfs-0.4.4 tree in to compliance with + the spl-0.4.4 packaging changes. The bottom line is 2 source + rpms and 4 binary rpms will now be generated when creating + packages there will be: + + zfs-.src.rpm + - Fully rebuildable source rpm for libzfs and utils. + zfs-modules-.src.rpm + - Fully rebuildable source rpm for kernel modules. + + zfs-..rpm + - Binary rpm for libzfs and utils. The utils in this package are + compatible with all zfs-module rpms of the same version. + zfs-devel-..rpm + - Binary rpm containing headers for building against libzfs libraries. + + zfs-modules--.arch.rpm + - Binary rpm containing the kernel modules for a specific kernel build. + The package name contains the kernel version and you should have one + of these packages installed to match every kernel on your system. + zfs-modules-devel--.arch.rpm + - Binary rpm containing development header and module symbols needed + for building additional kernel modules which are dependent on the + zfs module stack. + + * : SLES9/10 distro support added: Several significant updates to + the build system were required to support this including: + + - Autoconf macros updated to be aware of the standard install + locations for SLES kernel sources and build objects. Additionally + changes were made to support multiple names for Module{s}.symvers. + - By default on SLES all user space builds are 32-bit even on + 64-bit arches. This means we need to be careful to pass -m64 in + the autoconf checks which probe the kernel. Additionally, + ioctl_compat handles still need to be added so 32-bit user + binarys can perform ioctls with the 64-bit kernel. + + * : Powerpc64 support added: + + - 64-bit user space atomic support for power64 was obtained from + an old version of OpenSolaris which offered minimal powerpc support. + The atomic support is not 100% fully implemented but it's a good + first step towards cleanly supporting the architecture. + - Added powerpc ISA type. + - Explicitly use signed char for portability. On x86/x86_64 + systems the default char type is signed, on ppc/ppc64 systems + the default char type is unsigned. + - Core target arch support for conditional compilation of SUBDIRs. + Required by libspl for its arch specific atomic implementations. + + * COPYRIGHT: Readded accidentally dropped COPYRIGHT, it just + references the OPENSOLARIS.LICENSE which was still in the project. + + * module/zfs/dmu_tx.c: Add EXPORT_SYMBOL(dmu_tx_callback_register). + + * config/user-zlib.m4: Add basic zlib autoconf check for user space. + +2009-03-20 Brian Behlendorf + + * : Tag zfs-0.4.3 - Use 'git log --no-merges' for full change log. + + * : Rebased to ZFS b108 from OpenSolaris + + * configure.ac, *Makefile.am: Build system update. This includes + resolving various build issues and adding support for the remaining + common build targets. Available targets now include: + + - make all # Build everything + - make install # Install everything + - make clean # Clean up build products + - make distclean # Clean up everything + - make dist # Create package tarball + - make srpm # Create package source RPM + - make rpm # Create package binary RPMs + - make tags # Create ctags and etags for everything + + Extra care was taken to ensure that the source RPMs are fully + rebuildable against Fedora/RHEL/Chaos kernels. To build binary + RPMs from the source RPM for your system simply run: + + rpmbuild --rebuild zfs-x.y.z-1.src.rpm + + This will produce two binary RPMs with correct 'requires' + dependencies for your kernel. One will contain all zfs modules + and support utilities, the other is a devel package for compiling + additional kernel modules which are dependent on the zfs. + + zfs-x.y.z-1_.x86_64.rpm + zfs-devel-x.y.2-1_.x86_64.rpm + + * config/kernel.m4, config/Rules.am: Removed the build options + -Wno-unused -Wno-missing -Wno-unused -Wno-parentheses and + -Wno-unknown-pragmas. All upstream warnings which were being + suppressed by these options are now handled in the matching + gcc-* topic branch. + + * zpios: Various zpios and test infrastructure improvements. + + * feature-commit-cb: New feature commit callbacks. + + * : Various bug fixes which are more clearly detailed in the + git commit logs. For a detailed summary of changes post version + 0.4.2, check out the 'top' topic branch and view the commit logs. + + git checkout top + git log --no-merges -53 + +2009-02-05 Brian Behlendorf + + * : Tag zfs-0.4.2 + + * lib/libzfs/libzfs_util.c: Remove mknod() this is now handled by + by the SPL with an upcall even without GPL-only symbols. + + * module/zfs/vdev_diskc: Updated linux vdev_disk interfaces to issue + multiple bios if needed due to the maximum request size being smaller + than the request size passed down from the spa. + + * scripts/zpool-config/lo-*: Add loopback based test configs. + +2009-01-21 Brian Behlendorf + + * : Tag zfs-0.4.1 + + * : Rebased to ZFS b105 from OpenSolaris + + * : Restructured project and build system to more closely + mirror the code organization used in OpenSolaris. This + allows us minimize the changes to upstream source needed, + and allows concurrent user+kernel space builds of all components. + +2008-11-19 Brian Behlendorf + + * : Tag zfs-0.4.0 + + * : ZFS project migrated from Subversion which leveraged a + quilt based patch stack to Git and a TopGit managed patch + stack. The new method treats all patches as Git branches + which can be more easily shared for distributed development. + Consult the top level GIT file for detailed information on + how to properly develop for this package using Git+TopGit. + +2008-11-12 Brian Behlendorf + + * : Tag zfs-0.3.4 + + * zfs-07-create-dev-zfs.patch: + Ricardo M. Correia + - Make libzfs create /dev/zfs if it doesn't exist. + + * zfs-05-check-zvol-size.patch: + Ricardo M. Correia + - Properly check zvol size under Linux. + + * zfs-04-no-openat-fdopendir.patch: + Ricardo M. Correia + - Do not use openat() and fdopendir() since they are not available + on older systems. + + * zfs-03-fix-bio-sync.patch: + Ricardo M. Correia + - Fix memory corruption in RHEL4 due to synchronous IO becoming + asynchronous. + +2008-11-06 Brian Behlendorf + + * zfs-02-zpios-fix-stuck-thread-memleak.patch: + Ricardo M. Correia + - Fix stuck threads and memory leaks when errors occur while writing. + + * zfs-01-zpios-arg-corruption.patch: + Ricardo M. Correia + - Fix zpios cmd line argument corruption problem. + + * zfs-00-minor-fixes.patch: + Ricardo M. Correia + - Minor build system improvements + - Minor script improvements + - Create a full copy and not a link tree with quilt + - ZPIOS_MAJOR changed from 231 to 232 + - BIO_RW_BARRIER flag removed from IO request + +2008-06-30 Brian Behlendorf + + * : Tag zfs-0.3.3 + + * : Minor script updates and tweaks to be compatible with + the latest version of the SPL. + +2008-06-13 Brian Behlendorf + + * vdev_disk.diff: Replace vdev_disk implementation which was + based on the kmalloc'ed logical address space with a version + which works with vmalloc'ed memory in the virtual address space. + This was done to support the new SPL slab implementation which + is based on virtual addresses to avoid the need for contigeously + allocated memory. + +2008-06-05 Brian Behlendorf + + * arc-vm-integration.diff: Reduce maximum default arc memory + usage to 1/4 of total system memory. Because all the bulk data + is still allocated on the slab memory fragmentation is a serious + concern. To address this in the short term we simply need to + leave lots of free memory. + + * fix-stack.diff: First step towards reducing stack usage so + we can run the full ZFS stack using a stock kernel. + +2008-06-04 Brian Behlendorf + + * : Tag zfs-0.3.2 + + * : Extensive improvements to the build system to detect kernel + API changes so we can flexibly build with a wider range of kernel + versions. The code has now been testing with the 2.6.18-32chaos + and 2.6.25.3-18.fc9 kernels, however we should also be compatible + with other kernels in the range of 2.6.18-2.6.25. The only + remaining issue preventing us from running with a stock + kernel is ZFS stack usage. + +2008-05-21 Brian Behlendorf + + * : Tag zfs-0.3.1 + + * : License headers including URCL added for release. + +2008-05-21 Brian Behlendorf + + * : Tag zfs-0.3.0 + + * configure.ac: Improved autotools support and configurable debug. + +2008-05-15 Brian Behlendorf + + * : Updating original ZFS sources to build 89 which + includes the new write throttling changes plus support + for using ZFS as your root device. Neither of which + will work exactly right without some more work but this + gets us much closers to the latest source. + + +2008-02-28 Brian Behlendorf + + * : First attempt based on SPL module and zfs-lustre sources -- cgit v1.2.3