Build Guide for HandBrake 20160130110114-6a00e46-master on Mac OS X ******************************************************************* 1 Introduction 2 Prerequisites 3 QuickStart 4 Overview 5 Building via Terminal 5.1 Checkout Sources 5.2 Configure 5.3 Build 5.4 Make Targets 5.4.1 Global 5.4.2 General Modules 5.4.3 Contrib Modules 5.4.4 Contrib Touch and Untouch 5.4.5 Contrib Aggregates 5.5 Customizing Make 5.6 Universal Binaries 6 Building via Xcode.app 6.1 Checkout Sources 6.2 Build 6.3 Note: Finding Built Products 6.4 Note: Workspace Log Behaviors 6.5 External Target 6.6 User-Defined Settings 7 Troubleshooting Appendix A Project Repository Details 1 Introduction ************** This guide documents the recommended process to build HandBrake on Mac OS X hosts from the official source-code repository. Building from any other source is not supported. 2 Prerequisites *************** Building on Mac OS X is well supported. It is the reference platform for HandBrake. The following are the recommended specifications for this platform; but is not necessarily the only configuration that is possible: * Mac Intel hardware * Mac OS X 10.9.4 or later * Xcode 6.0.1 * Xcode command-line tools (installed via Preferences > Downloads > Components) or download via Appla Developer website. * XQuartz (http://xquartz.macosforge.org/landing/) The following tools are compiled during the build process if necessary, but you can speed up your build by installing them: * yasm 1.3.0 or later * autoconf * automake * libtool Note: It is recommended to use the platform distribution's standard compiler for maximum C++ compatibility. If you build with a custom compiler it will likely introduce non-standard runtime requirements and have new/delete, exception and RTTI incompatibilities. There are of course many valid reasons to build with unbundled compilers, but be aware it is generally unsupported and left as an exercise to the reader. The following general tools are used on various platforms and it is recommended you use these versions or similar: * git - 2.4.3 * python - Python 2.7.1 * curl - curl 7.21.4 (or wget) * m4 - GNU M4 1.4.6 * make - GNU Make 3.81 * patch - Patch 2.5.8 * tar - GNU tar 1.26 * wget - GNU Wget 1.13.4 (or curl) 3 QuickStart ************ This chapter is for building from a terminal/shell environment in as few commands as possible. Upon completion of the following commands you should have a fresh build of HandBrake. Further instructions are available beginning with *note overview:: which describes procedures suitable for repeating builds. This chapter should be skipped by those seeking more than a minimalist build. git clone https://github.com/HandBrake/HandBrake.git hb-master cd hb-master ./configure --launch The special option '--launch' selects launch mode and performs the following steps: * assert scratch directory 'build/' does not exist * create scratch directory 'build/' * change to directory 'build/' * launch 'make' * capture build output to 'build/log/build.txt' * echo build output * print elapsed time * indicate if build ultimately succeeded or failed 4 Overview ********** The two general methods to build on Mac OS X are from terminal or Xcode.app. The preferred method for automated and repeatable builds is to use the terminal. Otherwise the choice is generally up to the individual. To be extra clear, building from the terminal by default actually invokes 'xcodebuild' to build the very same targets contained in the Xcode project. Think of it as building with Xcode but without the GUI. 5 Building via Terminal *********************** 5.1 Checkout Sources ==================== Checkout HandBrake from the official source-code repository. git clone https://github.com/HandBrake/HandBrake.git hb-master cd hb-master Sources are checked out from the 'master' branch. This document was generated from that very branch, and for example purposes, we will use exactly the same branch. If you have write-access to the repository, then you may add the appropriate login/password information as needed. 5.2 Configure ============= Configure the build system. ./configure Configure will automatically create a scratch build directory 'build' unless you use GNU-style build procedures and first 'cd' to a directory other than top-level source. Additionally you may use '--build' to specify the directory. The name of the directory is arbitrary but it is recommended to use something which indicates transient files which are not checked into the repository. The 'configure' utility accepts many options. It is recommended that you specify '--help' for the complete list of options. The following options are also documented here: '--help' List available options. '--src=DIR' Specify top-level source directory for HandBrake sources. '--build=DIR' Specify destination directory for final product install. The default is to use either 'build' if in the top-level source directory, otherwise '.' '--prefix=DIR' Specify destination directory for final product install. This defaults to a reasonable platform-specific value. '--launch' All-in-one option which launches the build and logs output automatically. Useful for novices and quick-start procedures. '--disable-gtk' Disable building the GTK GUI on applicable platforms such as Linux. '--debug=MODE' Select debug mode. Must be one of 'none', 'min', 'std', 'max'. This generally maps to gcc options '-g0', '-g1', '-g2', '-g3'. '--optimize=MODE' Select optimize mode. Must be one of 'none', 'speed', 'size'. This generally maps to gcc options '-g0', '-O0', '-O3', '-Os'. '--arch=MODE' Select build architecture. The available architectures vary by platform. Most platforms support exactly one architecture except Mac OS X which has support for various universal binary architectures. The available choices are hard-coded per platform and no sanity checks for the required tools are performed. '--disable-xcode' Disable shunting the build through 'xcodebuild'. If this option is applied, 'HandBrakeCLI' will be produced in a similar fashion as it is on other platforms; sans Xcode and the Cocoa application will not be produced. Mac OS X only. '--xcconfig=MODE' Select Xcode project configuration file. The available modes are the basenames of files located in 'macosx/xcconfig/*.xcconfig' which direct Xcode to build using various architecture and Mac OS X deployment options. Mac OS X only. Clean-room procedures dictate that when certain factors change, old builds should be scrapped and new builds configured. This is the main reason for requiring a scratch directory; to promote consistent, reliable and clean software builds. The following is a short list of some of the reasons why someone may choose to scrap an existing build: * configure with different options * git working dir is updated and you want configure to re-evaluate working dir metadata. * build corruption is suspected There are generally two methods for scrapping a build. The 'build' directory can be recursively removed which has the effect of loosing your existing configuration but does guarantee no residuals are left behind. The other method is to ask the build system to perform an 'make xclean'. This is known to work well but will leave empty directories behind. However, the configuration is left intact. 5.3 Build ========= Build main product. All necessary dependencies are also built if required. make Parallel builds may optionally be enabled. Be aware that while a parallel build may save time on systems with additional cores, the output is often mixed, overlapped and sometimes even corrupted with binary characters. Thus if you experience a build issue, you should clean and redo the build in default serial mode to produce a readable log. The following command allows for up to 4 concurrent jobs via make: make -j4 5.4 Make Targets ================ The build system supports passing many kinds of targets some of which become very useful in normal development cycles. The targets by convention are lower-case words passed to 'make'. Global targets are one-word targets. Scoped targets are usually two-words separated by a period. 5.4.1 Global ------------ 'make' Alias for 'make build'. 'make build' Build main product. All necessary dependencies are also built if required. 'make clean' Clean all build output excluding contrib modules. Configuration is retained. 'make install' Perform final product(s) install. This will install build products to a standard directory or one specified via 'configure --prefix' option. 'make uninstall' Perform final product(s) uninstall. This will uninstall any products which may have been previously installed. 'make xclean' Clean all build output including contrib modules. Configuration is retained. 'make doc' Build auto-generated project documentation. Various articles are produced and may be found in 'build/doc/articles'. 'make doc.post' Build auto-generated project documentation and post produced articles directly to source tree. 'make report.help' Print list of available makefile vars report targets. These reports detail var definitions and expanded values used by the build system. For experts only. 'make report.all' Convenience target which aggregates all reports. For experts only. 5.4.2 General Modules --------------------- General modules such as 'libhb', 'test' and 'gtk' have the following scoped targets: 'make MODULE.build' Build MODULE. 'make MODULE.clean' Clean build output for MODULE. 5.4.3 Contrib Modules --------------------- Contrib modules such as 'bzip2', 'ffmpeg', 'fontconfig', 'freetype', 'fribidi', 'lame', 'libass', 'libbluray', 'libdvdnav', 'libdvdread', 'libiconv', 'libogg', 'libsamplerate', 'libtheora', 'libvorbis', 'libxml2', 'x264', 'yasm' and 'zlib' have the following scoped targets: 'make MODULE.fetch' Download source tarball from the Internet and save to 'TOP/downloads' directory. No check-summing is performed. 'make MODULE.extract' Extract source tarball into 'build' tree. 'make MODULE.patch' Apply appropriate patches (if any) to module sources. 'make MODULE.configure' Configure module sources. This usually invokes autotool configure. 'make MODULE.build' Build module. This usually invokes autotool build. 'make MODULE.install' Install module products such as headers and libraries into 'build' tree. This usually invokes autotool install. 'make MODULE.uninstall' Uninstall module products; generally the reverse of install. This usually invokes autotool uninstall. 'make MODULE.clean' Clean module; generally the reverse of build. This usually invokes autotool clean. 'make MODULE.xclean' Extra clean module; first invokes uninstall then recursively removes the module build directory. 5.4.4 Contrib Touch and Untouch ------------------------------- Also available are some very granular targets which help force builds from specific cycle points. The following targets are available to touch and untouch the respective module target; this will force the build system to treat the target as satisfied after a touch or unsatisfied after an untouch: * make MODULE.extract.touch * make MODULE.extract.untouch * make MODULE.patch.touch * make MODULE.patch.untouch * make MODULE.configure.touch * make MODULE.configure.untouch * make MODULE.build.touch * make MODULE.build.untouch * make MODULE.install.touch * make MODULE.install.untouch 5.4.5 Contrib Aggregates ------------------------ For convenience, the following targets aggregate the all contrib modules' respective targets together: * make contrib.fetch * make contrib.extract * make contrib.patch * make contrib.configure * make contrib.build * make contrib.install * make contrib.uninstall * make contrib.clean * make contrib.xclean 5.5 Customizing Make ==================== If the need arises to override settings in the build system (essentially gnu-make variables) the recommended method is to create optional include files which are automatically included if present and follow this naming convention; Do not check these files into the repository: '_SRC_/custom.defs' Custom makevar definitions outside 'build'. Suitable for settings which apply across all builds for a particular checkout; or which survives manual removal of 'build'. '_SRC_/custom.rules' Custom make rules outside 'build'. Suitable for rules which apply across all builds for a particular checkout; or which survives manual removal of 'build'. '_BUILD_/GNUmakefile.custom.defs' Custom makevar definitions specific to a 'build' directory. '_BUILD_/GNUmakefile.custom.rules' Custom makevar rules specific to a 'build' directory. The purpose is to allow a place to store local build settings for testing, tweaking, and experimenting with build configuration without losing your settings if 'configure' is invoked; ie: 'configure' would overwrite 'GNUmakefile' and any customizations contained therein would be lost. Here is a short example of what the contents of '_SRC_/custom.defs' might contain: ## bump to gcc-4.6 in current path GCC.gcc = /usr/bin/gcc-4.6 ## replace optimize for 'speed' with more aggressive settings GCC.args.O.speed = -O3 -fomit-frame-pointer -msse4.2 See also 'make report.help' which displays a set of reports used to dump makefile vars. 5.6 Universal Binaries ====================== This section outlines convenience procedures for creating Universal Binaries for all the architectures. Note: The dummy (container) build configuration uses '--disable-xcode'; but the nested architecture builds will all make full use of Xcode. Create a dummy (container) build configuration and use it to launch a nested-build for each architecture: ./configure --disable-xcode cd build/ make ub.build make ub.combine The list of architectures is hard coded to HandBrake's desired product and currently is composed of combining the binaries produced from two xcconfigs: osx106.i386 and osx106.x86_64. The following example shows how to specify a different list of xcconfigs: ./configure --disable-xcode cd build/ make UB.xcconfigs="osx107.i386 osx107.x86_64" ub.build make UB.xcconfigs="osx107.i386 osx107.x86_64" ub.combine 6 Building via Xcode.app ************************ 6.1 Checkout Sources ==================== Checkout HandBrake from the official source-code repository. git clone https://github.com/HandBrake/HandBrake.git hb-master cd hb-master Sources are checked out from the 'master' branch. This document was generated from that very branch, and for example purposes, we will use exactly the same branch. If you have write-access to the repository, then you may add the appropriate login/password information as needed. 6.2 Build ========= Perform the following steps to build: * Finder - navigate to 'macosx/' in the HandBrake source tree * Finder - open 'HandBrake.xcodeproj' * Xcode workspace - select scheme HandBrake [RELEASE] * Xcode menu - select Product -> Build * Xcode workspace - Show the Log navigator * Xcode workspace Log navigator - select top Build item 6.3 Note: Finding Built Products ================================ Under default Xcode.app options the products from a build are managed by the Xcode Organizer. Perform the following steps to open Finder at top of build tree and navigate to release products: * Xcode menu - select Window -> Organizer * Xcode organizer - select Projects tab * Xcode organizer Projects - select HandBrake item * HandBrake item - click Derived Data location arrow (immediately right of path) * Finder - navigate to Build -> Products -> release Note: There is a bug with Xcode Organizer. The very first time an Xcode project is opened the Project view Derived Data is greyed-out. Workaround glitch by selecting any other tab and then reselecting Projects tab. 6.4 Note: Workspace Log Behaviors ================================= The default Workspace behavior does not display latest Build log in the navigator and quickly becomes tedious. To automatically switch to Log navigator and show current log: * Xcode menu - select Behaviors -> Edit Behaviors * Xcode behaviors - select Build starts * navigator - enable, select Show, select Log Navigator * nagivate to - select current log Note: The Log navigator supports some possibly confusing options. It is recommended to only show results for the last build by selecting Recent. If All is selected then it will look as though Xcode is performing a build, but in reality it is bringing forward log output from prior builds and it becomes impossible to tell if any single log entry represents actual work performed or if it was brought forward from history. Note: When building external target, many 3rd-party contributed modules have warnings and errors which may safely be ignored and are ignored by the external build system. Ultimately, look to the workspace status indicator for Build Succeeded. 6.5 External Target =================== The external target mechanism is used to launch a full terminal-based build from within Xcode. Unfortunately, we do not have hooks in place to offer finer-grained control over per-module make actions. However, you can still use terminal to accomplish those tasks after launching the build at least once or doing a clean from within Xcode. Be careful to not issue terminal commands simultaneously with Xcode tasks. Invoking a clean from Xcode always destroys the entire external build tree and subsequently configures it. Changing settings in Xcode such as selecting xcconfig files should always be followed by a clean. This allows the external build system configuration to accurately reflect Xcode project changes. The following are some examples of using 'make' from the terminal to effect various components of the external build. But first, you must open a terminal at the top of the external build output tree. Here we navigate to external build configured for release: * Xcode menu - select Window -> Organizer * Xcode organizer - select Projects tab * Xcode organizer Projects - select HandBrake item * HandBrake item - click Derived Data location arrow (immediately right of path) * Finder - navigate to Build -> Products -> release -> external Example; external build failed but error is buried in a parallelized log; redo build sequentially: make xclean make BUILD.jobs=1 Example; build external x264 module: make x264.clean make x264 Example; extract, configure, build and install external x264 module: make x264.xclean make x264.install Example; something in a big module is failing; redo build sequentially: make ffmpeg.clean make BUILD.jobs=1 ffmpeg 6.6 User-Defined Settings ========================= The following user defined settings are visible in Xcode project and are used for the external build system. 'EXTERNAL_BUILD' Do not modify; used to specify the build (scratch) directory. 'EXTERNAL_DRIVER' Do not modify; used for internal/external build coordination and must always be 'xcode'. 'EXTERNAL_JOBS' Specifies the concurrency factor for the external build system when builds are launched from within Xcode. Modify for faster external builds if your system has the horsepower and resources. Specifying a value greater than the number of CPU cores (or virtual cores) in your system is unlikely to produce gains and will needlessly consume extra resources. A special string value of auto sets the factor to the number of active CPUs on the host system. 'EXTERNAL_SRC' Do not modify; specifies the top-level source directory for HandBrake, relative to Xcode project. 'EXTERNAL_XCCONFIG' Do not modify; specifies which xcconfig file is active. Defined inside xcconfig file. 7 Troubleshooting ***************** When troubleshooting build issues, the following files relative to the 'build/' directory may be especially useful: 'GNUmakefile' Top-level makefile which contains build settings generated via configure. 'log/config.info.txt' Record of output from configure. 'log/config.verbose.txt' Record of verbose output from configure. 'log/build.txt' Record of output from 'configure --launch'. Similar output may be recorded using 'make' depending on which shell is in use, eg: 'make >& log/build.txt' or 'make > log/build.txt 2>&1'. 'log/xcodemake.env.txt' Environment (variables) dump as seen when Xcode forks 'make'. Mac OS X only. Appendix A Project Repository Details ************************************* url: https://github.com/HandBrake/HandBrake.git branch: master tag: rev: 6989 hash: 6a00e46d7888cd49b6a8671784788129d8139835 date: 2016-01-30 11:01:14 type: developer