summaryrefslogtreecommitdiffstats
path: root/cmd/arcstat
Commit message (Collapse)AuthorAgeFilesLines
* FreeBSD: Update usage of py-sysctlRyan Moeller2020-12-231-2/+3
| | | | | | | | | | | | py-sysctl now includes the CTLTYPE_NODE type nodes in the list returned by sysctl.filter() on FreeBSD head. It also provides descriptions now. Eliminate the subprocess call to get descriptions, and filter out the nodes so we only deal with values. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #11318
* arcstat: Add -a and -p options from FreeNASRyan Moeller2020-10-301-6/+32
| | | | | | | | | | | | Added -a option to automatically print all valid statistics. Added -p option to suppress scaling of printed data. Reviewed-by: George Melikov <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Authored by: Nick Principe <[email protected]> Ported-by: Ryan Moeller <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #11090
* Include scatter_chunk_waste in arc_sizeMatthew Ahrens2020-08-171-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The ARC caches data in scatter ABD's, which are collections of pages, which are typically 4K. Therefore, the space used to cache each block is rounded up to a multiple of 4K. The ABD subsystem tracks this wasted memory in the `scatter_chunk_waste` kstat. However, the ARC's `size` is not aware of the memory used by this round-up, it only accounts for the size that it requested from the ABD subsystem. Therefore, the ARC is effectively using more memory than it is aware of, due to the `scatter_chunk_waste`. This impacts observability, e.g. `arcstat` will show that the ARC is using less memory than it effectively is. It also impacts how the ARC responds to memory pressure. As the amount of `scatter_chunk_waste` changes, it appears to the ARC as memory pressure, so it needs to resize `arc_c`. If the sector size (`1<<ashift`) is the same as the page size (or larger), there won't be any waste. If the (compressed) block size is relatively large compared to the page size, the amount of `scatter_chunk_waste` will be small, so the problematic effects are minimal. However, if using 512B sectors (`ashift=9`), and the (compressed) block size is small (e.g. `compression=on` with the default `volblocksize=8k` or a decreased `recordsize`), the amount of `scatter_chunk_waste` can be very large. On a production system, with `arc_size` at a constant 50% of memory, `scatter_chunk_waste` has been been observed to be 10-30% of memory. This commit adds `scatter_chunk_waste` to `arc_size`, and adds a new `waste` field to `arcstat`. As a result, the ARC's memory usage is more observable, and `arc_c` does not need to be adjusted as frequently. Reviewed-by: Pavel Zakharov <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: George Wilson <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Matthew Ahrens <[email protected]> Closes #10701
* Centralize variable substitutionArvind Sankar2020-07-143-13/+6
| | | | | | | | | | | | A bunch of places need to edit files to incorporate the configured paths i.e. bindir, sbindir etc. Move this logic into a common file. Create arc_summary by copying arc_summary[23] as appropriate at build time instead of install time. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Arvind Sankar <[email protected]> Closes #10559
* arcstat: add 'avail', fix 'free'Matthew Ahrens2020-06-261-11/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The meaning of the `free` field is currently `zfs_arc_sys_free`, which is the target amount of memory to leave free for the system, and is constant after booting. This commit changes the meaning of `free` to arc_free_memory(), the amount of memory that the ARC considers to be free. It also adds a new arcstat field `avail`, which tracks `arc_available_memory()`. Since `avail` can be negative, it also updates the arcstat script to pretty-print negative values. example output: $ arcstat -f time,miss,arcsz,c,grow,need,free,avail 1 time miss arcsz c grow need free avail 15:03:02 39K 114G 114G 0 0 2.4G 407M 15:03:03 42K 114G 114G 0 0 2.1G 120M 15:03:04 40K 114G 114G 0 0 1.8G -177M 15:03:05 24K 113G 112G 0 0 1.7G -269M 15:03:06 29K 111G 110G 0 0 1.6G -385M 15:03:07 27K 110G 108G 0 0 1.4G -535M 15:03:08 13K 108G 108G 0 0 2.2G 239M 15:03:09 33K 107G 107G 0 0 1.3G -639M 15:03:10 16K 105G 102G 0 0 2.6G 704M 15:03:11 7.2K 102G 102G 0 0 5.1G 3.1G 15:03:12 42K 103G 102G 0 0 4.8G 2.8G Reviewed-by: George Melikov <[email protected]> Reviewed-by: Pavel Zakharov <[email protected]> Reviewed-by: Tony Nguyen <[email protected]> Signed-off-by: Matthew Ahrens <[email protected]> Closes #10494
* Fix check for sed --in-placeArvind Sankar2020-06-241-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | The test added in commit 4313a5b4c51e ("Detect if sed supports --in-place") doesn't work at least on my system (autoconfig-2.69). The issue is that SED has already been found and cached before this function is evaluated, with the result that the test is completely skipped. ... checking for a sed that does not truncate output... /usr/bin/sed ... checking for sed --in-place... (cached) /usr/bin/sed The first test is executed by libtool.m4. This looks to have been around in libtool for at least 15 years or so, not sure why this was not encountered at the time of the original commit. Fix this by caching the value of the ac_inplace flag rather than the path to SED. Also use $SED and add AC_REQUIRE to ensure that we use the sed that was located by the standard configure test. Reviewed-by: Ryan Moeller <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Arvind Sankar <[email protected]> Closes #10493
* Fix outdated comment headerAJ Jordan2020-05-111-8/+13
| | | | | | | Reviewed-by: Richard Laager <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: AJ Jordan <[email protected]> Closes #10288
* Fix inconsistent capitalization in `arcstat -v`AJ Jordan2020-05-111-11/+11
| | | | | | | Reviewed-by: Richard Laager <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: AJ Jordan <[email protected]> Closes #10288
* Changed decimals to integers in the arcstat scriptAvatat2020-03-181-13/+4
| | | | | | | | | | | | | | Changed interval value type from decimal to integer, because of deprecation warning in Python 3.8 and above. Also changed kstat values type from decimal to integer, because all the values are integers. Fixed behavior of arcstat when run without args. Reviewed-by: George Melikov <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Bartosz Zieba <[email protected]> Closes #10132 Closes #10142
* Add FreeBSD code to arc_summary and arcstatRyan Moeller2019-11-301-1/+23
| | | | | | | | Adding the FreeBSD code allows arc_summary and arcstat to be used on FreeBSD. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #9641
* Isolate code specific to Linux in cmd/Ryan Moeller2019-11-111-19/+21
| | | | | | | | | | | | | Use sys.platform to choose the correct implementation of functions and values of variables for the platform being run on. Reword some comments to avoid describing implementation details in the wrong places. Reviewed-by: Kjeld Schouten <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Jorgen Lundman <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #9561
* Detect if sed supports --in-placeRyan Moeller2019-10-161-1/+1
| | | | | | | | | | | | | | Not all versions of sed have the --in-place flag. Detect support for the flag during ./configure and provide a fallback mechanism for those systems where sed's behavior differs. The autoconf variable ${ac_inplace} can be used to choose the correct flags for editing a file in place with sed. Replace violating usages in Makefile.am with ${ac_inplace}. Reviewed-by: Chris Dunlop <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #9463
* Canonicalize Python shebangsRyan Moeller2019-09-122-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | /usr/bin/env python3 is the suggested[1] shebang for Python in general (likewise for python2) and is conventional across platforms. This eases development on systems where python is not installed in /usr/bin (FreeBSD for example) and makes it possible to develop in virtual environments (venv) for isolating dependencies. Many packaging guidelines discourage the use of /usr/bin/env, but since this is the canonical way of writing shebangs in the Python community, many packaging scripts are already equipped to handle substituting the appropriate absolute path to python automatically. Some RPM package builders lacking brp-mangle-shebangs need a small fallback mechanism in the package spec to stamp the appropriate shebang on installed Python scripts. [1]: https://docs.python.org/3/using/unix.html?#miscellaneous Reviewed-by: Richard Laager <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: George Melikov <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #9314
* Fix typos in cmd/Andrea Gelmini2019-08-301-1/+1
| | | | | | | Reviewed-by: Ryan Moeller <[email protected]> Reviewed-by: Richard Laager <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Andrea Gelmini <[email protected]> Closes #9234
* pyzfs: python3 support (build system)Brian Behlendorf2019-01-062-8/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Almost all of the Python code in the respository has been updated to be compatibile with Python 2.6, Python 3.4, or newer. The only exceptions are arc_summery3.py which requires Python 3, and pyzfs which requires at least Python 2.7. This allows us to maintain a single version of the code and support most default versions of python. This change does the following: * Sets the default shebang for all Python scripts to python3. If only Python 2 is available, then at install time scripts which are compatible with Python 2 will have their shebangs replaced with /usr/bin/python. This is done for compatibility until Python 2 goes end of life. Since only the installed versions are changed this means Python 3 must be installed on the system for test-runner when testing in-tree. * Added --with-python=<2|3|3.4,etc> configure option which sets the PYTHON environment variable to target a specific python version. By default the newest installed version of Python will be used or the preferred distribution version when creating pacakges. * Fixed --enable-pyzfs configure checks so they are run when --enable-pyzfs=check and --enable-pyzfs=yes. * Enabled pyzfs for Python 3.4 and newer, which is now supported. * Renamed pyzfs package to python<VERSION>-pyzfs and updated to install in the appropriate site location. For example, when building with --with-python=3.4 a python34-pyzfs will be created which installs in /usr/lib/python3.4/site-packages/. * Renamed the following python scripts according to the Fedora guidance for packaging utilities in /bin - dbufstat.py -> dbufstat - arcstat.py -> arcstat - arc_summary.py -> arc_summary - arc_summary3.py -> arc_summary3 * Updated python-cffi package name. On CentOS 6, CentOS 7, and Amazon Linux it's called python-cffi, not python2-cffi. For Python3 it's called python3-cffi or python3x-cffi. * Install one version of arc_summary. Depending on the version of Python available install either arc_summary2 or arc_summary3 as arc_summary. The user output is only slightly different. Reviewed-by: John Ramsden <[email protected]> Reviewed-by: Neal Gompa <[email protected]> Reviewed-by: loli10K <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #8096
* Added column definitions to arcstat.pyGregor Kopka2018-10-291-1/+8
| | | | | | | | | | | | | grow: ARC Grow enabled (!arc_no_grow) free: ARC Free memory (arc_sys_free) need: ARC Reclaim need (arc_need_free) Fixed alignment issues (mread had wrong width). Reviewed-by: George Melikov <[email protected]> Reviewed-by: Giuseppe Di Natale <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Gregor Kopka <[email protected]> Closes #8058
* Fix flake8 "invalid escape sequence 'x'" warningBrian Behlendorf2018-10-241-1/+0
| | | | | | | | | | | | | | | | From, https://lintlyci.github.io/Flake8Rules/rules/W605.html As of Python 3.6, a backslash-character pair that is not a valid escape sequence now generates a DeprecationWarning. Although this will eventually become a SyntaxError, that will not be for several Python releases. Note 'float_pobj' was simply removed from arcstat.py since it was entirely unused. Reviewed-by: John Kennedy <[email protected]> Reviewed-by: Richard Elling <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #8056
* Fix arcstat.py handling of unsupported optionsLOLi2018-08-181-1/+1
| | | | | | | | | | | This change allows the arcstat.py script to handle unsupported options gracefully and print both error and usage messages when one such option is provided. Reviewed-by: Giuseppe Di Natale <[email protected]> Reviewed-by: George Melikov <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: loli10K <[email protected]> Closes #7799
* Fix some typosJohn Eismeier2018-02-281-2/+2
| | | | | | | Reviewed-by: Giuseppe Di Natale <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed by: George Melikov <[email protected]> Signed-off-by: John Eismeier <[email protected]> Closes #7237
* arcstat: flush stdout / outfile after each lineFabian-Gruenbichler2017-10-261-0/+1
| | | | | | | | | | | Otherwise, if arcstat gets interrupted before the desired number of iterations is reached, the output file will be empty (both if set via '-o' or via shell redirection). Reviewed-by: George Melikov <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Giuseppe Di Natale <[email protected]> Signed-off-by: Fabian Grünbichler <[email protected]> Closes #6775
* Correct flake8 errors after STYLE builder updateGiuseppe Di Natale2017-10-231-1/+1
| | | | | | | | | Fix new flake8 errors related to bare excepts and ambiguous variable names due to a STYLE builder update. Reviewed-by: George Melikov <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Giuseppe Di Natale <[email protected]> Closes #6776
* Correct exit code for dbufstat -v and arcstat -vGiuseppe Di Natale2016-10-061-1/+1
| | | | | | | | Both scripts were returning an error code of 1 when using the -v argument. -v should exit with an error code of 0. Signed-off-by: Giuseppe Di Natale <[email protected]>
* Correct style in arcstat and arc_summaryGiuseppe Di Natale2016-10-061-1/+6
| | | | | | | Fix arcstat and arc_summary so they pass flake8 python code style checks. Signed-off-by: Giuseppe Di Natale <[email protected]>
* Update arcstat.py to remove deprecated rmis reference.cable29992015-12-041-2/+2
| | | | | | | | | Running arcstat.py -x currently throws KeyError due to rmis being absent, it was removed in commit ca0bf58. Signed-off-by: cable2999 <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #3931
* Support parallel build trees (VPATH builds)Turbo Fredriksson2015-07-171-2/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Build products from an out of tree build should be written relative to the build directory. Sources should be referred to by their locations in the source directory. This is accomplished by adding the 'src' and 'obj' variables for the module Makefile.am, using relative paths to reference source files, and by setting VPATH when source files are not co-located with the Makefile. This enables the following: $ mkdir build $ cd build $ ../configure \ --with-spl=$HOME/src/git/spl/ \ --with-spl-obj=$HOME/src/git/spl/build $ make -s This change also has the advantage of resolving the following warning which is generated by modern versions of automake. Makefile.am:00: warning: source file 'xxx' is in a subdirectory, Makefile.am:00: but option 'subdir-objects' is disabled Signed-off-by: Turbo Fredriksson <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #1082
* Illumos 5497 - lock contention on arcs_mtxPrakash Surya2015-06-111-2/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Reviewed by: George Wilson <[email protected]> Reviewed by: Matthew Ahrens <[email protected]> Reviewed by: Richard Elling <[email protected]> Approved by: Dan McDonald <[email protected]> Porting notes and other significant code changes: The illumos 5368 patch (ARC should cache more metadata), which was never picked up by ZoL, is mostly reverted by this patch. Since ZoL relies on the kernel asynchronously calling the shrinker to actually reap memory, the shrinker wakes up arc_reclaim_waiters_cv every time it runs. The arc_adapt_thread() function no longer calls arc_do_user_evicts() since the newly-added arc_user_evicts_thread() calls it periodically. Notable conflicting ZoL commits which conflicted with this patch or whose effects are either duplicated or un-done by this patch: 302f753 - Integrate ARC more tightly with Linux 39e055c - Adjust arc_p based on "bytes" in arc_shrink f521ce1 - Allow "arc_p" to drop to zero or grow to "arc_c" 77765b5 - Remove "arc_meta_used" from arc_adjust calculation 94520ca - Prune metadata from ghost lists in arc_adjust_meta Trace support for multilist_insert() and multilist_remove() has been added and produces the following output: fio-12498 [077] .... 112936.448324: zfs_multilist__insert: ml { offset 240 numsublists 80 sublistidx 63 } fio-12498 [077] .... 112936.448347: zfs_multilist__remove: ml { offset 240 numsublists 80 sublistidx 29 } The following arcstats have been removed: recycle_miss - Used by arcstat.py and arc_summary.py, both of which have been updated appropriately. l2_writes_hdr_miss The following arcstats have been added: evict_not_enough - Number of times arc_evict_state() was unable to evict enough buffers to reach its target amount. evict_l2_skip - Number of times arc_evict_hdr() skipped eviction because it was being written to the l2arc. l2_writes_lock_retry - Replaces l2_writes_hdr_miss. Number of times l2arc_write_done() failed to acquire hash_lock (and re-tries). arc_meta_min - Shows the value of the zfs_arc_meta_min module parameter (see below). The "index" column of the "dbuf" kstat has been removed since it doesn't have a direct analog in the new multilist scheme. Additional multilist- related stats could be added in the future but would likely require extensions to the mulilist API. The following module parameters have been added: zfs_arc_evict_batch_limit - Number of ARC headers to free per sub-list before moving on to the next sub-list. zfs_arc_meta_min - Enforce a floor on the amount of metadata in the ARC. zfs_arc_num_sublists_per_state - Number of multilist sub-lists per ARC state. zfs_arc_overflow_shift - Controls amount by which the ARC must exceed the target size to be considered "overflowing". Ported-by: Tim Chase <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]
* Print header properly when terminal resizesIsaac Huang2014-11-141-6/+15
| | | | | | | | | Added a handler for SIGWINCH, so that one header is printed per screen even when the terminal resizes. Signed-off-by: Isaac Huang <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #2847
* Fix inaccurate field descriptionsIsaac Huang2014-11-141-5/+5
| | | | | | | | | | The field descriptions from arcstat.py -v for the demand accesses are inaccurate. They all begin with "Demand Data" yet the fields actually covered both demand data and demand meta-data accesses. Signed-off-by: Isaac Huang <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #2842
* sighandler() should take 2 argumentsIsaac Huang2014-03-201-6/+2
| | | | | | | | | | | | | Stopping arcstat.py with ^C always ends up with error: TypeError: sighandler() takes no arguments (2 given) Since no special signal handling was done in sighandler(), it's simpler to just set SIGINT handler to SIG_DFL, which terminates the script. Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: Isaac Huang <[email protected]> Closes #2179
* Revert "sighandler() should take 2 arguments"Brian Behlendorf2014-03-201-1/+1
| | | | | | | | This reverts commit 0bb89b6c594259829556f6dea5a89e722f214fd3 in favor of a cleaner implementation. Signed-off-by: Brian Behlendorf <[email protected]> Issue #2182
* sighandler() should take 2 argumentsIsaac Huang2014-03-121-1/+1
| | | | | | | | | | | | | | Stopping arcstat.py with ^C always ends up with error: TypeError: sighandler() takes no arguments (2 given) This patch corrects the error by updating the signal handler to take the correct number of arguments. Signed-off-by: Isaac Huang <[email protected]> Signed-off-by: Prakash Surya <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #2182
* Make arcstat.py default to one header per screenIsaac Huang2014-03-121-0/+13
| | | | | | | | | | | | Today arcstat.py prints one header every hdr_intr (20 by default) lines. It would be more consistent with out utilities like vmstat if hdr_intr defaulted to terminal window size, i.e. one header per screenful of outputs. Signed-off-by: Isaac Huang <[email protected]> Signed-off-by: Prakash Surya <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #2183
* Include l2asize in arcstatcburroughs2014-03-041-0/+2
| | | | | | | | | | | | | For consistency with upstream pull in the l2asize update after reworking it from Perl to Python. References: https://github.com/mharsch/arcstat/pull/11 https://github.com/mharsch/arcstat/pull/12 Signed-off-by: cburroughs <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #2122
* Python 3 fixesMatthew Thode2013-11-081-6/+6
| | | | | | | | Future proofing for compatibility with newer versions of Python. Signed-off-by: Matthew Thode <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Issue #1838
* pep8 code readability changesMatthew Thode2013-11-081-12/+11
| | | | | | | | | | | Update the code to follow the pep8 style guide. References: http://www.python.org/dev/peps/pep-0008/ Signed-off-by: Matthew Thode <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Issue #1838
* Add dbufstat.py commandBrian Behlendorf2013-10-251-5/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | The dbufstat.py command was added to provide a conveniant way to easily determine what ZFS is caching. The script consumes the raw /proc/spl/kstat/zfs/dbufs kstat data can consolidates it in to a more human readable form. This was designed primarily as a tool to aid developers but it may also be useful for advanced users who want more visibility in to what the ARC is caching. When run without options dbufstat.py will default to showing a list of all objects with at least one buffer present in the cache. The total cache space consumed by that object will be printed on the right along with the object type. Similar to the arcstats.py command the -x option may used to display additional fields. Two other modes of operation are also supported by dbufstat.py and the expectation is additional display modes may be added as needed. The -t option will summerize the total number of bytes cached for each object type, and the -b option will show every dbuf currently cached. The script was designed to be consistent with arcstat.py and includes most of the same options and funcationality. Signed-off-by: Prakash Surya <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]>
* Modified arcstat.py to run on linuxChrist Schlacta2013-06-182-19/+7
| | | | | | | | | * Modified kstat_update() to read arcstats from proc. * Fix shebang. * Added Makefile.am entries for arcstat.py Signed-off-by: Brian Behlendorf <[email protected]> Closes #1506
* Added arcstat.py from FreeNASChrist Schlacta2013-06-181-0/+454
Original source: http://support.freenas.org/browser/nanobsd/Files/usr/local/bin/arcstat.py Signed-off-by: Brian Behlendorf <[email protected]> Issue #1506
generated by cgit v1.2.3 (git 2.39.1) at 2024-12-04 12:42:11 +0000