The ARC cache is similar to the buffer cache, but just dedecated to ZFS, so there is generally nothing to worry about it.
On ZOL 0.6.3(didn't check other version yet), it's set max to use half of memory on the host.
The difference between buffer cache memory and ARC cache one is that for regular application, the first one is immediately available to allocation while the ARC cache one is not. In general ZFS monitors the free RAM available and when too low, it releases RAM to other consumers.
According to you application, you may want to set ZFS ARC maximum.
Set ZFS ARC MAX, zfs_arc_max
This setting is done in the /etc/modprobe.d/zfs.conf file.
For example, should you want the ARC never to exceed 24 GB, add this line:
options zfs zfs_arc_max=25769803776
The number is for my server(32GB memory), which is a file server, no other application running on it. So leave 8GB to OS and cluster file system.
This paramater is the most popular one that people like to tune their ZFS. Similarily, you can set other zfs tuning parameters in zfs.conf too. The list of tunable parameters you can find in
Note: Before you touch other parameters, I suggest you read this article first
ZFS tunable parameters, though it is for Open Solaris ZFS, still worth to read.
Get ZFS ARC statistics
To get the current ARC size and various other ARC statistics, run this command:
The size metric will show the current size of the ARC.
Tool to use for ZFS ARC monitoring
There is a tool called arcstat.pl on Solaris, what about ZOL?
It's called arcstat.py, right in the zfs linux package
# rpm -qf /usr/bin/arcstat.py
# arcstat.py 10 10
time read miss miss% dmis dm% pmis pm% mmis mm% arcsz c
13:23:43 2 0 0 0 0 0 0 0 0 24G 24G
13:23:53 7.2K 2.8K 38 0 0 2.8K 100 0 0 23G 24G
13:24:03 7.5K 2.8K 37 0 0 2.8K 100 0 0 24G 24G
13:24:13 7.9K 2.9K 36 0 0 2.9K 100 0 0 24G 24G
13:24:23 6.3K 2.5K 40 0 0 2.5K 100 0 0 24G 24G
13:24:33 7.1K 2.8K 39 0 0 2.8K 100 0 0 24G 24G
# arcstat.py -h
Usage: arcstat.py [-hvx] [-f fields] [-o file] [-s string] [interval [count]]
-h : Print this help message
-v : List all possible field headers and definitions
-x : Print extended stats
-f : Specify specific fields to print (see -v)
-o : Redirect output to the specified file
-s : Override default field separator with custom character or string
arcstat.py -o /tmp/a.log 2 10
arcstat.py -s "," -o /tmp/a.log 2 10
arcstat.py -f time,hit%,dh%,ph%,mh% 1
Free up ARC
Idealy, ZFS will release ARC memory reservation when needed by other application. If you want to force it to do so, then you have to export zfs pool which is using ARC, umount a zfs filesystem may not force zfs release ARC reserve.
zpool export <zfspool>