Linux内存监控工具

Linux内存监控工具

一、free

该工具主要是显示系统里可用和已用的内存

Linux通常按一定的算法把常用的数据加载到系统的虚拟内存buffers和cached中，以便于用户程序在访问系统资源更快。而由free 查看到的buffers是用于存放元数据，而cached是用于存放真实的文件内容。

由上图free -k的输出结果中可知：

系统总物理内存(total)是4144656K(约4G);

已用(Mem行对应的used)的物理内存是3871932K(约3.8G,注:这里包含了buffers的152460K(约152M)和cached的2253060K(2.2G).)，他包含系统的buffers和cached的。

-/+ buffers/cache对应的used是14612K(约1.4G),也就是Mem行used(3871932K)-Mem行buffers(152460K)-

Mem行cached(2253060K)=14612K(约1.4G).所以实际上可用于分配的物理内

存(-/+ buffers/cache行对应的free)是2678244K(约2.6G).

Shared在man手册里提示应该忽略(man free:The shared memory column should be ignored; it is obsolete.)。

Mem行对应的free对应的274220K(约274M).其实这个free是有一定的:不能低于min_free_kbytes。

min_free_kbytes用于计算系统里lowmem zone(物理内存0-6MB之间的zone)的值(This is used to force the Linux VM to keep a minimum number of kilobytes free. The VM uses this number to compute a pages_min value for each lowmem zone in the system. Each lowmem zone gets a number of reserved free pages based proportionally on its size.).

计算方式参见mm/page_alloc.c的min_free_kbytes = sqrt(lowmem_kbytes * 16)

上述值是一定的公式计算

系统的lowmem是872656KB

[root@crm_10 /root]grep LowTotal /proc/meminfo

LowTotal: 872656

min_free_kbytes=sqrt(872656*16) 约等于 3797

二、ps,top

这两个工具在内存监视方面有很大的相似性，所以一并说一下:

下面top里的VIRT相当于ps 里的VSZ:指用于这个任务的总虚拟内存(虚拟内存包括物理内存和swap交换分区),包括所有的代码、数据、共享库以及已经被out到swap分区的数据。/* The total amount of virtual memory used by the task. It includes all code, data and shared libraries plus pages that have been swapped out.*/

而top里的RES 相当于ps 里的RSS: 指用于这个任务的没被out到swap分区的总物理内存/* resident set size, the non-swapped physical memory that a task has used */

top里的%MEM: 指这个任务的RES占总物理内存的比率/* Memory usage (RES) A task's currently used share of available physical memory.*/

三、vmstat

显示的值跟用free工具查看到的值相似。一般情况下:只要swap一列的si/so数值不超过1024即可。

Swap

si: Amount of memory swapped in from disk (/s).

so: Amount of memory swapped to disk (/s).

四:VFS里的meminfo信息:

Dirty：是指数据已写入内存，但还没同步到外存(磁盘)的数据量.

Slab:为了提供内核空间以页分配对有些调用(只需小内存)不合适的一种内存分配方式,提出Pool的概念。

Vmalloc:为了解决非连续性内存的使用，提供的一种内存分配方式(采用链表)。

CommitLimit:指当前可以分配给程序使用的虚拟内存(只有当vm.overcommit_memory的值设置为2时，CommitLimit才有意义)

CommitLimit: Based on the overcommit ratio ('vm.overcommit_ratio'),

this is the total amount of memory currently available to

be allocated on the system. This limit is only adhered to

if strict overcommit accounting is enabled (mode 2 in

'vm.overcommit_memory').

The CommitLimit is calculated with the following formula:

CommitLimit = ('vm.overcommit_ratio' * Physical RAM) + Swap

For example, on a system with 1G of physical RAM and 7G

of swap with a `vm.overcommit_ratio` of 30 it would

yield a CommitLimit of 7.3G.

For more details, see the memory overcommit documentation

in vm/overcommit-accounting.

Committed_AS:指当前已分配给程序使用的总虚拟内存(包含已分配给进程但还没使用的内存)

Committed_AS: The amount of memory presently allocated on the system.

The committed memory is a sum of all of the memory which

has been allocated by processes, even if it has not been

\"used\" by them as of yet. A process which malloc()'s 1G

of memory, but only touches 300M of it will only show up

as using 300M of memory even if it has the address space

allocated for the entire 1G. This 1G is memory which has

been \"committed\" to by the VM and can be used at any time

by the allocating application. With strict overcommit

enabled on the system (mode 2 in 'vm.overcommit_memory'),

allocations which would exceed the CommitLimit (detailed

above) will not be permitted. This is useful if one needs

to guarantee that processes will not fail due to lack of

memory once that memory has been successfully allocated.

HugePagesize:在X86架构下,通常linux给内存分页时，默认是每页是4KB,而有些应用自己可以管理内存(例如db,java……),所以可以把页的大小分大些(在32位下:4K或4M,在PAE模式下可以把每页设置为2M.在位下: 4K, 8K, K, 256K, 1M, 4M, 16M,256M),这样可以增加TLB(一个存储线性地址和物理地址对应表的高速缓冲器)存储的条目,这样就可减少线性地址到物理地址转换的过程.可通过vm.hugetlb_shm_group 和

vm.nr_hugepages参数进行调整。具体可查看/usr/share/doc/kernel-doc-`uname –r|cut -d- -f1`/Documentation/vm/hugetlbpage.txt(如果你的机器上已经安装了kernel-doc)

The intent of this file is to give a brief summary of hugetlbpage support in

the Linux kernel. This support is built on top of multiple page size support

that is provided by most of modern architectures. For example, IA-32

architecture supports 4K and 4M (2M in PAE mode) page sizes, IA-

architecture supports multiple page sizes 4K, 8K, K, 256K, 1M, 4M, 16M,

256M. A TLB is a cache of virtual-to-physical translations. Typically this

is a very scarce resource on processor. Operating systems try to make best

use of limited number of TLB resources. This optimization is more critical

now as bigger and bigger physical memories (several GBs) are more readily

available.

Linux C/C++ 内存泄漏检测工具：Valgrind

Valgrind 是一款 Linux下（支持 x86、x86_和ppc32）程序的内存调试工具，它可以对编译后的二进制程序进行内存使用监测（C语言中的malloc和free，以及C++中的new和delete），找出内存泄漏问题。

Valgrind 中包含的 Memcheck 工具可以检查以下的程序错误：

使用未初始化的内存 (Use of uninitialised memory)

使用已经释放了的内存 (Reading/writing memory after it has been free’d)

使用超过malloc分配的内存空间(Reading/writing off the end of malloc’d blocks)

对堆栈的非法访问 (Reading/writing inappropriate areas on the stack)

申请的空间是否有释放 (Memory leaks – where pointers to malloc’d blocks are lost forever)

malloc/free/new/delete申请和释放内存的匹配(Mismatched use of

malloc/new/new [] vs free/delete/delete [])

src和dst的重叠(Overlapping src and dst pointers in memcpy() and related functions)

重复free

1、编译安装 Valgrind：

wget http://valgrind.org/downloads/valgrind-3.4.1.tar.bz2

tar xvf valgrind-3.4.1.tar.bz2

cd valgrind-3.4.1/

./configure --prefix=/usr/local/webserver/valgrind

make

make install

2、使用示例：对“ls”程序进程检查，返回结果中的“definitely lost: 0 bytes in 0 blocks.”表示没有内存泄漏。

[root@xoyo42 /]# /usr/local/webserver/valgrind/bin/valgrind

--tool=memcheck --leak-check=full ls /

==1157== Memcheck, a memory error detector.

==1157== Copyright (C) 2002-2008, and GNU GPL'd, by Julian Seward et al.

==1157== Using LibVEX rev 1884, a library for dynamic binary translation.

==1157== Copyright (C) 2004-2008, and GNU GPL'd, by OpenWorks LLP.

==1157== Using valgrind-3.4.1, a dynamic binary instrumentation framework.

==1157== Copyright (C) 2000-2008, and GNU GPL'd, by Julian Seward et al.

==1157== For more details, rerun with: -v

==1157==

bin data0 dev home lib media mnt opt root selinux sys tcsql.db.idx.pkey.dec ttserver.pid var

boot data1 etc lib lost+found misc net proc sbin srv tcsql.db tmp usr

==1157==

==1157== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 5 from 1)

==1157== malloc/free: in use at exit: 28,471 bytes in 36 blocks.

==1157== malloc/free: 166 allocs, 130 frees, 51,377 bytes allocated.

==1157== For counts of detected errors, rerun with: -v

==1157== searching for pointers to 36 not-freed blocks.

==1157== checked 174,0 bytes.

==1157==

==1157== LEAK SUMMARY:

==1157== definitely lost: 0 bytes in 0 blocks.

==1157== possibly lost: 0 bytes in 0 blocks.

==1157== still reachable: 28,471 bytes in 36 blocks.

==1157== suppressed: 0 bytes in 0 blocks.

==1157== Reachable blocks (those to which a pointer was found) are not shown.

==1157== To see them, rerun with: --leak-check=full --show-reachable=yes

3、使用示例：对一个使用libevent库编写的“httptest”程序进程检查，返回结果中的“definitely lost: 255 bytes in 5 blocks.”表示发生内存泄漏。

[root@xoyo42 tcsql-0.1]# /usr/local/webserver/valgrind/bin/valgrind

--tool=memcheck --leak-check=full ./httptest

==1274== Memcheck, a memory error detector.

==1274== Copyright (C) 2002-2008, and GNU GPL'd, by Julian Seward et al.

==1274== Using LibVEX rev 1884, a library for dynamic binary translation.

==1274== Copyright (C) 2004-2008, and GNU GPL'd, by OpenWorks LLP.

==1274== Using valgrind-3.4.1, a dynamic binary instrumentation framework.

==1274== Copyright (C) 2000-2008, and GNU GPL'd, by Julian Seward et al.

==1274== For more details, rerun with: -v

==1274==

==1274== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 1005 from 2)

==1274== malloc/free: in use at exit: 402,291 bytes in 74 blocks.

==1274== malloc/free: 15,939 allocs, 15,865 frees, 6,281,523 bytes allocated.

==1274== For counts of detected errors, rerun with: -v

==1274== searching for pointers to 74 not-freed blocks.

==1274== checked 682,468,160 bytes.

==1274==

==1274== 255 bytes in 5 blocks are definitely lost in loss record 17 of 32

==1274== at 0x4A05FBB: malloc (vg_replace_malloc.c:207)

==1274== by 0x3C1D809BC6: evhttp_decode_uri (http.c:2105)

==1274== by 0x401C75: tcsql_handler /data0/tcsql/cankao/tcsql-0.1/tcsql)

==1274== by 0x3C1D80C88F: evhttp_get_body (http.c:1582)

==1274== by 0x3C1D8065F7: event_base_loop (event.c:392)

==1274== by 0x403E2F: main (in /data0/tcsql/cankao/tcsql-0.1/tcsql)

(in

==1274==

==1274== LEAK SUMMARY:

==1274== definitely lost: 255 bytes in 5 blocks.

==1274== possibly lost: 0 bytes in 0 blocks.

==1274== still reachable: 402,036 bytes in 69 blocks.

==1274== suppressed: 0 bytes in 0 blocks.

==1274== Reachable blocks (those to which a pointer was found) are not shown.

==1274== To see them, rerun with: --leak-check=full --show-reachable=yes

检查httptest程序，发现有一处“char *decode_uri =

evhttp_decode_uri(evhttp_request_uri(req));”中的“decode_uri”没有被free，再程序处理完成后加上“free(decode_uri);”后，再使用Valgrind检查，结果已经是“definitely lost: 0 bytes in 0 blocks.”。