用Table在页面上显示2万条数据，占用内存300M。用windbg看看内存都给了谁（一）Viewstate

近日，客户说在某检索画面检索大量数据后，再执行CSV下载，出现服务器异常，显示OutOfMemoryException。

先用任务管理器/性能查看了下CSV下载页面的内存使用情况，并在检索及CSV下载完成后各dump了一次。

执行前：

检索及画面表示后：

CSV下载后：

从图上看，在检索及页面表示时一口气吃了280M可用内存。dump的文件大小，检索：370M，CSV下载：400M。

用windbg分析第一个dump文件（检索）

(部分略)

dumpheap -stat

total 8,597,930 objects
Statistics:
        MT      Count     TotalSize Class Name
......
......
0x040ca234          1            80 System.Data.DataSet
......
......
0x044297f4     23,170       278,040 System.Web.UI.WebControls.TableCellCollection
0x040c7920     23,176       556,224 System.Web.UI.WebControls.Unit
0x0442a45c     23,170       648,760 System.Web.UI.WebControls.TableRow/CellControlCollection
0x040cc3a4     23,169       926,760 System.Data.DataRow
0x04429094     23,170     2,317,000 System.Web.UI.WebControls.TableRow
0x011d26b0      3,020     4,097,936 System.Int32[]
0x79baaff0    486,796     5,841,552 System.Int32
0x0442252c    509,744     6,116,928 System.Web.UI.WebControls.FontInfo
0x04428c9c    517,396     6,208,752 System.Web.UI.WebControls.HorizontalAlign
0x0015aa80    112,105     6,622,976      Free
0x040c6c90    509,761     8,156,176 System.Web.UI.StateBag
0x03c8c450    509,764    10,195,280 System.Collections.Specialized.HybridDictionary
0x03c8ca90    509,762    12,234,288 System.Collections.Specialized.ListDictionary
0x040c4b64    509,744    14,272,832 System.Web.UI.ControlCollection
0x040c6ff4 1,019,485    16,311,760 System.Web.UI.StateItem
0x0442a334    509,740    20,389,600 System.Web.UI.WebControls.TableItemStyle
0x03c8cc24 1,019,488    20,389,760 System.Collections.Specialized.ListDictionary/DictionaryNode
0x79b946b0    686,345    24,009,348 System.String
0x011d2c3c        877    25,943,368 System.Byte[]
0x011d209c    544,614    29,757,316 System.Object[]
0x040c3f14    509,761    38,741,836 System.Web.UI.LiteralControl
0x044293b4    509,740    48,935,040 System.Web.UI.WebControls.TableCell

Total 8,597,930 objects, Total size: 303,663,236

可以看到，一共有23，170条TableRow，509，740个TableCell，每行数据22列。 TableCell、LiteralControl这两项就占用了86M，

为什么会有686,345个String、686,345个Object[]呢，一步步分析。

从上面可以看到Dataset只有80bytes，这只是它结构本身的大小，不包括其下DataRow等的实际大小。

用!objsize查看它的实际大小

!dumpheap -mt 0x040ca234
Using our cache to search the heap.
   Address         MT     Size Gen
0x1632d88 0x40ca234       80    2 System.Data.DataSet
Statistics:
        MT      Count     TotalSize Class Name
0x040ca234          1            80 System.Data.DataSet

!objsize 0x1632d88

sizeof(0x1632d88) = 24,741,124 (0x1798504) bytes (System.Data.DataSet)

总共24M。

首先引起我注意的是System.Web.UI.StateBag，MSDN上是这么说的：管理 ASP.NET 服务器控件（包括页）的视图状态。

ViewState机制由浅入深3 中的表述：StateBag中保存Key/Value对，Key是String类型，Value是Object类型。但是在StateBag内部保存Value不是Object类型，而是将Object类型转换为StateItem类型然后保存，从StateBag中取出的时候再将StateItem类型转换为Object类型，也就是说StateBag中的Key/Value对实际上是String/StateItem类型。转换过程是在StateBag内部实现客户感觉不到。

MT Count TotalSize Class Name
0x040c6c90 509,761 8,156,176 System.Web.UI.StateBag

!dumpheap -mt 0x040c6c90 -random ，然后用!objsize查看了几个

   Address         MT     Size Gen
0x161eaa4 0x40c6c90       16    2 System.Web.UI.StateBag    168
0x161ec14 0x40c6c90       16    2 System.Web.UI.StateBag    160
0x1630070 0x40c6c90       16    2 System.Web.UI.StateBag    72
0x162a764 0x40c6c90       16    2 System.Web.UI.StateBag    36
0x161f888 0x40c6c90       16    2 System.Web.UI.StateBag    256
0x161f77c 0x40c6c90       16    2 System.Web.UI.StateBag    252

最后一列是该对象的实际大小。

拿最多的那个看了看。

!do 0x161f888

Name: System.Web.UI.StateBag
MethodTable 0x040c6c90
EEClass 0x040f1bc0
Size 16(0x10) bytes
GC Generation: 2
mdToken: 0x020001ba (c:"windows"assembly"gac"system.web"1.0.5000.0__b03f5f7f11d50a3a"system.web.dll)
FieldDesc*: 0x040c6904
        MT      Field     Offset                 Type       Attr      Value Name
0x040c6c90 0x4000d86      0x4                CLASS   instance 0x0161f898 bag
0x040c6c90 0x4000d87      0x8       System.Boolean   instance 1 marked
0x040c6c90 0x4000d88      0x9       System.Boolean   instance 0 ignoreCase

bag是个什么东东呢

!do 0x0161f898

Name: System.Collections.Specialized.HybridDictionary
MethodTable 0x03c8c450
EEClass 0x03ca2e8c
Size 20(0x14) bytes
GC Generation: 2
mdToken: 0x02000172 (c:"windows"assembly"gac"system"1.0.5000.0__b77a5c561934e089"system.dll)
FieldDesc*: 0x03c8c254
        MT      Field     Offset                 Type       Attr      Value Name
0x03c8c450 0x4000a77      0x4                CLASS   instance 0x0161f8bc list
0x03c8c450 0x4000a78      0x8                CLASS   instance 0x00000000 hashtable
0x03c8c450 0x4000a79      0xc       System.Boolean   instance 0 caseInsensitive
0x03c8c450 0x4000a75        0                CLASS     shared   static hashCodeProvider
    >> Domain:Value 0x00155980:NotInit 0x00193830:0x0162d6a4 <<
0x03c8c450 0x4000a76      0x4                CLASS     shared   static comparer
    >> Domain:Value 0x00155980:NotInit 0x00193830:0x0155d574 <<

原来bag是ystem.Collections.Specialized.HybridDictionary，MSDN：在集合较小时，使用 ListDictionary 来实现 IDictionary，

然后当集合变大时，切换到Hashtable。（集合较小是指key/value键值对的个数较少）

很明显，list就是上面提到的ListDictionary，hashtable嘛就是Hashtable了。继续查看list成员。

!do 0x0161f8bc

Name: System.Collections.Specialized.ListDictionary
MethodTable 0x03c8ca90
EEClass 0x03ca32a4
Size 24(0x18) bytes
GC Generation: 2
mdToken: 0x02000174 (c:"windows"assembly"gac"system"1.0.5000.0__b77a5c561934e089"system.dll)
FieldDesc*: 0x03c8c8fc
        MT      Field     Offset                 Type       Attr      Value Name
0x03c8ca90 0x4000a7a      0x4                CLASS   instance 0x0161f8d4 head
0x03c8ca90 0x4000a7b      0xc         System.Int32   instance 2 version
0x03c8ca90 0x4000a7c     0x10         System.Int32   instance 2 count
0x03c8ca90 0x4000a7d      0x8                CLASS   instance 0x00000000 comparer

head是个什么东东呢？

!do 0x0161f8d4

Name: System.Collections.Specialized.ListDictionary/DictionaryNode
MethodTable 0x03c8cc24
EEClass 0x03ca3394
Size 20(0x14) bytes
GC Generation: 2
mdToken: 0x02000178 (c:"windows"assembly"gac"system"1.0.5000.0__b77a5c561934e089"system.dll)
FieldDesc*: 0x03c8cbd4
        MT      Field     Offset                 Type       Attr      Value Name
0x03c8cc24 0x4000a89      0x4                CLASS   instance 0x0155d610 key
0x03c8cc24 0x4000a8a      0x8                CLASS   instance 0x0161f8ac value
0x03c8cc24 0x4000a8b      0xc                CLASS   instance 0x0161f8f8 next

再看一下next

!do 0x0161f8f8

Name: System.Collections.Specialized.ListDictionary/DictionaryNode
MethodTable 0x03c8cc24
EEClass 0x03ca3394
Size 20(0x14) bytes
GC Generation: 2
mdToken: 0x02000178 (c:"windows"assembly"gac"system"1.0.5000.0__b77a5c561934e089"system.dll)
FieldDesc*: 0x03c8cbd4
        MT      Field     Offset                 Type       Attr      Value Name
0x03c8cc24 0x4000a89      0x4                CLASS   instance 0x0155d930 key
0x03c8cc24 0x4000a8a      0x8                CLASS   instance 0x0161f8e8 value
0x03c8cc24 0x4000a8b      0xc                CLASS   instance 0x00000000 next

从head、next可以看出是个链表结构。

继续查看head的key、value

!do 0x0155d610

String: CssClass

!do 0x0161f8ac

Name: System.Web.UI.StateItem
MethodTable 0x040c6ff4
EEClass 0x040f1d14
Size 16(0x10) bytes
GC Generation: 2
mdToken: 0x020001bb (c:"windows"assembly"gac"system.web"1.0.5000.0__b03f5f7f11d50a3a"system.web.dll)
FieldDesc*: 0x040c6f70
        MT      Field     Offset                 Type       Attr      Value Name
0x040c6ff4 0x4000d89      0x4                CLASS   instance 0x0155d8f8 value
0x040c6ff4 0x4000d8a      0x8       System.Boolean   instance 0 isDirty

!do 0x0155d8f8

String: button

原来是CssClass=button，key是string对象，value是StateItem对象，这与前面ViewState机制由浅入深3 的表述吻合。

StateBag-->bag(HybridDictionary)-->list(ListDictionary)-->head(ListDictionary/DictionaryNode)-->key(string)
value(StateItem)
next(ListDictionary/DictionaryNode)

到这里可以知道为什么托管堆上会有那么多StateBag、HybridDictionary、ListDictionary、Dictionary/DictionaryNode、StateItem、string对象了。

至于其他的，等下回了。

-----------------------------------------

2009/4/1

-----------------------------------------

近日在调查一个高内存+高cpu的处理，昨日终于找到症结所在。

先前展示了ListDictionary类的内部结构，key/value键值对以单向链表形式存放。那HashTable内部结构又是什么样的呢，带着这个疑问我继续了探查。

!dumpheap -mt 0x79bab9b4
Loaded Son of Strike data table version 5 from "C:"WINDOWS"Microsoft.NET"Framework"v1.1.4322"mscorwks.dll"
Loading the heap objects into our cache.
   Address         MT     Size Gen
0x014c191c 0x79bab9b4       52 2 System.Collections.Hashtable
0x014c22a0 0x79bab9b4       52 2 System.Collections.Hashtable
0x014c2390 0x79bab9b4       52 2 System.Collections.Hashtable
0x014c2564 0x79bab9b4       52 2 System.Collections.Hashtable
0x014c4124 0x79bab9b4       52 2 System.Collections.Hashtable
0x014c64a0 0x79bab9b4       52 2 System.Collections.Hashtable
0x014ca4e8 0x79bab9b4       52 2 System.Collections.Hashtable
0x014cb3d8 0x79bab9b4       52 2 System.Collections.Hashtable
0x014ccfcc 0x79bab9b4       52 2 System.Collections.Hashtable
0x014cd500 0x79bab9b4       52 2 System.Collections.Hashtable
0x014cdbe8 0x79bab9b4       52 2 System.Collections.Hashtable
0x014ce0d8 0x79bab9b4       52 2 System.Collections.Hashtable

从里面随意挑了个Hashtable对象

!do 0x014c191c
Name: System.Collections.Hashtable
MethodTable 0x79bab9b4
EEClass 0x79babaf4
Size 52(0x34) bytes
GC Generation: 2
mdToken: 0x02000118 (c:"windows"microsoft.net"framework"v1.1.4322"mscorlib.dll)
FieldDesc*: 0x79babb58
        MT      Field     Offset                 Type       Attr      Value Name
0x79bab9b4 0x400039e      0x4                CLASS   instance 0x014c1b90 buckets
0x79bab9b4 0x400039f     0x1c         System.Int32   instance 2 count
0x79bab9b4 0x40003a0     0x20         System.Int32   instance 0 occupancy
0x79bab9b4 0x40003a1     0x24         System.Int32   instance 16 loadsize
0x79bab9b4 0x40003a2     0x28        System.Single   instance 0.720000 loadFactor
0x79bab9b4 0x40003a3     0x2c         System.Int32   instance 2 version
0x79bab9b4 0x40003a4      0x8                CLASS   instance 0x00000000 keys
0x79bab9b4 0x40003a5      0xc                CLASS   instance 0x00000000 values
0x79bab9b4 0x40003a6     0x10                CLASS   instance 0x00000000 _hcp
0x79bab9b4 0x40003a7     0x14                CLASS   instance 0x00000000 _comparer
0x79bab9b4 0x40003a8     0x18                CLASS   instance 0x00000000 m_siInfo
0x79bab9b4 0x400039d        0                CLASS     shared   static primes
    >> Domain:Value 0x00155980:0x014c1950 0x00193830:0x014cdc1c <<

count字段说明只有2对key/value键值对，buckets则是接下来要探查的

!do 0x014c1b90
Name: System.Collections.Hashtable/bucket[]
MethodTable 0x011d2970
EEClass 0x011d28ec
Size 288(0x120) bytes
GC Generation: 2
Array: Rank 1, Type VALUETYPE
Element Type: System.Collections.Hashtable/bucket
Content: 23 items

很明显，buckets 是个System.Collections.Hashtable/bucket 数组，对象本身占用288个字节。

*这里有点要注意，content字段说明该数组有23个元素，这与前面count字段的2 看上去有出入，怎么解释呢。

bucket数组占用了23个元素的内存空间，但实际上只用到2个。

dd 0x014c1b90
014c1b90 011d2970 00000017 00000000 00000000
014c1ba0 00000000 00000000 00000000 00000000
014c1bb0 00000000 00000000 00000000 00000000
014c1bc0 00000000 00000000 00000000 00000000
014c1bd0 00000000 00000000 00000000 00000000
014c1be0 00000000 00000000 00000000 00000000
014c1bf0 00000000 00000000 00000000 00000000
014c1c00 00000000 00000000 00000000 00000000

看到这边我傻眼了，按照前面所说应该有2个key/value键值对，怎么这里一个也没有呢...

且慢，前面说到该数组占用288个字节，而这里只显示了128个字节，后面还有呢...

整个是这样子的：

014c1b90 011d2970 00000017 00000000 00000000
014c1ba0 00000000 00000000 00000000 00000000
014c1bb0 00000000 00000000 00000000 00000000
014c1bc0 00000000 00000000 00000000 00000000
014c1bd0 00000000 00000000 00000000 00000000
014c1be0 00000000 00000000 00000000 00000000
014c1bf0 00000000 00000000 00000000 00000000
014c1c00 00000000 00000000 00000000 00000000
014c1c10 00000000 00000000 00000000 00000000
014c1c20 00000000 00000000 00000000 00000000
014c1c30 00000000 014c9c38 014ca1d8 035e2a40
014c1c40 00000000 00000000 00000000 00000000
014c1c50 00000000 00000000 00000000 00000000
014c1c60 00000000 00000000 00000000 00000000
014c1c70 00000000 00000000 00000000 00000000
014c1c80 00000000 00000000 00000000 00000000
014c1c90 00000000 01563530 01563574 040f65bc
014c1ca0 00000000 00000000 00000000 80000000

再次看得一头雾水，网上看了篇回复才明白。bucket对象占用12字节，也就是这里的3列。第一列是key，第二列是指向Value对象的指针，第三列没说(囧)...

探查了下第一对

!do 014c9c38
Name: System.RuntimeType
MethodTable 0x79ba98b8
EEClass 0x79ba9e90
Size 16(0x10) bytes
GC Generation: 2
mdToken: 0x020000c4 (c:"windows"microsoft.net"framework"v1.1.4322"mscorlib.dll)
FieldDesc*: 0x79ba9ef4
        MT      Field     Offset                 Type       Attr      Value Name
0x79ba6184 0x400025d      0x4                CLASS   instance 0x00000000 m_cachedData
0x79ba86d0 0x400025f        0                CLASS     shared   static FilterAttribute
    >> Domain:Value 0x00155980:0x014c223c 0x00193830:0x014cfb0c <<
0x79ba86d0 0x4000260      0x4                CLASS     shared   static FilterName
    >> Domain:Value 0x00155980:0x014c2258 0x00193830:0x014cfb28 <<
0x79ba86d0 0x4000261      0x8                CLASS     shared   static FilterNameIgnoreCase
    >> Domain:Value 0x00155980:0x014c2274 0x00193830:0x014cfb44 <<
0x79ba86d0 0x4000262      0xc                CLASS     shared   static Missing
    >> Domain:Value 0x00155980:0x014c21e4 0x00193830:0x014cfae4 <<
0x79ba86d0 0x4000263     0x24          System.Char     shared   static Delimiter
    >> Domain:Value 0x00155980:0x2e 0x00193830:0x2e <<
0x79ba86d0 0x4000264     0x10                CLASS     shared   static EmptyTypes
    >> Domain:Value 0x00155980:0x014c21f0 0x00193830:0x014cfaf0 <<
0x79ba86d0 0x4000265     0x14                CLASS     shared   static defaultBinder
    >> Domain:Value 0x00155980:0x014c9dac 0x00193830:0x0150d118 <<
0x79ba86d0 0x4000266     0x18                CLASS     shared   static valueType
    >> Domain:Value 0x00155980:0x014c2200 0x00193830:0x014c2200 <<
0x79ba86d0 0x4000267     0x1c                CLASS     shared   static enumType
    >> Domain:Value 0x00155980:0x014c2210 0x00193830:0x014c2210 <<
0x79ba86d0 0x4000268     0x20                CLASS     shared   static objectType
    >> Domain:Value 0x00155980:0x014c2220 0x00193830:0x014c2220 <<
0x79ba98b8 0x4000273      0x8         System.Int32   instance 56502848 _pData
0x79ba98b8 0x4000274        0                CLASS     shared   static valueType
    >> Domain:Value 0x00155980:0x014c2200 0x00193830:0x014c2200 <<
0x79ba98b8 0x4000275      0x4                CLASS     shared   static s_ForwardCallBinder
    >> Domain:Value 0x00155980:0x00000000 0x00193830:0x00000000 <<

!do 014ca1d8
Name: System.Security.PermissionToken
MethodTable 0x79bae5f0
EEClass 0x79bae648
Size 16(0x10) bytes
GC Generation: 2
mdToken: 0x020003cc (c:"windows"microsoft.net"framework"v1.1.4322"mscorlib.dll)
FieldDesc*: 0x79bae6ac
        MT      Field     Offset                 Type       Attr      Value Name
0x79bae5f0 0x400111e      0x4         System.Int32   instance 9 m_index
0x79bae5f0 0x400111f      0x8       System.Boolean   instance 1 m_isUnrestricted
0x79bae5f0 0x400111d        0                CLASS     shared   static s_theTokenFactory
    >> Domain:Value 0x00155980:0x014c18bc 0x00193830:0x014c18bc <<
0x79bae5f0 0x4001120      0x4                CLASS     shared   static s_reflectPerm
    >> Domain:Value 0x00155980:0x00000000 0x00193830:0x00000000 <<

对Hashtable 内部结构的探查就到这里吧。这里贴篇.Net类库中实现的HashTable，有兴趣的朋友可以继续深入

参考：

ViewState机制由浅入深3

.Net类库中实现的HashTable