I have written earlier about how to track down exceptions using configuration scripts for adplus.
Most of the time the methods are short enough that just knowing what function you got an exception in, is enough for you to track down the why, but we all know that we don't live in that perfect world where we write completely modular applications and everything is nicely lined up:)
Let's say you have found this exception on the heap...
0:025> !dumpobj 02b7191c
Name: System.NullReferenceException
MethodTable: 7915ec4c
EEClass: 791ea18c
Size: 72(0x48) bytes (C:\WINDOWS\assembly\GAC_32\mscorlib\2.0.0.0__b77a5c561934e089\mscorlib.dll)
Fields: MT Field Offset Type VT Attr Value Name
790fa3e0 40000b5 4 System.String 0 instance 00000000 _className 79109208 40000b6 8 ...ection.MethodBase 0 instance 00000000 _exceptionMethod
790fa3e0 40000b7 c System.String 0 instance 00000000 _exceptionMethodString
790fa3e0 40000b8 10 System.String 0 instance 02b719bc _message 79113dfc 40000b9 14 ...tions.IDictionary 0 instance 00000000 _data
790fa9e8 40000ba 18 System.Exception 0 instance 00000000 _innerException
790fa3e0 40000bb 1c System.String 0 instance 00000000 _helpURL
790f9c18 40000bc 20 System.Object 0 instance 02b71a38 _stackTrace
790fa3e0 40000bd 24 System.String 0 instance 00000000 _stackTraceString
790fa3e0 40000be 28 System.String 0 instance 00000000 _remoteStackTraceString
790fed1c 40000bf 34 System.Int32 0 instance 0 _remoteStackIndex
790f9c18 40000c0 2c System.Object 0 instance 00000000 _dynamicMethods
790fed1c 40000c1 38 System.Int32 0 instance -2147467261 _HResult
790fa3e0 40000c2 30 System.String 0 instance 00000000 _source
790fe160 40000c3 3c System.IntPtr 0 instance 34270984 _xptrs
790fed1c 40000c4 40 System.Int32 0 instance -1073741819 _xcode 0:025> !printexception 02b7191c
Exception object: 02b7191c Exception type: System.NullReferenceException
Message: Object reference not set to an instance of an object.
InnerException: (none)
StackTrace (generated):
SP IP Function
020AF378 029C3269 DisplayUserInfo.Page_Load(System.Object, System.EventArgs)
StackTraceString: (none)
We know from the stack that the NullReference exception occurred in the DisplayUserInfo.Page_Load function, but how do you know exactly where inside that function? and how do you know what caused it?
The first thing I usually do is look at the code if I have it... And if the code is not readily available I extract the dll from the dump using !savemodule or !saveallmodules (!sam) in sos.dll, which will give me an exact copy of the dll loaded in memory when the dump was taken. (Slight caveat... the !sam functionality does not exist in the 2.0 version of sos.dll yet so for 2.0 you will have to use savemodule)
So for this exception I know that we are faulting at IP (instruction pointer) 0x029C3269, and I can use this to first get the method descriptor for the method (using !ip2md - Instruction Pointer to Method Descriptor)...
0:025> !ip2md 0x029C3269
MethodDesc: 0ee335b8
Method Name: DisplayUserInfo.Page_Load(System.Object, System.EventArgs)
Class: 0297a5b8
MethodTable: 0ee335ec
mdToken: 06000013
Module: 0ee329c4
IsJitted: yes
m_CodeOrIL: 029c3110
..and then dump out the method table to find out what dll this code is compiled to.
0:025> !dumpmt 0ee335ec
EEClass: 0297a5b8
Module: 0ee329c4
Name: DisplayUserInfo
mdToken: 02000004 (C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\Temporary ASP.NET Files\debuggersamples\e7443224\5232f845\App_Web_dmjhn1yn.dll)
BaseSize: 0x180
ComponentSize: 0x0
Number of IFaces in IFaceMap: c
Slots in VTable: 130
Once we know the dll, we can find the loading address by running lmv m
0:025> lmv mApp_Web_dmjhn1yn
start end module name
0f280000 0f288000 App_Web_dmjhn1yn (deferred)
Image path: C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\Temporary ASP.NET Files\debuggersamples\e7443224\5232f845\App_Web_dmjhn1yn.dll
Image name: App_Web_dmjhn1yn.dll
Using CLR debugging support for all symbols
Has CLR image header, track-debug-data flag not set
Timestamp: Thu May 18 13:24:36 2006 (446C5974)
CheckSum: 00000000
ImageSize: 00008000
File version: 0.0.0.0
Product version: 0.0.0.0
File flags: 0 (Mask 3F)
File OS: 4 Unknown Win32
File type: 2.0 Dll
File date: 00000000.00000000
Translations: 0000.04b0
InternalName: App_Web_dmjhn1yn.dll
OriginalFilename: App_Web_dmjhn1yn.dll
ProductVersion: 0.0.0.0
FileVersion: 0.0.0.0
FileDescription:
LegalCopyright:
And given this we can now extract this assembly (App_Web_dmjhn1yn.dll) from the memory dump
0:025> !savemodule 0f280000 f:\App_Web_dmjhn1yn.dll
3 sections in file
section 0 - VA=2000, VASize=1af4, FileAddr=200, FileSize=1c00
section 1 - VA=4000, VASize=2c8, FileAddr=1e00, FileSize=400
section 2 - VA=6000, VASize=c, FileAddr=2200, FileSize=200
Ok... so now we have the dll... but how do we get to the actual code?
Well... we could open it with ildasm.exe and browse to the DisplayUserInfo.Page_Load function which would give us somewhat readable IL code. It's actually not half-bad, but it doesn't give you as clear of a picture as the code would.
The tool I like to use is Lutz Roeder's reflector from http://www.aisto.com/roeder/dotnet/ which gives me the following output if I browse to the DisplayUserInfo.Page_Load function
protected void Page_Load(object sender, EventArgs e)
{
try
{
this.LblWelcomeMsg.Text = "Welcome " + this.Session["username"];
if (this.Session["role"].ToString() == "Administrator")
{
this.btnEditRoll.Enabled = true;
}
else
{
this.btnEditRoll.Enabled = false;
}
TableHeaderRow row1 = new TableHeaderRow();
TableHeaderCell cell1 = new TableHeaderCell();
cell1.Text = "Blogs";
row1.Cells.Add(cell1);
this.tblBlogRoll.Rows.Add(row1);
ArrayList list1 = (ArrayList) this.Session["BlogRoll"];
for (int num1 = 0; num1 < list1.Count; num1++)
{
TableCell cell2 = new TableCell();
cell2.Text = list1[num1].ToString();
TableRow row2 = new TableRow();
row2.Cells.Add(cell2);
tblBlogRoll.Rows.Add(row2);
}
}
catch (Exception)
{
base.Response.Write("An exception occurred");
}
}
Now how cool is that:) that is pretty much an exact replica of the original code... but... that alone doesn't really tell us where the exception occurred, so let's go back to the instruction pointer and un-assemble the function in the dump using !u, and then we can search for the instruction closest to our current instruction pointer to see exactly where we are at...
0:025> !u
029c3269 Normal JIT generated code DisplayUserInfo.Page_Load(System.Object, System.EventArgs) Begin
029c3110, size 214
029c3110 55 push ebp
029c3111 8bec mov ebp,esp
029c3113 57 push edi
029c3114 56 push esi
029c3115 53 push ebx
029c3116 83ec20 sub esp,20h
029c3119 33c0 xor eax,eax
029c311b 8945e8 mov dword ptr [ebp-18h],eax
029c311e 894ddc mov dword ptr [ebp-24h],ecx
029c3121 8b45dc mov eax,dword ptr [ebp-24h]
029c3124 8945d4 mov dword ptr [ebp-2Ch],eax
029c3127 8bb86c010000 mov edi,dword ptr [eax+16Ch]
029c312d 8b354444a30a mov esi,dword ptr ds:[0AA34444h]
029c3133 8bc8 mov ecx,eax
029c3135 8b01 mov eax,dword ptr [ecx]
029c3137 ff90a8010000 call dword ptr [eax+1A8h]
029c313d 8bc8 mov ecx,eax
029c313f 8b151844a30a mov edx,dword ptr ds:[0AA34418h]
029c3145 3909 cmp dword ptr [ecx],ecx
029c3147 e8c4b7de65 call System_Web_ni!System.Web.SessionState.HttpSessionState.get_Item(System.String) (687ae910)
029c314c 8bd0 mov edx,eax
029c314e 8bce mov ecx,esi
029c3150 e863e79a76 call USERENV!ProcessGPORegistryPolicy+0xdf (769ae763) (USERENV!ProcessGPORegistryPolicy)
029c3155 8bd0 mov edx,eax
029c3157 8bcf mov ecx,edi
029c3159 8b01 mov eax,dword ptr [ecx]
029c315b ff90fc010000 call dword ptr [eax+1FCh]
029c3161 8b4dd4 mov ecx,dword ptr [ebp-2Ch]
029c3164 8b01 mov eax,dword ptr [ecx]
029c3166 ff90a8010000 call dword ptr [eax+1A8h]
029c316c 8bc8 mov ecx,eax
029c316e 8b152044a30a mov edx,dword ptr ds:[0AA34420h]
029c3174 3909 cmp dword ptr [ecx],ecx
029c3176 e895b7de65 call System_Web_ni!System.Web.SessionState.HttpSessionState.get_Item(System.String) (687ae910)
029c317b 8bc8 mov ecx,eax
029c317d 8b01 mov eax,dword ptr [ecx]
029c317f ff5028 call dword ptr [eax+28h]
029c3182 8b152444a30a mov edx,dword ptr ds:[0AA34424h]
029c3188 8bc8 mov ecx,eax
029c318a e861879876 call USERENV!MyRegLoadKeyEx+0x21a (76988761) (USERENV!MyRegLoadKeyEx)
029c318f 25ff000000 and eax,0FFh
029c3194 7418 je App_Web_dmjhn1yn!DisplayUserInfo.Page_Load(System.Object, System.EventArgs)+0x9e (
029c31ae)
029c3196 8b45d4 mov eax,dword ptr [ebp-2Ch]
029c3199 8b8874010000 mov ecx,dword ptr [eax+174h]
029c319f ba01000000 mov edx,1
029c31a4 8b01 mov eax,dword ptr [ecx]
029c31a6 ff9098010000 call dword ptr [eax+198h]
029c31ac eb13 jmp App_Web_dmjhn1yn!DisplayUserInfo.Page_Load(System.Object, System.EventArgs)+0xb1 (
029c31c1)
029c31ae 8b45d4 mov eax,dword ptr [ebp-2Ch]
029c31b1 8b8874010000 mov ecx,dword ptr [eax+174h]
029c31b7 33d2 xor edx,edx
029c31b9 8b01 mov eax,dword ptr [ecx]
029c31bb ff9098010000 call dword ptr [eax+198h]
029c31c1 b9c86ca568 mov ecx,offset System_Web_ni+0x496cc8 (68a56cc8)
029c31c6 e851ee78ff call 0215201c (JitHelp: CORINFO_HELP_NEWSFAST)
029c31cb 8bf0 mov esi,eax
029c31cd 8bce mov ecx,esi
029c31cf e82448ef65 call System_Web_ni!System.Web.UI.WebControls.TableHeaderRow..ctor() (688b79f8)
029c31d4 b9a08ca668 mov ecx,offset System_Web_ni+0x4a8ca0 (68a68ca0)
029c31d9 e83eee78ff call 0215201c (JitHelp: CORINFO_HELP_NEWSFAST)
029c31de 8bf8 mov edi,eax
029c31e0 8bcf mov ecx,edi
029c31e2 e86144ef65 call System_Web_ni!System.Web.UI.WebControls.TableHeaderCell..ctor() (688b7648)
029c31e7 8b156844a30a mov edx,dword ptr ds:[0AA34468h]
029c31ed 8bcf mov ecx,edi
029c31ef 8b01 mov eax,dword ptr [ecx]
029c31f1 ff9014020000 call dword ptr [eax+214h]
029c31f7 8bce mov ecx,esi
029c31f9 8b01 mov eax,dword ptr [ecx]
029c31fb ff90f0010000 call dword ptr [eax+1F0h]
029c3201 8bc8 mov ecx,eax
029c3203 8bd7 mov edx,edi
029c3205 3909 cmp dword ptr [ecx],ecx
029c3207 e85c3fef65 call System_Web_ni!System.Web.UI.WebControls.TableCellCollection.Add(System.Web.UI.WebControls.TableCell) (688b7168)
029c320c 8b45d4 mov eax,dword ptr [ebp-2Ch]
029c320f 8b8870010000 mov ecx,dword ptr [eax+170h]
029c3215 8b01 mov eax,dword ptr [ecx]
029c3217 ff9028020000 call dword ptr [eax+228h]
029c321d 8bc8 mov ecx,eax
029c321f 8bd6 mov edx,esi
029c3221 3909 cmp dword ptr [ecx],ecx
029c3223 e8a048ef65 call System_Web_ni!System.Web.UI.WebControls.TableRowCollection.Add(System.Web.UI.WebControls.TableRow) (688b7ac8)
029c3228 8b4dd4 mov ecx,dword ptr [ebp-2Ch]
029c322b 8b01 mov eax,dword ptr [ecx]
029c322d ff90a8010000 call dword ptr [eax+1A8h]
029c3233 8bc8 mov ecx,eax
029c3235 8b156c44a30a mov edx,dword ptr ds:[0AA3446Ch]
029c323b 3909 cmp dword ptr [ecx],ecx
029c323d e8ceb6de65 call System_Web_ni!System.Web.SessionState.HttpSessionState.get_Item(System.String) (687ae910)
029c3242 8bf0 mov esi,eax
029c3244 85f6 test esi,esi
029c3246 7418 je App_Web_dmjhn1yn!DisplayUserInfo.Page_Load(System.Object, System.EventArgs)+0x150 (
029c3260)
029c3248 813eb0361079 cmp dword ptr [esi],offset mscorlib_ni+0x436b0 (791036b0)
029c324e 7502 jne App_Web_dmjhn1yn!DisplayUserInfo.Page_Load(System.Object, System.EventArgs)+0x142 (
029c3252)
029c3250 eb0e jmp App_Web_dmjhn1yn!DisplayUserInfo.Page_Load(System.Object, System.EventArgs)+0x150 (
029c3260)
029c3252 8bd6 mov edx,esi
029c3254 b9b0361079 mov ecx,offset mscorlib_ni+0x436b0 (791036b0)
029c3259 e84e895277 call mscorwks!JIT_ChkCastClassSpecial (79eebbac)
029c325e 8bf0 mov esi,eax
029c3260 8975d8 mov dword ptr [ebp-28h],esi
029c3263 33db xor ebx,ebx
029c3265 8b4dd8 mov ecx,dword ptr [ebp-28h]
029c3268 8b01 mov eax,dword ptr [ecx]
029c326a ff5040 call dword ptr [eax+40h]
029c326d 85c0 test eax,eax
029c326f 0f8e87000000 jle App_Web_dmjhn1yn!DisplayUserInfo.Page_Load(System.Object, System.EventArgs)+0x1ec (
029c32fc)
029c3275 b9cc00a568 mov ecx,offset System_Web_ni+0x4900cc (68a500cc)
029c327a e89ded78ff call 0215201c (JitHelp: CORINFO_HELP_NEWSFAST)
029c327f 8bf8 mov edi,eax
029c3281 8bcf mov ecx,edi
029c3283 e89897e765 call System_Web_ni!System.Web.UI.WebControls.TableCell..ctor() (6883ca20)
029c3288 8bd3 mov edx,ebx
029c328a 8b4dd8 mov ecx,dword ptr [ebp-28h]
029c328d 8b01 mov eax,dword ptr [ecx]
029c328f ff5054 call dword ptr [eax+54h]
029c3292 8bc8 mov ecx,eax
029c3294 8b01 mov eax,dword ptr [ecx]
029c3296 ff5028 call dword ptr [eax+28h]
029c3299 8bd0 mov edx,eax
029c329b 8bcf mov ecx,edi
029c329d 8b01 mov eax,dword ptr [ecx]
029c329f ff9014020000 call dword ptr [eax+214h]
029c32a5 b9843da668 mov ecx,offset System_Web_ni+0x4a3d84 (68a63d84)
029c32aa e86ded78ff call 0215201c (JitHelp: CORINFO_HELP_NEWSFAST)
029c32af 8bf0 mov esi,eax
029c32b1 8bce mov ecx,esi
029c32b3 e8b057e665 call System_Web_ni!System.Web.UI.WebControls.TableRow..ctor() (68828a68)
029c32b8 8bce mov ecx,esi
029c32ba 8b01 mov eax,dword ptr [ecx]
029c32bc ff90f0010000 call dword ptr [eax+1F0h]
029c32c2 8bc8 mov ecx,eax
029c32c4 8bd7 mov edx,edi
029c32c6 3909 cmp dword ptr [ecx],ecx
029c32c8 e89b3eef65 call System_Web_ni!System.Web.UI.WebControls.TableCellCollection.Add(System.Web.UI.WebControls.TableCell) (688b7168)
029c32cd 8b45d4 mov eax,dword ptr [ebp-2Ch]
029c32d0 8b8870010000 mov ecx,dword ptr [eax+170h]
029c32d6 8b01 mov eax,dword ptr [ecx]
029c32d8 ff9028020000 call dword ptr [eax+228h]
029c32de 8bc8 mov ecx,eax
029c32e0 8bd6 mov edx,esi
029c32e2 3909 cmp dword ptr [ecx],ecx
029c32e4 e8df47ef65 call System_Web_ni!System.Web.UI.WebControls.TableRowCollection.Add(System.Web.UI.WebControls.TableRow) (688b7ac8)
029c32e9 83c301 add ebx,1
029c32ec 8b4dd8 mov ecx,dword ptr [ebp-28h]
029c32ef 8b01 mov eax,dword ptr [ecx]
029c32f1 ff5040 call dword ptr [eax+40h]
029c32f4 3bc3 cmp eax,ebx
029c32f6 0f8f79ffffff jg App_Web_dmjhn1yn!DisplayUserInfo.Page_Load(System.Object, System.EventArgs)+0x165 (
029c3275)
029c32fe 8b4ddc mov ecx,dword ptr [ebp-24h]
029c3301 e84a2be165 call System_Web_ni!System.Web.UI.Page.get_Response() (687d5e50)
029c3306 8bc8 mov ecx,eax
029c3308 8b155c44a30a mov edx,dword ptr ds:[0AA3445Ch]
029c330e 3909 cmp dword ptr [ecx],ecx
029c3310 e8e3b4cd65 call System_Web_ni!System.Web.HttpResponse.Write(System.String) (6869e7f8)
029c3315 e8df2f5d77 call mscorwks!JIT_EndCatch (79f962f9)
029c331a 8d65f4 lea esp,[ebp-0Ch]
029c331d 5b pop ebx
029c331e 5e pop esi
029c331f 5f pop edi
029c3320 5d pop ebp
029c3321 c20400 ret 4
The line:
029c3268 8b01 mov eax,dword ptr [ecx]
...is the line right prior to our current IP, which means that it was the line that caused the NullReferenceException.
I have marked a few lines around it in gray that will help us compare the disassembly to code.
Disassembly:
029c3223 e8a048ef65 call System_Web_ni!System.Web.UI.WebControls.TableRowCollection.Add(System.Web.UI.WebControls.TableRow) (688b7ac8)
029c3228 8b4dd4 mov ecx,dword ptr [ebp-2Ch]
029c322b 8b01 mov eax,dword ptr [ecx]
029c322d ff90a8010000 call dword ptr [eax+1A8h]
029c3233 8bc8 mov ecx,eax
029c3235 8b156c44a30a mov edx,dword ptr ds:[0AA3446Ch]
029c323b 3909 cmp dword ptr [ecx],ecx
029c323d e8ceb6de65 call System_Web_ni!System.Web.SessionState.HttpSessionState.get_Item(System.String) (687ae910)
029c3242 8bf0 mov esi,eax
029c3244 85f6 test esi,esi
029c3246 7418 je App_Web_dmjhn1yn!DisplayUserInfo.Page_Load(System.Object, System.EventArgs)+0x150 (
029c3260)
029c3248 813eb0361079 cmp dword ptr [esi],offset mscorlib_ni+0x436b0 (791036b0)
029c324e 7502 jne App_Web_dmjhn1yn!DisplayUserInfo.Page_Load(System.Object, System.EventArgs)+0x142 (
029c3252)
029c3250 eb0e jmp App_Web_dmjhn1yn!DisplayUserInfo.Page_Load(System.Object, System.EventArgs)+0x150 (
029c3260)
029c3252 8bd6 mov edx,esi
029c3254 b9b0361079 mov ecx,offset mscorlib_ni+0x436b0 (791036b0)
029c3259 e84e895277 call mscorwks!JIT_ChkCastClassSpecial (79eebbac)
029c325e 8bf0 mov esi,eax
029c3260 8975d8 mov dword ptr [ebp-28h],esi
029c3263 33db xor ebx,ebx
029c3265 8b4dd8 mov ecx,dword ptr [ebp-28h]
029c3268 8b01 mov eax,dword ptr [ecx]
029c326a ff5040 call dword ptr [eax+40h]
029c326d 85c0 test eax,eax
029c326f 0f8e87000000 jle App_Web_dmjhn1yn!DisplayUserInfo.Page_Load(System.Object, System.EventArgs)+0x1ec (
029c32fc)
029c3275 b9cc00a568 mov ecx,offset System_Web_ni+0x4900cc (68a500cc)
029c327a e89ded78ff call 0215201c (JitHelp: CORINFO_HELP_NEWSFAST)
029c327f 8bf8 mov edi,eax
029c3281 8bcf mov ecx,edi
029c3283 e89897e765 call System_Web_ni!System.Web.UI.WebControls.TableCell..ctor() (6883ca20)
Code from reflector:
this.tblBlogRoll.Rows.Add(row1);ArrayList list1 = (ArrayList) this.Session["BlogRoll"];
for (int num1 = 0; num1 < list1.Count; num1++)
{
TableCell cell2 = new TableCell();
cell2.Text = list1[num1].ToString();
TableRow row2 = new TableRow();
We can see the call to this.tblBlogRoll.Rows.Add (System_Web_ni!System.Web.UI.WebControls.TableRowCollection.Add), and the call to this.Session["BlogRoll"] (System_Web_ni!System.Web.SessionState.HttpSessionState.get_Item), followed by the cast to ArrayList (mscorwks!JIT_ChkCastClassSpecial)
After our bolded line we can see the call to new TableCell (System_Web_ni!System.Web.UI.WebControls.TableCell..ctor), which means that the bolded line must be one of the instructions on the for line
for (int num1 = 0; num1 < list1.Count; num1++)
More specifically it is the list1.Count that is causing the nullref... in other words list1 is null because this.Session["BlogRoll"] was empty, and we nullreference when we try to get to the Count property, so to avoid this we need to do a null check on the Session["BlogRoll"] before assigning it to the ArrayList.
Btw, this method doesn't only apply to exceptions. You can use it to figure out where exactly you take a lock and other similar things as well, but you'll probably need it mostly for exceptions.
Til next time...