Case of the Hanging App on Windows 7

An application after migrated to Windows 7 was hanging all the time. It didn’t hang on Windows XP.

The first question I asked was the entire system freezing or just the application? i.e. when app was hung, were other applications and the Desktop / Start Menu responsive?

It was confirmed issue was limited to application only.

We configured ProcDump (http://live.sysinternals.com/procdump.exe) in hang detection mode to capture some dump files procdump -h -n 10 –ma <process name>

The 10 was to capture up to 10 dump files.

We opened the dmp files with WinDbg from Windows SDK.

In this case !analyze –v –hang we could see the hang was on opening a file. It is important to always remember in Windows “CreateFile” is used for opening files for reading or writing, and does not necessarily refer to “Creating a file”

004eda18 7643c5ef 004edab4 80100080 004eda58 ntdll!ZwCreateFile+0x12 <- hang here
004edabc 76ea3f66 00000060 80100080 00000003 KERNELBASE!CreateFileW+0x35e
004edae8 76ea53c4 0c723818 80000000 00000003 kernel32!CreateFileWImplementation+0x69
004edb18 0028e245 04296bf0 80000000 00000003 kernel32!CreateFileA+0x37
004edbf4 0028dce7 004edc34 004edce0 04296bf0 AQIRE32!_tsopen_nolock+0x4c5
004edc54 0028ee30 04296bf0 00000000 00000040 AQIRE32!_sopen_helper+0x1b7
004edc74 0028797d 004edce0 04296bf0 00000000 AQIRE32!_sopen_s+0x20
004edce4 00271cb5 04296bf0 048705f4 00000040 AQIRE32!_openfile+0x6cd
004edd3c 00271d04 04296bf0 048705f3 00000040 AQIRE32!_fsopen+0x1d5
004edd50 002ff421 04296bf0 048705f3 cccccccc AQIRE32!fopen+0x14
004edd70 001ea10f 096bdb98 def83a3c 004ee2d0 AQIRE32!File_Open+0x1a1

Because vendor gave me PDB file was could simply dump stack with parameters:

0:000> kp
ChildEBP RetAddr 
004eda18 7643c5ef ntdll!ZwCreateFile+0x12
004edabc 76ea3f66 KERNELBASE!CreateFileW+0x35e
004edae8 76ea53c4 kernel32!CreateFileWImplementation+0x69
004edb18 0028e245 kernel32!CreateFileA+0x37
004edbf4 0028dce7 AQIRE32!_tsopen_nolock(int * punlock_flag = 0x004edc34, int * pfh = 0x004edce0, char * path = 0x04296bf0 “H:\GV_EMAIL.LST“, int oflag = 0n0, int shflag = 0n64, int pmode = 0n384, int bSecure = 0n1)+0x4c5 [f:\dd\vctools\crt_bld\self_x86\crt\src\open.c @ 399]
004edc54 0028ee30 AQIRE32!_sopen_helper(char * path = 0x04296bf0 “H:\GV_EMAIL.LST”, int oflag = 0n0, int shflag = 0n64, int pmode = 0n384, int * pfh = 0x004edce0, int bSecure = 0n1)+0x1b7 [f:\dd\vctools\crt_bld\self_x86\crt\src\open.c @ 167]
004edc74 0028797d AQIRE32!_sopen_s(int * pfh = 0x004edce0, char * path = 0x04296bf0 “H:\GV_EMAIL.LST”, int oflag = 0n0, int shflag = 0n64, int pmode = 0n384)+0x20 [f:\dd\vctools\crt_bld\self_x86\crt\src\open.c @ 901]
004edce4 00271cb5 AQIRE32!_openfile(char * filename = 0x04296bf0 “H:\GV_EMAIL.LST”, char * mode = 0x048705f4 “”, int shflag = 0n64, struct _iobuf * str = 0x0036fe40)+0x6cd [f:\dd\vctools\crt_bld\self_x86\crt\src\_open.c @ 272]
004edd3c 00271d04 AQIRE32!_fsopen(char * file = 0x04296bf0 “H:\GV_EMAIL.LST”, char * mode = 0x048705f3 “r”, int shflag = 0n64)+0x1d5 [f:\dd\vctools\crt_bld\self_x86\crt\src\fopen.c @ 86]
004edd50 002ff421 AQIRE32!fopen(char * file = 0x04296bf0 “H:\GV_EMAIL.LST”, char * mode = 0x048705f3 “r”)+0x14 [f:\dd\vctools\crt_bld\self_x86\crt\src\fopen.c @ 125]

However without symbols we could also dumped the first parameter of CreateFileA to get the filename access (da for CreateFileA, du for CreateFileW)

004edb18 0028e245 04296bf0 80000000 00000003 kernel32!CreateFileA+0x37 (FPO: [Non-Fpo])

0:000> da 04296bf0
04296bf0  “H:\GV_EMAIL.LST”

All the dump files had something in common – they were accessing files on a network share. When investigating differences between Windows 7 and Windows XP I identified the following:

  • In Windows XP the network share had mapped to a Novell Netware Share
  • In Windows 7 the network share was mapped to a Windows Server share
  • In Windows 7 only one site was affected by this hang, other sites were not affected.

So I ran a robocopy test 3 times between the different environments:

Location

Test 1

Test 2

Test 3

Brisbane (Hangs occur here) – Windows 7 – Windows Share (100 MB/s connection)

4.47 MB/sec

7.60 MB/sec

1.45 MB/sec

Brisbane – Windows XP – Novell Share

(100 MB/s connection)

8.01 MB/sec

8.46 MB/sec

8.31 MB/sec

Sydney – Windows 7 – Windows Share

(100 MB/s connection)

8.34 MB/sec

7.10 MB/sec

8.33 MB/sec

Melbourne – Windows 8.1  – Windows Share (1 GB/s connection)

37.19 MB/sec

57.03 MB/sec

57.49 MB/sec

From this test we could see the Windows share in Brisbane was having poor performance, and this likely caused the hangs. While this root cause was being investigated the developers advised the application did not require to store these data on the network, and the developer transferred the files to the local drive, and the hanging disappeared.


 

 

About chentiangemalc

specializes in end-user computing technologies. disclaimer 1) use at your own risk. test any solution in your environment. if you do not understand the impact/consequences of what you're doing please stop, and ask advice from somebody who does. 2) views are my own at the time of posting and do not necessarily represent my current view or the view of my employer and family members/relatives. 3) over the years Microsoft/Citrix/VMWare have given me a few free shirts, pens, paper notebooks/etc. despite these gifts i will try to remain unbiased.
This entry was posted in WinDbg, Windows 7. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s