After following a similar approach as below:
https://chentiangemalc.wordpress.com/2014/09/12/case-of-the-8-minute-windows-8-1-first-logon/
I still hadn’t found any obvious cause of slow logon, except that it was in Win Logon Init phase.
CPU, Storage, Memory, and Disk usage are all low:
Expanding Generic Events, looking at Microsoft-Windows-GroupPolicy section we could see delays of 2,000 seconds apart. Another big clue was servers started with SYD prefix for SYDNEY, but these clients were located in Melbourne.
The Microsoft-Windows-GroupPolicy/Operational event log showed frequent delays, and repeatedly contacting the wrong domain controllers, and we also saw many delays in here. There is a ton of useful diagnostic info in this log, so in Domain Environments you should be well acquainted with it
Using a quick PowerShell script we can check domain controllers, this machine has connected to, and sort from most popular to least popular.
$xpath=@' <QueryList> <Query Id="0" Path="Microsoft-Windows-GroupPolicy/Operational"> <Select Path="Microsoft-Windows-GroupPolicy/Operational">*[System[(EventID=5310)]]</Select> </Query> </QueryList> '@ $DCs=@{} $events=Get-WinEvent -LogName "Microsoft-Windows-GroupPolicy/Operational" -FilterXPath $xpath ForEach ($event in $events) { $DCs[$event.Properties[2].Value]++ } $DCs.GetEnumerator() | Sort-Object Value -Descending
Here we saw frequency of going to domain controller in another site. On top of this DFS was in use, and was resulting in unnecessary WAN traffic due to wrong Active Directory site detection.
The root cause was the Melbourne site had new subnets added to it, but the Active Directory Sites & Services had not been updated, so the machines were frequently in the wrong site. Once this was corrected Windows logon times improved to more normal levels.
Well normal for the amount of crap they were required to have running at logon to meet business requirements.