App-V Visio and Thick Excel Collide

A 3rd party Excel add-in had stopped working for a group of users, and they all received an error message on launching Excel:

Microsoft Visual Basic for Applications
System Error &H8007007E (-2147024770)

image

Unfortunately the VBA code in question was from the 3rd party add-in and password protected so there was no “Debug” option available. Thankfully however VBA password protection isn’t really protection at all I removed the protection with my PowerShell script here https://chentiangemalc.wordpress.com/2012/01/17/powershell-script-to-remove-office-macro-protection/

With this done we a slightly more friendly, but misleading, error message:

Microsoft Visual Basic for Applications
Compile error: Out of memory

image

I did a quick check on modules loading into Excel. I used ProcMon with filter to include

  • Operation is Load Image
  • Process is Excel.exe

I noticed FM20.dll was being loaded from C:\Windows\SysWOW64 folder and an App-V folder.

It was then confirmed that all affected users had a Visio App-V package installed on their machines.

image

Looks like we have another case of DLL HELL. (https://chentiangemalc.wordpress.com/2011/04/06/iphone-configuration-utility-and-dll-hell/ )

Using ProcMon we can easily see how this DLL got loaded. For DLLs not in the “path” they will typically be loaded by registrations under HKCR.

To find them we changed our procmon filter to:

  • Operation is RegQueryValue
  • Detail contains X86\FM20.dll

And we can find the reg keys we need to change for a fix:

image

However an easier way to make the bulk change is to use Nirsoft RegScanner (http://www.nirsoft.net/utils/regscanner.html 

Note: make sure you download the program regscanner.zip or regscanner-x64.zip at bottom of the page, not the “free scan” or “registry booster”

RegScanner offers very fast registry searching, and you can export your search as a reg file. This found the App-V registration in 141 places (I didn’t include search results in \Virtual folder):

image

I then searched and replaced this with C:\Windows\System32\FM20.dll and re-imported the registry file.

Problem solved.

The App-V package / deployment method can now be reviewed as to why it is registering files outside of it’s bubble….

About chentiangemalc

specializes in end-user computing technologies. disclaimer 1) use at your own risk. test any solution in your environment. if you do not understand the impact/consequences of what you're doing please stop, and ask advice from somebody who does. 2) views are my own at the time of posting and do not necessarily represent my current view or the view of my employer and family members/relatives. 3) over the years Microsoft/Citrix/VMWare have given me a few free shirts, pens, paper notebooks/etc. despite these gifts i will try to remain unbiased.
This entry was posted in App-V, Office, ProcMon, SysInternals. Bookmark the permalink.

Leave a comment