When attempting to launch Visual Basic 4.0 from Windows 3.1 running in DosBox the following error appeared:
SHARE.exe must be installed in order to run Visual Basic
This was despite already extracting share.exe from MS-DOS 6.22 install files, using command ver set 6 22 to make machine believe it’s running MS-DOS v6.22, and having it run in the background.
So the question is how does Visual Basic 4 detect share.exe installation?
Using NirSoft SearchMyFiles I searched the file contents to find files containing this error message:
Opening the DLL in IDA pro we can find the error message:
Initial analysis of VB.exe startup process through IDA didn’t give immediate clues and was going to take a long time to completely walk through the startup process.
The resource string containing error message would likely be loaded by a call to LOADSTRING API but there was quite a few references to this and none were immediately clear which one loaded this particular error message.
Normally I would set a breakpoint on MESSAGEBOX or break into debugger when error was displayed. However the keyboard shortcut to break into CodeView debugger from Windows 3.1 SDK wasn’t working in Windows 3.1 on DosBox. Also I didn’t have any symbols so it was more difficult to set a breakpoint on MESSAGEBOX.
What I did instead is using IDA Pro, used the Assemble function to replace each CALL MESSAGEBOX with int 3 (Break into debugger)
From the surrounding code I immediately identified which MESSAGEBOX was displaying the error message.
The same section of code in IDA:
cseg47:297E push bp cseg47:297F mov bp, sp cseg47:2981 push di cseg47:2982 push si cseg47:2983 mov di, [bp+6] cseg47:2986 mov es, word ptr [bp+8] cseg47:2989 push word ptr es:[di] cseg47:298C push word ptr es:[di+4] cseg47:2990 push word ptr es:[di+2] cseg47:2994 push word ptr es:[di+8] cseg47:2998 push word ptr es:[di+6] cseg47:299C push word ptr es:[di+0Ah] cseg47:29A0 call MESSAGEBOX
However there was nothing directly calling this function, it seemed to be called by some kind of event handler set up with
push seg cseg47
push 297Eh
etc
call sub_847C
Next I tested the EXE on Windows 7 32-bit and it worked fine. I then used API Monitor run as local Administrator and configured to intercept new processes, monitoring for Data Access and Storage APIs.
First I ensured no ntvdm.exe processes were currently running, then when Visual Basic 4 launched, ntvdm.exe process launched in Windows 7 to emulate the 16-bit layer, and I monitored this process.
Here we can see a temporary file is created, locked/unlocked, and then deleted during startup:
The problem is our VB.exe does not make any calls to an API named LockFile or UnlockFile, and I don’t think this API existed yet in Windows 3.1.
So if not a Windows API, maybe a MS-DOS API was used….
The INT 2fH vector (at 0000:00bc) was a portal to many varied services; some installed by DOS and others by device drivers, utility programs, Windows, etc. It is sometimes call the Multiplex or MUX Interrupt.
By setting AX register on CPU to a specified value many services could be called such as:
- 01xxH Print spooler services
- 0600H Assign installed?
- 1000H Share installed?
- 1100H is network installed?
- 1400H Nlsfunc installed?
- 15xxH Mscdex services
- 16xxH Windows 386Enh-Mode services
- 1680H AppIdle Release timeslice
- 1a00H ANSI.SYS installed?
- 17xxH Windows Clipboard access
- 40xxH Windows VDD fns for VM-aware apps
- 4a11H DoubleSpace API services
- 43xxH HIMEM.SYS / XMS Services
- 48xxH Doskey services
- 4axxH HMA Allocations (undoc’d)
- 4bxxH Dosshell / Switcher API
- 54xxH POWER.EXE (undocumented)
- adxxH Keyb services
- aexxH COMMAND.COM hook services
- b000H Graftabl installed?
- b7xxH Append services
- xxfbH FaxBIOS Services
Scanning the program I could confirm the INT 2FH vector method was not used to detect Share being installed.
So I checked the standard MS-DOS functions executed via INT 21h.
Here we found the lock file / unlock file functions:
Documentation for the lock function specifies:
Compatibility: 3.0+
Expects: AX 5c00H
BX file handle
CX:DX file offset from start of file (CX * 65536)+DX
SI:DI length in bytes of region to lock (SI * 65536)+DI
Returns: AX error code if CF is set to CY
This function locks access to a region of the file identified by the file handle in BX. The region of the file that begins at file logical offset CX:DX extending for a length of SI:DI is locked. File sharing support must be present (usually installed by your network software, or Windows, or by using the DOS Share command). If not installed, the call returns an error code of “invalid function number.” Locks apply to reads, writes, and opens of a file by a child or concurrent process. When another process attempts such an access (and the Access Mode defined during the file OPEN is a sharing mode which disallows such access), then DOS will fail the operation via an INT 24H Critical Error handler. The correct way to avoid a lock violation is to attempt to lock the region yourself and examine the returned error code. By default, any attempt to violate a lock causes DOS to display the “Abort, Retry, Ignore” message after three tries. You can change the retry count via fn 440bH. And you can intercept INT 24H to cause a less ghastly response. To lock an entire file: It is legal to lock beyond the end of the file. You can lock an entire file by setting CX=0, DX=0, SI=0ffffH, and DI=0ffffH.
When you unlock (via fn 5c01H), the offset and length must match exactly with the offset and length that was locked.
- DUPing a file handle via Fn 45H or 46H will duplicate any locks.
- Even if an Access Mode of “Inherit” is used when opened, the locking mechanism does not give access privileges to child processes created via 4bH EXEC.
- It is important that all locks be removed from a file before a program is terminated. You should intercept INT 23H (Ctrl-Break) and INT 24H (Critical Error) to ensure that locks are removed before you terminate.
- It is recommended that you unlock as soon as possible. Always try to lock, access the file, and unlock in one operation.
Searching for locking files we can see mov word ptr [bp+var_12], 5C00h sets up the local file operation executed in the following call __intdos
With mov word ptr [bp+var_12], 5C01h setting up the file unlock operation.
cseg58:0000 LOCKFILETEST proc far ; CODE XREF: sub_388FE+772↑P cseg58:0000 cseg58:0000 var_12A = OFSTRUCT ptr -12Ah cseg58:0000 var_A2 = byte ptr -0A2h cseg58:0000 var_20 = _REGS ptr -20h cseg58:0000 var_12 = _REGS ptr -12h cseg58:0000 var_4 = word ptr -4 cseg58:0000 var_2 = word ptr -2 cseg58:0000 arg_0 = word ptr 6 cseg58:0000 arg_2 = word ptr 8 cseg58:0000 cseg58:0000 enter 12Ah, 0 cseg58:0004 push di cseg58:0005 mov di, [bp+arg_0] cseg58:0008 xor ax, ax cseg58:000A mov [bp+var_4], ax cseg58:000D mov es, [bp+arg_2] cseg58:0010 mov es:[di], ax cseg58:0013 lea ax, [bp+var_A2] cseg58:0017 push ss cseg58:0018 push ax ; LPSTR cseg58:0019 push 80h ; int cseg58:001C call sub_6301C cseg58:0021 lea ax, [bp+var_A2] cseg58:0025 push ss cseg58:0026 push ax ; LPCSTR cseg58:0027 lea ax, [bp+var_12A] cseg58:002B push ss cseg58:002C push ax ; OFSTRUCT far * cseg58:002D push 1010h ; UINT cseg58:0030 call OPENFILE cseg58:0035 mov [bp+var_2], ax cseg58:0038 inc ax cseg58:0039 jnz short loc_DDF45 cseg58:003B mov es, [bp+arg_2] cseg58:003E mov word ptr es:[di], 1 cseg58:0043 jmp short loc_DDFB2 cseg58:0045 ; --------------------------------------------------------------------------- cseg58:0045 cseg58:0045 loc_DDF45: ; CODE XREF: LOCKFILETEST+39↑j cseg58:0045 mov word ptr [bp+var_12], 5C00h cseg58:004A mov ax, [bp+var_2] cseg58:004D mov word ptr [bp+var_12+2], ax cseg58:0050 xor ax, ax cseg58:0052 mov word ptr [bp+var_12+4], ax cseg58:0055 mov word ptr [bp+var_12+6], ax cseg58:0058 mov word ptr [bp+var_12+8], ax cseg58:005B mov word ptr [bp+var_12+0Ah], 1 cseg58:0060 lea cx, [bp+var_20] cseg58:0063 push ss cseg58:0064 push cx ; union _REGS * cseg58:0065 lea cx, [bp+var_12] cseg58:0068 push ss cseg58:0069 push cx ; union _REGS * cseg58:006A call __intdos cseg58:006F add sp, 8 cseg58:0072 cmp word ptr [bp+var_20+0Ch], 0 cseg58:0076 jnz short loc_DDFAA cseg58:0078 mov word ptr [bp+var_12], 5C01h cseg58:007D mov ax, [bp+var_2] cseg58:0080 mov word ptr [bp+var_12+2], ax cseg58:0083 xor ax, ax cseg58:0085 mov word ptr [bp+var_12+4], ax cseg58:0088 mov word ptr [bp+var_12+6], ax cseg58:008B mov word ptr [bp+var_12+8], ax cseg58:008E mov word ptr [bp+var_12+0Ah], 1 cseg58:0093 lea ax, [bp+var_20] cseg58:0096 push ss cseg58:0097 push ax ; union _REGS * cseg58:0098 lea ax, [bp+var_12] cseg58:009B push ss cseg58:009C push ax ; union _REGS * cseg58:009D call __intdos cseg58:00A2 add sp, 8 cseg58:00A5 mov [bp+var_4], 1 cseg58:00AA cseg58:00AA loc_DDFAA: ; CODE XREF: LOCKFILETEST+76↑j cseg58:00AA push [bp+var_2] ; HFILE cseg58:00AD call _LCLOSE cseg58:00B2 cseg58:00B2 loc_DDFB2: ; CODE XREF: LOCKFILETEST+43↑j cseg58:00B2 lea ax, [bp+var_12A.szPathName] cseg58:00B6 push ss cseg58:00B7 push ax ; char * cseg58:00B8 call far ptr __unlink cseg58:00BD add sp, 4 cseg58:00C0 mov ax, [bp+var_4] cseg58:00C3 pop di cseg58:00C4 leave cseg58:00C5 retf 4 cseg58:00C5 LOCKFILETEST endp
Checking the __intdos function we find the following code:
cseg01:0580 ; int __cdecl _intdos(union _REGS *, union _REGS *) cseg01:0580 __intdos proc far ; CODE XREF: __dosexterr+18↓P cseg01:0580 ; sub_DDF00+6A↓P ... cseg01:0580 cseg01:0580 arg_0 = dword ptr 6 cseg01:0580 arg_4 = dword ptr 0Ah cseg01:0580 cseg01:0580 mov ax, ds cseg01:0582 nop cseg01:0583 inc bp cseg01:0584 push bp cseg01:0585 mov bp, sp cseg01:0587 push ds cseg01:0588 mov ds, ax cseg01:058A push si cseg01:058B push di cseg01:058C push ds cseg01:058D lds di, [bp+arg_0] cseg01:0590 mov ax, [di] cseg01:0592 mov bx, [di+2] cseg01:0595 mov cx, [di+4] cseg01:0598 mov dx, [di+6] cseg01:059B mov si, [di+8] cseg01:059E mov di, [di+0Ah] cseg01:05A1 pop ds cseg01:05A2 test cs:off_10, 1 cseg01:05A9 jz short loc_5B2 cseg01:05AB call DOS3CALL cseg01:05B0 jmp short loc_5B4 cseg01:05B2 ; --------------------------------------------------------------------------- cseg01:05B2 cseg01:05B2 loc_5B2: ; CODE XREF: __intdos+29↑j cseg01:05B2 int 21h ; DOS - cseg01:05B4 cseg01:05B4 loc_5B4: ; CODE XREF: __intdos+30↑j cseg01:05B4 push ds cseg01:05B5 push di cseg01:05B6 lds di, [bp+arg_4] cseg01:05B9 mov [di], ax cseg01:05BB mov [di+2], bx cseg01:05BE mov [di+4], cx cseg01:05C1 mov [di+6], dx cseg01:05C4 mov [di+8], si cseg01:05C7 pop word ptr [di+0Ah] cseg01:05CA jb short loc_5D0 cseg01:05CC xor si, si cseg01:05CE jmp short loc_5DE cseg01:05D0 ; --------------------------------------------------------------------------- cseg01:05D0 cseg01:05D0 loc_5D0: ; CODE XREF: __intdos+4A↑j cseg01:05D0 pop ds cseg01:05D1 push ds cseg01:05D2 push cs cseg01:05D3 call near ptr __maperror cseg01:05D6 lds di, [bp+arg_4] cseg01:05D9 mov si, 1 cseg01:05DC mov ax, [di] cseg01:05DE cseg01:05DE loc_5DE: ; CODE XREF: __intdos+4E↑j cseg01:05DE mov [di+0Ch], si cseg01:05E1 pop ds cseg01:05E2 pop di cseg01:05E3 pop si cseg01:05E4 sub bp, 2 cseg01:05E7 mov sp, bp cseg01:05E9 pop ds cseg01:05EA pop bp cseg01:05EB dec bp cseg01:05EC retf cseg01:05EC __intdos endp
In this case we can see the error handler called with jb short loc_5D0
So using patch bytes feature in IDA pro we patched out the two bytes of instruction to two No Operation “NOP” (90 90)”
The error handler never gets called now, if the lock / unlock APIs are unavailable:
These changes were then saved using Edit –> Patch Program –> Apply patches to input file.
Visual Basic now loads the UI, but we do get a Device I/O error.
Several OCX files fail to load.
Despite the OCX controls failing to load creating a program, compiling and running works fine now!
While this fix bypasses the error it will make the application less stable as it no longer has any file locking capability, and corruption of files is likely.
So the best fix would to configure/fix DosBox to support the lock / unlock MS-DOS API.
chentiangemalc 
Well done, respectable work. But not a good tradeoff if you lose OCX capability (no COMDLG = no Open/Save/Print dialogs, for example).
Instead, while waiting for the DOSBOX team, use VSHARE.386. That’s the Windows-integrated 32bit version of SHARE.EXE:
https://jeffpar.github.io/kbarchive/kb/112/Q112025/
It’s installed by default on Windows for Workgroups 3.11.
Otherwise, people suggest Visual Basic 3 as is the recommended version for Windows 3.1 because it’s faster and produces smaller native binaries.