Microsoft Exchange Server - FIPS-FS Error 0x800706BE, 0x80010105

We started encountering this issue on our on-prem (Hybrid) exchange server all of a sudden which was preventing emails from being scanned by the anti malware module, and hence were being held in the submission queue and not being delivered.

You may see errors in the event log such as;

The FIP-FS Scan Process failed initialization. Error: 0x80010105. Error Details: The server threw an exception.

The FIP-FS Scan Process failed initialization. Error: 0x800706BE. Error Details: The remote procedure call failed.

Fault bucket , type 0
Event Name: APPCRASH
Response: Not available
Cab Id: 0
Problem signature:
P1: scanningprocess.exe

The usual server and service restarts did not fix the problem.

The issue appears to be related to updates not being installed correctly for the exchange anti malware scanning module.

You can check/confirm this by running these commands from Powershell;

Add-PsSnapin Microsoft.Forefront.Filtering.Management.Powershell

Get-EngineUpdateInformation

We got a response like below - note the "UpdateAttemptFailed" status

You can also check the update engine settings by running the command

get-engineupdatecommonsettings

Note the PrimaryUpdatePath - when we attempted to access this URL we were getting a 404 error - this is potentially what the problem is?

Update 4/1/22

I found the following article from the Exchange Team Blog that references this issue and provides a script to fix it

https://techcommunity.microsoft.com/t5/exchange-team-blog/email-stuck-in-exchange-on-premises-transport-queues/ba-p/3049447/page/2

Before implementing the provided fix, I can ran the default script to Enable-AntimalwareScanning (located in C:\Program Files\Microsoft\Exchange Server\V15\Scripts) - mail delivery operated for a brief period of time before mails began queuing again.

I ran the script provided in the blogpost, and even after a full server restart, the problem still persists so I have disabled the antimalware scanning again

Fix/Solution/Workaround

Disable the anti-malware scanning option. This is easily done by running the included powershell script in your Exchange installation directory

C:\Program Files\Microsoft\Exchange Server\V15\Scripts\Disable-AntimalwareScanning.ps1

Run the script then restart the Microsoft Exchange Transport Service

Once we did this the submission queue immediately started clearing.

You can reverse the change easily by running the Enable-AntimalwareScanning.ps1 script in the same folder later on.

We've also found this recent MS article outlining steps to manually update the scan engines used by Exchange Server, but we found it still failed to update the signatures after running it

https://docs.microsoft.com/en-us/exchange/troubleshoot/setup/manually-update-scan-engines

How to reclaim space from deleted files on windows server deduplication volume

Windows server has great deduplication functionality allowing for increased usable capacity on non-operating system volumes, by performing block level deduplication to increase storage efficiency. This is particularly useful for storing data such as backups, which often involves minimal changing data, so a prime candidate for effective deduplication.

The problem often encountered with deduplication volumes is that when files are deleted, the space is not immediately reclaimed by the operating system. A "garbage collection" process needs to be run in order for the deduplication engine to reclaim this space.

Thankfully, this is incredibly easy to do and can be done from a powershell command prompt - here's the command you need to run;

start-dedupjob -type GarbageCollection -full -path e: -FastStart $true

Be sure to change the -path variable from E: to whatever drive you wish to run the garbage collection on. Also note that depending on the size of your volume, the garbage collection can take some time to initiate and complete, even with using the -FastStart switch.

You can check the progress of your deduplication jobs by running the command

get-dedupjob