Microsoft Exchange Server - FIPS-FS Error 0x800706BE, 0x80010105

We started encountering this issue on our on-prem (Hybrid) exchange server all of a sudden which was preventing emails from being scanned by the anti malware module, and hence were being held in the submission queue and not being delivered.

You may see errors in the event log such as;

The FIP-FS Scan Process failed initialization. Error: 0x80010105. Error Details: The server threw an exception.

The FIP-FS Scan Process failed initialization. Error: 0x800706BE. Error Details: The remote procedure call failed.

Fault bucket , type 0
Event Name: APPCRASH
Response: Not available
Cab Id: 0
Problem signature:
P1: scanningprocess.exe

The usual server and service restarts did not fix the problem.

The issue appears to be related to updates not being installed correctly for the exchange anti malware scanning module.

You can check/confirm this by running these commands from Powershell;

Add-PsSnapin Microsoft.Forefront.Filtering.Management.Powershell

Get-EngineUpdateInformation

We got a response like below - note the "UpdateAttemptFailed" status

You can also check the update engine settings by running the command

get-engineupdatecommonsettings

Note the PrimaryUpdatePath - when we attempted to access this URL we were getting a 404 error - this is potentially what the problem is?

Update 4/1/22

I found the following article from the Exchange Team Blog that references this issue and provides a script to fix it

https://techcommunity.microsoft.com/t5/exchange-team-blog/email-stuck-in-exchange-on-premises-transport-queues/ba-p/3049447/page/2

Before implementing the provided fix, I can ran the default script to Enable-AntimalwareScanning (located in C:\Program Files\Microsoft\Exchange Server\V15\Scripts) - mail delivery operated for a brief period of time before mails began queuing again.

I ran the script provided in the blogpost, and even after a full server restart, the problem still persists so I have disabled the antimalware scanning again

Fix/Solution/Workaround

Disable the anti-malware scanning option. This is easily done by running the included powershell script in your Exchange installation directory

C:\Program Files\Microsoft\Exchange Server\V15\Scripts\Disable-AntimalwareScanning.ps1

Run the script then restart the Microsoft Exchange Transport Service

Once we did this the submission queue immediately started clearing.

You can reverse the change easily by running the Enable-AntimalwareScanning.ps1 script in the same folder later on.

We've also found this recent MS article outlining steps to manually update the scan engines used by Exchange Server, but we found it still failed to update the signatures after running it

https://docs.microsoft.com/en-us/exchange/troubleshoot/setup/manually-update-scan-engines

How to reclaim space from deleted files on windows server deduplication volume

Windows server has great deduplication functionality allowing for increased usable capacity on non-operating system volumes, by performing block level deduplication to increase storage efficiency. This is particularly useful for storing data such as backups, which often involves minimal changing data, so a prime candidate for effective deduplication.

The problem often encountered with deduplication volumes is that when files are deleted, the space is not immediately reclaimed by the operating system. A "garbage collection" process needs to be run in order for the deduplication engine to reclaim this space.

Thankfully, this is incredibly easy to do and can be done from a powershell command prompt - here's the command you need to run;

start-dedupjob -type GarbageCollection -full -path e: -FastStart $true

Be sure to change the -path variable from E: to whatever drive you wish to run the garbage collection on. Also note that depending on the size of your volume, the garbage collection can take some time to initiate and complete, even with using the -FastStart switch.

You can check the progress of your deduplication jobs by running the command

get-dedupjob

Microsoft Exchange - 451 4.7.0 Temporary server error. Please try again later. PRX5 - FIX

I encountered this error on my on-premise/Hybrid exchange server today after noticing that internal mail relay was not working. When attempting to send a test email via telnet, I got the error 451 4.7.0 Temporary server error. Please try again later. PRX5

After rebooting the server, the error still persisted - so a quick bit of searching quickly lead to me adjusting the Exchange servers DNS settings.

From within Exchange Control Panel (ECP) - go to Servers (left hand menu) and double click your Exchange server to open it's properties

Select DNS Lookups from the left menu.

By default, the option for External and Internal DNS lookups is usually All network adapters (All available IPv4). I changed this to Microsoft Hyper-V Network Adapter and the problem was immediately resolved.

Google Classroom - How to get a list of all classes/classrooms

You can use the GAM utility to easily export the details of all Google Classrooms in your domain by following the commands below. In this example we'll be exporting the details to a CSV file

(Be sure to update the path to gam.exe and the output CSV file location)

$allclasses = C:\admin\gam\gam.exe print courses > "C:\Admin\Google Classroom\allclasses.csv"

This works well, but if you look at your CSV file you'll notice that firstly, it isn't formatted into columns properly, and secondly, there are a heap of fields called "coursematerial" that aren't required. We can filter out those fields by using the command below

$allclassesfiltered = import-csv "C:\Admin\GoogleClassroom\allclasses.csv" | select-object * -ExcludeProperty "courseMaterial*"

We can then re-export the CSV file with the command below which will actually export it correctly as a CSV file with data separated into columns correctly

$allclassesfiltered | export-csv "C:\Admin\Google Classroom\allclasses-filtered.csv" -NoTypeInformation

GAM - How To Sync Google Classroom Students from CSV File

GAM is great command line utility that fills a huge void in the management of Google Classroom - since Google haven't bothered after all these years to create any kind of centralised management interface for the product despite a huge uptake in use from COVID-19 and remote learning.

One of the downfalls of GAM is it can be a little slow - if you wanted to add students individually to a class, the command to add each student takes several seconds to execute - which doesn't sound like much, but in a large school with lots of students and lots of classes - it could take hours to complete.

Thankfully, they have included the ability to synchronise members (students) of a class from a CSV file. What they don't mention in their documentation though is how the CSV file should be formatted - kind of important, right?

The CSV file should be formatted as a basic list - with a single column and no column headings - containing the email addresses of the students you wish to add. If you opened it in Notepad - it would look like the screenshot below

Example CSV file format to sync students to Google Classroom with GAM

One thing to note - if you are going to sync from a CSV file, it will add all the students in the CSV file to the classroom, but it will also remove any students already in the classroom who aren't listed in the CSV file, so it's important your CSV file contains all the students in the class.

The command to run the sync is below;

C:\admin\gam\gam.exe course "googlecourseid"  sync students file "C:\admin\GoogleClassroom\class1.csv"

(Be sure to update the path to your gam.exe file, the google course ID and the CSV file location)

Microsoft Teams - Add Members from Active Directory Group to Team

Teams may often need to be populated with the members of an existing Active Directory security or distribution group. You can use the powershell commands below to easily do this.

Firstly, we need to get the members of our Active Directory group - and pipe the result to the get-aduser cmdlet so we can include the mail property as this is the identifying field we need to use to add users to Teams.

$grpmembers = Get-ADGroupMember -identity | Get-ADUser -properties mail

We will then enable connect to Microsoft Teams to enable the cmdlets, cycle through the list of members and add them to the team. Make sure you update the $teamgrpid to be the unique group ID for the team you wish to add the users to.

Connect-MicrosoftTeams

$teamgrpid = "123456"

foreach ($grpmember in $grpmembers)
    {
    Add-TeamUser -groupid $teamgrpid -user $($grpmember.mail) -role user
    }

You can change the role variable from user to member if you wish to add the users as members instead.

Install Active Directory Powershell Module in Windows 10

For whatever reason, Microsoft has now changed the way the powershell module for Active Directory Domain Services and Lightweight Directory Services tools.

Most documentation suggests you install this from the "Turn Windows features on or off" menu located under Programs and Features within Control Panel - but in later Windows 10 releases, the feature is no longer available. It was previously called "Role Administration Tools".

Role Administration Tools is no longer visible under Turn Windows Features on or off

Likewise, the previously used powershell command is no longer recognised

Enable-WindowsOptionalFeature -Online -FeatureName RSATClient-Roles-AD-Powershell

You now get an error when running this command

Enable-WindowsOptionalFeature : Feature name RSATClient-Roles-AD-Powershell is unknown.

So, how do you do it? Follow the steps below

1. Click the Start button then select Settings
2. Select Apps
3. Select Optional Features
4. Click Add a Feature
5. Search for "RSAT" and select the option for RSAT: Active Directory Domain Services and Lightweight Directory Services Tools. 
   
   
   
   

   

   Select the option for RSAT: Active Directory Domain Services and Lightweight Directory Services

   
   
6. Click Install

If you get an error saying the install failed, make sure you aren't using a WSUS server as windows will try and install it from there by default if you are.

To check, open regedit and go to HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\WindowsUpdate\AU and set the UseWUServer key to 0