Tech - Veeam Backup & Replication 12 - Failed to wait mutex sandiscover

Joshua D

Nov 27, 2023 • 2 min read

Over the last few months, my colleagues and I have been upgrading to Veeam 12.

NOW, after upgrading to Veeam 12, everything went well.

However, after a few weeks, we noticed that backups would still run there was something else going on!

Error: Failed to prepare VM for processing: [MachineSemaphore] Failed to wait for semaphore Global\PREPARING_SAN_VM_: timeout 10800000 ms exceeded

Until one day...

A: Ummm what the hell is this error about?

J: What error?

A: Mutex sandiscover error on veeam

A: All backups are failing...

J: S***.

Let the long week begin.

All backups failed to create snapshots from SAN.

In my case using HPE Nimble Replicated Snapshots.

We would see these errors in the logs:

Error: Failed to prepare VM for processing: [MachineSemaphore] Failed to wait for semaphore Global\PREPARING_SAN_VM_: timeout 10800000 ms exceeded

Or:

Failed to create snapshots on primary storage: Failed to wait mutex sandiscover: timeout 600 sec exceeded

Failed to create storage snapshot for datastore : Failed to wait mutex sandiscover: timeout 600 sec exceeded

Resolution:

The issue was down to the old decommissioned backup infrastructure being queried and the query failed and timed out!

The resolution for us was nice and simple!

Ensure that all your objects under Backup Infrastructure are still able to be reached by Veeam!

If not remove them as they will timeout.

This was weeks after the upgrade to Veeam.

So if you have done the upgrade and start getting this week later don't make the mistake we did and look everywhere else first!

Bad few days...

Just like to say thanks to Veeam Support for helping us find this issue! We normally have very little issues with Backup and Replication so this was a strange one!

I've also seen a lot of other people have this issue since the upgrade so it's worth a share!

#backups #veeam #backupreplication #mutex #san #timeout