- power goes out at a regional data center and backup power apparently fails.
- Domino server on a VM host won't come back up (we told them not to put high I/O Domino servers on VM )
- There is no failover server (also contrary to our recommendations)
- local admin says a drive failed (hard to believe, but maybe RAID was never configured properly on the VM host)
- the local admin eventually restores Domino app, data, and DAOS from the backup done the night before (backups are incremental daily with a full Friday nights)
- mail from between the date/time of crash and date/time of last backup is gone but the admin doesn't realize that
- users all have local replicas so some missing mail and attachments are probably still in the local replicas
- users receive numerous "file not found" messages, but local admin has no idea that this means nlo files are missing
- "file not found" messages prevent complete replications so users think they are not getting any mail
- users have no clue as to how to open their server replicas to see new mail that is not yet on their local replicas. Regional helpdesk also has no clue.
- users have no idea how to connect to the server with iNotes and most have no idea what their http password is
- local admin rushed to hospital for emergency surgery
- global admin (that's us in North America) is finally contacted and we quickly figure out DAOS is missing all kinds of files and some nlo files seem corrupt
- we get a local contact who we know speaks English and we show her how to get to iNotes so she can help the VIPs in their native language
- we figure we should have another go at restoring the DAOS drive/folders/files, but nobody but the guy in surgery knows the password for the backup software
- we know we will eventually get the password which we do, but in the meantime it is chaos for the users and the regional helpdesk
- a new shot at restore gets us some additional nlo files, we resync the DAOS catalog, but many users still are confused and frustrated
- we figure maybe we can find some more missing nlo's from the last Friday full backup.
- the tapes are far off-site with a third party and will take a couple of days.
- we figure they have two choices, neither of which are good, i.e. we wait for the tapes or we flush out the references to the missing nlo files
- the first choice would be the best choice if we could repress the "file not found" messages that are preventing replication
- IBM is no help on this situation
So, folks, does anyone have any ideas how to suppress the "file not found" message" so we can get replication working?