This forum is closed to new posts and responses. Individual names altered for privacy purposes. The information contained in this website is provided for informational purposes only and should not be construed as a forum for customer support requests. Any customer support requests should be directed to the official HCL customer support channels below:

HCL Software Customer Support Portal for U.S. Federal Government clients
HCL Software Customer Support Portal


~Julia Prejipymarakol
Feb 10, 2016, 6:51 PM
23 Posts

Domino Transaction Logging Issue

  • Category: Domino Server
  • Platform: Windows
  • Release: 9.0.1
  • Role: Administrator
  • Tags: Domino Transaction Logging Issue
  • Replies: 2

Hi,

Last week we had issue on Domino Mail server as getting Message Recovery Manager Full and ' The database D:\Lotus\Domino\Data\mail\AMIIE.nsf was unable to write
to file D:\DAOS\0084\A894831CF04DD0BCC1257F4E007F5D2512667A894831CF0P.nlo: 'and further we can see DB corruption issue on domino server therefore we have created back to back NSD and restarted server and further PMR has been raised with IBM after investigation by IBM saying issue with the infra related to Hardward/I/O etc.

Windows team has completely Scan Hw and OS however did not seen any issue Transaction logging set on Same drive which is physical also SAN sfp has been increased from 4 GB to 8 GB at the moment we can see DAOS issue has been fixed and no more error message about Recovery manger full however still issue with the Transaction logging as Database.RM.SinceStartup.Aborts = 3171 can you please let us know how we can fix these type of issue?

Database.RM.Logger.IO.Avg.OS.Write.Time = 1
Database.RM.Logger.IO.Avg.Write.Time = 1
Database.RM.Logger.IO.Max.OS.Write.Time = 35
Database.RM.Logger.IO.Max.Write.Time = 35
Database.RM.SinceStartup.Aborts = 3171

Many thanks for your assistant.

kind regards,

Mahesh Dhuri. 

 

 

~Michelle Elhipilitli
Feb 11, 2016, 7:42 AM
103 Posts
Disk Latency

I have seen this issue before on servers which have very high average disk queue lengths, some using SAN and some physical drives.  It happens when the server is processing lots of transactions but the disk on the server cannot keep up with the volume of transactions generated.

You can check your servers by running:

sh stat platform.logicaldisk.*

You will see:

[0A8C:0009-0DF4]   Platform.LogicalDisk.1.AssignedName = HarddiskVolume1
[0A8C:0009-0DF4]   Platform.LogicalDisk.1.AvgQueueLen = 0
[0A8C:0009-0DF4]   Platform.LogicalDisk.1.AvgQueueLen.Avg = 0
[0A8C:0009-0DF4]   Platform.LogicalDisk.1.AvgQueueLen.Peak = 0.01
[0A8C:0009-0DF4]   Platform.LogicalDisk.1.BytesReadPerSec = 0
[0A8C:0009-0DF4]   Platform.LogicalDisk.1.BytesWrittenPerSec = 0
[0A8C:0009-0DF4]   Platform.LogicalDisk.1.PctUtil = 0
[0A8C:0009-0DF4]   Platform.LogicalDisk.1.PctUtil.Avg = 0
[0A8C:0009-0DF4]   Platform.LogicalDisk.1.PctUtil.Peak = 0.52
[0A8C:0009-0DF4]   Platform.LogicalDisk.1.ReadsPerSec = 0
[0A8C:0009-0DF4]   Platform.LogicalDisk.1.WritesPerSec = 0

Monitor the drive which is housing your Domino data folder. If it is a Windows server you can also use perfmon or resource monitor. Ideally the AvgQueueLen.Peak should be under 1.0

You have two options.

1) Reduce the volume of transactions

     You can do this by using the notes.ini settings Server_Max_Concurrent_Trans, SERVER_POOL_TASKS and <PortName>_PortThreadPoolCount= <number>

     http://www-01.ibm.com/support/docview.wss?uid=swg21324864

     http://www-01.ibm.com/support/docview.wss?uid=swg21207456

     http://www-01.ibm.com/support/docview.wss?rs=203&uid=swg21220856

     This will impact users who access databases directly on the server so be careful.  Not such an issue if it's a mail server and you are using MMR.

 

2) Improve the disk performance

     This will be up to your SAN team but if they have no clue, check if you are using a single LUN for the whole server (OS + Domino + Domino data + Transaction files + DAOS).  If this is the case then split the LUN up into multiple LUNs.  One for the OS and Domino program files.  One for the Domino data folder.  One for Transaction files and one for DAOS.

 

Other info on performance https://www-10.lotus.com/ldd/dominowiki.nsf/dx/Domino_Server_performance_troubleshooting_best_practices

 

~Tanita Zeklutexakol
Feb 11, 2016, 4:31 PM
32 Posts
Those write times are awfully high
but they are not the cause of the aborts. Unfortunately we don't log anything about the aborts, but there might be something in the console logs. Are the aborts causing issues for the users or is it just the stat that has you concerned ? If it is causing issues that would be a clue as to what is going on.

If you have a pmr opened you can continue to work it through that, or if you want to can send me a console log where aborts have been known to happen & I can take a look. But, as I said, if there are issues that are seen, send them along as well.

--Steve
swatts@notesdev.ibm.com

This forum is closed to new posts and responses. Individual names altered for privacy purposes. The information contained in this website is provided for informational purposes only and should not be construed as a forum for customer support requests. Any customer support requests should be directed to the official HCL customer support channels below:

HCL Software Customer Support Portal for U.S. Federal Government clients
HCL Software Customer Support Portal