Skip to main content link. Accesskey S
  • HCL Logo
  • HCL Notes and Domino wiki
  • THIS WIKI IS READ-ONLY. Individual names altered for privacy purposes.
  • HCL Forums and Blogs
  • Home
  • Product Documentation
  • Community Articles
  • Learning Center
  • API Documentation
Search
Community Articles > Lotus Domino > Domino memory > "POOL IS FULL" Memory Errors on Various Blocks
  • Share Show Menu▼
  • Subscribe Show Menu▼

Recent articles by this author

AccessAllProtected Crashes Involving Shared Memory

Avoid PANIC: Insufficient memory, PANIC: Cannot attach to shared memory region, due to insufficient access (probably owned by another user or group), PANIC: AccessAllProtected() error (107 or 1B1 or 1F5) from MapSharedRegion on pool of pools.

"POOL IS FULL" Memory Errors on Various Blocks

When the server is reporting pool full errors, it is experiencing a possible memory leak in the block for the listed pool. The top 5 commonly reported memory blocks with this condition are discussed in this article
Community article"POOL IS FULL" Memory Errors on Various Blocks
Added by ~Kirk Nonhipi | Edited by ~Kirk Nonhipi on October 24, 2012 | Version 9
  • Actions Show Menu▼
expanded Abstract
collapsed Abstract
When the server is reporting pool full errors, it is experiencing a possible memory leak in the block for the listed pool. The top 5 commonly reported memory blocks with this condition are discussed in this article
Tags: memory, pool, leak, insufficient

This article is part of a larger series: Preventative Actions and Troubleshooting Common Domino Memory Issues


When the server is reporting pool full errors, as listed below, it is experiencing a possible memory leak in the block for the listed pool. The top 5 commonly reported memory blocks with this condition is listed below:
Insufficient memory - NSF directory manager pool is full
Insufficient memory - Index pool is full
Insufficient memory - Event pool is full
Insufficient memory - NSF Monitor pool is full  
Insufficient memory - Network pool is full

Once the errors are reported on the server console, the Domino administrator should take the following actions:

1. Capture two manual NSDs:
How to run NSD manually on a Domino server for UNIX platforms (Technote #1214298)
How to run a manual NSD for Notes/Domino on Windows (Technote #1204263)
  
2. Bring down the Domino server

3. Enable the following debug in the server's notes.ini:
CONSOLE_LOG_ENABLED=1
DEBUG_THREADID=1
DEBUG_SHOWLEAKS=1
DEBUG_TRAPLEAKS_SHOWSTACK=1
DEBUG_DUMP_FULL_HANDLE_TABLE=1
DEBUG_SHOW_MEMORY=1
DEBUG_TRAPLEAKS=[block_type]

4. Bring the Domino server back up

5. Set up memory dumps for collection every 4 hours to capture memory allocations
How to automate the collection of memory dumps (Technote #1104943)
   
6. Submit the manually executed NSD(s) and console log file from the incident to support.

7. If the issue reoccurs with the debug enabled then perform step 1 and restart server. Afterwards submit the NSD(s), console log and memory dumps from the incident to Lotus support.


Proactive Actions


The Domino administrator can monitor for any of the above error messages reported to the Domino console by using Domino Domain Monitoring (DDM) probes to alert them once they occur so that the listed actions can be performed.  
Notes/Domino Best Practices: Domino Domain Monitoring (DDM) (Technote #7009312)


The Domino administrator can also dynamically enable the following through the Domino server console to monitor for the error messages:
set config DEBUG_CONWRITE=pool is full
set config DEBUG_CONWRITE_NOBREAK=1

Additional Information on the listed blocks


When entering the block types for any of the listed blocks in the DEBUG_TRAPLEAKS parameter, you will use the below hexadecimal type IDs (dropping 0x) for the corresponding block.

Example: To set up trapleaks on the NSF directory manager pool block BLK_NSF_DIRMANPOOL, the parameter will look like the following:

DEBUG_TRAPLEAKS=826d

NSF directory manager pool corresponds to memory block: BLK_NSF_DIRMANPOOL (Type: 0x826d)
Index pool corresponds to memory block: BLK_NIF_POOL (Type:0x8311)
Event pool corresponds to memory block: BLK_EVENT_POOL (Type: 0x9506)
NSF Monitor pool corresponds to memory block: BLK_NSF_MONITOR_POOL (Type:0x82df)
Network pool corresponds to memory block: BLK_NETBUFFER (Type:0x8a03)


Debug Parameters in use


DEBUG_TRAPLEAKS prints out information about private memory that is still allocated when a process terminates. It also includes some information about shared memory as well, which is allocated by a process but accessible to all Domino processes, when the last process terminates. Therefore, to get information about a leak, the process (if the leak is on private handles/blocks) or the server (if the leak is on shared handles/blocks) must quit cleanly because trap_leaks compares what was allocated to what is left at program end.  If there is a crash, trap_leaks will not be able to report leaks.

By setting DEBUG_CONWRITE, the Domino console checks console message and a stack trace is printed to the console when the specified message is reported. By default, the Domino server halts when the message is reported; therefore, the parameter, DEBUG_CONWRITE_NOBREAK=1, will need to be set to keep the Domino server running normally. The value of DEBUG_CONWRITE will never display in a console log. This is to intentionally stop the server from generating a stack/NSD at the wrong time. You will need to check the notes.ini, NSD or the sysinfo file if you need to confirm the value of this setting.

 

Related Documents

"Insufficient Memory - NSF Directory Manager Pool is Full" or "The Buffer Pool is Too Full to Bring in Another Page" (Technote #1209395)

Error: "Insufficient memory - index pool is full" results in server hang (Technote #1405762)

Error: 'Warning: Cannot record event - cannot keep up with event occurrence rate!" (Technote #1097225)

Domino server panic with BLK_NETBUFFER memory leaks (Technote #1407257)

BMI Causes a Memory leak on BLK_NSF_POOL (Technote #14008593)

INTERMITTENT MEMORY LEAK ISSUE IN BLOCK BLK_NSF_DIRMANPOOL (APAR #LO47470)

Server Crash with error message PANIC OSBBLOCKADDR: BAD BBLOCK HANDLE (0) Domino Crash when calling function Nifviewregisterrebuild (Technote #1501337)

Server crashed with the panic message " PANIC: OSBBlockAddr: Bad BBlock handle (0)" (Technote #1408586)


Go back to: Preventative Actions and Troubleshooting Common Domino Memory Issues

About the Author
Shermeker Sanders is an IBM Certified IT Specialist currently working as an Advisory Software Engineer Lead in the IBM Collaborative Solutions (ICS) division on the Domino UNIX Crash & Performance team in Atlanta, GA. Previously she worked on the AIX pSeries Kernel and Crash teams in Dallas, TX.

  • Actions Show Menu▼


expanded Attachments (0)
collapsed Attachments (0)
Edit the article to add or modify attachments.
expanded Versions (1)
collapsed Versions (1)
Version Comparison     
VersionDateChanged by              Summary of changes
This version (9)Oct 24, 2012, 10:03:10 PM~Kirk Nonhipi  
expanded Comments (0)
collapsed Comments (0)
Copy and paste this wiki markup to link to this article from another article in this wiki.
Go ElsewhereStay ConnectedAbout
  • HCL Software
  • HCL Digital Solutions community
  • HCL Software support
  • BlogsDigital Solutions blog
  • Community LinkHCL Software forums and blogs
  • About HCL Software
  • Privacy
  • Accessibility