This forum is closed to new posts and responses. Individual names altered for privacy purposes. The information contained in this website is provided for informational purposes only and should not be construed as a forum for customer support requests. Any customer support requests should be directed to the official HCL customer support channels below:

HCL Software Customer Support Portal for U.S. Federal Government clients
HCL Software Customer Support Portal


Apr 9, 2014, 9:12 PM
2 Posts

Unable to extend an ID Table - insufficient memory

  • Category: Notes Client
  • Platform: Windows
  • Release: 9.0.1
  • Role: Administrator,Developer
  • Tags:
  • Replies: 3

One of our R and D databases now has over 2 million documents.  The database is not utilizing transaction logging.

In the Notes Client, when attempting to select all documents in some of the views in the database, we are seeing the error "Unable to extend an ID Table - insufficient memory".

I do not see this error for all views.  For example, I can successfully select all documents in a view of all documents by UNID, but then I see the error for other views containing a subset of 1 million documents or 500,000 documents.  I haven't seen this error for views containing less than 100,000 documents.

We've also seen this error in the Domino Server's Log file (log.nsf) after we've run a Java agent to update a data field on a large set of documents.  After this, one of our web applications fails to successfully search / display information from some views of documents.  At this time we have not run this agent against the large database in question, and we are not seeing any errors in the web application.

I have seen older Notes / Domino forum posts for this error associated with compacting a database.  One of the causes of this error is said to be a high number of deletions.  This database only has about 400 deletions, so I question if this is an issue with our database.

In years past it has been suggested to make a non-replica copy from a good backup.  We have gone this route but we still see the problem.

Also, there has been a recent reference to SPR JPAI6W8KUJ indicating that there is a new compact -REPLICA option for 9.x or for 8.5.3 FP3. We have tried this new style of compact without success.

First, I understand that a Domino database has multiple ID tables.  Are ID tables created for each view, or for sets of documents based on form type? Is there any way to gather any information about the ID tables in the database and which one(s) might be bad?

Second, is there any additional debugging or information gathering that I should be doing?

Thanks.

 

Apr 10, 2014, 1:18 PM
26 Posts
we split our database

I was working for a company about 2-3 years ago, we ran into the same issue.  I forgot how many docs the db had but I know it was 1m+, it may have been over 2m.  I worked with IBM for over 6 months on it trying every suggestion they had.  In the end we decided to split the database so that all open/active records and closed records under 1 year old were in 1 database and everything else went into the secod database.  We used an archive agent to copy closed records over 1 yr to the second db.  In the end it kept the doc counts about the same in each database.  We also purged the deletion stubs and lowered the purge interval.

Apr 11, 2014, 4:28 AM
32 Posts
idtables
IDTABLES are heavily used in nsf/nif. They are a compressed list of note ids & are built & passed between clients & servers & within the server/client code (as well as stored in different places in the db). We have a limited amount of memory for each idtable & depending on the density of the documents in the table, it can hold a larger number of a smaller number of documents. So for instance if all the note ids are in sequence, the IDTABLE can hold a lot of note ids, but if there are large gaps or holds both normal docs & deleted docs, it can hold fewer docs (deleted docs typically have the high order bit turned on).

Since they are stored as well as passed from client/application to server we have to be very careful about any changes in the idtable code since the down level client/application/server won't be able to understand the changed storage. In the short term it would be best to
1) open a pmr
2) follow the suggestion of the other response about possibly splitting the db.

--Steve
swatts@notesdev.ibm.com
Apr 21, 2014, 8:36 PM
9 Posts
I suggest you recreate the databas
I suggest you recreate the database fresh using this steps:

        **Before following this, have a back up of the original database.

        Pls follow these steps:

        1. Create a new database from scratch based off of the same template.
             Make sure you place it to the correct directory of the database on the server.

        2. Run ' load fixup -f directory/databasename.nsf ' and
            ' load updall -r directory/databasename.nsf ' to the corrupted database.

        3. Run ' load convert -m directory/databasename.nsf ' to the new database
            and to the corrupted one.

        4. Next download the tools database from the technote

        Title: All-in-one Admin Tool for agent-based troubleshooting & problem solving
        Doc #: 1459332
        URL: http://www.ibm.com/support/docview.wss?uid=swg21459332

             and follow the directions in the About page of the tool to install it to the server (the instructions
                     should be easy to follow starting with opening the tools database from the technote)

        5. After you install it to your server, open it and click on to the 'Create Mail Database Copy'
            Supply the database you want to copy from and you want to copy to, in your case
            the database that you will copy from is the corrupted mailfile and the database you will copy to
            is the mailfile you newly created. After you supply the db paths, click the last button to
            perform the copy process.

        6. Edit the ACL of the new database and add users that should have access to it.

        ***Reminder: Make a new replica of this new database to the cluster servers you have, and
                             you may want to remove the corrupted database on the server directory and store
                           it on a different directory.  But don't delete it, We might need it as a back up
                           for the docs you have in it.

In this process we can be sure that the id Table is fresh and new

This forum is closed to new posts and responses. Individual names altered for privacy purposes. The information contained in this website is provided for informational purposes only and should not be construed as a forum for customer support requests. Any customer support requests should be directed to the official HCL customer support channels below:

HCL Software Customer Support Portal for U.S. Federal Government clients
HCL Software Customer Support Portal