This forum is closed to new posts and responses. Individual names altered for privacy purposes. The information contained in this website is provided for informational purposes only and should not be construed as a forum for customer support requests. Any customer support requests should be directed to the official HCL customer support channels below:

HCL Software Customer Support Portal for U.S. Federal Government clients
HCL Software Customer Support Portal


~Yentl Mintooly
Mar 3, 2014, 9:01 AM
21 Posts

View Indexing Always Out of Date

  • Category: Notes Client
  • Platform: Windows
  • Release: 9.0.1
  • Role: Developer
  • Tags:
  • Replies: 7

Hi all,

We have started to experience an issue with view indexing in one of our databases.

Indexing never seems to keep up now with changes and hence updated documents don't appear in the view. And even older documents are not showing.

Database is 12.5 GB in size with around 188K documents.

This has only been reported in 2 views but may affect others I presume. These 2 views show around 10K and 7K documents and responses.

There are over 300 views in the database and we have 1 indexer task on the server.

Also, overnight, the number of documents in these views is greatly reduced from previous day. I have to Shift-F9 to get them up to date. Very strange behaviour.

I have also purged these indexes but no change.

This seems to have only started happening since Domino server was moved to a new host but I don't see how this could affect DB view indexing.

Can anyone give me any idea what the problem might be please .. ?

Do we need to add another indexer task perhaps which I understand can be created via Admin client which then takes effect after server restart.  .... ?

Any advise greatly appreciated.

Regards

Andy

~Sean Eljumigon
Mar 3, 2014, 1:27 PM
326 Posts
Question

Are there any date formulas in the selection formula?   Can you post the selection formula for these views?  

~Yentl Mintooly
Mar 3, 2014, 2:34 PM
21 Posts
Selection Formulas

No date formulas but here you go below .... there is an agent in this DB which for views with selection formulas that have @Today etc ..  runs and replaces '@Today' with today's actual date DD/MM/YY.

But there views aren't processed in that agent.

SELECT Form = "PolYear" & PolStatus = "Active" & DataStatus = "Clean" | Form = "Cover" & PolStatus = "Active"

and

SELECT Form = "PolYear" & PolStatus = "Active" | Form = "Renewal"  & PolStatus = "Active" | Form = "Cover" & PolStatus = "Active"

There are 3 columns that have column header sorting in first view and 5 with column header sorting in second view.

Still seems odd this only seems to have started happening recently.

Thanks

Andy

~Xagra Minponeburobu
Mar 3, 2014, 3:45 PM
9 Posts
Need to gather more information
Need to gather more information first.

We have started to experience an issue with view indexing in one of our databases.

        Is it all views or only views with certain properties?  And - only this database?

This has only been reported in 2 views but may affect others I presume. These 2 views show around 10K and 7K documents and responses.

        If ok - can you open a PMR and paste a design only copy of the database in the PMR - or if there is an issue with that then please post the general view properties that effect indexing.   Is form optimization on the database and do you use forms alot in the workflow?

        I assume this is not easy to reproduce on a smaller test copy of the database.

Also, overnight, the number of documents in these views is greatly reduced from previous day. I have to Shift-F9 to get them up to date. Very strange behaviour.

I have also purged these indexes but no change.

        Is it happing with all 3 varities of change?

                        * Update existing.

                        * Add new

                        *Delete existing

This seems to have only started happening since Domino server was moved to a new host but I don't see how this could affect DB view indexing.

        Is the machine significantly faster?  What architecture did you go from/to?  Could be timing related.

        Was the file copied in is entirety from the OS console or a new replica created?

Do we need to add another indexer task perhaps which I understand can be created via Admin client which then takes effect after server restart.  .... ?

        I am assuming a manual "updall" on the database does not correct the situation.  Another indexer will not help.                        

       

~Sean Eljumigon
Mar 3, 2014, 5:11 PM
326 Posts
By Chance

Is the new host in a diffrent time zone then the other host was? 

~James Revelulyader
Mar 3, 2014, 8:05 PM
145 Posts
Try UPDATERS=2 in Server notes.ini
I'm assuming this new host has multiple CPU's

I like to set UPDATERS=4 on my Domino Servers so 4 Indexer's are run....

Start by trying UPDATERS=2  on your server and (UPDATERS=4 if you have lots of CPUs) and see if problem gets better..
~Fritz Bretumimanakoi
Mar 4, 2014, 2:44 PM
15 Posts
Try a few of these things on the server.
Note that these are general recommendations going off of the data you have provided. It may be that these parameters need to be set to different values for your environment.

UPDATERS=2 (number of update tasks to use) Start with going to 2. This will likely be what helps you out since you are just running one updater. You can set this in the Configuration Document and it will take effect on the next restart. You can run load update with the server online and another updater task will be spawned as you would like. One thing to note is there is a bug in which you can see corruption due to multiple indexers running over the same view. Info is (was) here. However, the technote is no longer public. - TITLE: LO73693: MULTIPLE INDEXES WORK ON THE SAME VIEW LIKE IN SPR GRCE6YVM2F RESULTING BTREE ERRORS FOR THE VIEW.
- URL: http://www.ibm.com/support/docview.wss?uid=swg1LO73693&myns=swglotus&mynp=OCSSKTMJ&mync=E
- ABSTRACT: If you run multiple indexes there is a chance that both will update one index a time, see SPR GRCE6YVM2F for example, turning on debug_nif will also allow you to see in console.

There are some other factors/settings to consider on a database with that many views (fyi, the above APAR was specifically documenting a problem one of my customers was having with running 4 or 5 updaters. Setting it to 3 seems to be ok for them, so your mileage may vary). But I digress, here are some of the other things you should consider:

Where are you allowing the server to do index REBUILDS? If you don't have VIEW_REBUILD_DIR=D:\ViewRebuild (or similar) set, then the rebuilds are happening in %TMP%. If this disk is slower than the Domino Data disk (or TransLog, etc...), then this can have a big impact on performance and keeping views updated should they need to be rebuilt.

How much RAM do you have? Under normal operations, view UPDATES will run directly in RAM (this is my understanding from many discussions with IBM). It is also the case that REBUILDS tend to happen in RAM first (and will not always get sent to the rebuild directory as specified above). Make sure you have adequate RAM available.

The following three settings may also need to be set, but this may only need to happen if updaters=2 does not help you out.
UPDATE_ACCESS_FREQUENCY=2 (for views that haven't been accessed in x number of days, they will be kept updated, the default is 7)
UPDATE_NOTE_MINIMUM=30 (view updates will not be triggered until x number of documents have been modified, the default is 20 and this should always be higher than 20)
UPDATE_IDLE_TIME_MS=11000 (this is 11 seconds and means that update will wait at least this long before issuing an update. This can improve responsiveness. The default is 5 seconds)

Hope this is helpful. I feel your pain, but you should be able to get through it. For the most part...
~Yentl Mintooly
Mar 11, 2014, 9:01 AM
21 Posts
All Seems Well now ......

Hi all,

We have now found out the cause of our indexing issues.
I checked through the log databases and saw several thousand documents being pulled to our server at 0800 each day.

We work with a third party company who look after our website and Lotus Extranet.
Each morning they have some agents running which update various entries at their end.

My index counts were always out by a few thousand and this is around the number being processed by one of the agents.
My extra Updall is scheduled for 0500 and so when this didn't help, I was perplexed.

Replicaiton with external server begins at 0800, pulling in all the updated documents from their server.

I'm still not sure why the index didn't catch up though.
Is that due to the number of updates perhaps .. ?

Anyway, the offending agent is apparently a legacy job from some time ago and is not required now, so they have disabled it. All good now although I am keeping an eye on it this week.

 

Many thanks to all for your help and suggestions.

Regards
Andy


This forum is closed to new posts and responses. Individual names altered for privacy purposes. The information contained in this website is provided for informational purposes only and should not be construed as a forum for customer support requests. Any customer support requests should be directed to the official HCL customer support channels below:

HCL Software Customer Support Portal for U.S. Federal Government clients
HCL Software Customer Support Portal