This forum is closed to new posts and responses. Individual names altered for privacy purposes. The information contained in this website is provided for informational purposes only and should not be construed as a forum for customer support requests. Any customer support requests should be directed to the official HCL customer support channels below:

HCL Software Customer Support Portal for U.S. Federal Government clients
HCL Software Customer Support Portal



Dec 1, 2013, 6:33 PM
45 Posts

Storing Data for Charts etc.

  • Category: Server Side JavaScript
  • Platform: All
  • Release: All
  • Role: Developer
  • Tags:
  • Replies: 10

Hi,

I have developed several charts and tables showing company performance for viewing on a 'dash board'. The data is retrieved from views of Sales invoices, customer orders, support requests etc. using, in the main, View Navigators and repeats etc. iterating through a few thousand records and several categories. The problem I have is the time to load the data and render the chart or table in XPages.

This data does not have to be 'live' it can be generated overnight. Do I need to run an Agent, store the data in an Application Scope (but I am not sure if that can be done) for rapid retrieval during the day. I am obviously missing some basic concepts. Could someone point me in the right direction.

Any examples would be most welcome.

 

Many thanks

 

Mark.

Dec 2, 2013, 12:19 AM
366 Posts
I wouldn't put...

that much data in an application scoped variable.  You didn't say what the data result was.  JSON?  XML?  Domino Documents?

 

 

Dec 2, 2013, 1:42 PM
47 Posts
JSON - documents

What I've done in the past is to have "cache" documents that have a key and a text field with JSON data.  Agents run at certain times to populate/refresh the "cached" data.  Then, when building the charts I just ask for the data by key and get the JSON payload to use to create the charts (with dojo).

Dec 2, 2013, 2:19 PM
45 Posts
Storing Data for Charts etc.

Hi Paul,

The data is used in Repeats and Computed Fields sometimes it is used for the JSON values for charts. This example simply produces a column of our top 30 spending customers. The data is collected from 2 views, a view of invoices  categorised on the partent ID (the account ID) along with a second view of Accounts to extract the name from the ID. In this instance we are only using the sub totals rather than each entry, however, the time to generate the table, - or chart, if we then format to provide JSON format. for use in a chart, - is around 60 seconds. 30 results are returned as the repeat has 30 rows. Other examples use all of the data for Pie Charts.

This is a typical example, my objective it to try to carry out the process overnight and then point the results at a system stored variable of field to eliminate the processing during the day.

 

 

<xp:repeat id="repeat1" rows="30" var="rowDataSpend">

<xp:this.value><![CDATA[#{javascript:var db:NotesDatabase = session.getDatabase("Titan", "NCLCMS.nsf", false);

var v = db.getView("($ xPages Invoices By Account)");

var nav:NotesViewNavigator = v.createViewNav();

var entry:NotesViewEntry = nav.getFirst();

var vAccnt = db.getView("($ xPages Accounts by ID)");

var values = []; // create an empty array

while (entry!=null && !entry.isTotal()){

var obj={};

var acctID = entry.getColumnValues().get(0);

obj.account= vAccnt.getEntryByKey(acctID).getColumnValues().get(1);

obj.total=entry.getColumnValues().get(1);

values.push(obj);

var tmpentry:NotesViewEntry = nav.getNextSibling(entry);

entry.recycle();

entry = tmpentry;

}

// define comparator function, sorts by lastname + firstname

var comparator = function(a,b){

var v1 = a.total;

var v2 = b.total;

if (v1 > v2)

return -1;

else if (v1 < v2)

return 1;

else

return (a.total).compareTo(b.total);

}

values.sort(comparator)

}

 

]]></xp:this.value>

 

<xp:text escape="true" id="computedField3" value="#{javascript:rowDataSpend.account}"></xp:text>

</xp:panel>

<xp:panel styleClass="col2CS">

<xp:text escape="true" id="computedField4" value="#{javascript:rowDataSpend.total}">

</xp:text>

</xp:repeat>

Dec 2, 2013, 4:02 PM
45 Posts
Storing Data for Chart's etc.

Michael,

As I don't always want JSON data, e.g. as in the example, I need a 'repeat' to produce a listing, is this still the correct way to achieve my objective? i.e. would it be an array in a multivalue field.

 

Any examples would  be really appreciated.

 

 

Many thanks.

Dec 2, 2013, 9:55 PM
47 Posts
Documents in your bound view

From your example it looks as if you are only concerned with the Account and total spent.  So could you build a temporary view that gets refreshed each time by an agent that gets the total spent for each account and stores it in a document.  The view would show the account and the amount.  Then just use the repeat control bound to that temporary view.

Dec 3, 2013, 10:07 AM
45 Posts
Storing Data For Charts etc.

Hi Michael.

Yes I probably could do that in this instance and it is a good idea,  but this is one of many examples in others creating another view would not help.

I try not to create too many views the application already has 140 views. I believe the advice is not to have 'too many views in Notes' , but for the life of me, I cannot ever establish what 'too many' means, 10,100, 1000 ? I know it depends on the complexity of the view, sorts, no. of records, categorisation etc. But if I knew what I should keep it below, then it would help.

Currently I am going down the path of an Agent using Lotusscript and attempting to store the results in a multivalue field in one document created for this purpose. Then using the multivalue area in a repeat. Is this the direction you advocate? I am a little concerned about the length of data in the field.  

As the number of accounts is a variable, in your suggestion below, are you creating a document for each account by the agent and then deleting them the next time it is run?

 

Many thanks.

Dec 3, 2013, 1:24 PM
366 Posts
I would store the data in documents by..

some sort of key that was specific to the graph that was being used to display the data and then either updated the fields in that document or replace the document every time.

You can programatically set the summery flag on a notes field to false and it can then store more then the total of a summary field up to the size of a document (1 gig I think)  Your json and xml data should never exceed that.

 

 

 

Dec 3, 2013, 1:32 PM
33 Posts
Do you have access to a RDBMS?

In the past, when doing complex reporting, I have found that it is much quicker for the end user to store the report data in a SQL table. 

On a macro scale, you have an agent that gets the data from Domino and writes it to the SQL table via O/JDBC and then query that data with another application e.g. hand rolled Notes App, Excel, Crystal Reports, XPages app etc.  An alternative to the agent is DECS / LEI / NotesPump or whatever name it goes by these days.  The benfits of doing it with an agfent is that you can create a "flat" table of all the data you need rather than "normalised" ones that need to be linked via joins in SQL.

Dec 3, 2013, 5:54 PM
47 Posts
Yes..storying as a document then removing

Mark, i was thinking about creating the documents, one per account, and removing /updating when new data is run.  If you are concerned about too many views, why not create a second database/application specific to storing this data.  If it's on the same server then it should be easy enough to access from your main application.

Dec 3, 2013, 8:56 PM
45 Posts
Storing Data for Charts etc.

Many thanks Paul, Michael and Rob I have plenty to go at there.

I will work with both Paul's single document and Michael's multiple documents and see which works for me (I am not an expert as you may have guessed) Thanks for the tip on the summary flag as I was concerned about the size and thought I would have to use Rich Text.

Rob, many thanks, it seems a little too advanced for me at this stage, but I appreciate you taking the time out.

 

Mark.

 

 


This forum is closed to new posts and responses. Individual names altered for privacy purposes. The information contained in this website is provided for informational purposes only and should not be construed as a forum for customer support requests. Any customer support requests should be directed to the official HCL customer support channels below:

HCL Software Customer Support Portal for U.S. Federal Government clients
HCL Software Customer Support Portal