Domino Access Services (DAS) is a family of REST APIs built into IBM Domino and available from the XPages Extension Library on OpenNTF.  The DAS data API is used to create, read, update and delete data in any Domino server application.  This article describes some techniques for optimizing the performance of the data API.  It assumes you already have some experience with this API.  For more general information on the data API, see this reference.

The DAS data API is designed to provide access over HTTP/HTTPS to virtually any Domino server application.  When you use the data API to read a view, an individual document, or other data object, the default response often contains more data than you actually need to implement your application.  Since the data API cannot infer what's important to you, it returns the maximum amount of data.  Fortunately, you can often boost performance by requesting less data.  It's simply a matter of adding one or more query parameters to the request URL.

One example of this is the compact parameter.  This is a boolean query parameter you can use in any GET request.  Consider, for example, the database collection request:

GET /api/data

The response is a potentially large array of databases like this:

[

  {

    "@title": "Administration Requests",

    "@filepath": "admin4.nsf",

    "@replicaid": "852573910361A2F4",

    "@template": "StdR4AdminRequests",

    "@href": "/admin4.nsf/api/data/collections"

  },

  {

    "@title": "Java AgentRunner",

    "@filepath": "AgentRunner.nsf",

    "@replicaid": "8525671400725208",

    "@template": "",

    "@href": "/AgentRunner.nsf/api/data/collections"

  },

  ...

]

By default, the response include lots of white space including spaces and new line characters.  This white space increases readability and is helpful when you are learning about the data API, but it's not necessary as you move your application into production.  You can remove the white space from the response by adding the compact parameter to your request:

GET /api/data?compact=true

Depending on your network bandwidth and the number of databases on your server, this can reliably shave several milliseconds off the database collection response time.  Best of all, the compact parameter is available for all data API GET requests.

You use the view entry collection resource to read entries in a view or folder.  For example, the following is a simple view read request:

GET /XPagesExt.nsf/api/data/collections/name/AllContacts

The response is an array of view entries like this:

[

  {

    "@href": "/XPagesExt.nsf/api/data/collections/name/AllContacts/unid/39A95ECD82D0DEB485257AB5005BFEE0",

    "@link": {

      "rel": "document",

      "href": "/XPagesExt.nsf/api/data/documents/unid/39A95ECD82D0DEB485257AB5005BFEE0"

    },

    "@entryid": "1-39A95ECD82D0DEB485257AB5005BFEE0",

    "@unid": "39A95ECD82D0DEB485257AB5005BFEE0",

    "@noteid": "5D26",

    "@position": "1",

    "@read": true,

    "@siblings": 10000,

    "@form": "Contact",

    "Id": "CN=Aaron Goodman/O=renovations",

    "FirstName": "Aaron",

    "LastName": "Goodman",

    "EMail": "aaron_goodman@renovations.com",

    "City": "Philadelphia",

    "State": "PA",

    "Created": "2012-11-13T16:44:50Z",

    "$10": "Aaron Goodman"

  },

  {

    "@href": "/XPagesExt.nsf/api/data/collections/name/AllContacts/unid/32898140B9DB45F485257AB5005A9394",

    "@link": {

      "rel": "document",

      "href": "/XPagesExt.nsf/api/data/documents/unid/32898140B9DB45F485257AB5005A9394"

    },

    "@entryid": "2-32898140B9DB45F485257AB5005A9394",

    "@unid": "32898140B9DB45F485257AB5005A9394",

    "@noteid": "D7E",

    "@position": "2",

    "@read": true,

    "@siblings": 10000,

    "@form": "Contact",

    "Id": "CN=Aaron Guerrero/O=renovations",

    "FirstName": "Aaron",

    "LastName": "Guerrero",

    "EMail": "aaron_guerrero@renovations.com",

    "City": "Coral Springs",

    "State": "FL",

    "Created": "2012-11-13T16:29:20Z",

    "$10": "Aaron Guerrero"

  },

  ...

]

By default, the response includes the first page of 10 entries.  You can increase the page size (e.g. count=50) and send subsequent requests for more pages (page=1, page=2, etc.).  

Prior to Domino 9.0.1 FP8, there was virtually no limit to the number entries you could ask for in a single request.  If the view had tens of thousands of entries, you could receive ten thousand (count=10000) or more entries in one response.  However, as of Domino 9.0.1 FP8, the maximum page size is 100 entries by default.  IBM made this change in response to problem reports citing poor performance when the server was handling hundreds of simultaneous requests for ten thousand or more rows of data.  Since the data API was never designed to support that kind of load, IBM decided to enforce a maximum page size.  This encourages paging through large views and improves overall performance.

NOTE:  We realize this change in behavior has the potential to break existing applications.  You can relax the page size limit with a server Notes.ini setting.  For example, use the following setting to increase the maximum page size to one thousand entries:

DataServiceMaxViewEntries=1000

For best performance under high load, it's best to keep the maximum page size as low as possible.

Of course when optimizing performance, the size of each entry may matter as much as the total number of entries in each page.  Referring to the sample response at the beginning of this section, you will notice two distinct kinds of properties for each entry.  There are system columns (@href, @link, @unid) -- the names of which each begin with an @ -- and there are application-specific properties (Id, FirstName, LastName, etc.).  By default, the response includes all available system columns, but you can improve performance by using the systemcolumns parameter.

Consider a request like this:

GET /XPagesExt.nsf/api/data/collections/name/AllContacts?systemcolumns=0

The response is similar to the previous example, but most system columns are excluded:

[

  {

    "@entryid": "1-39A95ECD82D0DEB485257AB5005BFEE0",

    "Id": "CN=Aaron Goodman/O=renovations",

    "FirstName": "Aaron",

    "LastName": "Goodman",

    "EMail": "aaron_goodman@renovations.com",

    "City": "Philadelphia",

    "State": "PA",

    "Created": "2012-11-13T16:44:50Z",

    "$10": "Aaron Goodman"

  },

  {

    "@entryid": "2-32898140B9DB45F485257AB5005A9394",

    "Id": "CN=Aaron Guerrero/O=renovations",

    "FirstName": "Aaron",

    "LastName": "Guerrero",

    "EMail": "aaron_guerrero@renovations.com",

    "City": "Coral Springs",

    "State": "FL",

    "Created": "2012-11-13T16:29:20Z",

    "$10": "Aaron Guerrero"

  },

  ...

]

Depending on the view design, this can reduce the size of the response by more than 50%.  Not only does it take less time to transfer the response across the network, but the server spends less time reading data from the underlying view -- data you may not need any way.

The value of the systemcolumns property is actually a bit mask where each bit represents a specific system column:

• @noteid -- 0x0001
• @unid -- 0x0002
• @position -- 0x0004
• @read -- 0x0008
• @siblings -- 0x0010
• @descendants -- 0x0020
• @children -- 0x0040
• @indent -- 0x0080
• @form -- 0x0100
• @category -- 0x0200
• @response -- 0x0400
• @href -- 0x0800
• @link -- 0x1000
• @score -- 0x2000

So to request the @unid property and no other system columns, you might send a request like this:

GET /XPagesExt.nsf/api/data/collections/name/AllContacts?systemcolumns=0x0002

Or to request just the @unid and @link properties, you might send a request like this:

GET /XPagesExt.nsf/api/data/collections/name/AllContacts?systemcolumns=0x1002

In other words, you can request just the set of system columns you need for your particular application.

You use the document resource to read a single document by UNID.  For example, the following is a document read request:

GET /XPagesExt.nsf/api/data/documents/unid/39A95ECD82D0DEB485257AB5005BFEE0

The response is a document object in JSON format like this:

{

  "@href": "/XPagesExt.nsf/api/data/documents/unid/39A95ECD82D0DEB485257AB5005BFEE0",

  "@unid": "39A95ECD82D0DEB485257AB5005BFEE0",

  "@noteid": "5D26",

  "@created": "2012-11-13T16:44:50Z",

  "@modified": "2013-07-16T21:39:28Z",

  "@authors": "CN=Duke Lawson/O=Peaks",

  "@form": "Contact",

  "Id": "CN=Aaron Goodman/O=renovations",

  "FirstName": "Aaron",

  "LastName": "Goodman",

  "City": "Philadelphia",

  "State": "PA",

  "EMail": "aaron_goodman@renovations.com"

}

As with the view entries response, the document response includes both system properties (beginning with @) and application-specific properties (Id, FirstName, etc.).  But in this case, it's the application-specific properties that often increase the size of the response.  By default the data API includes a JSON property for each item on the document.  This can be a performance problem when:

• the document contains a large number of items
• one or more items is a large list (e.g. a text list)
• the document contains a large rich text item
• a rich text item contains a large attachment

The good news is you can control the amount of application-specific data returned with two query parameters -- fields and attachmentlinks.

The fields parameter is new as of Domino 9.0.1 FP8.  The value of the parameter is a comma separated list of field names.  For example, the following request uses the fields property:

GET /XPagesExt.nsf/api/data/documents/unid/39A95ECD82D0DEB485257AB5005BFEE0?fields=id,email

The response is a document object with fewer application-specific properties -- in this case just Id and EMail:

{

  "@href": "/XPagesExt.nsf/api/data/documents/unid/39A95ECD82D0DEB485257AB5005BFEE0",

  "@unid": "39A95ECD82D0DEB485257AB5005BFEE0",

  "@noteid": "5D26",

  "@created": "2012-11-13T16:44:50Z",

  "@modified": "2013-07-16T21:39:28Z",

  "@authors": "CN=Duke Lawson/O=Peaks",

  "@form": "Contact",

  "Id": "CN=Aaron Goodman/O=renovations",

  "EMail": "aaron_goodman@renovations.com"

}

In this example, we are only filtering out a small set of fields from the default response -- FirstName, LastName, City and State.  While the example is somewhat trivial, the principle can be applied to documents with large items or a large number of items.

The attachmentlinks parameter is new as of Domino 9.0.1 FP9.  This is a boolean parameter that is useful when requesting a document containing a rich text item which may include a large attachment.  By default the data API includes all the attachment data with the rich text item.  For example, the following is an example of a rich text item in JSON format:

  "Body": {

    "type": "multipart",

    "content": [

      {

        "contentType": "multipart/mixed; Boundary=\"0__=0ABB0BFADFC919E78f9e8a93df938690918c0ABB0BFADFC919E7\"",

        "data": ""

      },

      {

        "contentType": "text/html; charset=US-ASCII",

        "contentDisposition": "inline",

        "data": "<html><body><font size=\"2\" face=\"sans-serif\">test</font><br>\r\n<div id=\"MIMEAttachInfoDiv\" style=\"display:none\" title=\"|avatar.png\"></div><br>\r\n<i>(See attached file: avatar.png)</i></body></html>\r\n",

        "boundary": "--0__=0ABB0BFADFC919E78f9e8a93df938690918c0ABB0BFADFC919E7"

      },

      {

        "contentType": "image/png; name=\"avatar.png\"",

        "contentID": "<1__=0ABB0BFADFC919E78f9e8a93df93869091@local>",

        "contentDisposition": "attachment; filename=\"avatar.png\"",

        "contentTransferEncoding": "base64",

        "data": "iVBORw0KGgoAAAANSUhEUgAAAIAAAACACAIAAABMXPacAAAACXBIWXMAAAsSAAALEgHS3X78AAAf ...",

        "boundary": "--0__=0ABB0BFADFC919E78f9e8a93df938690918c0ABB0BFADFC919E7"

      }

    ]

  }

Notice the third part in the content array.  This is an image part containing a base64 encoding of the image data.  For this example, we have truncated the image data with an ellipsis ("...").  However, the data can obviously be quite large.  The following request reduces the size of the response:

GET /XPagesExt.nsf/api/data/documents/unid/39A95ECD82D0DEB485257AB5005BFEE0?attachmentlinks=true

Now the third part should look more like this:

      {

        "contentType": "image/png; name=\"avatar.png\"",

        "contentID": "<1__=0ABB0BFADFC919E78f9e8a93df93869091@local>",

        "contentDisposition": "attachment; filename=\"avatar.png\"",

        "contentLocation": "/XPagesExt.nsf/api/data/documents/unid/3E46BB507C145A8D8525816F005665EA/Body/avatar.png",

        "boundary": "--0__=0ABB0BFADFC919E78f9e8a93df938690918c0ABB0BFADFC919E7"

      }

In other words, the data property is replaced with a contentLocation property -- the value of which is the relative URL of a separate attachment resource.  The response is much smaller and you can still retrieve the attachment data with a subsequent request (if necessary).

NOTE:  As mentioned, the fields property is new to Domino 9.0.1 FP8  and the attachmentlinks property is new to 9.0.1 FP9.  Therefore, these parameters may not yet be included in the official document resource documentation.  However, they are included in the DAS OpenAPI specifications repository on GitHub.  See Swagger UI on Domino for information on generating your own documentation from the latest OpenAPI specs.

Because the Domino data API cannot infer what's important to your application, it sometimes returns more data than your application needs.  This can increase the time it takes for any single request to complete.  It can also impact overall server performance under high load.  Fortunately, you can boost performance by requesting just the data you need.

In summary, use the following parameters as necessary:

• compact=true eliminates white space in all responses to GET requests
• when reading a view, the systemcolumns parameter selects just the system properties your application needs 
• when reading a document, the fields and attachmentlinks parameters can dramatically reduce the size of the response