Introduction

The Domino calendar service is a REST API first introduced in the 9.0.1 release. The API is accessible from virtually any browser, device, programming language or OS capable of making an HTTP request. It allows an application to create, read, update and delete entries on a user's calendar. It also provides simple commands for processing calendar notices (e.g. Accept, Decline, Delegate, and so on).

This article describes performance tests IBM designed to stress the API and discover potential bottlenecks. It also summarizes the performance test results. For more information on the API itself, see the Domino calendar service reference.

Test Scenario Description

Register 1000/2000/3000 users with Domino Administrator. For every user, Create one meeting with 6 attendees within a two week period. This means there are 6 invitations and 7 calendar events in the calendar of every user. Then accept 4 invitations for every user. This means there are now 2 invitations and 7 events (including ghost events) in the calendar of every user. NOTE: In this article, the word “event” just means a calendar entry(a meeting, an appointment, etc.). A ghost event is a meeting which has not yet been accepted.

In this article, we only test the REST calendar service of the Domino. We design a test scenario as below. Every cycle spends about 12 minutes.

Activity	Range of Wait (s – How long to send this request after get the previous response)
Step1 - Log in	5 ~ 15
Step2 - Read all invitations and events(two requests)	5 ~ 15
Step3 - Schedule a single meeting to 6 people in one day of two weeks	60 ~ 180
Step4 - Read all events in this month	30 ~ 90
Step5 - Read all invitations	15 ~ 45
Step6 - Open one invitation	15 ~ 45
Step7 - Accept the invitation	5 ~ 15
Step8 - Open one invitation	15 ~ 45
Step9 - Accept the invitation	5 ~ 15
Step10 - Open one invitation	15 ~ 45
Step11 - Decline the invitation	5 ~ 15
Step12 - Open one invitation	15 ~ 45
Step13 - Delegate the invitation	10 ~ 30
Step14 - Open one invitation	15 ~ 45
Step15 - Accept the invitation	5 ~ 15
Step16 - Open one invitation	15 ~ 45
Step17 - Accept the invitation	5 ~ 15
Step18 - Read all events in this month	30 ~ 90
Step19 - Open one event	15 ~ 45
Step20 - Delegate the event	10 ~ 30
Repeat	0

Test method

1) Start with the IBM Domino 9.0.1 gold build.

2) Execute the test scenario against the Domino with IBM Rational Performance Tester 8.2

3) Load 1000 virtual users in 12 minutes, hold this load level for 4 cycles, then finish the test loading 1000 virtual users, clean up all the data in every user's mail database.

4) Load 2000 virtual users in 12 minutes, hold this load level for 4 cycles, then finish the test loading 2000 virtual users, clean up all the data in every user's mail database.

5) Load 3000 virtual users in 12 minutes, hold this load level for 4 cycles, then finish the test loading 3000 virtual users, clean up all the data in every user's mail database.

Server Configuration

The test platform is Windows, the configuration of the server is as below:

Model	Intel 64 bit platform(virtual machine)
Processor for test/speed	2 processors * 4 cores at 2.77 GHZ
Memory	16G installed
Disk	1Tb disk - SCSI
Operator system	Windows 2008 Enterprise x64 SP2

The configuration of Domino is as below:

Domino version	9.0.1
Calendar service	Enable
Log level	Min
HTTP Number active threads	80
Mail database size per user	30 M
Other configurations	Default

Test results

# of Users	Transactions	Average TPS	TPS
1000 users	94500	29.7	28.9
2000 users	189000	58.6	57.6
3000 users	283500	86.2	86.1

Average response time

# of Users	Average response time
1000	52.5 ms
2000	172.8 ms
3000	305.8 ms

Server performance

	1000 users	2000 users	3000 users
Testing elapse time	53m57s	54m41s	54m00s
%CPU Utilization	8.15%	14.40%	21.60%
Disk transfers / sec	312.6	528.3	820.5
Disk total (K Bytes) /sec	4934.5 K	7614.3 K	11974.5 K
Disk split IO/sec	2.24	6.08	12.6
Disk queue length	0.59	0.8	1.61
Disk busy time	15.00%	25.90%	37.80%
Memory %committed In use	11.00%	12.10%	13.10%
Available M Bytes	12621.6 M	11606.9 M	10834.6 M
Network K Bytes total / sec	86.2 K	166.9 K	254.5 K

Highest average response times for 1000, 2000 & 3000 users

request	1000 users (ms)	2000 users (ms)	3000 users (ms)
Step2 - Read all invitations and events	96.2	244.2	406.6
Step3 - Schedule a single meeting to 6 people in one day of two weeks	70.5	220.3	379.3
Step4 - Read all events in this month	46.9	195.1	354.0
Step5 - Read all invitations	144.7	293.8	454.7
Step10 - Open one invitation	46.9	184	334.1
Step13 - Delegate the invitation	103.2	184.3	336.2
Step19 - Open one event	51.4	254.4	419.0
Step20 - Delegate the event	90.5	223.2	367.0

Conclusion

(1) For a load of 1000 virtual users, we found the bottleneck is the read invitations request. The average response time for reading invitations is 144.7 ms (the minimum is 31 ms and the maximum is 702 ms). However, for a load of 2000 or 3000 virtual users, the 8 highest average response times are similar. So if there are many invitations to read, we recommend two ways to improve performance. First, read invitations by paging with the receivedsince and since parameters. Second, implement a client side cache mechanism to minimize the number of read invitations requests.

(2) If there are many users to request the REST calendar service, It will result in better performance to increase the value of http number active threads. Refer to this web page for guidance on tuning the http number active thread (http://www-01.ibm.com/support/docview.wss?uid=swg21173877).

(3) For disk utilization, the change of disk parameters is stable. Use disk busy time as an example,when load 1000 users, it is 15.00%; when load 2000 users, it is 25.90%; when load 3000 users, it is 37.80%.

About the Author

Wei Jie Lu is a member of the IBM Domino team with a focus on Domino REST services. You can reach Wei Jie Lu at weijielu@cn.ibm.com.