ShowTable of Contents
Introduction
With additional IDVault support for Citrix® XenApp
TM, IBM® is committed to continuing to improve the experience of running IBM Notes® 9.0, IBM iNotes® 9.0 or IBM Notes Browser Plugins 9.0 in Citrix XenApp.
The new support for Microsoft® Windows® 2008 R2 with IBM Notes 9.0, together with additional features such as IDVault do not cost memory, which means you can leverage the new features without adding to your hardware budget.
In this article we present the testing environment, tuning parameters, and the workflows that were executed to aquire the capacity data on IBM Notes 9.0, IBM iNotes 9.0 and IBM Notes Browser Plugins across various hardware configurations and browser versions.
IBM Notes 9.0
IBM Notes and IBM Domino® 9.0 include a vast array of end-user feature enhancements to increase personal productivity of IBM Notes, IBM iNotes, IBM Notes Traveler, and IBM Notes Browser Plugins for developers and users
of IBM Domino Designer. Here are some highlights:
Significant usability enhancements to support social businesses and help users to get work done quickly. The enhancements include:
◦ Mail
▪ See messages in your Inbox grouped by date.
▪ Abbreviated dates in Inbox view.
▪ Message snippets available in Inbox view.
▪ New action bar button for Read/Unread mail.
▪ Image preview for attachments.
◦ Calendar
▪ Scroll through One Month view in Calendar.
▪ New Weekly Planner view.
▪ Color-code calendar entries by category.
▪ Check Calendar dialog box remains in view.
▪ Calendar and Scheduling APIs for C SDK.
◦ OpenSocial embedded experiences allow you to access and work with embedded content from applications inside OpenSocial containers, like a gadget or a simple web page, directly in your Notes email.
◦ Embedded Sametime® version 8.5.2 IFR1 installed with Notes. The embedded version of Sametime installed with Notes has been upgraded to Sametime 8.5.2 IFR1.
◦ Widgets and Live Text. Call phone numbers and start chats through Live Text. All users with Live Text enabled will see new Live Text actions upon upgrading to Notes 9.0 Social Edition, if they also use embedded Sametime. The new embedded Sametime version provides Live Text actions to call phone numbers and start chats with people's name that Live Text identifies. If users did not previously have Live Text actions (or widgets) wired to the "person" or "phone number" data type, the Sametime actions will cause Live Text to start underlining this content after upgrading.
◦ Search
▪ Improved search experience with Quick Find.
▪ Search results more easily sorted by selected view sort order.
▪ Search Mail and Archives at the same time.
◦ Other new features
▪ Discover page. The Notes 9.0 Social Edition client introduces a new home page called the Discover page. With the Discover page, users can find targeted Notes client information more quickly and easily, including new features in the release, introductory material for new users, and helpful hints and tips. There is also a Quick Links tab that allows users to launch their workspace, Mail, Calendar, and other Notes applications they have recently used, in addition to providing links to other resources, such as the Notes and Domino wiki.
▪ Support for IBM Social theme. This theme provides a new, cleaner, more modern look to the Notes client interface, and is being adopted across the IBM product line for UI consistency.
▪ Some default preferences have changed.
▪ Open Mail, Calendar, and other views with one click.
▪ Use the mini-view to see work at a glance.
▪ New toolbar option.
▪ New keyboard shortcuts.
• IBM Domino 9.0 has the option of running the IBM HTTP Server (IHS) on the same computer as a Domino HTTP server. The purpose of this enhancement is to support the Transport Layer Security (TLS) protocol.
• Domino Designer 9.0 continues to deliver innovation for developers with enhancements focused on performance, capacity, and enhanced user experience.
• IBM Notes Traveler support for BlackBerry 10 devices, Windows Phone devices, Windows RT tablet, Windows 8 Pro tablet, allows you to expand the reach of the business to more devices. IBM Notes Traveler now supports additional operating system platforms, including: IBM i (V6R1/V7R1), Windows Server 2012, Redhat 6 64-bit mode, SUSE 11 64-bit mode.
• IBM® iNotes® 9.0 Social Edition delivered a number of new features and functionality in the IBM Domino® 9.0 Social Edition release.
◦ Configuring Domino Social Edition Open Social component for iNotes clients.
◦ Using a credential store to share credentials
◦ Configuring widgets for specific IBM Notes 9.0 client releases
◦ iNotes integration with IBM Docs
◦ iNotes integration with IBM Connections files
◦ Return receipt generation control
• IBM Notes Browser Plug-in allows you to work with your IBM Notes applications directly in a browser. The plug-in consists of components of the Notes client that allow you to work with your Notes applications, as well as components that allow integration inside a browser. The components from the Notes Client which are part of this plug-in are specific to the Notes Basic client; any features in an application that require Standard Client components will not be available when running that application under the Notes Browser Plug-in. Essentially, if your Notes application runs successfully in the Notes Basic client, it should run unmodified in the Notes Browser Plug-in.
Citrix XenApp Server and Client
Citrix XenApp 6.5 now delivers an even better user experience for on-demand application delivery, featuring turbocharged app launches and a seamless multimedia experience over any network. With Citrix Receiver providing access from over one billion devices and enhanced management features that simplify deployment and support, it's easy to see why XenApp is the de facto standard in on-demand application delivery. Now XenApp 6.5 makes it even easier for IT to control costs by centralizing applications and delivering them securely to any user, anywhere.
• Fast, mobile access to applications
◦ Instant App Access. Three new Instant App Access features dramatically reduce the time it takes for users to launch applications delivered by XenApp. Session Pre-Launch, Session Linger and Fast Reconnect shorten app launch time by initializing a XenApp ICA session immediately upon user login and keeping that session alive for a time period determined by the IT admin.
◦ Any device, anywhere with Citrix Receiver. XenApp 6.5 takes advantage of the latest Citrix Receiver enhancements for extreme multitasking, faster Windows app performance and advanced Linux device support. Users have ultimate mobility with access to corporate applications from almost any smartphone or tablet—over one billion devices in all.
◦ Native Device Experience. XenApp 6.5 delivers hosted shared desktops that are automatically re-skinned for touch-friendly use on mobile devices and hosted applications that enable features such as auto-keyboard popup. With the newly available software developer kit, developers can customize Windows applications to take advantage of unique mobile device capabilities such as camera and GPS.
• Seamless multimedia experience with HDX
◦ HDX MediaStream for Flash. Adobe Flash content can be rendered locally in many more cases than before, resulting in even higher server capacity and a great user experience even at up to 300 ms round-trip latency.
◦ Multi-stream ICA. IT administrators now have the option of delivering HDX ICA traffic over up to four TCP/IP streams. This enables full flexibility for Quality of Service (QoS) routing and provides superior audio-visual quality over the WAN without disrupting other HTTP traffic.
◦ Enhanced desktop experience. IT can take advantage of the ease of management and cost effectiveness of publishing a hosted shared desktop delivered by XenApp on Windows 2008 R2, while users enjoy the look and feel of an updated Window 7 desktop.
• Simplified deployment and management
◦ Desktop Director. Now IT help desk admins can monitor, troubleshoot and provide assistance with virtual desktops and apps for 1,000 users as easily as for one. The same Desktop Director console can be used to assist users with XenDesktop delivered desktops as well as XenApp applications and server-hosted desktops.
◦ Dynamic datacenter provisioning. XenApp 6.5 deployments can be scaled up quickly by creating "controller" or "worker" roles in a XenApp server farm. Because workers need to sync much less data, fewer database transactions are required and the process of joining a large number of servers to a XenApp farm is faster and easier.
Rational Performance Tester
Rational Performance Tester v8.3 (RPT) with the Citrix extension was used for the testing
capacity owing to its benefits over AutoIT, which was used for testing previous releases. The benefits of RPT Citrix Extensions include easy-to-use wizard-based recording, with automated reporting and integrated resource monitoring.
RPT captures ICA protocol traffic to emulate the end-user experience and records response times through its ability to synchronize on window events. It not only can perform image synchronizations, but it can also carry out optical character recognition on text displayed. Use of this tool enabled us to move from a qualitative to a quantitative measurement of user experience.
Test strategy
This sections describes our test strategy.
Ensuring continuity of results
For capacity testing we need to determine what we measure, how we measure it, and how we analyze the data and report on it.
We measure the changes in memory with load, and use that to determine the optimum and maximum number of users a particular system can handle.
Capacity testing with response times
Response times are a measure of the time taken to process a transaction from start to finish, that is, the time taken for a message to be sent and a response received from a server (for example, the time taken for a new window to be created when a button is pressed).
As load increases, the time taken to process a transaction increases. Ideally a set of acceptance criteria for a response should be determined for each transaction, to easily determine pass or fail conditions, for example, as shown in figure 1.
Figure 1. Example acceptance criteria for response time
Script considerations
In scripting for these tests we took into account the following things:
• Think times should be minimal, to maximize throughput per user. We are looking to see how many transactions we can put through the system.
• The choice of transactions is restricted to the most-popular transactions.
• Verification points and synchronizations should be implemented and stable.
• The workload model should be ramping, giving sufficient time in each stage so that a good set of metrics are gathered for each load level.
Test methodology
This section contains information on the methodology employed by the test team, to measure the capacity of the systems under test so as to produce consistent, reproducible results.
Test planning
Capacity planning involves the following:
• Analyzing the application to determine hardware and software components, system configuration, and typical usage patterns.
• Defining testing objectives (for example, maximum user load, application upgrade compatibility, and identifying bottlenecks).
• Defining the launch sequence of the test users.
Users are added gradually until the system reaches critical thresholds or when the scripts fail to respond.
User profile
The Virtual Users (VUs) emulate production users interacting with the system under test conditions. The VUs scripts contain the actions that each virtual user will perform during a scenario execution. For this test, the scripts were based on real-world usage. These scripts directed VUs to perform common functional activities using IBM Notes, and iNotes. The VUs launched an Independent Computing Architecture (ICA) session to the XenApp server and opened the IBM Notes or iNotes on browser. A standard user profile is employed by test teams throughout IBM.
Standard Notes Client
•
Read mail from inbox. The user clicks on the inbox, presses the down arrow for a random number of times, and then presses Return to view the message. The user then pauses for 15 seconds and presses Escape to close the message.
•
Send new message. The user clicks the New Message button from the inbox, types in a random recipient from the inbox, tabs to the Subject line and types a subject, and then tabs again and types 1KB of text before clicking the Send button.
•
Send new message with attachment. The user clicks the New Message button from the inbox, types in a random recipient from the inbox, tabs to the Subject line and types a subject, and then tabs again and types 1KB of text. Afterward, attach a text file (less than 100KB) then clicking the Send button
•
Reply to a message. The user clicks on the inbox, then presses the down arrow for a random number of times and then clicks the Reply button. The user then types 1KB of text as a reply and clicks Send.
•
Delete a message. The user clicks on the inbox, presses the down arrow for a random number of times, and then clicks the Trash button. The user then opens the Trash folder, clicks Empty Trash, and confirms the delete before returning to the inbox folder.
•
Standard Notes Client with Embedded Experience
•
Read mail from inbox. The user clicks on the inbox, presses the down arrow for a random number of times, and then presses Return to view the message. The user then pauses for 15 seconds and presses Escape to close the message.
•
Send new message. The user clicks the New Message button from the inbox, types in a random recipient from the inbox, tabs to the Subject line and types a subject, and then tabs again and types 1KB of text before clicking the Send button.
•
Send new message with attachment. The user clicks the New Message button from the inbox, types in a random recipient from the inbox, tabs to the Subject line and types a subject, and then tabs again and types 1KB of text. Afterward, attach a text file (less than 100KB) then clicking the Send button
•
Reply to a message. The user clicks on the inbox, then presses the down arrow for a random number of times and then clicks the Reply button. The user then types 1KB of text as a reply and clicks Send.
•
Delete a message. The user clicks on the inbox, presses the down arrow for a random number of times, and then clicks the Trash button. The user next opens the Trash folder, clicks Empty Trash, and confirms the delete before returning to the inbox folder.
•
Send MIME mail. The user clicks Open button. Then Click "Lotus Labs Tests" button. When the new panel is opened, double click the "Compose MIME email". First, the user need to change type to "application/embed+json", then add following text:
{
"gadget" : "http://<shindig server name>/EE_Context_Displayer.xml",
"context" : "EE Context Displayer!"
}
EE_Context_Displayer is a gadget which can display simple text. Then the user need to change type to "text/html". Add text e.g.
TESTING . Finally, click Send MIME email and OK to confirm send
•
Open The Connections Activity Stream Gadget. The user clicks the "Open Updates" button on the left top corner. Then navigate to the "I'm following". Click an activity entry field and then press "Esc". Repeat 3-5 times.
iNotes on Firefox and IE
•
Read mail from inbox. The user clicks on the inbox, presses the down arrow for a random number of times, and then presses Return to view the message. The user then pauses for 15 seconds and then close the message.
•
Read MIME mail from inbox. The user clicks on the inbox, look for a MIME mail, and then presses Return to view the message. The user then pauses for 15 seconds and then close the message.
•
Send new message. The user clicks the New Message button from the inbox, types in a random recipient from the inbox, tabs to the Subject line and types a subject, and then tabs again and types 1KB of text before clicking the Send button.
•
Reply to a message. The user clicks on the inbox, then presses the down arrow for a random number of times and then clicks the Reply button. The user then types 1KB of text as a reply and clicks Send.
•
Delete a message. The user clicks on the inbox, presses the down arrow for a random number of times, and then clicks the Trash button. The user then opens the Trash folder, clicks Empty Trash, and confirms the delete before returning to the inbox folder.
•
Notes Plugin on IE
•
Read a blog post. Left click on Blog Icon and press "Return" to open blog application. Arrow down a random number of times (1-10). Press "Return" to open a blog post. Press "Escape" to close the post after 15 sec. Press "Escape" to close blog application.
•
Create new blog post. Left click on Blog Icon and press "Return" to open blog application. Left click on New Post Icon. Left click on Subject Field and type in the Subject. Left click on Body Field and type in the Body text. Left click on Save and Post button. Press "Escape" to close blog application.
•
Edit a Blog post. Left click on Blog Icon and press "Return" to open blog application. Arrow down a random number of times (1-10). Press "Return" to open a blog post. Left click on Edit Icon. Left click on Body Field and type in the Body text. Left click on Save and Post button. Press "Escape" to close blog application.
•
Delete a Blog post. Left click on Blog Icon and press "Return" to open blog application. Arrow down a random number of times (1-10). Press "Delete" key (to delete a post). Press "Escape" to close blog application. Press "Return" to confirm deleting.
•
Create new Discussion topic. Left click on Discussion Icon and press "Return" to open Discussion application. Left click on New Topic Icon. Left click on Subject Field and type in the Subject. Left click on Body Field and type in the Body text. Left click on Save and Post button. Press "Escape" to exit Discussion application.
•
Edit a Discussion topic. Left click on Discussion Icon and press "Return" to open Discussion application. Arrow down a random number of times (1-10). Press "Return" to open a Discussion topic. Left click on Edit Icon. Left click on Body Field and type in the Body text. Left click on Save and Post button. Press "Escape" to exit Discussion application.
•
Read a Discussion topic. Left click on Discussion Icon and press "Return" to open Discussion application. Arrow down a random number of times (1-10). Press "Return" to open a Discussion topic. Press "Escape" to close the Discussion topic after 15 sec. Press "Escape" to exit Discussion application.
•
Delete a Discussion topic. Left click on Discussion Icon and press "Return" to open Discussion application. Arrow down a random number of times (1-10). Press "Delete" key to delete a post. Press "Escape" to exit Discussion application. Press "Return" to confirm deleting.
•
Create new Library Document. Left click on Library Icon and press "Return" to open Library application. Left click on New Document Icon. Left click on Subject Field and type in the Subject. Left click on Body Field and type in the Body text. Left click on Save and Post button. Press "Escape" to exit Library application.
•
Edit a Library Document. Left click on Library Icon and press "Return" to open Library application. Arrow down a random number of times (1-10). Press "Return" to open a Library Document. Left click on Edit Icon. Left click on Body Field and type in the Body text. Left click on Save and Post button. Press "Escape" to exit Library application.
•
Read a Library Document. Left click on Library Icon and press "Return" to open Library application. Arrow down a random number of times (1-10). Press "Return" to open a Library Document. Press "Escape" to close the Library Document after 15 sec. Press "Escape" to exit Library application.
•
Delete a Library Document. Left click on Library Icon and press "Return" to open Library application. Arrow down a random number of times (1-10). Press "Delete" key to delete a post. Press "Escape" to exit Library application. Press "Return" to confirm deleting.
Metrics and monitoring
Table 1 lists the recommended performance objects and counters along with their descriptions.
Table 1. Performance objects and counters
Counter | Description |
Memory: Available bytes | Amount of physical memory available to processes, measured in megabytes. Paging should be monitored if less than 25% of physical memory is available, as excessive paging may occur. |
Paging file: % Usage | The percentage of page file in use. If greater than 75% of the page file is in use, physical memory (RAM) should be increased. |
Processor: % Processor Time | Percentage of elapsed time a CPU is busy executing a non-idle thread. A high value is a concern, only if accompanied by a Processor Queue Length sum greater than <2 x # of CPU’s> or growing with % Processor Time greater than 80-90%. |
Terminal Services: Active Sessions | Number of active Terminal Server sessions. |
Defining acceptable response time is a difficult task that depends on the user and the task that user is trying to accomplish. For example, if a user is performing a search function, then it's likely the user will find it acceptable to wait for a longer time than when the user is requesting a new document.
Response time measurement is a measure of application responsiveness and is graded as shown in table 2.
Table 2. Application responsiveness
Application Responsiveness | Response time range | Description |
GOOD | 0-3 seconds | No contention for resources, response user workflow unaffected. |
POOR | > 3 seconds | Response times unacceptable, content reaches a level to which usability has suffered. |
Environment optimization
XenApp tuning
• Download the
XA6/XA65 Server Tuning Policy x64. Unzip the package and follow the guide inside of it to apply.
System tuning
• Page file was set to 1.5 times installed RAM
• Disable the Windows Search process from running
IBM Notes optimization
• Disable discover page. Open Notes client, you should see the "Discover" page is opened. Close it and shutdown Notes. Then modify the notes.ini. Change the $DISCOVER_HOMEPAGE value from 2 to 1. Finally, open the Notes again. You should see the "Home" page now.
Browser tuning
• Disable pop-up blocker.
• Disable auto-update
• Disable auto-complete
• Clean all history and cookies after exit
Systems under test
Now let's look at our systems under test.
Test environment
Figure 2 illustrates the test environment infrastructure.
Figure 2. Test environment
XenApp 6.5 Servers. (allowing multi-user access to published resources.) For this
test, IBM Notes, iNotes and Notes Browser Plugin were published as applications on the Citrix server, and each virtual user logged in and launched this published application. Four servers were used to host the Citrix XenApp server, each with varying hardware configurations.
Citrix License Server. This server is responsible for maintaining and providing
concurrent user licenses to ICA sessions.
SQL Server: It serves the data store that stores the static information for all Citrix XenApp servers in the farm.
File Store Server: It stores data directory for all users.
Active Directory Server: Responsible for user registrations and controlling rights.
Domino 9.0 Mail Server: Stores users' mail files that are accessed via the IBM Notes client.
Domino 9.0 Shindig Server: Hosts widget catalog database and renders open social gadget.
Hardware specifications
Table 3 list the hardware specifications for the XenApp servers and the other four servers, respectively.
Table 3. XenApp server specs
Facet | XenApp 6.5 Server | XenApp 6.5 Server |
Hardware System Type | Hyper-V | Hyper-V |
Number of Processors | 2 Dual Core – 4 processors | 2 Dual Core – 4 processors |
Memory (GB) | 8 GB | 16 GB |
Page file Size | 12 GB | 24 GB |
Disk Speed | 10000 RPM | 10000 RPM |
Operating System | Windows Server 2008 R2 Enterprise 64 bit | Windows Server 2008 R2 Enterprise 64 bit |
Browsers specifications
Table 3 list the hardware specifications for the XenApp servers and the other four servers, respectively.
Internet Explorer 9
Internet Explorer 9 provides the best web experience for Windows. Here are the top reasons why you should get it today.
• It's all-around fast — hardware-accelerated text, video, and graphics speed up performance and make websites perform like programs that are installed on your computer.
• Streamlined and simplified design — gives you the basic controls you need and puts the focus on your websites.
• One-click access to your favorite sites — with Pinned Sites, you can get to your favorite sites directly from the Windows taskbar without having to open your browser first.
• A combined search and Address bar — one place to either navigate to a website or start a search.
• Redesigned New Tab page — now you have quick access to the sites you visit most often, and you can also reopen closed tabs or your last browsing session, or start an InPrivate Browsing session.
• Integration with Windows 7 — with Snap, Pinned Sites, Jump Lists, and thumbnail preview controls, Internet Explorer 9 and Windows 7 bring you the best web experience for Windows.
• Improved add-on management — Add-on Performance Advisor tells you if an add-on is slowing down your browser and allows you to disable or remove it, helping ensure that your browser stays fast over time.
• Helps control your privacy — Tracking Protection enables you to limit the browser's communication with certain websites to help keep your information private.
• Keeps your downloads in order — Download Manager keeps a running list of the files you download from the Internet, notifies you if a file could be malicious, and allows you to pause and restart a
Firefox 19
Firefox 19 has involved a list of improvements:
• Built-in PDF viewer
• Canvas elements can export their content as an image blob using canvas.toBlob()
• Startup performance improvements
• Experience better image quality with our new HTML scaling algorithm
• Debugger now supports pausing on exceptions and hiding non-enumerable properties
• Remote Web Console is available for connecting to Firefox on Android or Firefox OS
• There is now a Browser Debugger available for add-on and browser developersWeb Console CSS links now open in the Style Editor
• CSS @page is now supported
• CSS viewport-percentage length units implemented (vh, vw, vmin and vmax)
• CSS text-transform now supports full-width
Capacity results and analyses
This section presents the results for each system under test for IBM Notes Standard Client 9.0, IBM Notes Browser Plugin and IBM iNotes 9.0 accessed through both Mozilla Firefox and Internet Explorer.
IBM Notes Standard Client on XenApp 6.5 on 64-bit Win2008 SP2 Enterprise Server
This subsection reports the results from the tests of the capacity analysis performed against XenApp 6.5 servers. Table 4 shows the maximum user sessions that were achieved before a bottleneck was identified on the system.
Table 4. Summary of the capacity achieved over the two hardware platforms
IBM Notes 9.0 Standard Client | Recommend Sessions | Limiting Factor |
Windows 2008 R2 Enterprise 64 bit 2 Dual Core 8 GB RAM | 50 - 55 | Memory |
Windows 2008 R2 Enterprise 64 bit 2 Dual Core 16 GB RAM | 60 - 70 | CPU |
IBM Notes Standard Client on 8GB RAM platform
Table 5 shows the results for this test.
Table 5. Results of XenApp 6.5 on 64-bit Win2008 SP2 Enterprise Server, 8GB RAM
Users | Available
Memory
(MB) | Paging File
Usage
(%) | Processor
Time
(%) | User
Experience |
0 | 6990 | 0 | 0 | GOOD |
10 | 5011 | 0 | 8 | GOOD |
20 | 2771 | 7 | 19 | GOOD |
30 | 1733 | 11 | 25 | GOOD |
40 | 427 | 19 | 35 | GOOD |
50 | 489 | 42 | 39 | GOOD |
55 | 286 | 49 | 43 | GOOD |
60 | 423 | 58 | 47 | POOR |
The Citrix Server is not able to add new users when the max sessions reach 60.
In this test, MEMORY was the bottleneck when this capacity test was performed (see figure 3).
Figure 3. Available memory vs. user load
As load was increased, memory was consumed to a point at which the paging file was increasingly used (see figure 4). At 50 users the memory was exhausted and paging file increase to over 40% full. at 60 users, the performance degrade and reached an unacceptable level.
Figure 4. Paging file usage vs. user load
In figure 5 we can see that processor time increases steadily with load going. At 60 users, the processor was being used more than 45% of the time.
Figure 5. Processor time vs. user load
From the figure we can see that, once the system is beyond 50 users, contention increases, while any more workload past 60 users would make the system unusable with respect to the response time criteria set earlier.
IBM Notes Standard Client on 16GB RAM platform
Table 6 shows the results for this test.
Table 6. Results of XenApp 6.5 on 64-bit Win2008 SP2 Enterprise Server, 16GB RAM
Users | Available
Memory
(MB) | Paging File
Usage
(%) | Processor
Time
(%) | User
Experience |
0 | 14688 | 0 | 0 | GOOD |
10 | 12616 | 0 | 11 | GOOD |
20 | 10514 | 1 | 26 | GOOD |
30 | 8779 | 2 | 34 | GOOD |
40 | 7164 | 6 | 46 | GOOD |
50 | 5601 | 10 | 58 | GOOD |
60 | 4096 | 13 | 67 | GOOD |
70 | 2846 | 16 | 72 | GOOD |
80 | 1178 | 19 | 78 | POOR |
Above chart shows that the performance degrade when load reach to 80 users. An acceptable performance level would be between 60 and 70 users.
In this test, CPU was the bottleneck when this capacity test was performed (see figure 4). In this test the amount of processor usage increased steadily as user sessions increased on the Citrix XenApp Server.
Figure 6. Processor time vs. user load
Though much of the available memory was consumed in the test, there was still about 1GB of available physical memory and over 80% of paging file remaining before response times began to degrade.
Figure 7. Available memory vs. user load
Figure 8. Paging file usage vs. user load
IBM Notes Standard Client with Embedded Experience on XenApp 6.5 on 64-bit Win2008 SP2 Enterprise Server
IBM Notes Standard Client with Embedded Experience on 8GB RAM platform
Table 7 shows the results for this test.
Table 7. Results of XenApp 6.5 on 64-bit Win2008 SP2 Enterprise Server, 8GB RAM
Users | Available
Memory
(MB) | Paging File
Usage
(%) | Processor
Time
(%) | User
Experience |
0 | 6885 | 0 | 2 | GOOD |
10 | 4240 | 0 | 16 | GOOD |
20 | 1897 | 12 | 32 | GOOD |
30 | 448 | 23 | 42 | GOOD |
35 | 210 | 37 | 45 | GOOD |
40 | 138 | 51 | 52 | GOOD |
45 | 118 | 64 | 58 | GOOD |
50 | 97 | 71 | 57 | POOR |
Above chart shows that the performance degrade when load reach to 50 users. An acceptable performance level would be between 40 and 45 users.
The response time keep stable and low as the load going until reach the bottleneck. In this test, memory was the bottleneck when this capacity test was performed (see figure 9). The paging file usage keep increasing as the user load increase.
Figure 9. Available memory vs. user load
Figure 10. Paging file usage vs. user load
Figure 11. Processor time vs. user load
IBM Notes Standard Client with Embedded Experience on 16GB RAM platform
Table 8 shows the results for this test.
Table 8. Results of XenApp 6.5 on 64-bit Win2008 SP2 Enterprise Server, 16GB RAM
Users | Available
Memory
(MB) | Paging File
Usage
(%) | Processor
Time
(%) | User
Experience |
0 | 14798 | 0 | 1 | GOOD |
10 | 12138 | 0 | 15 | GOOD |
20 | 9450 | 1 | 31 | GOOD |
30 | 7144 | 3 | 41 | GOOD |
40 | 4922 | 7 | 53 | GOOD |
50 | 2938 | 12 | 63 | GOOD |
60 | 1072 | 16 | 73 | GOOD |
70 | 565 | 31 | 83 | GOOD |
80 | 310 | 43 | 89 | GOOD |
90 | 198 | 57 | 93 | POOR |
Above chart shows that the performance degrade when load reach to 90 users. An acceptable performance level would be between 70 and 80 users.
In this test, CPU was the bottleneck when this capacity test was performed (see figure 12). In this test the amount of processor usage increased steadily as user sessions increased on the Citrix XenApp Server. Performance began to degrade as CPU consumption over 90%.
Figure 12. Processor time vs. user loadFigure 13. Available memory vs. user load
Figure 14. Paging file usage vs. user load
IBM iNotes with Internet Explorer on XenApp 6.5 on 64-bit Win2008 SP2 Enterprise Server
This subsection reports the results from the tests of IBM iNotes with Internet Explorer 9. Table 9 shows the capacity that can be achieved on IBM iNotes.
Table 9. Summary of the capacity achieved over the two hardware platforms
IBM iNotes 9.0 Internet Explorer | Recommend Sessions | Limiting Factor |
Windows 2008 R2 Enterprise 64 bit 2 Dual Core 8 GB RAM | 80 - 90 | Memory |
Windows 2008 R2 Enterprise 64 bit 2 Dual Core 16 GB RAM | 130 - 140 | CPU |
IBM iNotes with Internet Explorer on 8GB RAM platform
Table 10 shows the results for this test.
Table 10. Results of XenApp 6.5 on 64-bit Win2008 SP2 Enterprise Server, 8GB RAM
Users | Available
Memory
(MB) | Paging File
Usage
(%) | Processor
Time
(%) | User
Experience |
0 | 6483 | 0 | 0 | GOOD |
10 | 5706 | 0 | 4 | GOOD |
20 | 4379 | 0 | 12 | GOOD |
30 | 2916 | 0 | 14 | GOOD |
40 | 1360 | 0 | 18 | GOOD |
50 | 207 | 3 | 24 | GOOD |
60 | 130 | 18 | 31 | GOOD |
70 | 144 | 34 | 36 | GOOD |
80 | 94 | 49 | 42 | GOOD |
90 | 92 | 66 | 48 | GOOD |
100 | 88 | 81 | 53 | POOR |
In this test the MEMORY was the limiting factor. By the level at 100 users, the performance degrade. An acceptable performance level would be between 80 - 90
Once the memory was exhausted, as load increased, paging file usage increased. As a result, the CPU usage increase to a point at which performance degrade to unacceptable levels (see figure 15, figure 16 and figure 17).
Figure 15. Available memory vs. user load
Figure 16. Processor time vs. user load
Figure 17. Paging file usage vs. user load
IBM iNotes with Internet Explorer on 16GB RAM platform
Table 11 shows the results for this test.
Table 11. Results of XenApp 6.5 on 64-bit Win2008 SP2 Enterprise Server, 16 GB RAM
Users | Available
Memory
(MB) | Paging File
Usage
(%) | Processor
Time
(%) | User
Experience |
0 | 14771 | 0 | 2 | GOOD |
10 | 13652 | 0 | 5 | GOOD |
20 | 12376 | 0 | 12 | GOOD |
30 | 10903 | 0 | 15 | GOOD |
40 | 9382 | 0 | 18 | GOOD |
50 | 7691 | 0 | 23 | GOOD |
60 | 5867 | 0 | 30 | GOOD |
70 | 3941 | 0 | 37 | GOOD |
80 | 1931 | 0 | 42 | GOOD |
90 | 680 | 4 | 46 | GOOD |
100 | 614 | 16 | 53 | GOOD |
110 | 507 | 27 | 56 | GOOD |
120 | 336 | 38 | 64 | GOOD |
130 | 319 | 51 | 66 | GOOD |
140 | 242 | 64 | 75 | GOOD |
150 | 199 | 76 | 80 | POOR |
For the 16GB system with iNotes on Internet Explorer 9 we found that CPU contention was limiting. At 150 users the response times were unacceptable. The acceptable level should be 130 to 140 users.
Figure 18. Processor time vs. user load
Figure 19. Available memory vs. user load
Figure 20. Paging file usage vs. user load
IBM iNotes with Firefox on XenApp 6.5 on 64-bit Win2008 SP2 Enterprise Server
This subsection reports the results from the tests of IBM iNotes with Firefox 19. Table 12 shows the capacity that can be achieved on IBM iNotes.
Table 12. Summary of the capacity achieved over the two hardware platforms
IBM iNotes 9.0 Firefox | Recommend Sessions | Limiting Factor |
Windows 2008 R2 Enterprise 64 bit 2 Dual Core 8 GB RAM | 60 - 65 | Memory |
Windows 2008 R2 Enterprise 64 bit 2 Dual Core 16 GB RAM | 105 - 115 | CPU |
IBM iNotes with Firefox on 8GB RAM platform
Table 13 shows the results for this test.
Table 13. Results of XenApp 6.5 on 64-bit Win2008 SP2 Enterprise Server, 8GB RAM
Users | Available
Memory
(MB) | Paging File
Usage
(%) | Processor
Time
(%) | User
Experience |
0 | 6950 | 0 | 0 | GOOD |
10 | 5723 | 0 | 3 | GOOD |
20 | 3913 | 0 | 8 | GOOD |
30 | 2959 | 0 | 11 | GOOD |
40 | 1852 | 0 | 15 | GOOD |
50 | 417 | 6 | 24 | GOOD |
60 | 303 | 21 | 30 | GOOD |
65 | 219 | 30 | 33 | GOOD |
70 | 139 | 38 | 36 | POOR |
The response time keep stable and low as the load going until reach the bottleneck. When users pass 65 would make the system unusable with respect to the response time criteria set earlier (see figure 21).
Figure 21. Available memory vs. user load
At 50 users the memory was exhausted and paging file start to increase (see figure 22). At 70 users, the performance degrade and reached an unacceptable level.
Figure 22. Paging file usage vs. user load
In figure 23 we can see that processor time increases steadily with load going. At 70 users, the processor was being used more than 35% of the time.
Figure 23. Processor time vs. user load
IBM iNotes with Firefox on 16 GB RAM platform
Table 14 shows the results for this test.
Table 14. Results of XenApp 6.5 on 64-bit Win2008 SP2 Enterprise Server, 16 GB RAM
Users | Available
Memory
(MB) | Paging File
Usage
(%) | Processor
Time
(%) | User
Experience |
0 | 14369 | 0 | 1 | GOOD |
10 | 13563 | 0 | 3 | GOOD |
30 | 10948 | 0 | 15 | GOOD |
50 | 8132 | 0 | 22 | GOOD |
65 | 5584 | 0 | 32 | GOOD |
80 | 2806 | 0 | 40 | GOOD |
95 | 348 | 0 | 50 | GOOD |
105 | 465 | 17 | 57 | GOOD |
115 | 271 | 24 | 60 | GOOD |
120 | 268 | 31 | 66 | POOR |
In this test the processor was the limiting factor. By the level at 120 users, the performance was degrade. An acceptable performance level would be between 105 and 115 users.
In this test the amount of processor usage increased steadily as user sessions increased on the Citrix XenApp Server (see figure 24). Performance began to degrade as CPU consumption over 60%.
Figure 24. Processor time vs. user load
Though much of the available memory was consumed in the test, there was still over 65% of paging file remaining before response times began to degrade.
Figure 25. Available memory vs. user load
Figure 26. Paging file usage vs. user load
IBM Notes Browser Plugin with Internet Explorer on XenApp 6.5 on 64-bit Win2008 SP2 Enterprise Server
This subsection reports the results from the tests of IBM Notes Browser Plugin with Internet Explorer 9. Table 15 shows the capacity that can be achieved on IBM Notes Browser Plugin.
Table 15. Summary of the capacity achieved over the two hardware platforms
IBM Notes Browser plugin 9.0 IE | Recommend Sessions | Limiting Factor |
Windows 2008 R2 Enterprise 64 bit 2 Dual Core 8 GB RAM | 60 - 70 | Memory |
Windows 2008 R2 Enterprise 64 bit 2 Dual Core 16 GB RAM | 100- 110 | CPU |
IBM Notes Browser Plugin on 8GB RAM platform
Table 16 shows the results for this test.
Table 16. Results of XenApp 6.5 on 64-bit Win2008 SP2 Enterprise Server, 8GB RAM
Users | Available
Memory
(MB) | Paging File
Usage
(%) | Processor
Time
(%) | User
Experience |
0 | 6915 | 0 | 1 | GOOD |
10 | 5533 | 0 | 4 | GOOD |
20 | 4027 | 1 | 11 | GOOD |
30 | 2820 | 2 | 14 | GOOD |
40 | 1594 | 4 | 18 | GOOD |
50 | 382 | 7 | 23 | GOOD |
60 | 430 | 25 | 27 | GOOD |
70 | 333 | 37 | 32 | POOR |
The response time keep stable and low as the load going until reach the bottleneck. When users pass 60 would make the system unusable.
Figure 27. Available memory vs. user load
As load was increased, memory was consumed to a point at which the paging file was increasingly used (see figure 6). At 60 users the memory was exhausted and paging file start to increase sharply.
Figure 28. Paging file usage vs. user load
In figure 29 we can see that processor time increases steadily with load going. At 70 users, the processor was being used more than 30% of the time.
Figure 29. Processor time vs. user load
IBM Notes Browser Plugin on 16 GB RAM platform
Table 17 shows the results for this test.
Table 17. Results of XenApp 6.5 on 64-bit Win2008 SP2 Enterprise Server, 16GB RAM
Users | Available
Memory
(MB) | Paging File
Usage
(%) | Processor
Time
(%) | User
Experience |
0 | 14770 | 0 | 1 | GOOD |
10 | 13376 | 0 | 5 | GOOD |
30 | 10398 | 0 | 17 | GOOD |
50 | 7468 | 0 | 23 | GOOD |
70 | 4850 | 3 | 31 | GOOD |
90 | 2446 | 5 | 43 | GOOD |
100 | 889 | 6 | 50 | GOOD |
110 | 983 | 17 | 52 | GOOD |
120 | 601 | 24 | 59 | POOR |
In this test the CPU was the limiting factor. By the level at 120 users, the performance degrade. An acceptable performance level would be between 100 and 110 users.
In this test, CPU was the bottleneck when this capacity test was performed (see figure 30). In this test the amount of processor usage increased steadily as user sessions increased on the Citrix XenApp Server. Performance began to degrade as CPU consumption over 50%, although the paging file were not all consumed.
Figure 30. Processor time vs. user load
Though much of the available memory was consumed in the test, there was still over 80% of paging file remaining before response times began to degrade.
Figure 31. Available memory vs. user load
Figure 32. Paging file usage vs. user load
Conclusions
The analyses presented in this white paper yield the following key conclusions:
• During our testing, as more processor and memory capacity was added, near-linear capacity was observed for the Notes client in terms of capacity. Therefore it is anticipated that, for large implementations, there is potential value in using hardware and growing it to even more CPUs and memory.
•
• Given that 8GB environment and 16GB environment using the same amount processor, it was no surprise that CPU becomes the main bottleneck in 16 GB 64-bit environments.
•
• If Embedded Experience (like Discover Homepage and MIME mail) is enabled, the Notes Client will consume more memory.
Appendix A: Enable IBM Notes Shared Class Cache
When running your Notes Standard 9.0 client in a XenApp environment, it is recommended to enable the following optimization:
IBM Notes Shared Class Cache for Citrix XenApp running IBM Notes 9.0.
The Notes Shared Class Cache for Citrix allows the multiple Notes users on a Citrix server to share some of the Java classes, ensuring that each Notes instance does not load its own copy of the class files. This in turn allows you to reduce memory usage for each Notes user.
To enable the Java Shared Cache Class for Notes 9.0:
In the [notes install path]\framework\rcp\deploy\
jvm.properties file remove the ",singleJVM" option from the line:
vmarg.Dshare=-Xshareclasses:name=xpdplat,controlDir=${prop.jvm.shareclasses.loc},groupAccess,keep,nonfatal
Then, change the Shared Class cache location to a shared/common path in the following line from:
jvm.shareclasses.loc=${rcp.data}/.config/org.eclipse.osgi
to something like
:
jvm.shareclasses.loc=c:/temp/xpdplat
This directory should allow write access for all users. Note that the Notes client will not start if this is not done correctly.
Appendix B: Test environment preparations
Test-user provisioning involved the following:
• Microsoft Active Directory 6.1 was installed on a Windows 2008 R2 Server in the same domain as the Citrix servers and populated with virtual users from a batch file via the “dsadd user” command.
• Each added user corresponded to a IBM Domino virtual user with a mail file already registered on the Domino server.
• A group was created on the Active Directory containing all users, and this was added to the Remote Desktop Users group locally on each Citrix server.
• The published application on each Citrix server was then set to contain the Remote Desktop Users group local to that machine.
• Another Windows 2008 R2 Server with 8GB RAM was set up as a File Server. Because this machine was to act as the repository for the IBM Notes Data directory for all users, a large amount of disk space was essential.
• A 1GB network connection was available between the Citrix server and the file server, which was to reduce any potential latency issues with having a remote
data directory.
• We initialized all users by creating a script that logged in via an ICA client, launched the IBM Notes client, and walked through the steps to configure that user, such as pointing the user to the Domino server on which the user’s mail file resides and setting a common Home Page for all users.
• Once all users were set to launch their IBM Notes clients in a common state, the test was ready to begin.
Appendix C: IBM Notes install
IBM has invested significant development time in improving the Notes 9.0 Client install on Citrix
XenApp.
Installing IBM Notes
To install Notes 9.0 on a clean system, first set this property for each user on the Active Directory server:
Property: "Active Directory Users and Computers" - "Terminal Services Profile" tab -Terminal
Services Home Folder
Setting: "Connect W: To \\fileservername\directory\%USERNAME%"
Also, ensure that you have updated the Terminal Services Profile tab in Active Directory with
the file server mapping that you intend on hosting the data for each user.
1. Log on to the Citrix server (directly, not remotely) as Administrator.
2. As Administrator, map w: to \\fileservername\directory\Administrator (using Windows Explorer or the “net use” command).
3. Open a command prompt, browse to the location of setup.exe, and type this command:
setup /v"SETMULTIUSER=1 MULTIUSERBASEDIR=w:\ MULTIUSERCOMMONDIR=c:\Notes\common CITRIX=1"
4. During the installation, select the Multi-user Install option.
5. Optional: If testing Notes Browser Plug-in scenarios, need enable Notes Browser Plug-in option in Custom Setup step, as figure 33.
Figure 33. Custom Setup – enable Notes Browser Plug-in
6. Optional: If testing OpenSocial Embedded Experiences scenarios, remain the default enabled status of OpenSocial component option. Otherwise, disable OpenSocial component option, as figure 34.
Figure 34. Custom Setup – disable OpenSocial component
7. Other steps through the installation, selecting the defaults or customizing, if desired. If prompted to set Notes as the default Mail Program, select Yes.
Resources
• IBM Notes and Domino 9.0 Social Edition Forum
• Citrix Support Forums
• Rational Performance Tester Extension for Citrix Presentation Server
• developerWorks® article, “
Using IBM Rational Performance Tester V8.2 to load test IBM Notes Standard Client in a Citrix XenApp environment”
• Technote #1386866: “
Upgrading a Notes deployment on the Citrix platform -- release-specific upgrade matrix.”
• Technote #1330177, "
How to improve memory usage of Notes Standard clients on Citrix"
• Technote #1627450, “
IBM Notes Social Edition clients using Citrix and OpenSocial gadgets need to be configured to render gadgets remotely”
• Technote #1592345, “
Is the Notes Preloader feature support on the XenApp/Terminal Services Platform?”
• Technote #1635483, “
MAC OS X 10.8.x: Calendar entries display incorrect times”
11 About the authors
Shenggao Ou joined IBM in 2004,started from SVT engineer, then SVT leader, now he is SVT architect for Domino,IBM Docs, Forms and ICS integration. You can reach Shenggao Ou at
oushengg@cn.ibm.com.
Xiaoguang Luo joined IBM in 2006. You can reach Xiaoguang Luo at
luoxg@cn.ibm.com.
Xiran Li joined IBM in 2010, working with the Domino System Test Reliability team. You can reach Xiran Li at
lixiran@cn.ibm.com.
Rong Sun joined IBM in 2009 as a part of IBM Lotus Software Group, working with Domino System Test Reliability Team. She has a strong background in Information Technologies and experience in automation development. You can reach Rong Sun at
rongsun@cn.ibm.com.
Trademarks
• developerWorks, Domino, IBM, iNotes, Lotus, Notes, and Rational are trademarks or registered trademarks of IBM Corporation in the United States, other countries, or both.
• Microsoft and Windows are registered trademarks of Microsoft Corporation in the United States, other countries, or both.
• Java and all Java-based trademarks and logos are trademarks or registered trademarks of Sun Microsystems, Inc., in the United States, other countries, or both.
• Citrix, XenServer, XenApp, and XenDesktop are trademarks of Citrix Systems, Inc.
• Other company, product, and service names may be trademarks or service marks of others.