Requirement

For an Extensive File Based usage in connections, there could be cases wherein you would need to move the files application from the Existing cluster to Separate Files cluster.

This article provides the highlevel steps to move the Files Application to its own separate cluster

Environment : IC 4.5 CR 5

Medium Deployment :
Cluster1
Cluster2
InfrastructureCluster

Files application resides in Cluster2 along with other connections applications

Current Connection Setup

New Connections Setup with Separate Files Cluster

 

Preparation of Connection File Application Servers

1. Provision 2 servers for the File cluster . Ensure its from the Same Datacenter and other details like VLAN , network bandwidth etc should be same as the other connections Server Nodes
2. The OS versions should be same as the other connections Server Nodes
3. Install VNC or any X11 forwarding enabled for the GUI access of the new file Server 
4. Ensure resolve.conf is updated to match the appropriate DNS
5. Ensure that the File Shared mount point is mounted on the new File Servers.
   Update the /etc/fstab of the Filecluster Nodes for the connection shared folder
   This should automount the shared mount point which holds the files after reboot.
6. Ensure hosts file are updated in all the servers in the env with the new conn file server details
7. Start NTP services

 

Backup of the servers

1. Take the Image Backup of the Complete Connections servers including the DBs
2. Take the backup of the Customization folder
3. Screenshot details of the File Application from Admin console to get the security mapping etc
4. Take the backup of the LotusConfig folders from the DMgr Cell
5. Get the version details of the IIM and WAS and IBM connections
6. Take the Backup of the Installation Manger binaries and data path ie /var/ibm/installationmanager
7. Dmgr Profile Backup
8. Backup of the /opt/IBM/Connections/version  folder
9. Backup of the Config Logs and SystemOut Logs of All server before implementation

 

NOTE : After the backup is completed , Stop all the Connection Clusters and the Nodeagents. But keep Dmgr running .

 

Install Separate WAS instance for Files and Federate it

Step 1: Commission two New Server with the same OS , Platform etc for the two File Cluster members

Step2 : On the Filecluster servers , Install Websphere Application server and the necessary fixpacks similar to the Websphere running on other clusters
In my case I have installed WebSphere 8.0.0.8 on both the file clusters

Step3 : Create Custom Profiles on these servers using manage Profile or through the GUI mode

Step4 : Add the 2 Newly created profiles on Connfile1Node01 and Connfile2Node01 in the connection Dmgr Cell using AddNode Command

 

NOTE : The Time difference between the Dmgr and the File Nodes should be less than 5 min otherwise the addNode command will fail

 

Step5: Get the value of the File_JDBC_DRIVER_HOME from the Environment 

WebSphere variable from the connections dmgr

The path mentioned is /opt/IBM/JDBC  

1. Create a directory /opt/IBM/JDBC    on both Connfile1Node01 and Connfile2Node01
2. Using scp or any other methods copy the db2jcc4.jar & db2jcc_license_cu.jar from the other Connections nodes and save it in /opt/IBM/JDBC  on both Connfile1Node01 and Connfile2Node01
3. Provide appropriate read permissions to /opt/IBM/JDBC  

 

NOTE: if the db drivers are not in the same path as mentioned in File_JDBC_DRIVER_HOME.  After the installation of Files application on the new Servers there will be some JDBC exceptions which will prevent from assessing the Files application “During the restarts of the file Server few JDBC exception for jdbc/oauthp is generated  00000018 J2CUtilityCla E   J2CA0036E: An exception occurred while invoking method setDataSourceProperties on com.ibm.ws.rsadapter.spi.WSManagedConnectionFactoryImpl used by resource jdbc/oauthp” Also when I try to access the files apps it gives "EJPVJ9104E: Unable to initialize application platform."

 

Note : Recheck the WasRemoteHostName within the wkplc.properties under /opt/IBM/WebSphere/AppServer/profiles/Dmgr01/ConfigEngine/properties/ to ensure that the WasRemoteHostName is set for the proper Dmgr Hostname cat /opt/IBM/WebSphere/AppServer/profiles/Dmgr01/ConfigEngine/properties/wkplc.properties | grep -i hostname # Property: WasRemoteHostName WasRemoteHostName= dmgrhostname.com # Property: VirtualHostName VirtualHostName=default_host

 

Removing the File Application from the Cluster

Step6: Removing the File Application from the Cluster

1. Launch the Installation Manager from the Dmgr

         b. Click on Modify

 

          c. Select for package : IBM connections and click Next

  

    d. Here all the applications currently installed on the Connection will be displayed .

      

          e. To remove the files application from the connections uncheck the Files application and click Next

 

        f)    We have to provide the Dmgr Details like the installation path , the dmgr credentials etc and then click “Validate”.
       Only if the validation is successful the “Next” button will get highlighted

 

            g. Once the validation is completed click Next . the summary would be displayed

 

Reconfirm all the details . Ensure that only Files application is under the Removing Feature list

Then click Modify

This may take a few min

          h. Ensure that the Removal is successful

 

If the Removal was not successful check the IIM logs for the reason for the failure.
 

 

                i. Only Start the Nodeagent and then Restart the Dmgr and Full Sync the Nodes

            j. Stop the Nodeagent

            k. Login to the dmgr console to check if the file application is removed

 

NOTE : DO NOT REMOVE THE FILES DATABASE . THE SAME WILL BE USED BY THE NEW FILES APPLICATION

 

Reinstall Files application

Step7: Reinstall Files application on the new FileNodes which were created above

1. Ensure that the Dmgr is Running and the Connection Servers and Nodeagents are Stopped
    
2. Launch the Installation Manager from the DMGR servers and click on modify

           c. Select the IBM Connection Package and click Next

 

   d. Currently Files application will be unchecked

 

           e. Select the checkbox for the Files Application and click Next

         f.  Enter the Dmgr Details like the Location of the Installation , Profile Name , the admin username and password then click Validate

Then click Next

 

          
g. Select the deployment type . For the Files installation on its separate Cluster we should select Large Deployment .
Here we can provide the name of the cluster .. I have kept it as FileCluster .

For the members I have selected the two new added Nodes of the Connfile1Node01 and Connfile2Node01

Provide the File Cluster member names ie
FilesCluster_server1 &  FilesCluster_server2  for Connfile1Node01 and Connfile2Node01 Respectively

 

 

        h .  Provide the details of the FILES Database server like the db hostname , db name , port nos , db lcusers credentials etc

Then click “Validate”

 

 

    i.  Recheck that only files application is listed in the Adding Feature Section . Then click Modify

This process will add the files application to the New FileCluster
It would take some time though for the installation to be completed. Then click Finish  on sucessful completion

          j)    Restart DMGR. and start the Nodeagents on all the connection nodes including the Connfile1Node01 and Connfile2Node01

          k)    Login to the admin console and check under Cluster to view the FileCluster and Applications > Files

DO NOT START THE SERVERS YET  
 

 

Up-gradation of the CR  level for Files Application

Step8: Up-gradation of the CR  level for the Files Application

As the we have installed the base version of the Connection for File Application on the File Cluster we need to upgrade it to the Same Version as the other Application ie IC 4.5 CR5

To upgrade File application to IC 4.5 CR5
              1)    Ensure that Services Shut down all Nodes and Nodeagents and DMGR.

                        a) Rename/ Remove  ifix CR5 and CR4 for FILES ie log, backupjar, efixDriver .     
                       This will make the updateInstaller think the Files CR5 isn't installed.  

                        b) Details Mentioned Below

...........................................................................................................................
Log Name              : /connections/IBM/version/log/
Log Name              : /connections/IBM/version/log/
...........................................................................................................................
Product Directory     : /connections/IBM/version/xxxxxx.efix
History Directory     : /connections/IBM/version/history/xxxxxx.efixDriver
History Directory     : /connections/IBM/version/history/xxxxxx.efixApplied
Backup File Name      : /connections/IBM/version/backup/xxxxxxxx_xxxxxx_xxxxx_undo.jar
Where xxxxxx pertains to  
...........................................................................................................................

 

NOTE : If there are multiple files for the same Application within the above folders remove all the files for the specific application .. in this case files [root@/ ]$pwd /opt/IBM/Connections/version [root@/ ]$ ls -l | grep Files -rw-rw-rw- 1 root root 246 Jun 12 2014 LO78429-IC4500-CR04-Files.efix -rw-rw-rw- 1 root root 246 Sep 14 2015 LO79880-IC4500-CR05-Files.efix

Remove/move  all the previous version of efix for Files ie  IC4500-CR04 & IC4500-CR05

              2. Delete all Temp and wstemp  folders for DMGR and all Profiles/Nodes.

NOTE : Please refer to the below link before deleting the temp https://www.ibm.com/developerworks/community/blogs/aimsupport/entry/deleting_temporary_files_in_websphere_application_server?lang=en For DMGR: /opt/IBM/WebSphere/Appserver/profiles/Dmgr01/Config/ /opt/IBM/WebSphere/Appserver/profiles/Dmgr01/ For Nodes/App Profiles: /opt/IBM/WebSphere/Appserver/profiles/AppServer/

3. Start the DMGR only

4. Copy the CR5 jar only for FILE Application in the fixes folder in updateInstaller

5.Install CR5 using the updateInstaller lauchpad

1.  Execute setupCmdline.sh from the /opt/IBM/WebSphere/Appserver/profiles/Dmgr01/bin   to load the environment variables
2.   Execute the /opt/IBM/Connections/updateInstaller/updateWizard.sh for the GUI

And Click Next

 

        c. Select the Install Option and the provide the path where the Files

              d. Ensure that the ifix for Files for CR5 is selected and then click Next

            e.  Click Install to proceed with the installation if ifix

           Click “Finish”

       f)    Check the version of the files

6)    Restart DMGR

7)    Start all Nodeagents then Run Full Sync against all Nodes.

8)    Check the IC versions using “./updateSilent.sh -fix -installDir /opt/IBM/Connections/”

 

File Application Configurations

Step9 : Configure the Files Application
1)    Ensure only Dmgr is started

2)    Checkout LotusConnections-config.xml file and update the entry for files cluster entry with appropriate webserver urls and ports

Before:

After changes:

           
3. From the dmgr console Files web module will need to be mapped to the webserver and Files cluster 

4. Generate the Plugin and propagated the plugin

       

5. Start Nodeagent and Sync the Nodes

6. Now start the Connections Servers & View the logs of all the Servers

7. Ensure the File Servers are started properly with no error in logs   

8. Increase the JVM Heap Memory appropriately  for the File App JVMs

9. Login to the Connections url and try to access the files application of different test users

10.Stop each of the File cluster members such that the request is getting processed by either of the File servers

These are the highlevel steps to separate the File Application to a separate Cluster

 

Errors Occurred

You could see similar errors for the FilesScheduler in the System Out logs when starting the Files Application Servers   [4/4/16 11:18:36:460 CDT] 0000002f AlarmListener E   SCHD0076E: Scheduler resource FilesScheduler (scheduler/files) was unable to fire a notification event of FIRING for task with task id 4 because the following error occurred: SCHD0137E: Unable to create EJB instance for NotificationSink: java.rmi.NoSuchObjectException: CORBA OBJECT_NOT_EXIST 0x4942fc14 No; nested exception is:         org.omg.CORBA.OBJECT_NOT_EXIST:           SERVER (id=2a24696b, host=connections2.myorg.com) TRACE START:               org.omg.CORBA.OBJECT_NOT_EXIST: SERVANT_NOT_FOUND (4) for key 0x49454a500201be8c2c5908436c75737465723203454a4200000069acac0002000100290000005f5f686f6d654f66486f6d65 73235f5f686f6d654f66486f6d6573235f5f686f6d654f66486f6d65730846696c6573236c636f6e6e2e736368656475 6c65722e656a622e6a617223436f6d6d6f6e4e6f74696669636174696f6e53696e6b  vmcid: IBM  minor code: C14  completed: No   Solution : Clear the Scheduler for the Files Application . Running the clearScheduler.sql on the Db2 Server hosting FILES DB   $db2 -td@ -vf clearScheduler.sql   Refer below link for more information http://www-01.ibm.com/support/docview.wss?uid=swg21414001