Maintainance of WebSphere Extended deployment server

When applying maintenance to upgrade the level of WebSphere Application Server Extended Deployment and WebSphere Network Deployment it is important that the deployment manager remain inactive until both upgrades are complete because if the deployment manager is active before both upgrades are complete it may detect an incompatible version of WebSphere Application Server Extended Deployment and remove some required resources from the dynamic cluster. Keeping the deployment manager inactive until both Network Deployment and WebSphere Application Server Extended Deployment updates have been completed will insure that this potential problem does not occur.

Finding out the cluster member that is handling your request

Every now and then we have this issue where the bug is not consistent instead it is reproducible only when request goes to particular cluster member. For example in my last project we had this portlet that use to invoke shell script on the server to perform some task. Now if the shell script is not setup properly it used to fail. Problem was that out of 3 requests 2 use to work right and problem was only when the request would go to particular cluster where shell script is not setup properly.

You can use one of these methods to find out which cluster member is handling your request

Using ServerName class


You can add this code to the JSP that is generating markup for the request.

<%@page import="com.ibm.websphere.runtime.ServerName"%>
<table>
<tr>
<td>Dispaly Name</td>
<td><%=ServerName.getDisplayName()%></td>
</tr>
<tr>
<td>Full Name</td>
<td><%=ServerName.getFullName()%></td>
</tr>
<tr>
<td>Server Id</td>
<td><%=ServerName.getServerId()%></td>
</tr>
</table>


In my case i do have ServerNamePortlet.war that i can add to the page and it displays the server name




Global settings portlet



The Global settings portlet displays host name of the server that is handling the current request in the top section



As you can see it is displaying dmgr.wpcerticiation.com as the host name of the server. But this might not be helpful if you have vertical cluster it will display same name irrespective of the which cluster is handling the request

Configure HTTP Server Plugin log



If you have access to the HTTP Server Plugin log files then you can also configure the logging level for HTTP Server Plugin to Detail and it will generate a detailed log for which cluster member is handling particular request and reason why

In the WAS Admin Console go to Servers -> Web Servers -> Web Server Name -> Plugin properties. Change the log level to detail. Save and restart the HTTP Server



Once HTTP server is restarted, make couple of requests and then check Plugin/log/webserver1/http_plugin.log. It will have very detail information about the request routing. This is sample log from my machine

[Tue May 26 17:38:48 2009] 00000fb5 0234bb90 - DETAIL: ws_common: websphereShouldHandleRequest: trying to match a route for: vhost='localhost'; uri='/wps/portal'
[Tue May 26 17:38:48 2009] 00000fb5 0234bb90 - DETAIL: ws_common: websphereBeginRequest: Request is: host='localhost'; uri='/wps/portal'
[Tue May 26 17:38:48 2009] 00000fb5 0234bb90 - DETAIL: ws_common: websphereFindServerGroup: Setting the server group: PortalCluster; highScore: 6; highExactMatch: 5; affinityCookie: JSESSIONID; affinityURL: jsessionid
[Tue May 26 17:38:48 2009] 00000fb5 0234bb90 - DETAIL: ESI: esiRequestPushUrl: '/wps/portal'
[Tue May 26 17:38:48 2009] 00000fb5 0234bb90 - DETAIL: ESI: esiRulesGetCacheId: cache miss; no rule for '/wps/portal'
[Tue May 26 17:38:48 2009] 00000fb5 0234bb90 - STATS: ws_server_group: serverGroupCheckServerStatus: Checking status of portal1_WebSphere_Portal_V1, ignoreWeights 0, markedDown 0, retryNow 0, wlbAllows 1 reachedMaxConnectionsLimit 0
[Tue May 26 17:38:48 2009] 00000fb5 0234bb90 - DETAIL: ws_common: websphereFindTransport: Setting the transport(case 2): dmgr.wpcertification.com on port 10048
[Tue May 26 17:38:48 2009] 00000fb5 0234bb90 - ERROR: ws_common: websphereGetStream: Failed to connect to app server on host 'dmgr.wpcertification.com', OS err=111
[Tue May 26 17:38:48 2009] 00000fb5 0234bb90 - ERROR: ws_common: websphereExecute: Failed to create the stream
[Tue May 26 17:38:48 2009] 00000fb5 0234bb90 - ERROR: ws_server: serverSetFailoverStatus: Marking portal1_WebSphere_Portal_V1 down
[Tue May 26 17:38:48 2009] 00000fb5 0234bb90 - STATS: ws_server: serverSetFailoverStatus: Server portal1_WebSphere_Portal_V1 : pendingRequests 0 failedRequests 1 affinityRequests 0 totalRequests 0.
[Tue May 26 17:38:48 2009] 00000fb5 0234bb90 - ERROR: ws_common: websphereHandleRequest: Failed to execute the transaction to 'portal1_WebSphere_Portal_V1'on host 'dmgr.wpcertification.com'; will try another one
[Tue May 26 17:38:48 2009] 00000fb5 0234bb90 - STATS: ws_server_group: serverGroupCheckServerStatus: Checking status of portal1_WebSphere_Portal, ignoreWeights 0, markedDown 0, retryNow 0, wlbAllows 1 reachedMaxConnectionsLimit 0
[Tue May 26 17:38:48 2009] 00000fb5 0234bb90 - DETAIL: ws_common: websphereFindTransport: Setting the transport(case 2): dmgr.wpcertification.com on port 10040
[Tue May 26 17:38:48 2009] 00000fb5 0234bb90 - ERROR: ws_common: websphereGetStream: Failed to connect to app server on host 'dmgr.wpcertification.com', OS err=111
[Tue May 26 17:38:48 2009] 00000fb5 0234bb90 - ERROR: ws_common: websphereExecute: Failed to create the stream
[Tue May 26 17:38:48 2009] 00000fb5 0234bb90 - ERROR: ws_server: serverSetFailoverStatus: Marking portal1_WebSphere_Portal down
[Tue May 26 17:38:48 2009] 00000fb5 0234bb90 - STATS: ws_server: serverSetFailoverStatus: Server portal1_WebSphere_Portal : pendingRequests 0 failedRequests 1 affinityRequests 0 totalRequests 0.
[Tue May 26 17:38:48 2009] 00000fb5 0234bb90 - ERROR: ws_common: websphereHandleRequest: Failed to execute the transaction to 'portal1_WebSphere_Portal'on host 'dmgr.wpcertification.com'; will try another one
[Tue May 26 17:38:48 2009] 00000fb5 0234bb90 - ERROR: ws_common: websphereWriteRequestReadResponse: Failed to find an app server to handle this request
[Tue May 26 17:38:48 2009] 00000fb5 0234bb90 - ERROR: ESI: getResponse: failed to get response: rc = 2
[Tue May 26 17:38:48 2009] 00000fb5 0234bb90 - DETAIL: ESI: esiRequestPopUrl: '/wps/portal'
[Tue May 26 17:38:48 2009] 00000fb5 0234bb90 - ERROR: ws_common: websphereHandleRequest: Failed to handle request
[Tue May 26 17:38:48 2009] 00000fb5 0234bb90 - DETAIL: ws_common: websphereEndRequest: Ending the request
[Tue May 26 17:38:52 2009] 00000fb7 04c6eb90 - DETAIL: ws_common: websphereShouldHandleRequest: trying to match a route for: vhost='localhost'; uri='/favicon.ico'
[Tue May 26 17:38:52 2009] 00000fb7 04c6eb90 - DETAIL: ws_common: websphereShouldHandleRequest: No route found
[Tue May 26 17:48:31 2009] 00000fb5 02d4cb90 - DETAIL: ws_common: websphereShouldHandleRequest: trying to match a route for: vhost='localhost'; uri='/wps/portal'
[Tue May 26 17:48:31 2009] 00000fb5 02d4cb90 - DETAIL: ws_common: websphereBeginRequest: Request is: host='localhost'; uri='/wps/portal'
[Tue May 26 17:48:31 2009] 00000fb5 02d4cb90 - DETAIL: ws_common: websphereFindServerGroup: Setting the server group: PortalCluster; highScore: 6; highExactMatch: 5; affinityCookie: JSESSIONID; affinityURL: jsessionid
[Tue May 26 17:48:31 2009] 00000fb5 02d4cb90 - DETAIL: ESI: esiRequestPushUrl: '/wps/portal'
[Tue May 26 17:48:31 2009] 00000fb5 02d4cb90 - DETAIL: ESI: esiRulesGetCacheId: cache miss; no rule for '/wps/portal'
[Tue May 26 17:48:31 2009] 00000fb5 02d4cb90 - STATS: ws_server_group: serverGroupCheckServerStatus: Checking status of portal1_WebSphere_Portal_V1, ignoreWeights 0, markedDown 1, retryNow 1, wlbAllows 0 reachedMaxConnectionsLimit 0
[Tue May 26 17:48:31 2009] 00000fb5 02d4cb90 - DETAIL: ws_common: websphereFindTransport: Setting the transport(case 2): dmgr.wpcertification.com on port 10048
[Tue May 26 17:48:31 2009] 00000fb5 02d4cb90 - DETAIL: ws_common: websphereGetStream: Created a new stream; queue was empty, socket = 11
[Tue May 26 17:48:31 2009] 00000fb5 02d4cb90 - DETAIL: GET /wps/portal HTTP/1.1
[Tue May 26 17:48:31 2009] 00000fb5 02d4cb90 - DETAIL: Host: localhost
[Tue May 26 17:48:31 2009] 00000fb5 02d4cb90 - DETAIL: User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.10) Gecko/2009042718 CentOS/3.0.10-1.el5.centos Firefox/3.0.10
[Tue May 26 17:48:31 2009] 00000fb5 02d4cb90 - DETAIL: Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
[Tue May 26 17:48:31 2009] 00000fb5 02d4cb90 - DETAIL: Accept-Language: en-us,en;q=0.5
[Tue May 26 17:48:31 2009] 00000fb5 02d4cb90 - DETAIL: Accept-Encoding: gzip,deflate
[Tue May 26 17:48:31 2009] 00000fb5 02d4cb90 - DETAIL: Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
[Tue May 26 17:48:31 2009] 00000fb5 02d4cb90 - DETAIL: Keep-Alive: 300
[Tue May 26 17:48:31 2009] 00000fb5 02d4cb90 - DETAIL: Cookie: TJE=; TE3=N0:C N1:E N2:C N3:C N4:C N5:C N6:C N7:C N8:C N9:C N10:C N11:C N12:C N13:C N14:C N15:C N16:C N17:C N18:C N19:C N20:C N21:C N22:C N23:C N24:C N25:C N26:C; LtpaToken2=RVs0qI0TZY1A5F/F64O7fOKte/Km9n5e9szaWOHbigepxKneWYF5zmGu3va/Zg3A/z0actIMUCL1zz4DYN8iSsc3jXrGE6w3XLlIqGlXvjDDCQJUF+ihenCkj7HPH/SokxxB7Jn2JKOcIjYlQVkysqDhcM6g+vtZwWL8ca5nj3I4mKI4rhZmoNmuxXihF0ZloI7QIg1Da71qqILj0dWsKCvToVZPrfACdVEWqeqzd+r4Spj+Ko2Spnp9aOFIPua3d6RQq6fTR1WWkfC1Ie6HtGN5lt+KaGmq9qSfFCWVJ7QzPA5/FxWtug8jQPXcsniy+gVuFmUMr0SbTfUhZpODrG2HN1w8JIE2Rncg3ZTeaNA=; LtpaToken=UAx8tmo+3t2C4YhDmSxy6C90fwSChm1sf9bPGJKeLkz3ZSbcbVT+5u9efRUHASEoguGIQ5kscugvdH8EgoZtbH/WzxiSni5MN/v6cH7uJbwqwoqM0cC6C9P7mXKOPeSMPwHYnsgoxPnaflmwB2XdH33n9Lsh001BPCOTMcVylkRO3QXOjub1S11SMwVb+YY7bxARWjRrzbkSKlikjFRVStnrMIWEEVqn2BC1rtZllE9IwIjR4wdf4CnMaZY/1JPuiioIpL4sZYSFLUrA/nXsyTBjdz69FtrxHz7mClOZaUIuAPU5IoEGwiN7YziVJJQWU+pbsU+m0Mu+2co+aV1GO+J10W3vGOCd
[Tue May 26 17:48:31 2009] 00000fb5 02d4cb90 - DETAIL: $WSIS: false
[Tue May 26 17:48:31 2009] 00000fb5 02d4cb90 - DETAIL: $WSSC: http
[Tue May 26 17:48:31 2009] 00000fb5 02d4cb90 - DETAIL: $WSPR: HTTP/1.1
[Tue May 26 17:48:31 2009] 00000fb5 02d4cb90 - DETAIL: $WSRA: 127.0.0.1
[Tue May 26 17:48:31 2009] 00000fb5 02d4cb90 - DETAIL: $WSRH: 127.0.0.1
[Tue May 26 17:48:31 2009] 00000fb5 02d4cb90 - DETAIL: $WSSN: localhost
[Tue May 26 17:48:31 2009] 00000fb5 02d4cb90 - DETAIL: $WSSP: 80
[Tue May 26 17:48:31 2009] 00000fb5 02d4cb90 - DETAIL: Surrogate-Capability: WS-ESI="ESI/1.0+"
[Tue May 26 17:48:31 2009] 00000fb5 02d4cb90 - DETAIL: _WS_HAPRT_WLMVERSION: -1
[Tue May 26 17:48:31 2009] 00000fb5 02d4cb90 - DETAIL: lib_htresponse: htresponseRead: Reading the response: 870d7c4
[Tue May 26 17:49:20 2009] 00000fb5 02d4cb90 - DETAIL: HTTP/1.1 200 OK
[Tue May 26 17:49:20 2009] 00000fb5 02d4cb90 - DETAIL: Content-Type: text/html; charset=UTF-8

[Tue May 26 17:49:20 2009] 00000fb5 02d4cb90 - DETAIL: IBM-Web2-Location: /wps/portal/!ut/p/c5/04_SB8K8xLLM9MSSzPy8xBz9CP0os3hnd0cPE3MfAwN3cyNTAyM_02AjX8cwAwNfQ_1wkA6zeAMcwNFA388jPzdVvyA7rxwAZPfmsg!!/dl3/d3/L2dBISEvZ0FBIS9nQSEh/

[Tue May 26 17:49:20 2009] 00000fb5 02d4cb90 - DETAIL: Cache-Control: no-cache

[Tue May 26 17:49:20 2009] 00000fb5 02d4cb90 - DETAIL: Expires: Thu, 01 Jan 1970 00:00:00 GMT

[Tue May 26 17:49:20 2009] 00000fb5 02d4cb90 - DETAIL: Pragma: no-cache

[Tue May 26 17:49:20 2009] 00000fb5 02d4cb90 - DETAIL: Vary: User-Agent,Cookie

[Tue May 26 17:49:20 2009] 00000fb5 02d4cb90 - DETAIL: Content-Language: en-US

[Tue May 26 17:49:20 2009] 00000fb5 02d4cb90 - DETAIL: Content-Length: 16671

[Tue May 26 17:49:20 2009] 00000fb5 02d4cb90 - DETAIL: Date: Wed, 27 May 2009 00:49:20 GMT

[Tue May 26 17:49:20 2009] 00000fb5 02d4cb90 - DETAIL: Server: WebSphere Application Server/6.1

[Tue May 26 17:49:20 2009] 00000fb5 02d4cb90 - STATS: ws_server: serverSetFailoverStatus: Server portal1_WebSphere_Portal_V1 : pendingRequests 0 failedRequests 1 affinityRequests 0 totalRequests 1.
[Tue May 26 17:49:20 2009] 00000fb5 02d4cb90 - DETAIL: ESI: esiRequestPopUrl: '/wps/portal'
[Tue May 26 17:49:20 2009] 00000fb5 02d4cb90 - DETAIL: ws_common: websphereEndRequest: Ending the request
[Tue May 26 17:49:44 2009] 00000fb7 0234bb90 - DETAIL: ws_common: websphereShouldHandleRequest: trying to match a route for: vhost='localhost'; uri='/wps/portal/!ut/p/c5/04_SB8K8xLLM9MSSzPy8xBz9CP0os3hnd0cPE3MfAwN3cyNTAyM_02AjX8cwAwNfQ_1wkA6zeAMcwNFA388jPzdVvyA7rxwAZPfmsg!!/dl3/d3/L0lDUWtpQ1NTUW9LVVFBISEvb0lvZ0FFQ1FRREdJUXBURE9DNEpuQSEhLzRDd2lSLXJmbTE2SWt5WGlnRUEhLzdfQ0dBSDQ3TDAwRzcyNTAyTjVTMk1BVjAwRTQvd3BzLnBvcnRsZXRzLmxvZ2lu/'
[Tue May 26 17:49:44 2009] 00000fb7 0234bb90 - DETAIL: ws_common: websphereBeginRequest: Request is: host='localhost'; uri='/wps/portal/!ut/p/c5/04_SB8K8xLLM9MSSzPy8xBz9CP0os3hnd0cPE3MfAwN3cyNTAyM_02AjX8cwAwNfQ_1wkA6zeAMcwNFA388jPzdVvyA7rxwAZPfmsg!!/dl3/d3/L0lDUWtpQ1NTUW9LVVFBISEvb0lvZ0FFQ1FRREdJUXBURE9DNEpuQSEhLzRDd2lSLXJmbTE2SWt5WGlnRUEhLzdfQ0dBSDQ3TDAwRzcyNTAyTjVTMk1BVjAwRTQvd3BzLnBvcnRsZXRzLmxvZ2lu/'
[Tue May 26 17:49:44 2009] 00000fb7 0234bb90 - DETAIL: ws_common: websphereFindServerGroup: Setting the server group: PortalCluster; highScore: 6; highExactMatch: 5; affinityCookie: JSESSIONID; affinityURL: jsessionid
[Tue May 26 17:49:44 2009] 00000fb7 0234bb90 - DETAIL: ESI: esiRequestPushUrl: '/wps/portal/!ut/p/c5/04_SB8K8xLLM9MSSzPy8xBz9CP0os3hnd0cPE3MfAwN3cyNTAyM_02AjX8cwAwNfQ_1wkA6zeAMcwNFA388jPzdVvyA7rxwAZPfmsg!!/dl3/d3/L0lDUWtpQ1NTUW9LVVFBISEvb0lvZ0FFQ1FRREdJUXBURE9DNEpuQSEhLzRDd2lSLXJmbTE2SWt5WGlnRUEhLzdfQ0dBSDQ3TDAwRzcyNTAyTjVTMk1BVjAwRTQvd3BzLnBvcnRsZXRzLmxvZ2lu/'
[Tue May 26 17:49:44 2009] 00000fb7 0234bb90 - DETAIL: ESI: esiRulesGetCacheId: cache miss; no rule for '/wps/portal/!ut/p/c5/04_SB8K8xLLM9MSSzPy8xBz9CP0os3hnd0cPE3MfAwN3cyNTAyM_02AjX8cwAwNfQ_1wkA6zeAMcwNFA388jPzdVvyA7rxwAZPfmsg!!/dl3/d3/L0lDUWtpQ1NTUW9LVVFBISEvb0lvZ0FFQ1FRREdJUXBURE9DNEpuQSEhLzRDd2lSLXJmbTE2SWt5WGlnRUEhLzdfQ0dBSDQ3TDAwRzcyNTAyTjVTMk1BVjAwRTQvd3BzLnBvcnRsZXRzLmxvZ2lu/'
[Tue May 26 17:49:44 2009] 00000fb7 0234bb90 - STATS: ws_server_group: serverGroupCheckServerStatus: Checking status of portal1_WebSphere_Portal_V1, ignoreWeights 0, markedDown 0, retryNow 0, wlbAllows 1 reachedMaxConnectionsLimit 0
[Tue May 26 17:49:44 2009] 00000fb7 0234bb90 - DETAIL: ws_common: websphereFindTransport: Setting the transport(case 2): dmgr.wpcertification.com on port 10048
[Tue May 26 17:49:44 2009] 00000fb7 0234bb90 - DETAIL: ws_common: websphereGetStream: Created a new stream; queue was empty, socket = 11
[Tue May 26 17:49:44 2009] 00000fb7 0234bb90 - DETAIL: POST /wps/portal/!ut/p/c5/04_SB8K8xLLM9MSSzPy8xBz9CP0os3hnd0cPE3MfAwN3cyNTAyM_02AjX8cwAwNfQ_1wkA6zeAMcwNFA388jPzdVvyA7rxwAZPfmsg!!/dl3/d3/L0lDUWtpQ1NTUW9LVVFBISEvb0lvZ0FFQ1FRREdJUXBURE9DNEpuQSEhLzRDd2lSLXJmbTE2SWt5WGlnRUEhLzdfQ0dBSDQ3TDAwRzcyNTAyTjVTMk1BVjAwRTQvd3BzLnBvcnRsZXRzLmxvZ2lu/ HTTP/1.1
[Tue May 26 17:49:44 2009] 00000fb7 0234bb90 - DETAIL: Host: localhost
[Tue May 26 17:49:44 2009] 00000fb7 0234bb90 - DETAIL: User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.10) Gecko/2009042718 CentOS/3.0.10-1.el5.centos Firefox/3.0.10
[Tue May 26 17:49:44 2009] 00000fb7 0234bb90 - DETAIL: Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
[Tue May 26 17:49:44 2009] 00000fb7 0234bb90 - DETAIL: Accept-Language: en-us,en;q=0.5
[Tue May 26 17:49:44 2009] 00000fb7 0234bb90 - DETAIL: Accept-Encoding: gzip,deflate
[Tue May 26 17:49:44 2009] 00000fb7 0234bb90 - DETAIL: Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
[Tue May 26 17:49:44 2009] 00000fb7 0234bb90 - DETAIL: Keep-Alive: 300
[Tue May 26 17:49:44 2009] 00000fb7 0234bb90 - DETAIL: Referer: http://localhost/wps/portal
[Tue May 26 17:49:44 2009] 00000fb7 0234bb90 - DETAIL: Cookie: TJE=; TE3=N0:C N1:E N2:C N3:C N4:C N5:C N6:C N7:C N8:C N9:C N10:C N11:C N12:C N13:C N14:C N15:C N16:C N17:C N18:C N19:C N20:C N21:C N22:C N23:C N24:C N25:C N26:C; LtpaToken2=RVs0qI0TZY1A5F/F64O7fOKte/Km9n5e9szaWOHbigepxKneWYF5zmGu3va/Zg3A/z0actIMUCL1zz4DYN8iSsc3jXrGE6w3XLlIqGlXvjDDCQJUF+ihenCkj7HPH/SokxxB7Jn2JKOcIjYlQVkysqDhcM6g+vtZwWL8ca5nj3I4mKI4rhZmoNmuxXihF0ZloI7QIg1Da71qqILj0dWsKCvToVZPrfACdVEWqeqzd+r4Spj+Ko2Spnp9aOFIPua3d6RQq6fTR1WWkfC1Ie6HtGN5lt+KaGmq9qSfFCWVJ7QzPA5/FxWtug8jQPXcsniy+gVuFmUMr0SbTfUhZpODrG2HN1w8JIE2Rncg3ZTeaNA=; LtpaToken=UAx8tmo+3t2C4YhDmSxy6C90fwSChm1sf9bPGJKeLkz3ZSbcbVT+5u9efRUHASEoguGIQ5kscugvdH8EgoZtbH/WzxiSni5MN/v6cH7uJbwqwoqM0cC6C9P7mXKOPeSMPwHYnsgoxPnaflmwB2XdH33n9Lsh001BPCOTMcVylkRO3QXOjub1S11SMwVb+YY7bxARWjRrzbkSKlikjFRVStnrMIWEEVqn2BC1rtZllE9IwIjR4wdf4CnMaZY/1JPuiioIpL4sZYSFLUrA/nXsyTBjdz69FtrxHz7mClOZaUIuAPU5IoEGwiN7YziVJJQWU+pbsU+m0Mu+2co+aV1GO+J10W3vGOCd; portalFlyoutIsOpen=; portalOpenFlyout=
[Tue May 26 17:49:44 2009] 00000fb7 0234bb90 - DETAIL: Content-Type: application/x-www-form-urlencoded
[Tue May 26 17:49:44 2009] 00000fb7 0234bb90 - DETAIL: Content-Length: 92
[Tue May 26 17:49:44 2009] 00000fb7 0234bb90 - DETAIL: $WSIS: false
[Tue May 26 17:49:44 2009] 00000fb7 0234bb90 - DETAIL: $WSSC: http
[Tue May 26 17:49:44 2009] 00000fb7 0234bb90 - DETAIL: $WSPR: HTTP/1.1
[Tue May 26 17:49:44 2009] 00000fb7 0234bb90 - DETAIL: $WSRA: 127.0.0.1
[Tue May 26 17:49:44 2009] 00000fb7 0234bb90 - DETAIL: $WSRH: 127.0.0.1
[Tue May 26 17:49:44 2009] 00000fb7 0234bb90 - DETAIL: $WSSN: localhost
[Tue May 26 17:49:44 2009] 00000fb7 0234bb90 - DETAIL: $WSSP: 80
[Tue May 26 17:49:44 2009] 00000fb7 0234bb90 - DETAIL: Surrogate-Capability: WS-ESI="ESI/1.0+"
[Tue May 26 17:49:44 2009] 00000fb7 0234bb90 - DETAIL: _WS_HAPRT_WLMVERSION: -1
[Tue May 26 17:49:44 2009] 00000fb7 0234bb90 - DETAIL: Expect: 100-Continue
[Tue May 26 17:49:44 2009] 00000fb7 0234bb90 - DETAIL: HTTP/1.1 100 Continue
[Tue May 26 17:49:44 2009] 00000fb7 0234bb90 - DETAIL: Content-Length: 0
[Tue May 26 17:49:44 2009] 00000fb7 0234bb90 - DETAIL: Date: Wed, 27 May 2009 00:49:44 GMT
[Tue May 26 17:49:44 2009] 00000fb7 0234bb90 - DETAIL: Server: WebSphere Application Server/6.1
[Tue May 26 17:49:44 2009] 00000fb7 0234bb90 - DETAIL: lib_htresponse: htresponseRead: Reading the response: 870daec
[Tue May 26 17:49:46 2009] 00000fb7 0234bb90 - DETAIL: HTTP/1.1 302 Found
[Tue May 26 17:49:46 2009] 00000fb7 0234bb90 - DETAIL: Date: Wed, 27 May 2009 00:49:46 GMT

[Tue May 26 17:49:46 2009] 00000fb7 0234bb90 - DETAIL: Server: WebSphere Application Server/6.1

[Tue May 26 17:49:46 2009] 00000fb7 0234bb90 - DETAIL: Location: http://localhost/wps/myportal/!ut/p/c5/0wcA1NLTeQ!!/

[Tue May 26 17:49:46 2009] 00000fb7 0234bb90 - DETAIL: Content-Language: en-US

[Tue May 26 17:49:46 2009] 00000fb7 0234bb90 - DETAIL: Set-Cookie: LtpaToken2=mIQa+HWLCT2FimDmKdD1LkuIKsWQ3OktZyySoCG4FUV5w1UGcd27Xu0FhVInen8YF0FJDntHNwuyTlE8OWHTjixTKExxdpR2s6oRV9ORwMCmhyucLZzibfo2JHM7jfX8ONT/QY2j774kAveeSUeWlSncJUzWgfHpRdB+0WYkdpsXIg2vR02kzE+X1Etdoryl/4m2yTIt3qcVv/UnMFIp9mt0APxjTyK6acyQfZr0S7XSH0uSU86UzOiQGEjODHzCdQSplts9F+LBzomeLWTu04vtJSIByR57dnYNYnar+mhGgitnv0nSiH26gQBTRDjgI6PQRPVJCLYgetduYGGUBGMpFa7NxBfMMxbF1K81uK3zpWi5chspMrpzxProLVje7waVfPZlcii1Q+NO2ACwD1pKYqRQmkS0P9U8cAGMKU6YNLX1gKDZW/2mkotVkjLedhQgNo3mf/JAkZORGG9Vz6pL+DWsT4o4179EsZbXGtKS0eohPE8YI6nw0ngTTXdNQ/k9o2kHvrHCyPYdxnyHbkjVtCpRAcXkb79d46DGaDP3wNJ7WIauNK1DD9omsE0O6bp91FvZmy+HN3EUOXOfbYCyyDL0KMl1rKNq0G1gxMVwY7aAdpnH7XWHkL8a/XQhoaU5P1zMoTNIfDa9j3icnicQNnB7wnn+dgpNZ+rQWhrckdZqk7OnXsW43QGg3lSFFbFQZKMr5kOVPmGPPFLuEA==; Path=/

[Tue May 26 17:49:46 2009] 00000fb7 0234bb90 - DETAIL: Set-Cookie: LtpaToken=UAx8tmo+3t2C4YhDmSxy6C90fwSChm1sf9bPGJKeLkzOk2c6GrnSIe9efRUHASEoguGIQ5kscugvdH8EgoZtbH/WzxiSni5Mt81K0nRqrVFutRlhQyihdUOi1A3QA+TkpB52U168KJK+Af/ivUrs3SfllUnmJBdQNNgpAkRzor3IfPpWA/9CYAwTLENylm30NR8Sw58ogo1O6g3aI0JOpHjQWFyJMe7zEgaK31To3k4rvZoi/DNveNEoU6uMunfs9zxs9i8yJ+4rv2DixYUEING7ZT+QDkz7UhgBzxXksanA5HiBUZm4DozYEtBC+IENx8L7twqAqrXd7c3B6C9rgQMLscIwKIAw; Path=/

[Tue May 26 17:49:46 2009] 00000fb7 0234bb90 - DETAIL: Set-Cookie: JSESSIONID=0000Q5jHUZSwpLygvRzlfdLa1Ie:145t799pf; Path=/

[Tue May 26 17:49:46 2009] 00000fb7 0234bb90 - DETAIL: Transfer-Encoding: chunked

[Tue May 26 17:49:46 2009] 00000fb7 0234bb90 - DETAIL: Expires: Thu, 01 Dec 1994 16:00:00 GMT

[Tue May 26 17:49:46 2009] 00000fb7 0234bb90 - DETAIL: Cache-Control: no-cache="set-cookie, set-cookie2"

[Tue May 26 17:49:46 2009] 00000fb7 0234bb90 - STATS: ws_server: serverSetFailoverStatus: Server portal1_WebSphere_Portal_V1 : pendingRequests 0 failedRequests 0 affinityRequests 0 totalRequests 1.
[Tue May 26 17:49:46 2009] 00000fb7 0234bb90 - DETAIL: ESI: esiRequestPopUrl: '/wps/portal/!ut/p/c5/04_SB8K8xLLM9MSSzPy8xBz9CP0os3hnd0cPE3MfAwN3cyNTAyM_02AjX8cwAwNfQ_1wkA6zeAMcwNFA388jPzdVvyA7rxwAZPfmsg!!/dl3/d3/L0lDUWtpQ1NTUW9LVVFBISEvb0lvZ0FFQ1FRREdJUXBURE9DNEpuQSEhLzRDd2lSLXJmbTE2SWt5WGlnRUEhLzdfQ0dBSDQ3TDAwRzcyNTAyTjVTMk1BVjAwRTQvd3BzLnBvcnRsZXRzLmxvZ2lu/'
[Tue May 26 17:49:46 2009] 00000fb7 0234bb90 - DETAIL: ws_common: websphereEndRequest: Ending the request
[Tue May 26 17:49:46 2009] 00000fb7 0234bb90 - DETAIL: ws_common: websphereShouldHandleRequest: trying to match a route for: vhost='localhost'; uri='/wps/myportal/!ut/p/c5/0wcA1NLTeQ!!/'
[Tue May 26 17:49:46 2009] 00000fb7 0234bb90 - DETAIL: ws_common: websphereBeginRequest: Request is: host='localhost'; uri='/wps/myportal/!ut/p/c5/0wcA1NLTeQ!!/'
[Tue May 26 17:49:46 2009] 00000fb7 0234bb90 - DETAIL: ws_common: websphereFindServerGroup: Setting the server group: PortalCluster; highScore: 6; highExactMatch: 5; affinityCookie: JSESSIONID; affinityURL: jsessionid
[Tue May 26 17:49:46 2009] 00000fb7 0234bb90 - DETAIL: ESI: esiRequestPushUrl: '/wps/myportal/!ut/p/c5/0wcA1NLTeQ!!/'
[Tue May 26 17:49:46 2009] 00000fb7 0234bb90 - DETAIL: ESI: esiRulesGetCacheId: cache miss; no rule for '/wps/myportal/!ut/p/c5/0wcA1NLTeQ!!/'
[Tue May 26 17:49:46 2009] 00000fb7 0234bb90 - STATS: ws_server_group: serverGroupCheckServerStatus: Checking status of portal1_WebSphere_Portal_V1, ignoreWeights 1, markedDown 0, retryNow 0, wlbAllows 0 reachedMaxConnectionsLimit 0
[Tue May 26 17:49:46 2009] 00000fb7 0234bb90 - DETAIL: ws_common: websphereFindTransport: Setting the transport(case 2): dmgr.wpcertification.com on port 10048
[Tue May 26 17:49:46 2009] 00000fb7 0234bb90 - DETAIL: ws_common: websphereGetStream: Using existing stream from transport dmgr.wpcertification.com:10048 queue, socket = 11
[Tue May 26 17:49:46 2009] 00000fb7 0234bb90 - DETAIL: GET /wps/myportal/!ut/p/c5/0wcA1NLTeQ!!/ HTTP/1.1
[Tue May 26 17:49:46 2009] 00000fb7 0234bb90 - DETAIL: Host: localhost
[Tue May 26 17:49:46 2009] 00000fb7 0234bb90 - DETAIL: User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.10) Gecko/2009042718 CentOS/3.0.10-1.el5.centos Firefox/3.0.10
[Tue May 26 17:49:46 2009] 00000fb7 0234bb90 - DETAIL: Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
[Tue May 26 17:49:46 2009] 00000fb7 0234bb90 - DETAIL: Accept-Language: en-us,en;q=0.5
[Tue May 26 17:49:46 2009] 00000fb7 0234bb90 - DETAIL: Accept-Encoding: gzip,deflate
[Tue May 26 17:49:46 2009] 00000fb7 0234bb90 - DETAIL: Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
[Tue May 26 17:49:46 2009] 00000fb7 0234bb90 - DETAIL: Keep-Alive: 300
[Tue May 26 17:49:46 2009] 00000fb7 0234bb90 - DETAIL: Referer: http://localhost/wps/portal
[Tue May 26 17:49:46 2009] 00000fb7 0234bb90 - DETAIL: Cookie: TJE=; TE3=N0:C N1:E N2:C N3:C N4:C N5:C N6:C N7:C N8:C N9:C N10:C N11:C N12:C N13:C N14:C N15:C N16:C N17:C N18:C N19:C N20:C N21:C N22:C N23:C N24:C N25:C N26:C; LtpaToken2=mIQa+HWLCT2FimDmKdD1LkuIKsWQ3OktZyySoCG4FUV5w1UGcd27Xu0FhVInen8YF0FJDntHNwuyTlE8OWHTjixTKExxdpR2s6oRV9ORwMCmhyucLZzibfo2JHM7jfX8ONT/QY2j774kAveeSUeWlSncJUzWgfHpRdB+0WYkdpsXIg2vR02kzE+X1Etdoryl/4m2yTIt3qcVv/UnMFIp9mt0APxjTyK6acyQfZr0S7XSH0uSU86UzOiQGEjODHzCdQSplts9F+LBzomeLWTu04vtJSIByR57dnYNYnar+mhGgitnv0nSiH26gQBTRDjgI6PQRPVJCLYgetduYGGUBGMpFa7NxBfMMxbF1K81uK3zpWi5chspMrpzxProLVje7waVfPZlcii1Q+NO2ACwD1pKYqRQmkS0P9U8cAGMKU6YNLX1gKDZW/2mkotVkjLedhQgNo3mf/JAkZORGG9Vz6pL+DWsT4o4179EsZbXGtKS0eohPE8YI6nw0ngTTXdNQ/k9o2kHvrHCyPYdxnyHbkjVtCpRAcXkb79d46DGaDP3wNJ7WIauNK1DD9omsE0O6bp91FvZmy+HN3EUOXOfbYCyyDL0KMl1rKNq0G1gxMVwY7aAdpnH7XWHkL8a/XQhoaU5P1zMoTNIfDa9j3icnicQNnB7wnn+dgpNZ+rQWhrckdZqk7OnXsW43QGg3lSFFbFQZKMr5kOVPmGPPFLuEA==; LtpaToken=UAx8tmo+3t2C4YhDmSxy6C90fwSChm1sf9bPGJKeLkzOk2c6GrnSIe9efRUHASEoguGIQ5kscugvdH8EgoZtbH/WzxiSni5Mt81K0nRqrVFutRlhQyihdUOi1A3QA+TkpB52U168KJK+Af/ivUrs3SfllUnmJBdQNNgpAkRzor3IfPpWA/9CYAwTLENylm30NR8Sw58ogo1O6g3aI0JOpHjQWFyJMe7zEgaK31To3k4rvZoi/DNveNEoU6uMunfs9zxs9i8yJ+4rv2DixYUEING7ZT+QDkz7UhgBzxXksanA5HiBUZm4DozYEtBC+IENx8L7twqAqrXd7c3B6C9rgQMLscIwKIAw; portalFlyoutIsOpen=; portalOpenFlyout=; JSESSIONID=0000Q5jHUZSwpLygvRzlfdLa1Ie:145t799pf
[Tue May 26 17:49:46 2009] 00000fb7 0234bb90 - DETAIL: $WSIS: false
[Tue May 26 17:49:46 2009] 00000fb7 0234bb90 - DETAIL: $WSSC: http
[Tue May 26 17:49:46 2009] 00000fb7 0234bb90 - DETAIL: $WSPR: HTTP/1.1
[Tue May 26 17:49:46 2009] 00000fb7 0234bb90 - DETAIL: $WSRA: 127.0.0.1
[Tue May 26 17:49:46 2009] 00000fb7 0234bb90 - DETAIL: $WSRH: 127.0.0.1
[Tue May 26 17:49:46 2009] 00000fb7 0234bb90 - DETAIL: $WSSN: localhost
[Tue May 26 17:49:46 2009] 00000fb7 0234bb90 - DETAIL: $WSSP: 80
[Tue May 26 17:49:46 2009] 00000fb7 0234bb90 - DETAIL: Surrogate-Capability: WS-ESI="ESI/1.0+"
[Tue May 26 17:49:46 2009] 00000fb7 0234bb90 - DETAIL: _WS_HAPRT_WLMVERSION: -1
[Tue May 26 17:49:46 2009] 00000fb7 0234bb90 - DETAIL: lib_htresponse: htresponseRead: Reading the response: 871181c
[Tue May 26 17:49:54 2009] 00000fb7 0234bb90 - DETAIL: HTTP/1.1 200 OK
[Tue May 26 17:49:54 2009] 00000fb7 0234bb90 - DETAIL: Content-Type: text/html; charset=UTF-8

[Tue May 26 17:49:54 2009] 00000fb7 0234bb90 - DETAIL: IBM-Web2-Location: /wps/myportal/!ut/p/c5/04_SB8K8xLLM9MSSzPy8xBz9CP0os_iwYBfDUG93QwP3MAsDA0_30FAnPzcfQwMDA_1wkA6zeGd3Rw8Tcx8DA3dzI1MDIz_TYCNfxzCgWmOIvAEO4Gig7-eRn5uqX5CdneboqKgIAJDBEFc!/dl3/d3/L2dBISEvZ0FBIS9nQSEh/

[Tue May 26 17:49:54 2009] 00000fb7 0234bb90 - DETAIL: Cache-Control: no-cache

[Tue May 26 17:49:54 2009] 00000fb7 0234bb90 - DETAIL: Expires: Thu, 01 Jan 1970 00:00:00 GMT

[Tue May 26 17:49:54 2009] 00000fb7 0234bb90 - DETAIL: Pragma: no-cache

[Tue May 26 17:49:54 2009] 00000fb7 0234bb90 - DETAIL: Vary: User-Agent,Cookie

[Tue May 26 17:49:54 2009] 00000fb7 0234bb90 - DETAIL: Content-Language: en-US

[Tue May 26 17:49:54 2009] 00000fb7 0234bb90 - DETAIL: Transfer-Encoding: chunked

[Tue May 26 17:49:54 2009] 00000fb7 0234bb90 - DETAIL: Date: Wed, 27 May 2009 00:49:54 GMT

[Tue May 26 17:49:54 2009] 00000fb7 0234bb90 - DETAIL: Server: WebSphere Application Server/6.1

[Tue May 26 17:49:54 2009] 00000fb7 0234bb90 - STATS: ws_server: serverSetFailoverStatus: Server portal1_WebSphere_Portal_V1 : pendingRequests 0 failedRequests 0 affinityRequests 1 totalRequests 2.
[Tue May 26 17:49:55 2009] 00000fb7 0234bb90 - DETAIL: ESI: esiRequestPopUrl: '/wps/myportal/!ut/p/c5/0wcA1NLTeQ!!/'
[Tue May 26 17:49:55 2009] 00000fb7 0234bb90 - DETAIL: ws_common: websphereEndRequest: Ending the request
[Tue May 26 17:49:56 2009] 00000fb7 0234bb90 - DETAIL: ws_common: websphereShouldHandleRequest: trying to match a route for: vhost='localhost'; uri='/searchfeed/myserver/scopes'
[Tue May 26 17:49:56 2009] 00000fb7 0234bb90 - DETAIL: ws_common: websphereBeginRequest: Request is: host='localhost'; uri='/searchfeed/myserver/scopes'
[Tue May 26 17:49:56 2009] 00000fb7 0234bb90 - DETAIL: ws_common: websphereFindServerGroup: Setting the server group: PortalCluster; highScore: 13; highExactMatch: 12; affinityCookie: JSESSIONID; affinityURL: jsessionid
[Tue May 26 17:49:56 2009] 00000fb7 0234bb90 - DETAIL: ESI: esiRequestPushUrl: '/searchfeed/myserver/scopes?locale=en&customLinks=true&timeStamp=1243385067143'
[Tue May 26 17:49:56 2009] 00000fb7 0234bb90 - DETAIL: ESI: esiRulesGetCacheId: cache miss; no rule for '/searchfeed/myserver/scopes'
[Tue May 26 17:49:56 2009] 00000fb7 0234bb90 - STATS: ws_server_group: serverGroupCheckServerStatus: Checking status of portal1_WebSphere_Portal_V1, ignoreWeights 1, markedDown 0, retryNow 0, wlbAllows 0 reachedMaxConnectionsLimit 0
[Tue May 26 17:49:56 2009] 00000fb7 0234bb90 - DETAIL: ws_common: websphereFindTransport: Setting the transport(case 2): dmgr.wpcertification.com on port 10048
[Tue May 26 17:49:56 2009] 00000fb7 0234bb90 - DETAIL: ws_common: websphereGetStream: Using existing stream from transport dmgr.wpcertification.com:10048 queue, socket = 11
[Tue May 26 17:49:56 2009] 00000fb7 0234bb90 - DETAIL: GET /searchfeed/myserver/scopes?locale=en&customLinks=true&timeStamp=1243385067143 HTTP/1.1
[Tue May 26 17:49:56 2009] 00000fb7 0234bb90 - DETAIL: Host: localhost
[Tue May 26 17:49:56 2009] 00000fb7 0234bb90 - DETAIL: User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.10) Gecko/2009042718 CentOS/3.0.10-1.el5.centos Firefox/3.0.10
[Tue May 26 17:49:56 2009] 00000fb7 0234bb90 - DETAIL: Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
[Tue May 26 17:49:56 2009] 00000fb7 0234bb90 - DETAIL: Accept-Language: en-us,en;q=0.5
[Tue May 26 17:49:56 2009] 00000fb7 0234bb90 - DETAIL: Accept-Encoding: gzip,deflate
[Tue May 26 17:49:56 2009] 00000fb7 0234bb90 - DETAIL: Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
[Tue May 26 17:49:56 2009] 00000fb7 0234bb90 - DETAIL: Keep-Alive: 300
[Tue May 26 17:49:56 2009] 00000fb7 0234bb90 - DETAIL: Content-Type: application/x-www-form-urlencoded
[Tue May 26 17:49:56 2009] 00000fb7 0234bb90 - DETAIL: X-Requested-With: XMLHttpRequest
[Tue May 26 17:49:56 2009] 00000fb7 0234bb90 - DETAIL: Referer: http://localhost/wps/myportal/!ut/p/c5/0wcA1NLTeQ!!/
[Tue May 26 17:49:56 2009] 00000fb7 0234bb90 - DETAIL: Cookie: TJE=; TE3=N0:C N1:E N2:C N3:C N4:C N5:C N6:C N7:C N8:C N9:C N10:C N11:C N12:C N13:C N14:C N15:C N16:C N17:C N18:C N19:C N20:C N21:C N22:C N23:C N24:C N25:C N26:C; LtpaToken2=mIQa+HWLCT2FimDmKdD1LkuIKsWQ3OktZyySoCG4FUV5w1UGcd27Xu0FhVInen8YF0FJDntHNwuyTlE8OWHTjixTKExxdpR2s6oRV9ORwMCmhyucLZzibfo2JHM7jfX8ONT/QY2j774kAveeSUeWlSncJUzWgfHpRdB+0WYkdpsXIg2vR02kzE+X1Etdoryl/4m2yTIt3qcVv/UnMFIp9mt0APxjTyK6acyQfZr0S7XSH0uSU86UzOiQGEjODHzCdQSplts9F+LBzomeLWTu04vtJSIByR57dnYNYnar+mhGgitnv0nSiH26gQBTRDjgI6PQRPVJCLYgetduYGGUBGMpFa7NxBfMMxbF1K81uK3zpWi5chspMrpzxProLVje7waVfPZlcii1Q+NO2ACwD1pKYqRQmkS0P9U8cAGMKU6YNLX1gKDZW/2mkotVkjLedhQgNo3mf/JAkZORGG9Vz6pL+DWsT4o4179EsZbXGtKS0eohPE8YI6nw0ngTTXdNQ/k9o2kHvrHCyPYdxnyHbkjVtCpRAcXkb79d46DGaDP3wNJ7WIauNK1DD9omsE0O6bp91FvZmy+HN3EUOXOfbYCyyDL0KMl1rKNq0G1gxMVwY7aAdpnH7XWHkL8a/XQhoaU5P1zMoTNIfDa9j3icnicQNnB7wnn+dgpNZ+rQWhrckdZqk7OnXsW43QGg3lSFFbFQZKMr5kOVPmGPPFLuEA==; LtpaToken=UAx8tmo+3t2C4YhDmSxy6C90fwSChm1sf9bPGJKeLkzOk2c6GrnSIe9efRUHASEoguGIQ5kscugvdH8EgoZtbH/WzxiSni5Mt81K0nRqrVFutRlhQyihdUOi1A3QA+TkpB52U168KJK+Af/ivUrs3SfllUnmJBdQNNgpAkRzor3IfPpWA/9CYAwTLENylm30NR8Sw58ogo1O6g3aI0JOpHjQWFyJMe7zEgaK31To3k4rvZoi/DNveNEoU6uMunfs9zxs9i8yJ+4rv2DixYUEING7ZT+QDkz7UhgBzxXksanA5HiBUZm4DozYEtBC+IENx8L7twqAqrXd7c3B6C9rgQMLscIwKIAw; portalFlyoutIsOpen=; portalOpenFlyout=; JSESSIONID=0000Q5jHUZSwpLygvRzlfdLa1Ie:145t799pf
[Tue May 26 17:49:56 2009] 00000fb7 0234bb90 - DETAIL: $WSIS: false
[Tue May 26 17:49:56 2009] 00000fb7 0234bb90 - DETAIL: $WSSC: http
[Tue May 26 17:49:56 2009] 00000fb7 0234bb90 - DETAIL: $WSPR: HTTP/1.1
[Tue May 26 17:49:56 2009] 00000fb7 0234bb90 - DETAIL: $WSRA: 127.0.0.1
[Tue May 26 17:49:56 2009] 00000fb7 0234bb90 - DETAIL: $WSRH: 127.0.0.1
[Tue May 26 17:49:56 2009] 00000fb7 0234bb90 - DETAIL: $WSSN: localhost
[Tue May 26 17:49:56 2009] 00000fb7 0234bb90 - DETAIL: $WSSP: 80
[Tue May 26 17:49:56 2009] 00000fb7 0234bb90 - DETAIL: Surrogate-Capability: WS-ESI="ESI/1.0+"
[Tue May 26 17:49:56 2009] 00000fb7 0234bb90 - DETAIL: _WS_HAPRT_WLMVERSION: -1
[Tue May 26 17:49:56 2009] 00000fb7 0234bb90 - DETAIL: lib_htresponse: htresponseRead: Reading the response: 8711804
[Tue May 26 17:49:59 2009] 00000fb7 0234bb90 - DETAIL: HTTP/1.1 200 OK
[Tue May 26 17:49:59 2009] 00000fb7 0234bb90 - DETAIL: Cache-Control: private, max-age=86400

[Tue May 26 17:49:59 2009] 00000fb7 0234bb90 - DETAIL: Content-Type: application/atom+xml

[Tue May 26 17:49:59 2009] 00000fb7 0234bb90 - DETAIL: Content-Language: en-US

[Tue May 26 17:49:59 2009] 00000fb7 0234bb90 - DETAIL: Transfer-Encoding: chunked

[Tue May 26 17:49:59 2009] 00000fb7 0234bb90 - DETAIL: Date: Wed, 27 May 2009 00:49:59 GMT

[Tue May 26 17:49:59 2009] 00000fb7 0234bb90 - DETAIL: Server: WebSphere Application Server/6.1

[Tue May 26 17:49:59 2009] 00000fb7 0234bb90 - STATS: ws_server: serverSetFailoverStatus: Server portal1_WebSphere_Portal_V1 : pendingRequests 0 failedRequests 0 affinityRequests 2 totalRequests 3.
[Tue May 26 17:49:59 2009] 00000fb7 0234bb90 - DETAIL: ESI: esiRequestPopUrl: '/searchfeed/myserver/scopes?locale=en&customLinks=true&timeStamp=1243385067143'
[Tue May 26 17:49:59 2009] 00000fb7 0234bb90 - DETAIL: ws_common: websphereEndRequest: Ending the request
[Tue May 26 18:37:28 2009] 00000fb5 0374db90 - DETAIL: ws_common: websphereShouldHandleRequest: trying to match a route for: vhost='localhost'; uri='/wps/themes/html/Portal/colors/default/menu_selected_disabled.gif'
[Tue May 26 18:37:28 2009] 00000fb5 0374db90 - DETAIL: ws_common: websphereBeginRequest: Request is: host='localhost'; uri='/wps/themes/html/Portal/colors/default/menu_selected_disabled.gif'
[Tue May 26 18:37:28 2009] 00000fb5 0374db90 - DETAIL: ws_common: websphereFindServerGroup: Setting the server group: PortalCluster; highScore: 6; highExactMatch: 5; affinityCookie: JSESSIONID; affinityURL: jsessionid
[Tue May 26 18:37:28 2009] 00000fb5 0374db90 - DETAIL: ESI: esiRequestPushUrl: '/wps/themes/html/Portal/colors/default/menu_selected_disabled.gif'
[Tue May 26 18:37:28 2009] 00000fb5 0374db90 - DETAIL: ESI: esiRulesGetCacheId: cache miss; no rule for '/wps/themes/html/Portal/colors/default/menu_selected_disabled.gif'
[Tue May 26 18:37:28 2009] 00000fb5 0374db90 - STATS: ws_server_group: serverGroupCheckServerStatus: Checking status of portal1_WebSphere_Portal_V1, ignoreWeights 1, markedDown 0, retryNow 0, wlbAllows 1 reachedMaxConnectionsLimit 0
[Tue May 26 18:37:28 2009] 00000fb5 0374db90 - DETAIL: ws_common: websphereFindTransport: Setting the transport(case 2): dmgr.wpcertification.com on port 10048
[Tue May 26 18:37:28 2009] 00000fb5 0374db90 - DETAIL: ws_common: websphereGetStream: Created a new stream; queue was empty, socket = 11
[Tue May 26 18:37:28 2009] 00000fb5 0374db90 - DETAIL: GET /wps/themes/html/Portal/colors/default/menu_selected_disabled.gif HTTP/1.1
[Tue May 26 18:37:28 2009] 00000fb5 0374db90 - DETAIL: Host: localhost
[Tue May 26 18:37:28 2009] 00000fb5 0374db90 - DETAIL: User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.10) Gecko/2009042718 CentOS/3.0.10-1.el5.centos Firefox/3.0.10
[Tue May 26 18:37:28 2009] 00000fb5 0374db90 - DETAIL: Accept: image/png,image/*;q=0.8,*/*;q=0.5
[Tue May 26 18:37:28 2009] 00000fb5 0374db90 - DETAIL: Accept-Language: en-us,en;q=0.5
[Tue May 26 18:37:28 2009] 00000fb5 0374db90 - DETAIL: Accept-Encoding: gzip,deflate
[Tue May 26 18:37:28 2009] 00000fb5 0374db90 - DETAIL: Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
[Tue May 26 18:37:28 2009] 00000fb5 0374db90 - DETAIL: Keep-Alive: 300
[Tue May 26 18:37:28 2009] 00000fb5 0374db90 - DETAIL: Referer: http://localhost/wps/myportal/!ut/p/c5/0wcA1NLTeQ!!/
[Tue May 26 18:37:28 2009] 00000fb5 0374db90 - DETAIL: Cookie: TJE=; TE3=N0:C N1:E N2:C N3:C N4:C N5:C N6:C N7:C N8:C N9:C N10:C N11:C N12:C N13:C N14:C N15:C N16:C N17:C N18:C N19:C N20:C N21:C N22:C N23:C N24:C N25:C N26:C; LtpaToken2=mIQa+HWLCT2FimDmKdD1LkuIKsWQ3OktZyySoCG4FUV5w1UGcd27Xu0FhVInen8YF0FJDntHNwuyTlE8OWHTjixTKExxdpR2s6oRV9ORwMCmhyucLZzibfo2JHM7jfX8ONT/QY2j774kAveeSUeWlSncJUzWgfHpRdB+0WYkdpsXIg2vR02kzE+X1Etdoryl/4m2yTIt3qcVv/UnMFIp9mt0APxjTyK6acyQfZr0S7XSH0uSU86UzOiQGEjODHzCdQSplts9F+LBzomeLWTu04vtJSIByR57dnYNYnar+mhGgitnv0nSiH26gQBTRDjgI6PQRPVJCLYgetduYGGUBGMpFa7NxBfMMxbF1K81uK3zpWi5chspMrpzxProLVje7waVfPZlcii1Q+NO2ACwD1pKYqRQmkS0P9U8cAGMKU6YNLX1gKDZW/2mkotVkjLedhQgNo3mf/JAkZORGG9Vz6pL+DWsT4o4179EsZbXGtKS0eohPE8YI6nw0ngTTXdNQ/k9o2kHvrHCyPYdxnyHbkjVtCpRAcXkb79d46DGaDP3wNJ7WIauNK1DD9omsE0O6bp91FvZmy+HN3EUOXOfbYCyyDL0KMl1rKNq0G1gxMVwY7aAdpnH7XWHkL8a/XQhoaU5P1zMoTNIfDa9j3icnicQNnB7wnn+dgpNZ+rQWhrckdZqk7OnXsW43QGg3lSFFbFQZKMr5kOVPmGPPFLuEA==; LtpaToken=UAx8tmo+3t2C4YhDmSxy6C90fwSChm1sf9bPGJKeLkzOk2c6GrnSIe9efRUHASEoguGIQ5kscugvdH8EgoZtbH/WzxiSni5Mt81K0nRqrVFutRlhQyihdUOi1A3QA+TkpB52U168KJK+Af/ivUrs3SfllUnmJBdQNNgpAkRzor3IfPpWA/9CYAwTLENylm30NR8Sw58ogo1O6g3aI0JOpHjQWFyJMe7zEgaK31To3k4rvZoi/DNveNEoU6uMunfs9zxs9i8yJ+4rv2DixYUEING7ZT+QDkz7UhgBzxXksanA5HiBUZm4DozYEtBC+IENx8L7twqAqrXd7c3B6C9rgQMLscIwKIAw; portalFlyoutIsOpen=; portalOpenFlyout=; JSESSIONID=0000Q5jHUZSwpLygvRzlfdLa1Ie:145t799pf
[Tue May 26 18:37:28 2009] 00000fb5 0374db90 - DETAIL: $WSIS: false
[Tue May 26 18:37:28 2009] 00000fb5 0374db90 - DETAIL: $WSSC: http
[Tue May 26 18:37:28 2009] 00000fb5 0374db90 - DETAIL: $WSPR: HTTP/1.1
[Tue May 26 18:37:28 2009] 00000fb5 0374db90 - DETAIL: $WSRA: 127.0.0.1
[Tue May 26 18:37:28 2009] 00000fb5 0374db90 - DETAIL: $WSRH: 127.0.0.1
[Tue May 26 18:37:28 2009] 00000fb5 0374db90 - DETAIL: $WSSN: localhost
[Tue May 26 18:37:28 2009] 00000fb5 0374db90 - DETAIL: $WSSP: 80
[Tue May 26 18:37:28 2009] 00000fb5 0374db90 - DETAIL: Surrogate-Capability: WS-ESI="ESI/1.0+"
[Tue May 26 18:37:28 2009] 00000fb5 0374db90 - DETAIL: _WS_HAPRT_WLMVERSION: -1
[Tue May 26 18:37:28 2009] 00000fb5 0374db90 - DETAIL: lib_htresponse: htresponseRead: Reading the response: 8711874
[Tue May 26 18:37:28 2009] 00000fb5 058f4b90 - DETAIL: ws_common: websphereShouldHandleRequest: trying to match a route for: vhost='localhost'; uri='/wps/themes/html/Portal/colors/default/menu_selected.gif'
[Tue May 26 18:37:28 2009] 00000fb5 058f4b90 - DETAIL: ws_common: websphereBeginRequest: Request is: host='localhost'; uri='/wps/themes/html/Portal/colors/default/menu_selected.gif'
[Tue May 26 18:37:28 2009] 00000fb5 058f4b90 - DETAIL: ws_common: websphereFindServerGroup: Setting the server group: PortalCluster; highScore: 6; highExactMatch: 5; affinityCookie: JSESSIONID; affinityURL: jsessionid
[Tue May 26 18:37:28 2009] 00000fb5 058f4b90 - DETAIL: ESI: esiRequestPushUrl: '/wps/themes/html/Portal/colors/default/menu_selected.gif'
[Tue May 26 18:37:28 2009] 00000fb5 058f4b90 - DETAIL: ESI: esiRulesGetCacheId: cache miss; no rule for '/wps/themes/html/Portal/colors/default/menu_selected.gif'
[Tue May 26 18:37:28 2009] 00000fb5 058f4b90 - STATS: ws_server_group: serverGroupCheckServerStatus: Checking status of portal1_WebSphere_Portal_V1, ignoreWeights 1, markedDown 0, retryNow 0, wlbAllows 1 reachedMaxConnectionsLimit 0
[Tue May 26 18:37:28 2009] 00000fb5 058f4b90 - DETAIL: ws_common: websphereFindTransport: Setting the transport(case 2): dmgr.wpcertification.com on port 10048
[Tue May 26 18:37:28 2009] 00000fb5 058f4b90 - DETAIL: ws_common: websphereGetStream: Created a new stream; queue was empty, socket = 13
[Tue May 26 18:37:28 2009] 00000fb5 058f4b90 - DETAIL: GET /wps/themes/html/Portal/colors/default/menu_selected.gif HTTP/1.1
[Tue May 26 18:37:28 2009] 00000fb5 058f4b90 - DETAIL: Host: localhost
[Tue May 26 18:37:28 2009] 00000fb5 058f4b90 - DETAIL: User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.10) Gecko/2009042718 CentOS/3.0.10-1.el5.centos Firefox/3.0.10
[Tue May 26 18:37:28 2009] 00000fb5 058f4b90 - DETAIL: Accept: image/png,image/*;q=0.8,*/*;q=0.5
[Tue May 26 18:37:28 2009] 00000fb5 058f4b90 - DETAIL: Accept-Language: en-us,en;q=0.5
[Tue May 26 18:37:28 2009] 00000fb5 058f4b90 - DETAIL: Accept-Encoding: gzip,deflate
[Tue May 26 18:37:28 2009] 00000fb5 058f4b90 - DETAIL: Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
[Tue May 26 18:37:28 2009] 00000fb5 058f4b90 - DETAIL: Keep-Alive: 300
[Tue May 26 18:37:28 2009] 00000fb5 058f4b90 - DETAIL: Referer: http://localhost/wps/myportal/!ut/p/c5/0wcA1NLTeQ!!/
[Tue May 26 18:37:28 2009] 00000fb5 058f4b90 - DETAIL: Cookie: TJE=; TE3=N0:C N1:E N2:C N3:C N4:C N5:C N6:C N7:C N8:C N9:C N10:C N11:C N12:C N13:C N14:C N15:C N16:C N17:C N18:C N19:C N20:C N21:C N22:C N23:C N24:C N25:C N26:C; LtpaToken2=mIQa+HWLCT2FimDmKdD1LkuIKsWQ3OktZyySoCG4FUV5w1UGcd27Xu0FhVInen8YF0FJDntHNwuyTlE8OWHTjixTKExxdpR2s6oRV9ORwMCmhyucLZzibfo2JHM7jfX8ONT/QY2j774kAveeSUeWlSncJUzWgfHpRdB+0WYkdpsXIg2vR02kzE+X1Etdoryl/4m2yTIt3qcVv/UnMFIp9mt0APxjTyK6acyQfZr0S7XSH0uSU86UzOiQGEjODHzCdQSplts9F+LBzomeLWTu04vtJSIByR57dnYNYnar+mhGgitnv0nSiH26gQBTRDjgI6PQRPVJCLYgetduYGGUBGMpFa7NxBfMMxbF1K81uK3zpWi5chspMrpzxProLVje7waVfPZlcii1Q+NO2ACwD1pKYqRQmkS0P9U8cAGMKU6YNLX1gKDZW/2mkotVkjLedhQgNo3mf/JAkZORGG9Vz6pL+DWsT4o4179EsZbXGtKS0eohPE8YI6nw0ngTTXdNQ/k9o2kHvrHCyPYdxnyHbkjVtCpRAcXkb79d46DGaDP3wNJ7WIauNK1DD9omsE0O6bp91FvZmy+HN3EUOXOfbYCyyDL0KMl1rKNq0G1gxMVwY7aAdpnH7XWHkL8a/XQhoaU5P1zMoTNIfDa9j3icnicQNnB7wnn+dgpNZ+rQWhrckdZqk7OnXsW43QGg3lSFFbFQZKMr5kOVPmGPPFLuEA==; LtpaToken=UAx8tmo+3t2C4YhDmSxy6C90fwSChm1sf9bPGJKeLkzOk2c6GrnSIe9efRUHASEoguGIQ5kscugvdH8EgoZtbH/WzxiSni5Mt81K0nRqrVFutRlhQyihdUOi1A3QA+TkpB52U168KJK+Af/ivUrs3SfllUnmJBdQNNgpAkRzor3IfPpWA/9CYAwTLENylm30NR8Sw58ogo1O6g3aI0JOpHjQWFyJMe7zEgaK31To3k4rvZoi/DNveNEoU6uMunfs9zxs9i8yJ+4rv2DixYUEING7ZT+QDkz7UhgBzxXksanA5HiBUZm4DozYEtBC+IENx8L7twqAqrXd7c3B6C9rgQMLscIwKIAw; portalFlyoutIsOpen=; portalOpenFlyout=; JSESSIONID=0000Q5jHUZSwpLygvRzlfdLa1Ie:145t799pf
[Tue May 26 18:37:28 2009] 00000fb5 058f4b90 - DETAIL: $WSIS: false
[Tue May 26 18:37:28 2009] 00000fb5 058f4b90 - DETAIL: $WSSC: http
[Tue May 26 18:37:28 2009] 00000fb5 058f4b90 - DETAIL: $WSPR: HTTP/1.1
[Tue May 26 18:37:28 2009] 00000fb5 058f4b90 - DETAIL: $WSRA: 127.0.0.1
[Tue May 26 18:37:28 2009] 00000fb5 058f4b90 - DETAIL: $WSRH: 127.0.0.1
[Tue May 26 18:37:28 2009] 00000fb5 058f4b90 - DETAIL: $WSSN: localhost
[Tue May 26 18:37:28 2009] 00000fb5 058f4b90 - DETAIL: $WSSP: 80
[Tue May 26 18:37:28 2009] 00000fb5 058f4b90 - DETAIL: Surrogate-Capability: WS-ESI="ESI/1.0+"
[Tue May 26 18:37:28 2009] 00000fb5 058f4b90 - DETAIL: _WS_HAPRT_WLMVERSION: -1
[Tue May 26 18:37:28 2009] 00000fb5 058f4b90 - DETAIL: lib_htresponse: htresponseRead: Reading the response: 8727914
[Tue May 26 18:37:28 2009] 00000fb5 0374db90 - DETAIL: HTTP/1.1 200 OK
[Tue May 26 18:37:28 2009] 00000fb5 0374db90 - DETAIL: Date: Wed, 27 May 2009 01:37:28 GMT

[Tue May 26 18:37:28 2009] 00000fb5 0374db90 - DETAIL: Content-Type: image/gif

[Tue May 26 18:37:28 2009] 00000fb5 0374db90 - DETAIL: Surrogate-Control: max-age=300,cacheid="URL",content="ESI/1.0+"

[Tue May 26 18:37:28 2009] 00000fb5 0374db90 - DETAIL: Last-Modified: Sun, 17 May 2009 16:55:19 GMT

[Tue May 26 18:37:28 2009] 00000fb5 0374db90 - DETAIL: Content-Length: 825

[Tue May 26 18:37:28 2009] 00000fb5 0374db90 - DETAIL: Content-Language: en-US

[Tue May 26 18:37:28 2009] 00000fb5 0374db90 - DETAIL: Server: WebSphere Application Server/6.1

[Tue May 26 18:37:28 2009] 00000fb5 0374db90 - STATS: ws_server: serverSetFailoverStatus: Server portal1_WebSphere_Portal_V1 : pendingRequests 1 failedRequests 1 affinityRequests 2 totalRequests 3.
[Tue May 26 18:37:29 2009] 00000fb5 0374db90 - DETAIL: ESI: esiRulesAddAndGetCacheId: URL = '/wps/themes/html/Portal/colors/default/menu_selected_disabled.gif', rules = 'URL'
[Tue May 26 18:37:29 2009] 00000fb5 0374db90 - DETAIL: ESI: esiRulesAddAndGetCacheId: cache id is 'GET_http_/wps/themes/html/Portal/colors/default/menu_selected_disabled.gif'
[Tue May 26 18:37:29 2009] 00000fb5 0374db90 - DETAIL: ESI: esiResponseWrite: Total response size 825
[Tue May 26 18:37:29 2009] 00000fb5 0374db90 - DETAIL: ESI: esiRequestPopUrl: '/wps/themes/html/Portal/colors/default/menu_selected_disabled.gif'
[Tue May 26 18:37:29 2009] 00000fb5 0374db90 - DETAIL: ws_common: websphereEndRequest: Ending the request
[Tue May 26 18:37:29 2009] 00000fb5 058f4b90 - DETAIL: HTTP/1.1 200 OK
[Tue May 26 18:37:29 2009] 00000fb5 058f4b90 - DETAIL: Date: Wed, 27 May 2009 01:37:28 GMT

[Tue May 26 18:37:29 2009] 00000fb5 058f4b90 - DETAIL: Content-Type: image/gif

[Tue May 26 18:37:29 2009] 00000fb5 058f4b90 - DETAIL: Surrogate-Control: max-age=300,cacheid="URL",content="ESI/1.0+"

[Tue May 26 18:37:29 2009] 00000fb5 058f4b90 - DETAIL: Last-Modified: Sun, 17 May 2009 16:55:19 GMT

[Tue May 26 18:37:29 2009] 00000fb5 058f4b90 - DETAIL: Content-Length: 57

[Tue May 26 18:37:29 2009] 00000fb5 058f4b90 - DETAIL: Content-Language: en-US

[Tue May 26 18:37:29 2009] 00000fb5 058f4b90 - DETAIL: Server: WebSphere Application Server/6.1

[Tue May 26 18:37:29 2009] 00000fb5 058f4b90 - STATS: ws_server: serverSetFailoverStatus: Server portal1_WebSphere_Portal_V1 : pendingRequests 0 failedRequests 1 affinityRequests 2 totalRequests 3.
[Tue May 26 18:37:29 2009] 00000fb5 058f4b90 - DETAIL: ESI: esiRulesAddAndGetCacheId: URL = '/wps/themes/html/Portal/colors/default/menu_selected.gif', rules = 'URL'
[Tue May 26 18:37:29 2009] 00000fb5 058f4b90 - DETAIL: ESI: esiRulesAddAndGetCacheId: cache id is 'GET_http_/wps/themes/html/Portal/colors/default/menu_selected.gif'
[Tue May 26 18:37:29 2009] 00000fb5 058f4b90 - DETAIL: ESI: esiResponseWrite: Total response size 57
[Tue May 26 18:37:29 2009] 00000fb5 058f4b90 - DETAIL: ESI: esiRequestPopUrl: '/wps/themes/html/Portal/colors/default/menu_selected.gif'
[Tue May 26 18:37:29 2009] 00000fb5 058f4b90 - DETAIL: ws_common: websphereEndRequest: Ending the request

Enabling NCSA loggin

If you want to view all the Http requests that your application server is handling then you can enable the NCSA log. This might be useful in case of clustered environment to see which server is actually handling particular request or you might be interested in looking at who is accessing the server at what time, load on the server during particular time

You can enable the National Center for Supercomputing Application(NCSA) using WAS Admin Console. Follow these steps

  • Log into the WAs ADMIn console

  • Go to the Servers -> Application Server. Select the server where you want to turn the NCSA log on. On the server details page click on HTTP error and NCSA access logging



  • On the "HTTP error and NCSA access logging" page, check the Enable access logging and Enable Service at startup check boxes. ALso set the Error log level

  • Now go back to the Application server details page, and go to Web Container transportation chain -> WCInboundDefault ->HTTP Inbound channel. Check the Enable access and error logging check box

  • Save your changes and restart the application server.

  • Now when you access the Application server every request would be logged in http_access.log file in the log folder of your application server.



This is sample log in my http_access.log file on websphere portal server

192.168.163.129 - - [26/May/2009:17:49:20 -0700] "GET /wps/portal HTTP/1.1" 200 16671
192.168.163.129 - - [26/May/2009:17:49:46 -0700] "POST /wps/portal/!ut/p/c5/04_SB8K8xLLM9MSSzPy8xBz9CP0os3hnd0cPE3MfAwN3cyNTAyM_02AjX8cwAwNfQ_1wkA6zeAMcwNFA388jPzdVvyA7rxwAZPfmsg!!/dl3/d3/L0lDUWtpQ1NTUW9LVVFBISEvb0lvZ0FFQ1FRREdJUXBURE9DNEpuQSEhLzRDd2lSLXJmbTE2SWt5WGlnRUEhLzdfQ0dBSDQ3TDAwRzcyNTAyTjVTMk1BVjAwRTQvd3BzLnBvcnRsZXRzLmxvZ2lu/ HTTP/1.1" 302 5
192.168.163.129 - - [26/May/2009:17:49:54 -0700] "GET /wps/myportal/!ut/p/c5/0wcA1NLTeQ!!/ HTTP/1.1" 200 49479
192.168.163.129 - - [26/May/2009:17:49:59 -0700] "GET /searchfeed/myserver/scopes?locale=en&customLinks=true&timeStamp=1243385067143 HTTP/1.1" 200 925
192.168.163.129 - - [26/May/2009:18:37:28 -0700] "GET /wps/themes/html/Portal/colors/default/menu_selected_disabled.gif HTTP/1.1" 200 825
192.168.163.129 - - [26/May/2009:18:37:28 -0700] "GET /wps/themes/html/Portal/colors/default/menu_selected.gif HTTP/1.1" 200 57

How to set up dynamic cluster

Follow the Creating a new dynamic cluster on Windows using WebSphere Virtual Enterprise document to setup a dynamic cluster.


The basic steps for creating dynamic cluster can be divided into three basic parts

Preparing DM and portal nodes



  • Install and configure Deployment Manager

  • Install WAS XD and augment the deployment manager profile

  • On all the WebSphere portal servers install WAS XD and augment wp_profile



Adding Primary Node to dynamic cluster




  • Prepare the node for dynamic cluster

    • cluster-node-config-pre-federation

    • Optional tasks if your using look aside repository or database backed user registry

    • cluster-node-config-post-federation

    • wp-change-portal-admin-user



  • Login into WAS Admin consol and Create a node group

  • Add primary portal server as node to the node group

  • Create Dynamic cluster, in the WAS Admin Console

  • Create the primary portal node as cluster member using the existing server as a template

  • Create dynamic cluster by executing cluster-node-config-dynamic-cluster-setup config task

  • Access the web content management content:Set the WCM_HOST and WCM_POST websphere variables at the primary portal server level to point to the web server host name and port name through which the WCM content would be accessed.

  • Propogate the changes

    • Stop DM

    • Stop Node Agent

    • Stop Server 1

    • Stop portal server

    • start DM

    • Start Node

    • Start Server1

    • Start WebSPhere POrtal





Adding additional nodes to the cluster




  • Setup the wkplc_dbtype.properties and wkplc_comp.properties files to point to correct database

  • Validate database settings by executing

    • validate-database-driver

    • validate-database-connections



  • Prepare the node for dynamic cluster

    • cluster-node-config-pre-federation

    • Optional tasks if your using look aside repository or database backed user registry

    • cluster-node-config-post-federation

    • wp-change-portal-admin-user



  • Login into the WAS Admin Console and add the new node as new member to the node group

  • Run the ConfigEngine.bat/sh cluster-node-config-dynamic-cluster-setup task to add the new member to the existing dynamic cluster

  • Apply changes by restarting the server

  • Access the Web Content Management Content by changing value of WCM_HOST and WCM_PORT websphere variables at the nodename scope to indicate the name of the fully qualified host and port

What is a dynamic Cluster

You can create IBM WebSphere Application Server Extended Deployment dynamic cluster to run IBM WebSphere Portal server. The Dynamic cluster is one of the features provided by WAS - Extended Deployment, to support dynamic operations.

The WebSphere XD has a concept of service policy that lets you define performance goals for application or actually application URL and decide the priority of application compaired to other applications.

Once an administrator has classified an application's URLs and connected them to service policies, then the ODR will begin examining incoming URLs and compare their performance characteristics to the goals defined in the corresponding service policies. The ODR can accomplish this through several mechanisms:

  • The ODR can use its knowledge of the relative priorities of different service policies to determine how long requests will wait in its own internal queues; higher-priority requests will receive service before lower-priority requests when resources are constrained.

  • The ODR can use its knowledge of response time goals and the relative availability of resources (CPU and memory) in different cluster members to perform dynamic workload management.

  • The ODR can work with other parts of the system to achieve dynamic placement within a dynamic cluster. Dynamic placement is a facility in which the XD autonomic managers determine how many copies of an application server are appropriate within a cluster. For example, if one application is performing well above its performance goals, while another higher- or equal-priority application is underperforming because the machines running clustered copies of that application are overtaxed, then the autonomic manager can decide to adjust the number of application servers running each application, decreasing the former and increasing the latter to improve the performance of the overwhelmed application.


Some of these facilities (such as the classification and queuing of work) are entirely the function of the ODR, and will work well with many different types of workloads. Others (such as dynamic placement) are more specialized and work in concert with other parts of the XD system. All require require the use of the ODR, which is the fundamental architectural linchpin of XD, and possesses still more features.

How load balancing works in case of WebSphere XD/ virtual enterprise

The On Demad Router (ODR) is an intelligent routing engine that forms the core of WebSphere XD topologies. It does many of the features of Web Server Plugin and the edge-of-network routers, like the WebSphere Edge Server Network dispatcher

It does following things


  • Listens for requests that it proxies
  • Classifies the requests according to the configuration of the WebSphere application server for which the requests are destined

  • Prioritizes the requests and uses its knowledge of the status of the application server to determine which application server the request should be routed to

  • Queues and issues the requests to the application server that are servicing them according to the weigth assigned to the request

  • Communicates with other parts of the XD system to coordinate its activities with other components and driver features like dynamic placement






There are a number of advantages that this topology holds over the standard Network Deployment topology:


  • Here, the routing information used by the ODR is dynamic; as applications are deployed (or undeployed), or as application servers are added to (or removed from) the topology, the ODR routing tables are automatically updated. This eliminates the need for a plugin-xml.cfg file, since the communication between the deployment manager and the ODR is direct.

  • Since the ODR is an active application server instance in its own right, it can use additional information from the application servers to make dynamic determinations about request weighting.

Configure external remote web server

I followed the steps defined in A Step-By-Step Guide to Configuring a WebSphere Portal v6.1.0.0 Cluster using WebSphere Application Server v6.1.0.15 document for setting up external web server. In short these are the steps that i had to follow


  • Install HTTP Server and HTTP Server plugin on same VMWare

  • Navigate to /bin and find the configurewebservername.bat script where webservername is the web server definition name you defined on step 8. In this case, we used webserver1 so our script is called: configurewebserver1.bat

  • Copy the configurewebserver1.bat script from the <plugin root>/bin directory to the <dmgr_profile>/bin directory on your Deployment Manager server.

  • Ensure that the DMGR is running.

  • . In a command line from the <dmgr_profile>/bin directory, run the following command: configurewebserver1.bat -user <was_admin_user> -password password. This script will create the web server definition in the DMGR configuration and map all of the installed applications to the web server.

  • 15. Regenerate the web server plugin by performing the following steps:
    1. Login to the DMGR Admin Console
    2. Navigate to Servers -> Web Servers
    3. Select the Checkbox for the new web server definition
    4. Click the “Generate Plug-in” button
    NOTE: This will be written to the
    <dmgr_profile>/config/cells/<cellname>/nodes/<nodename>/servers/webserver1/plugincfg.
    xml file.

  • Copy the plugin-cfg.xml file to the remote web server at the following directory, overwriting the existing one: <plugin_root>/config/webserver1

  • Restart the DMGR, web server, and cluster.

  • Verify that you can access the Portal cluster via the web server

How load balancing works in case of static cluster

In case of WebSphere Application/Portla Server Network deployment edition the Http Server is used for the load balancing. This diagram shows how the load balancing works.



As you can see requests are initially directed to an IP sprayer. The requests are then forwarded to the layer of Web Server, where the WebSphere Plug-in evaluates each request to figure out whether the request should be forwarded to the application server layer by matching the incoming request URL with the URLs defined in plugin-cfg.xml to check if there is a match if yes forward request to the application server.

The Plugin can be configured to use either of two load balancing algorithams for routing request


  • Round robin: The Round Robin is the default load balancing algorithm, in this case when the first request arrives the server is selected on random basis but thereafter the requests are forwarded on round robin basis.This implementation ensures that in multiple process based Web servers, all of the processes don't start up by sending the first request to the same Application Server.

  • Random:In case of Random load balancing the requests would be forwarded to the application servers on random basis



Important Note: Every time a web server receives a client request it checks if this is a new request or old request by checking for either session cookie or session url.If it finds a session it will forward request to the same application server which was used to handle previous request for the client. This is called session affinity

If you want you can change the load balancing algorithm from the WAS console by going to Server -> Web Servers -> Plug-in properties -> Request routing and changing value of Load balancing option



As you can see the load balancing in case of ND environment is pretty basic and it has following disadvantages

  • There is no communication path back from the application server machines to the Web server plug-in. As a result, the plug-in does not know how busy a server is; often, the plug-ins first indication that a server is overloaded is when the server stops responding to the requests

  • The information used by the plug-in to determine how to route requests to application server is static. That is the mapping between requests URLs and set of application servers that can service those URLs is provided by an XML file that is generated by the deployment manager and must be manually copied by the administrator to the web server machines. When the URLs change or when a new serveris added the plug-in file must be regenerated and replaced.

Setting service configuration properties

WebSphere Portal comprises a framework of services to accommodate the different scenarios that portals need to address. You can configure some of these services. You can set the service configuration properties by using the administrative console.

Follow these steps to change or set value of configuration service property

  • Login into the WAS Admin Console

  • Go to Resources -> Resource Environment -> Resource Environment Provider. It will display list of configuration service like this


  • In this list select the Configuration service that you want to customize. Here it will list out the list of properties for that configuration service. You can add new property or configure the existing property


  • Save your changes and they get saved in the resources.xml file in the corresponding scope.




If you look in the wp_profile/PortalServer/config directory you will notice that there is a .properties file for each of the configuration service in this folder but these files are not used any more. So making changes in them wont have any effect.

File Synchronization in clustered environment

In the clustered environment whenever you want to make any configuration changes your supposed to use WAS Admin Console and make those changes on the Deployment Manager, the deployment manager then takes care of Synchronizing those changes to all the nodes.

Please remember one important thing that the Deployment Manager will only synchronize changes under the config directory


In case of portal server there portal configuration service related files stored under PortalServer directory. So if you want to make any changes in these files you should always use WAS Admin Console. Those changes would get stored in resources.xml file under corresponding scope.

Similarly if you make any changes in wkplc.properties or wkplc_comp.properties or wkplc_dbtype.properties file you will have to copy those changes manually. Please note one more important thing that even in a cluster the wkplc.properties file will be specific to a server so you cannot copy it as it is to other machine. Instead you should copy the corresponding helper file from primary portal node and use it only copy necessary property values to the other nodes.

Steps for adding vertical cluster member

IBM recommends that you should user vertical clustering i.e. having more than one portal JVM's running on single machine to make full use of the resources. Also it looks like setting up vertical cluster is much easier than the horizontal cluster.

I did setup a vertical cluster on my VMWare by following these steps

  • Start the Deployment manager if it is not already started

  • Login into the WAS Admin Console of the deployment manager by going to https://localhost:9043/ibm/console URL.

  • In the WAS Admin Console Go to Server -> Clusters. And select the cluster in which you want to add new member


  • Click on the Cluster members, it will list out the members of that cluster. Click on New. It will open a Create additional cluster member page like this


  • On this page enter name of the new cluster and select the node where this vertical node should be added. Dont forget to check the Generate Unique HTTP ports check box. Click on next couple of pages and when you say finish it will create set of configuration files for the new server
    Important Note :You must update the virtual host entries for the new port created when adding a cluster member. You can do this by updating the default_host virtual host in the administrative console and adding a new alias entry for the port number (use an asterisk [*] wildcard character for the host name).

  • Once the new vertical cluster member is created you can verify it by going to Cluster topology


  • Next step would be to configure dynamic cache so that the cache entries created on this newly created server get copied to other members of the cluster. You can do that by going to Application Server -> Newly created server -> Dynamic Cache service.
    On this page check Enable cache replication check box, select name of your cluster as Full Group replication domain and set Replication Type as pull only


  • Next go to the wp_profile/ConfigEngine.sh directory on the target machine and execute ./ConfigEngine.sh cluster-node-config-vertical-cluster-setup -DServerName=WebSphere_Portal_V1 task to clean up the server-scoped resources, caches, and resource providers

  • Restart the newly added vertical cluster member

  • In the Admin Console change the value of WCM_HOST and WCM_PORT websphere variables scoped at the server level to point to the web server which will be used to serve WCM content

  • Re Synchronize changes from Deployment manager to the cluster members

  • Regenerate the Web Server plugin to include the newly added vertical cluster and copy the newly generated plugin-cfg.xml file to web server and restart web server so that it can start forwarding request to newly added vertical cluster member

Java Client for REST SPI

Starting from WPS 6.1 you can access the Portal SPI model even from outside the portal JVM. The SPI is accessed using REST API. Lets assume that you want to find out the portlet model of Login Portlet. You can do that by writing a simple Java Client like this.

I am using Apache Abdera library to access the ATOM feed and print the response out on System.out


import org.apache.abdera.Abdera;
import org.apache.abdera.model.Document;
import org.apache.abdera.model.Feed;
import org.apache.abdera.protocol.Response.ResponseType;
import org.apache.abdera.protocol.client.AbderaClient;
import org.apache.abdera.protocol.client.ClientResponse;
import org.apache.commons.httpclient.UsernamePasswordCredentials;

public class RESTSPIClient {
private static final String REALM_URL = "http://localhost:10040";
private static final String CONTENTHANDER_URL = REALM_URL +"/wps/mycontenthandler";
private static final String LOGIN_USERNAME ="wasadmin";
private static final String LOGIN_PASSWORD ="wasadmin";

public static void main(String[] args) {
try {
Abdera abdera = new Abdera();
AbderaClient client = new AbderaClient(abdera);
UsernamePasswordCredentials credential = new UsernamePasswordCredentials(LOGIN_USERNAME, LOGIN_PASSWORD);
client.addCredentials(REALM_URL, null, null,credential);
ClientResponse resp = client.get(CONTENTHANDER_URL+"?uri=pm:oid:wps.p.Login");
if (resp.getType() == ResponseType.SUCCESS) {
Document response = resp.getDocument();
Feed feed = response.getRoot();
abdera.getWriterFactory().getWriter("prettyxml").writeTo(feed, System.out);
} else {
System.out.println("Error in response " + resp.getStatusText());
}
}catch (Exception e) {
e.printStackTrace();
}
}
}


As you can see first you should login into the portal using the BASIC authentication client.addCredentials(REALM_URL, null, null,credential);. After that make a HTTP GET request to http://localhost:10040/wps/mycontenthandler?uri=pm:oid:wps.p.Login URL. Look at the last part of the uri which is uri=pm:oid:wps.p.Login. In this pm indicates that this request is for portlet mode. and the wps.p.Login is the unique name of the portlet.

After making the request i am checking if the response status was successful, if yes you can simply read the feed and pretty print it out console using abdera utilities

One important thing you will need to add quite a lot of .jar files in class path to get this sample working. If not you will get few difficult to understand exceptions like NullPointerException because it was not able to create abdera parser


Enabling security in clustered environment

Enabling security in clustered environment is little different from enabling security in the standalone environment. Follow these steps to enable security in clustered environment


  1. Copy the helper file appropriate to your ldap server from <wp_profile>/ConfigEngine/config/helpers directory and set values to match your LDAP configuration for the properties

  2. Execute the ConfigEngine.bat validate-standalone-ldap configuration task that will validate the values of the LDAP that you set in helper file

  3. Next execute the ConfigEngine.bat wp-modify-ldap-security task to make the actual changes in the portal security

  4. Restart the DMGR, All Node Agents and all cluster members.

  5. Copy the helper file that you changed on primary portal node to the secondary portal node.

  6. Copy the content of helper file into the main wkplc.properties by running the following command
    ConfigEngine.bat
    -DparentProperties=/ConfigEngine/config/helpers/wp_security_ids.p
    roperties -DsaveParentProperties=true

    Important Note: Did you notice that we did not mention any config task name here. Instead just specified parentProperties and savenParentProperties parameter. This will only change the wkplc.properties file.

  7. Update the Portal security information on the secondary node by executing the following ConfigEngine script from the /ConfigEngine directory on your secondary node:

    ConfigEngine.bat wp-change-portal-admin-user -DnewAdminId=admin ID> -DnewAdminPwd= -DnewAdminGroupId=Admin Group ID> -Dskip.ldap.validation=true

    Note: The -Dskip.ldap.validation=true flag can be used if the script fails during ldap validation.

  8. Restart the secondary node's WebSphere Portal server

Steps for setting up horizontal cluster

I followed the steps defined in A Step-By-Step Guide to Configuring a WebSphere Portal v6.1.0.0 Cluster using WebSphere Application Server v6.1.0.15 document for setting up 2 node horizontal cluster.

These are the high level steps

  1. Install Deployment manager. Check Enable administrative security check box

  2. Configure Deployment Manager: Change the request timeout for SOAP, Web Container Chain,.. Also create a WebSphere Portal Admin user in the local file system based repository using WAS Admin Console.

  3. Install Standalone portal server. This portal install will act as primary portal node.

  4. Configure the websphere portal server to use external database.

  5. Federate and cluster the primary portal node

  6. Collect files from portal node using ConfigEngine.bat collect-files-for-dmgr task

  7. Copy the filesforDmgr.zip from primary node to DMGR

  8. Expand the filesForDmgr.zip on DMGR and copy content to appropriate directories

  9. Add node to the deployment manager by executing following command on the primary portal node ConfigEngine.bat cluster-node-config-pre-federation

  10. Update the deployment manager configuration for new WPS by executing the following command ConfigEngine.bat cluster-node-config-post-federation

  11. Create the cluster definition and add the WebSphere_Portal server as a cluster member by executing following command ConfigEngine.bat cluster-node-config-cluster-setup

  12. Install Standalone portal server on secondary portal server

  13. Copy database configuration from primary node to secondary node

  14. Add node to the deployment manager by executing ConfigEngine.bat cluster-node-config-pre-federation command

  15. Update the deployment manager configuration for the new WebSphere Portal server by executing the following ConfigEngine script: ConfigEngine.bat cluster-node-config-post-federation

  16. Add this newly federated WebSphere_Portal server as a cluster member to the existing cluster by executing the following ConfigEngine script: ConfigEngine.bat cluster-node-config-cluster-setup

Using External security manager in cluster

If you are configuring security for IBM WebSphere Portal with an external security manager, review the additional considerations described in this section, depending on the external security manager that you are using. Perform any configuration for an external security manager after you have completed all other setup, including ensuring that the WebSphere Portal cluster is functional.
The following considerations apply to all external security managers:


  • When setting up security in a cluster to use an external security manager, ensure that you configure either TAM or Siteminder on each of the portal nodes.

  • If you make any changes to the external security manager configuration after initially setting it up, ensure that any changes you make to the wkplc.properties file are propagated to the other nodes in the cluster. Changes to the wkplc.properties file are not included as part of the cluster resynchronization process.

  • If you are using an external Web server, additional configuration is required before running any task to configure an external security manager with a WebSphere Portal cluster. Edit the wkplc.properties file on each node, and ensure that the values for the wp.ac.impl.JunctionHost and wp.ac.impl.JunctionPort properties are set to the backend server host name and port number you are using for your Web server.

Considerations for security during cluster installation and fedrations

When setting up a cluster, there are two scenarios that must be considered. The first scenario is when the default VMM file-based repository security is used at both the WebSphere Portal nodes and the deployment manager until after the WebSphere Portal cluster is completely set up. Prior to federating the first WebSphere Portal node into the cell, the required group for WebSphere Portal administrators must be defined in the deployment manager’s security repository. Once the cluster has been set up, you can modify the security settings of the cell. Although it is possible to modify security in the cell using the WebSphere Application Server administrative interfaces, you should use the WebSphere Portal security tasks to change cell security in order to ensure that the security configuration settings for WebSphere Application Server and WebSphere Portal are identical.

The second scenario is when the existing deployment manager cell has already modified its default security setting prior to the first WebSphere Portal node joining the cell. WebSphere Portal supports the capability of using two different sets of administrative user ID and password credentials when federating a WebSphere Portal node into a cell – one set for the WebSphere Portal node authentication and one set for deployment manager authentication. This means that it is not necessary to define a common administrative user ID before WebSphere Portal joins the cell. If the deployment manager cell is using federated VMM with additional repositories, WebSphere Portal will pick up this configuration dynamically from the deployment manager when it joins the cell. If the deployment manager cell is using standalone LDAP security, however, then it is necessary to configure the LDAP values into the WebSphere Portal property files before federation to enable WebSphere Portal to dynamically adapt to the existing standalone LDAP security settings of the cell. As with the first scenario, once the cluster has been set up then security changes to the deployment manager cell security settings can be made using the WebSphere Portal security tasks, and additional WebSphere Portal nodes may be added to the cell following the same procedures.

The tasks under Setting up a clustered production environment recommend configuring security before configuring your additional nodes but if you configure your security after configuring your additional nodes or if you need to update your security configuration after you have created your clustered environment, you will need to run an additional task to update the security settings on the secondary nodes; see Configuring security after cluster creation for information.

Cluster security options

There are many security options that can be used in a cluster. All of the VMM federated security options, including multiple LDAP repositories, database repositories, and the default file-based repository can be used. Additionally there is an option to use standalone LDAP security instead of the VMM federated security approach.

Note: It is not recommended to use the file-based repository in a production environment. The reason is that updates are only possible through the WebSphere Application Server administrative console, not through portal user management. These updates are sent to each node in the cell using deployment manager file synchronization. This can be time consuming for large volumes of users and groups. Also, synchronization does not occur at the same time for all nodes in a cell, so there will be time windows when the nodes in the cell have differing security definitions.

Cluster Security

WebSphere Portal 6.1 depends on Virtual Member Manager (VMM) for security. The VMM configurations are applied at the cell level and maintained at the Deployment manager level. When you federate a node into the Deployment manager it wont change any of the deployment manager security settings instead any changes that you made at the portal level will get lost and portal will inherit the security settings at the Deployment manager level.

Note: If administrative security is deselected during installation of the deployment manager or is disabled after the deployment manager is installed, it must be enabled prior to executing the security configuration tasks on the WebSphere Portal cluster members.

WebSphere Portal provides a number of security tasks, which can be used to modify the WebSphere Application Server security settings and make the required updates to the WebSphere Portal configuration in a single step. As soon as a WebSphere Portal node is federated into a deployment manager cell, all WebSphere Portal security tasks will execute on the deployment manager. Run security tasks after federating the WebSphere Portal node because the Deployment Manager cell does not contain the configuration resources required to run the security tasks.

Http Session Failover

User interactions with WebSphere Portal Server are maintained through the use of a
HttpSession. This provides a way to preserve data across multiple pages or requests on an individual user basis. The failure or outage, either scheduled or unscheduled, of a WebSphere Portal Server cluster member will result in the termination of the user's HttpSession.

You can configure WebSphere Application server so that in case of failure the Users Http Session is available to the other cluster, this is called distributed session. WAS provides following two options to do this

  • Database session persistence, where sessions are stored in the database specified.

  • Memory-to-memory session replication, where sessions are stored in one or more specified WebSphere Application Server instances or profiles.



When a session contains attributes that implement HttpSessionActivationListener, notification occurs anytime the session is activated (that is, session is read to the memory cache) or passivated (that is, session leaves the memory cache). Passivation can occur because of a server shutdown or when the session memory cache is full and an older session is removed from the memory cache to make room for a newer session. It is not guaranteed that a session is passivated in one application server prior to activation in another application.

When you store an attribute in the PortletSession it actually gets stored in the HttpSession only the attribute name is stored in the namespaced format.

LDAP for geographically deployed websphere portal

The implementation of a multi-clustered WebSphere Portal Server V6.0.x
architecture that sees the deployment of individual clusters in each geography to support a truly "global deployment" mandates the use of the same LDAP directory server in all geographies. The LDAP directory server may, however, be replicated for redundancy purposes. This requirement is necessary to maintain uniqueness between Portal users.

Database for geographically deployed architecture

In case of geographically deployed architecture each geography maintains its own set of databases. Each database would be highly available. In this case the shared databases would be replicate across wide area network using technique as queue replication or 2way sql replication

LDAP directory servers and firewalls

Problems can arise if a firewall is placed between WebSphere Portal Server and the chosen LDAP directory server. Under such circumstances, authentication can appear to stall after a long period of inactivity. This typically manifests itself in the morning after a night of inactivity,whereupon users may wait up to 30 minutes before authenticating into the Portal solution (unless the Portal is restarted or the LDAP Reuse Connection parameter is disabled from the WebSphere administrative console and WMM connection pooling mechanism is disabled).

After this initial period, subsequent users are authenticated in the normal fashion.
The origin of this problem is not with WebSphere Portal Server or the underlying WebSphere Application Server instance, but with the firewall idle timeout. System Administrators should ensure that the tcp_keepidle system setting on each of the servers is smaller than the firewall idle timeout. Failing this, when a client is left to idle for longer than the firewall idle timeout, a communications error will be encountered. Usually, a keepAlive packet is sent according to the tcp setting of tcp_keepidle.

LDAP directory server high availability

WebSphere Portal Server V6.0.x introduced support for multiple LDAP directory servers with respect to new multi-realm capabilities. Not surprisingly, this has lead to some confusion when deploying multiple LDAP directory servers in response to the requirements of high-availability. As such, when multiple LDAP directory servers are deployed in support of a multi-realm deployment, often used in conjuction with Virtual Portals, these LDAP directory servers need to be highly available in their own right.
For Tivoli Directory Server based implementations, high availability is achieveable through the deployment of two directory servers that operate in a master peer-to-peer topology. However, in a slight deviation from the standard peer-to-peer practice, which works on a concept that there are multiple master peers in an environment each being capable of processing read and write requests, the recommend solution is to utilize a load balancer to preference one master peer as the active member for all read and write requests. The reason for this decision is to eliminate any potential conflicts that would otherwise result from two-way replication.
As such, the load balancer should be configured to always route read and write requests to the nominated master peer during normal operation. However, should the load balancer detect a failure of the master peer, the load balancer will re-route all requests to the alternate master peer. During write requests, there will be replication from 'node 1' to 'node 2', not the other way round, as there should not be any write requests being distributed across both LDAP servers or peers. It follows that read only requests can be evenly distributed to both peer LDAP servers. This can be achieved by configuring a second load balancer cluster group, with a different virtual host name to make a distinction from the first load balancer cluster group and virtual host name.

Important Note When using LDAP over SSL (LDAPS), care should be taken when utilizing a load balancer as described above. LDAPS not only establishes a JNDI context against the target server, but also implements SSL handshaking between the client and target server (including key
negotiation). Whether the load balancer simply just redirects the SSL connection to the target directory server or whether the SSL connection is terminated at the load balancer, with the load balancer re-negotiating a secondary SSL connection to the target directory server, needs to be decided.

Guidelines for vertical and horizontal clustering

Please follow these guidelines while deciding about using either Vertical or horizontal clustering

Vertical clustering
To fully utilize the processing power of modern SMP servers
Local redundancy

Horizontal cluster
To achieve scalability beyond the limitation of individual server
Reduandancy and reliability
Hardware failover

Horizontal scaling is especially effective in environments that contain many smaller, less powerful machines. Client requests that would overwhelm a single small machine can be distributed over several machines in the solution. Failover is another benefit of horizontal scaling. If a machine becomes unavailable, its work can be routed to other machines
containing cluster members.


By contrast, vertical cloning benefits Symmetric Multi-Processing (SMP) systems and should be implemented when system resources are found to be underutilized. For Java, systems with multiple processors typically outperform multiple systems with fewer processors, when comparing the total number of processors.

A general rule of thumb has always been to allocate a single Java Virtual Machine (JVM) to a single processor. Traditionally, this was based on the constraint that the compaction phase of Garbage Collection (GC) was single threaded. Recent versions of the JVM have, however, minimized this restriction, potentially allowing a greater number of JVMs to run given the total number of available processors. It is also worth remembering that there are a number of JVMs associated with the Deployment Manager and NodeAgent (a NodeAgent is required for each individual physical node participating in the cell) when running Portal Server V6.0.1 in a clustered environment.

It is highly recommended to use combination of horizontal and vertical clustering to make full use of the system resources and still making sure that system is fault tolerent if one of the server hardware fails

Additional considerations for multiple lines of productions

In case of multiple lines of productions environment every cell + portal cluster will have its own release database but they will share customization database, because of this if user customizes any of the resource using one cluster those changes would be available when same user will access the resource even through other cluster.

Since multiple lines of production will share the customization database they should be exact replica of each other, i.e. the object ids of the resource such as portlet or page should be same in both lines of production. In order to make sure that object ids are same you will have follow some additional guidelines in case of multiple lines of production

First you will need to execute empty-portal configuration task on both lines of productions to clean it up during the setup process. Once that is done you will have to export configuration from the staging environment and import same configuration with ID generation mode set to false. this will make sure that the portal does not generate object ids while importing portal configuration.

You should use the release builder tool to build subsequent releases

Maintenance procedure for multiple cluster environment

In case of multiple cell+cluster environment i.e. WebSphere Portal Server Gold Availability Standard environment every cluster gets its own Release domain but the customization and community data is shared. As a result you can safely apply the major fixes that change the configuration stored in the release database.

Follow this procedure to update the multi cell environment

  • Change the IT Sprayer so that it stops forwarding requests to the one line of production

  • Once that is done you can update that line of production and during that time second line of production will service requests

  • Test the first line of production and once your satisfied you start the first line of production and reconfigure IP sprayer to forward request to first line of production

  • Follow similar procedure for all other lines of productions