Saturday, 20 October 2012

Some handy wsadmin tips

Set heap size
for server in AdminControl.queryNames('type=Server,processType=ManagedProcess,*').splitlines():        serverName=server.split(',')[0].split(':')[1].split('=')[1]
        nodeName=server.split(',')[3].split('=')[1]
        AdminTask.setJVMProperties('[-serverName '+serverName+' -nodeName '+nodeName+'  -initialHeapSize 768 -maximumHeapSize 2048]')

 AdminConfig.save()

Set connection pools
import re
for dataSource in AdminConfig.list('DataSource').splitlines():
        jndi= AdminConfig.showAttribute(dataSource, 'jndiName')
        if(re.search("PerformanceDB", jndi)):
                connPool=AdminConfig.showAttribute(dataSource, 'connectionPool')
                AdminConfig.modify(connPool, '[[maxConnections "200"]]')

AdminConfig.save()


Check if server is up or down

This will only return the servers that are running 

for server in AdminControl.queryNames('type=Server,*').splitlines():
        name=server.split(',')[0].split(':')[1].split('=')[1]
        print name

If you want to filter out the deployment manager and nodeagent to only show the App servers you're interested i, use the  ManagedProcess qualifier as below


for server in AdminControl.queryNames('type=Server,processType=ManagedProcess,*').splitlines():
        name=server.split(',')[0].split(':')[1].split('=')[1]
        print name



Prepared statements cache (and limiting to specific datasources)

import re
for dsName in 'BPEDB PerformanceDB PDWDB TwqlDB TeamWorksDB ProcessDB WPSDB'.split():
        for dataSource in AdminConfig.list('DataSource').splitlines():
                jndi= AdminConfig.showAttribute(dataSource, 'jndiName')
                if(re.search(dsName, jndi)):
                        print "Setting prepared statement cache "+jndi
                        AdminConfig.modify(dataSource, '[[statementCacheSize "100"]]')

AdminConfig.save()



Find a running cluster and stop it
for cluster in AdminControl.queryNames('type=Cluster,*').splitlines():        if(re.search("RMRS.WebApp",cluster)):
        AdminControl.invoke(cluster, 'stop')



Looking for a Type
import re
for line in AdminConfig.types().splitlines():
        if(re.search("Server",line)):
                print line

Actually a better way to do it might be

AdminControl.queryNames('type=*Security*,*')


Get the config object of the Mashups Config Service

import re
for rep in AdminConfig.list('ResourceEnvironmentProvider').splitlines():
        if(re.search("Mashups_ConfigService",rep)):
               mashup_cs=rep


List all of the custom properties for the Mashups Config Service
for prop in AdminConfig.show( AdminConfig.showAttribute(mashup_cs,"propertySet") ).split():        print prop

or

for prop in AdminConfig.show( AdminConfig.showAttribute(mashup_cs,"propertySet") ).split():print prop

Actually, it's a tad more tricky because of square brackets

for a in AdminConfig.show( AdminConfig.showAttribute(mashup_cs,"propertySet") ).split('resourceProperties')[1].split(']]')[0].split('[')[1].split(' '):


Find a running cluster using wildcard search and stop it

cluster = AdminControl.queryNames('type=Cluster,name=*WebApp*,*')
AdminControl.invoke(cluster, 'stop')

The cluster reference appears to remain after it's stopped (which is not the case for the server names) so you're able
to start with AdminControl.invoke(cluster, 'start')

You cant do this with a server though because the object reference dissapears and even if you kept it in a variable
If you try you get an error stating that the object instance doesn't exists
Instead use the fllowing

AdminControl.startServer(servername, nodename , time)

Example...
AdminControl.startServer('RMRS.WebApp.node1.0', 'node1',120)

(Not really sure what time is)


Queue depth
for Q in AdminControl.queryNames('type=*QueuePoint,SIBMessagingEngine=*BPC*,name=*HldQueue*,*').splitlines():

more specifically
q = AdminControl.queryNames('type=*QueuePoint,SIBMessagingEngine=*BPC*,name=*BPEHldQueue*,*').splitlines()[0]
then
AdminControl.getAttribute(q, "depth")

Thursday, 12 April 2012

Tivoli Performance Viewer without the clutter

Personally, I preferred TPV when is was a stand-alone application. The thing I dislike about embeding TPV into the Admin console is the the presence of the frames at the side and top. It's easy enough to drag across to remove the side frame but you're still left with the top frame. OK, maybe I'm a little OCD but to get yoursaelf a nice clean screen with just the data you want, enter this URL

https://<DMGR hostname>:<port>/ibm/console/tpvServerSelect.do

This will take you directly to the frame where you select which server you want to monitor (i.e .without the side and top frames). Selecting your server here will result in the main TPV page being displayed also without the annoying side and top frames. Much better.

Also, it's worth mentioning that this page also contains 2 frames (one for selecting modules and one for displaying tables and graphs). This being the case you can right click in the frame displaying the data and view only that frame, even better.


Thursday, 1 March 2012

Using Local Schema's

Problem Description

WSDL documents usually contain import statements that in many cases refer to remote locations. For example..
<wsdl:import location="http://docs.oasis-open.org/wsn/bw-2.wsdl" namespace="http://docs.oasis-open.org/wsn/bw-2">

In many cases, a WSDL document can contain 3 or 4 import statements pointing to remotely located schemas. Within each of these remote schemas there may be another 1 or 2 import statements also pointing to remote locations and on some occasions those "second level" documents may contain further remote imports. A scenario such as this could result in hierarchy of 15 to 20 documents all of which would need to be downloaded by the application at runtime.

During development time this doesn't present a problem (assuming the IDE being used can be configured to use a proxy) because download times are not critical. At runtime however, download times can be an issue and therefore it makes sense to have all required schemas located within the enterprise either on the local file system or on the internal network accessed via a shared file system or an internal HTTP server.

Although this is possible by configuring each application server JVM to use a proxy server (assuming such an approach didn't contravene any organisational standards), relying on internet based resources is not desirable from a performance point of view. This being the case it is recommended that all required schemas are downloaded and referenced locally.

Options for using local schemas

Option 1: Using a jax-ws catalog
Download all of the schemas to the local file system (i.e. using a browser or better still using a wget or curl script)
Create a jax-ws-catalog.xml file (this could easily be scripted) to map the remote schemaLocations to the local file system location - see sample contents below...
<catalog xmlns="urn:oasis:names:tc:entity:xmlns:xml:catalog"><system systemid="http://www.oasis-docs.org/foo.xsd uri="wsdl/foo.xsd"/></catalog>

Place the newly created jax-ws-catalog.xml file in the relevant location (e.g for a servlet based endpoint deployed this location is the WEB-INF directory (a Websphere example would be %WAS_HOME%\profiles\myProfile\installedApps\myCell\myAppAPP.ear\myAppWEB.war\WEB-INF). Standard locations for other types of endpoints can be found here (http://jax-ws.java.net/nonav/2.1.5/docs/catalog-support.html)


Option 2: Using Websphere Integration Developer (WID) to import schemas and resolve dependancies
Normally, to import a remote WSDL into WID (along with the subsequent XSD's that it imports and XSD's that they in turn import) the process would be to right click the relevant library or module and select Import > WSDL and XSD > remote WSDL or XSD file. However, this results in the WSDL and it's dependencies being imported but erronously does not result in the schemaLocation attributes being modified accordingly to point at the imported schemas. This appears to be a bug (observed using WID version 7.0.0.0)
The workaround to this is as follows...

  1. Import the top level WSDL to the local file system manually (e.g. via a browser or using a cmd line tool such as wget or curl) 
  2. Import that local WSDL into WID (Import > WSDL and XSD > remote WSDL or XSD file, or both).
  3. Manually update the schemaLocation attributes in the top level WSDL only (i.e. the WSDL that was downloaded manually in step 1

When using this method, WID resolves the remote dependencies and downloads them as before but it also changes the schemaLocation attribute as expected.

As described in step 3 above, the schemaLocation attribute in the top level WSDL still needs to be edited manually but given it will only be 3 or 4 lines it's a relatively easy task in comparison to editing all of the imported schemas.

Wednesday, 29 February 2012

BPM 7.5 Changing default TCP ports


This is quite a laborious process and I do intend to automate it at some point. for now though, here's the manual version (you'll need coffee)

Assumptions....
  • BPM 7.5 standard
  • Remote messaging topology (i.e 2 clusters)   

NOTE:  All changes in this section must be carried out on the Deployment Manager machine

The following files will be edited...

·       virtualhosts.xml at Cell scope
·       serverindex.xml at Node scope for all nodes in the Cell
·       99local.xml at AppTarget Server scope
·       50appserver.xml at AppTarget Server scope
·       80eventmanager.xml at AppTarget Server scope
·       resources.xml at AppTarget Cluster scope
·       wsadmin.props for both WAS profiles


Details of each change are as follows...

1.   Stop all JVM’s including the Deployment Manager
2.   Log on to the Deployment Manager machine

virtualhosts.xml

3.   Backup the file D:\IBM\profiles\dmgr\config\cells\<Cell name>\virtualhosts.xml and open the original for editing.
4.   Locate the line containing the string port=9443 and modify the port number to 15202
5.   Locate the line containing the string port=9060 and modify the port number to 15001
6.   Locate the line containing the string port=9043 and modify the port number to 15003

serverindex.xml

7.   Backup the file D:\IBM\profiles\dmgr\config\cells\<Cell name>\ nodes\<Node name>\serverindex.xml and open the original for editing.
8.   Modify the port numbers using data in the following tables


Endpoint Name
Port
BOOTSTRAP_ADDRESS
15N04
SOAP_CONNECTOR_ADDRESS
15N05
SAS_SSL_SERVERAUTH_LISTENER_ADDRESS
15N06
CSIV2_SSL_SERVERAUTH_LISTENER_ADDRESS
15N07
CSIV2_SSL_MUTUALAUTH_LISTENER_ADDRESS
15N08
WC_adminhost
15N01
WC_defaulthost
15N00
DCS_UNICAST_ADDRESS
15N10
WC_adminhost_secure
15N03
WC_defaulthost_secure
15N02
SIP_DEFAULTHOST
15N15
SIP_DEFAULTHOST_SECURE
15N16
SIB_ENDPOINT_ADDRESS
15N11
SIB_ENDPOINT_SECURE_ADDRESS
15N12
SIB_MQ_ENDPOINT_ADDRESS
15N13
SIB_MQ_ENDPOINT_SECURE_ADDRESS
15N14
ORB_LISTENER_ADDRESS
15N09
CELL_DISCOVERY_ADDRESS
15N17
DataPowerMgr_inbound_secure
15N18
IPC_CONNECTOR_ADDRESS
15N19
NODE_DISCOVERY_ADDRESS
15N20
NODE_IPV6_MULTICAST_DISCOVERY_ADDRESS
15N21
NODE_MULTICAST_DISCOVERY_ADDRESS
15N22
Component
Value of N
Deployment Manager
N = 0
Node Agents
N = 1
AppTarget
N = 2
Messaging
N = 3


9.   Repeat steps 2 and 3 for each node (including the DMGR node) within the Cell

99local.xml.xml

10. Log on to the Deployment Manager machine
11. Backup  Backup  D:\IBM\profiles\dmgr\config\cells \<Cell name>\nodes\<node name>\<AppTarget server>\<product>\config\system\99local.xml and open the original for editing

Where...
<AppTarget server> = The name of the AppTarget JVM
<product> = process-center or process-server

12.  Modify all instances of port 9443 to 15202 with the exception of portal-prefix which should be set to port 443 (i.e the webserver HTTPS port)


50AppServer.xml

13.  Backup  D:\IBM\profiles\dmgr\config\cells
\nodes\<node name>\<AppTarget server>\<product>\config\system\50AppServer.xml and open the original for editing.
14.  Change the two instances (provider-url and jndi-url) of port 9810 to15204
15. Backup  D:\IBM\profiles\dmgr\config\cells \<Cell name>\nodes\<node name>\<AppTarget server>\performance-data-warehouse\config\system\50AppServer.xml and open the original for editing.
16. Change the single instance (jndi-url) of port 9810 to15204

Where...
<AppTarget server> = The name of the AppTarget JVM
<product> = process-center or process-server


80EventManager.xml

17.  Backup  D:\IBM\profiles\dmgr\config\cells \<Cell name>\nodes\<node name>\<AppTarget server>\<product>\config\system\80EventManager.xml and open the original for editing.
18.  Change the single instance of port 9810 to15204


Resources.xml

19.  Backup D:\IBM\profiles\dmgr\config\cells\ <Cell name>\clusters\<Product>.AppTarget\resources.xml and open the original for editing

Where <Product> = ProcesServer01 or ProcesCenter01

20.  Change all instances of port 7278 to 15311 (i.e. the SIB_ENDPOINT_ADDRESS of your Messaging cluster members, assuming of course you only have one per nore, i.e you're NOT vertically clustering)


wsadmin.properties

21.  Backup D:\IBM\profiles\<profile name>\properties\wsadmin.properties and open the original for editing
22.  Change the value for com.ibm.ws.scripting.port from 8879 to 15<N>05

Where <N>  is 0 for the Deployment Manager Profile and 1 for a Node profile


Syncronise

23.  Start the Deployment Manager
24.  Once the Deployment Manager has initialised run the following command on each node

D:\ibm\profiles\<node profile name>\bin\sync_node <DMGR hostname> 15005 –username <admin username>
-password <admin password>

25.  Once synchronised, start each node, log on the Admin Console (remember the new ports of 15003 must be used) and start the each BPM cluster as normal.

Wednesday, 22 February 2012

Applying Refresh Pack 7.5.1 to an exising BPM 7.5.0.1 environment

I found the IBM instructions for this less than satisfactory, so I've written my own


This refresh pack consists of four software components as follows...

1.     WAS 7.0.0.19 fixpack
·         was.7.0.0.19.fp_Part1of2.zip
·         was.7.0.0.19.fp_Part2of2.zip

2.     Interim fixes for was 7.0.0.19 fixpack
·         was.7.0.0.19.fixes.for.bpm.7.5.1.0.zip

3.     BPM refresh pack
·         bpmPC.7510.delta.repository.zip
·         bpmPS.7510.delta.repository.zip

4.     BPM Specific BPM product fixes
·         7.5.1.0-WS-ATLAS-IFJR41201.zip
·         7.5.1.0-WS-ATLAS-IFJR41203.zip
·         7.5.1.0-WS-BPM-IFJR41195.zip
·         7.5.1.0-WS-BPMPC-IFJR41200.zip
·         7.5.1.0-WS-BPM-MultiOS-IFJR41549.zip


The process for upgrading is broken into the following steps...

1.     Install binaries and perform initial Profiles updates
2.     Generate SQL Scripts for schema upgrade
3.     Upgrade Profiles
4.     Upgrade Databases

NOTE: In practice the above steps are not cleanly isolated as their names imply and profile modifications take place in steps 1 and 4.

The following sections will guide the installing engineer through the upgrade process...


1.     Shutdown all JVM's within the target Cell
2.     Back up all WebSphere Profiles
3.     Back up CACerts (D:\IBM\procCentre\java\jre\lib\security\cacerts)
4.     Back up the BPMDB and PDWDB databases
5.   Download the Refresh pack to a nomintaed directory
    6.     Unpack all zips as follows...
a.     From Windows Explorer, right click and select Extract All
b.    The Extraction Wizzard will appear, click Next
c.     Accept the default destination for extraction. For example when unpacking the was.7.0.0.19.fp_Part1of2.zip archive, the Extraction Wizzard will chose a location of D:\Software\BPM 7.5 Install\IBMBPM7.5Adv\Fixpack 7.5.1\was.7.0.0.19.fp_Part1of2

7.     Merge the two parts of the WAS 7.0.0.19 fixpack by copying the contents of

D:\software\BPM 7.5 Install\IBMBPM7.5Adv\Fixpack 7.5.1\was.7.0.0.19.fp_Part2of2\Fixpack70019Delta\files

to

D:\software\BPM 7.5 Install\IBMBPM7.5Adv\Fixpack 7.5.1\was.7.0.0.19.fp_Part1of2\Fixpack70019Delta\files

8.     Start the IBM Installation Manager (Start > Programs > IBM Installation Manager)
9.     From the Installation Manager user interface select file > preferences
10.  In the resultant window, select Repositories from the left hand menu
11.  Highlight any exiting repositories listed and click the Remove Repository button. At this point the list of repositories should be empty.
12.  Click the Add Repository button
13.  Browse and add the following repositories


Common repositories for both Process Centre and Process Server
·         BASE DIR\was.7.0.0.19.fp_Part1of2\Fixpack70019Delta\repository.config
·         BASE DIR \was.7.0.0.19.fixes.for.bpm.7.5.1.0\repository.config
·         BASE DIR\7.5.1.0-WS-ATLAS-IFJR41201\7.5.1.0-WS-ATLAS-IFJR41201\repository.config
·         BASE DIR \7.5.1.0-WS-ATLAS-IFJR41203\7.5.1.0-WS-ATLAS-IFJR41203\repository.config
·         BASE DIR \7.5.1.0-WS-BPM-IFJR41195\7.5.1.0-WS-BPM-IFJR41195\repository.config
·         BASE DIR \7.5.1.0-WS-BPMPC-IFJR41200\7.5.1.0-WS-BPMPC-IFJR41200\repository.config
·         BASE DIR \7.5.1.0-WS-BPM-MultiOS-IFJR41549\7.5.1.0-WS-BPM-MultiOS-IFJR41549\repository.config
·         BASE DIR \bpmPC.7510.delta.repository\repository.config  (must be applied for both PC and PS)
·         BASE DIR \bpmPS.7510.delta.repository\repository.config  (must be applied for both PC and PS)

14.  Ensure that the checkbox labelled Search service repositories during installation and updates is unchecked.
15.  Click OK
16.  From the Installation Manager main screen, click Update
17.  On the next screen select the "update all" checkbox
18.  A warning will be displayed near the top of the Installation Manager windows. Clicking on the View Details link will display the following message...
(At time of writing this warning was under investigation by IBM (PMR 33156-999-866)

One or more fixes will be uninstalled when IBM WebSphere Application Server - ND from package group IBM WebSphere Application Server - ND is updated to 7.0.0.19.
The update does not address issues that were resolved previously by the maintenance packages.
The problems might return if fixes for the following issues are not reapplied or have new fixes applied to prevent these problems from returning.
PM37742 in the fix package 7.0.0.0-WS-WAS-IFPM37742.

19.  Click next
20.  Accept the license agreement and click Next
21.  Scroll to the bottom of the next screen and expand the Repository Information section by clicking on the small black arrow on the left. The table will show details of the repositories added in step 12. Confirm that the table contents matches the repositories added earlier, however, note that the 7.5.1.0-WS-BPM-MultiOS-IFJR41549 repository will be missing. This is expected and details of how to apply this fix are provided in a later step.
22.  Click Update. The update process will take approximately 1 hour
23.  Verify binary installation using the Installation Manager log viewer. This can be accessed from the Installation Manager main screen by navigating to File > View Log and selecting the relevant log file in the left hand panel.
24.  Finally, install the 7.5.1.0-WS-BPM-MultiOS-IFJR41549 fix pack by repeating steps 15 through 22 above. The Installation Manager will decipher that only this single patch needs to be applied and this will be indicated accordingly. This final patch takes approximately 2 minutes to install.


In this step, SQL scripts to update BPMDB and PDWDB schemas will be generated. These SQL scripts will be used in a later step.

1.     Log into the Deployment Manager machine and open a command prompt
2.     Change directory to D:\ibm\profiles\dmgr\bin and run the BPMGenerateUpgradeSchemaScripts command. The command syntax is as follows...

BPMGenerateUpgradeSchemaScripts.bat   -<action> <source version> <profile>

In this case the command should be invoked as follows...

BPMGenerateUpgradeSchemaScripts.bat   -upgrade   7.5.0.1  dmgr




3.     The script will prompt for confirmation to generate scripts for each database, enter Y on each occasion

The script will output the following errors... (These can be safely ignored)

CWMCO0903I: Potential problems found in the scripts. They are:
@ line 19 in file createProcedure_ProcessServer.sql
@ line 71 in file createProcedure_ProcessServer.sql
@ line 161 in file createProcedure_ProcessServer.sql
taskId, @ line 171 in file createProcedure_ProcessServer.sql
@ line 172 in file createProcedure_ProcessServer.sql
@ line 200 in file createProcedure_ProcessServer.sql
@ line 329 in file createProcedure_ProcessServer.sql
CWMCO0904I: Scripts have been generated in D:\IBM\profiles\dmgr\dbscripts\ProcessServer\SQLServer\BPMDB directory.
CWMCO0919I: The command BPMGenerateUpgradeSchemaScripts completed successfully.
 

4.     Verify success by checking for the existence of four new scripts in each of the following directories
·         D:\IBM\profiles\dmgr\dbscripts\PerformanceDW\SQLServer\PDWDB
·         D:\IBM\profiles\dmgr\dbscripts\ProcessServer\SQLServer\BPMDB

NOTE: The scripts do not need to be invoked at this stage, instead they will be utilised in a later step by the upgrade_7x .bat script.


1.     Log into the Deployment Manager machine and open a command prompt
2.     Change directory to D:\ibm\<PRODUCT>\bin  (where <PRODUCT> is procCentre or procServer)
3.     Invoke the following commands...

  • ws_ant.bat -f ..\util\BPMProfileUpgrade.ant -profileName dmgr -Dupgrade=true -Dcluster=<ProductType>.Messaging

  • ws_ant.bat -f ..\util\BPMProfileUpgrade.ant -profileName dmgr -Dupgrade=true -Dcluster=<ProductType>.AppTarget


Where <ProductType> is ProcessCenter01 or ProcessServer01

4.     Checking the installconfig_bpmPC_profileMaintenance.log and installconfig_server.log log files located in D:\ibm\<PRODUCT>\logs\wbi\install.RP7510. Success can be verified by checking for the following message and the bottom of the log file 

Returning with return code: INSTCONFSUCCESS


In this step, the SQL scripts generated earlier will be invoked by the upgrade_7x.bat script which in turn will upgrade the BPMDB and PDWDB databases.

NOTE: The upgrade_7x.bat script gathers database credentials from D:\IBM\profiles\dmgr\properties\db\ProcessCenter01.AppTarget\bpm.standard.nd.dbDesign. This being the case it may be necessary to update this file with new details if passwords have changed since this file was created.

1.     Log into the Deployment Manager machine and open a command prompt
2.     Change directory to D:\ibm\<PRODUCT>\BPM\Lombardi\tools\upgrade\upgrade_7x  (where <PRODUCT> is procCentre or procServer)
3.     Invoke the following command

upgrade_7x.bat -profileName dmgr -nodeName node1 -serverName \<ProductType>.AppTarget.node1.0

Where <ProductType> is ProcessCenter01 or ProcessServer01


4.     Detailed logs are located in D:\ibm\<PRODUCT>\BPM\Lombardi\tools\upgrade\upgrade_7x\logs
5.     Log levels can be set in D:\ibm\<PRODUCT>\BPM\Lombardi\tools\upgrade\upgrade_7x\logging.properties
6.     Succesful completion will be indicated by the following console message...

Executing upgrade step: Upgrade 7.5.0 schema to 7.5.1 for ProcessServerDatabase
Executing upgrade step: Upgrade 7.5.0 schema to 7.5.1 for PerfServerDatabase
Successfully executed 14 upgrade steps.
Process Server database is now version 7.5.1.
Performance Data Warehouse database is now version 7.5.1.
Database upgrade completed successfully.
IBM Business Process Manager V7.5.1 Enterprise database upgrade program finished.
Loading system data

Cleaning SIB tables


Amongst other things, the upgrade adds certain GUI enhancements which results in additional URI’s. This being the case the Websphere Plugin must be regenerated and copied to the Webserver(s).

For instructions on how to achieve this, refer to the Generate Websphere Plugin section


Because CACerts was overwritten during the upgrade, the original version will need to be restored.

Copy the original CACerts that was backed up at the start of the upgrade process to D:\IBM\procServer\java\jre\lib\security

Followers