Sunday 21 July 2013

Promoted Properties vs UDP

Broker properties are divided into four categories:
Properties that relate to a specific node
Properties that relate to nodes in general
Properties that relate to a message flow
Properties that relate to the execution group
Experiment1
* Promote Property for main flow - Input Mq node Queue property and Output node Queue property
The properties are seen when seeing properties of message flow
Properties are promoted to message flow level
After creating bar file ,just the overriding properties are changing with new values
After overriding with new values,the bar file is just saved WITHOUT RECOMPILATION.
If we recompile,again it get backs to the same old value;the overridden values are not getting effected.
Just deploy now;The overriding properties works fine :)
Experiment 2
* Subflow is used in the middle of MQInput and MQOutput node in the main flow
*The subflow has a MQOutput node whose Queuename is promoted
*As it is promoted,the queue name has become the subflow's configurable property
*Now,when subflow is referred in the main flow,the subflow's property is shown.But it says,the subflow's promoted property cannot be overridden ."Property override is not supported on the subflow node"

WMB and Error Handling

(WMB) Error handling
When we design any message flow, we often do not give more emphasis on error handling. Well as per my experience I found this error handling techniques & design principles more crucial than designing the happy path.
So here I am with few details on how to handle unhappy path in WMBV6.0 with information about the message flow error behavior.


Design Consideration
·                     Connect the Failure terminal of any node to a sequence of nodes that processes the node's internal exception (the Failure flow).
·                     Connect the Catch terminal of the input node or a TryCatch node to a sequence of nodes that processes exceptions that are generated beyond it (the Catch flow).
·                     Insert one or more TryCatch nodes at specific points in the message flow to catch and process exceptions that are generated by the flow that is connected to the Try terminal.
·                     Ensure that all messages that are received by an MQInput node are processed within a transaction or are not processed within a transaction.

Understanding the Flow Sequence

·                     When an exception is detected within a node, the message and the exception information are propagated to the node's Failure terminal ( Diagnostic information is available in the ExceptionList). 
·                     If the node does not have a Failure terminal or if it is not connected, the broker throws an exception and returns control to the closest previous node that can process the exception. This node can be a TryCatch node  (Root & LocalEnvironment are reset to the values they had before) or the MQInput node .
·  If the catch terminal of the MQInput node is connected, the message is propagated there (ExceptionList entries are available; Root & LocalEnvironment are reset to the values they had before).
      1) If message is transactional and catch terminal is connected ,then message goes to catch terminal and thrown to Input node due to Throw node in Catch.And then backout count is checked.And based upon backout count,message is retried.Finally message goes to Failure terminal if it is connected.If Failure terminal is not connected,then Backout Q or System DLQ is checked.
      2)  If message is not transactional and catch terminal is connected,then message goes to catch terminal and thrown to Input node due to Throw node in Catch. But backout count is not checked and also does not go to Failure terminal even if it is connected and it is discarded.


What if in Failure path,there is some error?
Error path should be properly handled by Failure terminal's subsequent nodes.Otherwise,successive errors will be discarded.

Otherwise if the catch terminal is not connected, the transactionality of the message is considered.
·                     If the message is not transactional, the message is discarded. Otherwise , if it is transactional, the message is returned to the input queue, and it is read again, whereupon the backout count is checked.
·                     If the backout count has not exceeded its threshold, the message is propagated to the output terminal of the MQInput node for reprocessing. Otherwise if it is exceeded & if the failure terminal of the MQInput node is connected then the message is propagated to that path. (Root is available but ExceptionList is empty)
·                     If the failure terminal of the MQInput node is not connected, the message is put on an available queue, in order of preference; message is put in backout queue, if one is defined; otherwise in dead-letter queue, if one is defined. If the message cannot be put on either of these queues, it remains on the input queue in a retry loop until the target queue clears.(It also records the error situation by writing errors to the local error log)

The below are the info that can be captured from the ExceptionList:
·         MessageId of the incoming message
·         Flow name(Name of the message flow in which exception has occurred)
·         Node name in which the exception has occurred
·         Error time (Time when the exception has occurred)
·         Error Number (Error Number for the exception which is generated by Message broker)
·         Error Severity(Severity of the exception)
·         ErrorDetails(Complete description of the error)



My testing scenarios:
With Javacompute node:
Input queue -àHas Backout queueàBackout threshold =3
1)      Scenario:
Javacompute node has exception catching code as below.
Message flow  nodes’ Failure and Catch  are not connected
 Transactionality is “YES”,
catch (SQLException sqx ){
           MbUserException mbue = new MbUserException(this, "evaluate()", "","", sqx.toString(), null);
             throw mbue;
         }
      Observation:
·         Error thrown to MQInput node
·         Message is retrying with Backout count in MQMD as 1,2,3 consecutively
2)Scenario:
   Javacompute node has exception catching code
    MQInput node’s catch is connected
      Transactionality is “YES”
     Exception handling subflow ends with MQOUTPUT node (ERRORQ)  
      Observation:
·         Error thrown to MQInput node
·         Exception once thrown to MQInput node,is sent to catch terminal
·         The message  is written to MQOUTPUT node (ERRORQ).This message written has only original message
·         Retry is not actually performed because the message is not thrown to the input queue.
         3)Scenario
                        Javacompute node does not have exception catching code .Just like below,
catch (SQLException sqx ){
           
            }
                   MQInput node’s catch is connected
      Transactionality is “YES”
     Exception handling subflow ends with MQOUTPUT node (ERRORQ)  
                 The below Exceptionlist populating message is written in subflow,  
                    DECLARE start REFERENCE TO InputExceptionList.*[1];
             WHILE  start.Number IS NOT NULL DO
                 
                  SET OutputRoot.XML.Exception.messageNumber = start.Number;
                  SET  OutputRoot.XML.Exception.messageText = start.Text;
           
                  MOVE start LASTCHILD;
             END WHILE;

Observation:
Behaved same as Scenario 2 but the message at output is,
hi
<Exception>
 <messageNumber>6233</messageNumber>
 <messageText>Problem encountered obtaining JDBC connection</messageText>
</Exception>
4)Scenario
  Only if it is connectivity failure,the code has to retry.
 Otherwise the code should not retry


 

               

 

  


Database & WMB

Database node vs Compute node
The actual difference between the Database node and the Compute node is NOT the accessing of databases as both nodes can do this, which does mean their names can be slightly confusing. The actual (and only) difference is that a ComputeNode always creates a new message (even if you never use it), whereas a Database node never creates a new message. You can obviously do more in a Compute node than in a DB node (like create a changed OutputRoot message which you can not do in a DB node). But if all you want to do in the node is access and/or change the Envrionment or LocalEnvrionment trees then you can do this in the Database node which is why the patterns used the DB node where they do as they are simply setting a flag in the LE.

JDBC vs ODBC
i think JDBC may be faster than ODBC, our supplier's ODBC is not multi-threaded yet whereas their JDBC is, there are SQL limitations in the ODBC which are over-come by using JDBC, and ODBC may be phased out by Microsoft. i would propbaly offer JDBC as an alternative rather that a straight replacement to our customers.


As discussed below are the differences between ODBC and JDBC connections 

• OBDC is Microsoft drivers and uses dlls, JDBC is Java drivers and uses jar files. 
• ODBC is procedural oriented and JDBC is object oriented. 
• ODBC can't be directly used with Java because it uses a C interface. 
• ODBC makes use of pointers which have been removed totally from Java. 
• ODBC mixes simple and advanced features together and has complex options for simple queries. But JDBC is designed to keep things simple while allowing advanced capabilities when required. 
• ODBC requires manual installation of the ODBC driver manager and driver on all client machines. JDBC drivers are written in java and JDBC code is automatically installable, secure, and portable on all platforms. 
• JDBC API is a natural Java Interface and is built on ODBC. JDBC retains some of the basic feature of ODBC. 
• JDBC is Java Database Connectivity. It enables java programs to execute SQL statements. JDBC makes it possible to write a single database application that can run on different platforms and interact with different DBMS. JDBC is language dependent i.e it is used only to make connectivity with java. 
• ODBC is open database connectivity. The goal of ODBC is to make it possible to access any data from any application, regardless of which database management system (DBMS) is handling the data so it is language independent. 
• ODBC is language independent and JDBC is language dependent. 
·         If you are creating ODBC connections to your user databases, ensure that you correctly create 32-bit, 64-bit, or both, connections for each DSN. If you are creating JDBC connections, their use is independent of 32-bit or 64-bit mode.


There are no performance issues for JDBC connections as compared to ODBC connections, we need to configure it on the server to use it in InfoView , in the jar files, and the end users cannot modify it like ODBC connections. 

Yes JDBC libraries are faster than ODBC because it uses JAVA jars and not windows dlls, so there can be a slight better performance die to platform independency if using any non windows Server, but again as I told earlier it completely depends on the requirement and scenario . 

There are no major issues it’s just the configuration which is a bit complex as compared to ODBC , but it is more reliable and once done it works perfectly. 

Connecting to Database from message broker:
·         For one database ,one dsn ,one message flow,we need one thread to access.
·         When connection gets broker?
The broker makes the connections when it needs to use them, and they remain open until one of the following events occurs:
  • The message flow has been idle for one minute
  • The message flow is stopped
  • The broker is stopped

ODBC.ini
ODBC .ini files that are supplied, and the information contained in these configuration topics, include all the connection parameters that are supported for connections to your databases.

ODBC connections –Creating DSNs

In Windows
Fro all databases like DB2,Mysql,… we just need to choose ODBC driver in User or System DSN and create a DSN.

But for ORACLE,we need to go to Windows registery REGEDIT ,then HKEY_LOCAL_MACHINE\SOFTWARE\ODBC\ODBC.INI. Expand that location, and right-click your DSN entry.Then give value in string(refer http://publib.boulder.ibm.com/infocenter/wmbhelp/v8r0m0/index.jsp?topic=%2Fcom.ibm.etools.mft.doc%2Fah14442_.htm)

In Linux/Unix (Using WMB ODBC Database extender)
Copy  the ODBC.ini and ODBCINST.ini  files from installation path(install_dir/ODBC/unixodbc/) and place in WMB /var/mqsi path.
Then point the environment variables ODBCINI  and ODBCSYSINI to this in
This copied path.
Then edit the ODBC.ini file with your required Database paramaters.

For DB2
The DB2 alias name configured during DB2 remote cataloguing for DB2 remote database in Client/server modal,should be used for DB2 alias name.

Enabling JDBC in WMB

First install the driver of the database if the driver is not in your broker machine.

Then mqsicreateconfigurable service to enable JDBC connection with that database.Include security identity.

Then we can use the JDBC only with DatabaseRetrieve node and DatabaseRoute nodes.

ODBC and JDBC drivers in Broker
Broker contains ODBC drivers for the Databases supported by Broker.
But the Broker will not contain the JDBC drivers for even the Databases supported by Broker.If any JDBC driver is required,it will has to be download and kept in BROKER lib folder and referred in jars url property of JDBCProvider.

Queue Manager Alias - Case Study

I was doing experiments to understand Queue Manager Alias concepts..
Hope my Case study would be useful to all who is involved in this Technology

Hints
·         Remote Queue Definition is only for the Queue manager resolution.It is not for Application which is connected to the first Queue manager.The Application will always try to give only the local queue name and Queue manager name to which it is intended to place  its message for the destiny Queue manager.
Transmission queues
·         In Remote Queue Definition, I am defining Transmission queue name.The same name is used in MCA (Channel definition) .This name could be anything now.It will work.
·         In RQD, I am not specifying the XMIT queue name.Then,the Queue manager will look for the XMIT queue defined with the same name of the Remote Queue Manager name in RQD.
·                If it is defined ,it will use that XMIT queue and it will transport.So,now also it will work.
·               Else if,it is not defined,it will place the message in default Transmission queue.
QueueManager Alias
Generally, queue-manager alias is used to resolve a queue-manager name. One of the uses of a queue-manager alias is to move a message down the network to one or more intermediary, non-adjacent, queue-managers. This network arrangement is often called a daisy-chain.

In a daisy-chain network arrangement, one queue-manager is networked to the next queue-manager, which is networked to the next queue-manager (QM1 -> QM2 -> QM3 -> QM4 -> QM5). If a message is created on QM1, and the destination queue-manager is on QM5, then the message must traverse QM2, QM3, and QM4. QM2 and QM3 will need a queue-manager aliases since they are non-adjacent (don't have a transmission queue) to the destination queue-manager. 

Simple daisy-chain example
QM1 -> QM2 -> QM3 -> QM4 -> QM5
When a message is created on QM1, it needs to be moved to QM2, on its way toward QM5. There is no hub in this configuration. QM2 needs a definition to move the message from QM2 to the next hop QM3. 

So, when the message arrives on QM2, a queue-manager alias must exist to resolve the queue-manager name in the XQH (transmission queue header): DEF QR(QM5) RQMname(QM5) XMITQ(QM3). 

This definition tells the MCA (message channel agent) on QM2 to move the message to transmission queue named QM3, so that the message will be sent to QM3. 

When the message arrives on QM3, a queue-manager alias will be needed to move the message to a transmission queue named QM4: DEF QR(QM5) RQMname(QM5) XMITQ(QM4).
Transmission Headers
The queue name used by the application, the logical queue name, is resolved by the queue manager to the destination queue name, that is, the physical queue name. This destination queue name travels with the message in a separate data area, the transmission header, until the destination queue has been reached after which the transmission header is stripped off.

Initial setup –Distributed Messaging

1)QM1 ,QM2,QM3,QM4  are in distributed queueing as shown below,
QM1 -
              àQM2
              àQM3 à QM4

Case Study 1

       A message has to be sent from QM1 to QM4

Create a message broker flow which acts as an application connected to a Queue manager.This Application wants to put message to some other Queue manager.
QM1 is broker Queue manager
QM4 is the destiny Queue manager

But anyhow,it cannot directly MQOPEN to the destiny Queuemanager name and Queue name.
It can only place its message to a Remote Queue definition in the queuemanager .So,since the Application wants to put message to different queue manager, as per the Application’s need,RQD is created with Remote Queue manager name as ‘destinyQM_name’ .

 In defining the XMIT Queue name àI am giving the XMIT queue as QM3.

Configuration
At QM1:
   Remote Queue Definition : QM4 as Remote Queue manager
                                                    QM3 as XMIT Queue
                                                       OUT as Remote Queue
  Now ,at this point,the message transmission header contains QM4 as Remote QM name and OUT as remote Queue name once it falls to QM3 from QM1.

At QM3:
Now it’s the task of resolving the transmission header of the message by QM3 Queue manager.It needs to resolve QM4 from the transmission message header.Though it can easily resolve the Queue manager as it is its adjacent Queue manager,certain naming conventions has to be followed.
There are two different  ways to configure here:
1)      Define the XMIT queue name QM4 same name as that of  queue manager name QM4.

By this way,the QM3 can easily resolve its adjacent queue manager through the XMIT queue name.

2)      Define a Queue manager alias for QM4 in QM3 with the same name.This has to be done if the transmission queue is not of the same name as that of QM4.
At QM4:
  Local queue OUT is defined.