Aqualogic Service Bus Tutorial

Weblogic - Aqualogic Service Bus - EAI - Part 1

This blog is to gather my learnings and understandings about Weblogic(WL)'s Aqualogic Service Bus (ALSB). Initially, i could not appreciate this awesome product from BEA. ALSB is meant mostly for Webservices and SOAP requests. I was trying to use these functionalities for normal GET and POST requests in RESTful services. Before we proceed further, lets look at some basic SOA jargons.

Service Bus:

Service Bus is considered as Enterprise Application Integration (EAI) tool, that would facilitate easy integration of various services (that are building blocks of any software system). This integration tool is created with various technologies that are opted for software middleware infrastructure. So, please note that ESB is not a programing language itself.

Also, ESB is NOT an app or web server. This is an application (could be web application) that runs in an web container. So, if you want to play with ALSB, you would need both WL server and ALSB. You could download trial version which you can use for 60days.

Basic Features of an ESB:

1. Loose coupling of services in a software system. This would provide great flexibility of modifying / enhancing any part of the system without affecting much on the other dependents.
2. ESB should be standard based and flexible, and should support many transport mediums like Messaging Service, SOAP, XML and text etc.
3. It should allow integration with very less coding. In other words, should be configurable.
4. ESB should be more scalable and highly revolvable.

All other features are added as vendors think it is necessary for an ESB. So, an Architect can select appropriate product based on their needs.

Ok! Lets get our hands dirty.

Installation:

ALSB setup can be (of course, trail version) downloaded from BEA site. I downloaded ALSB 3.0 which comes bundled with WL server 10.0.

There is nothing special about this installation. I just followed the installation setup wizard and it went perfect on my XP machine.

Basic Setup:

As basic setup, I created a user domain for my test project. You don't need to create a new domain, if you prefer to work on example domain, which is created with installation.

So, I created a domain using Start --> All Programs --> Bea Products --> Tools --> Config Wizard. Please follow this link if you have any questions on specific setting.

Once domain is created, start weblogic server from Start --> All Programs --> Bea Products --> User_projects --> --> "Start Server for AquaLogic Service Bus Domain"

Look for server state as running. When server is up and running, it would open Weblogic Admin console automatically. Since we are interested in ALSB and its console, I am not going to discuss about Weblogic Admin Console here.

Weblogic products are considered as one of the best and user friendly because of their Admin Console. All settings and cluster, server maintenance and deployments everything could be done through Admin console.As we expected, ALSB has a Web Admin console, where most of the configuration can be done by just few clicks of mouse.

Ok. I like to wind up this blog at here. So, we have completed the installation of ALSB and before I start next topic, you could play around with various options in admin console to get used to it.

Weblogic - Aqualogic Service Bus - EAI - Part 2

I hope you are familiar with ALSB Admin console. If you don't have Weblogic ALSB please visit my previous blog for seeting up ALSB on your window machine.

I assume that you have WL server running and you can open Admin console of ALSB (http://localhost:7001/sbconsole/) in a web browser. Please note, if you are installing WL10.0 bundle, the port number for ALSB may be 7021.

How ALSB works:

In a nutshell, ALSB acts as a proxy between client and service (any service lives in server). The proxying is done by creating a Proxy Service and Business Service in ALSB. I hope the below diagram could help you understand the "request" and "respose" flow.

As diagram illustrates, ALSB stands between client and service / server to route the requests and responses. Here, I have shown 2 proxy service and 2 business service. However, a project could have 'n' number of proxy services and business services. If you happen to have more proxy and business services, you may want to touch base with customer support folks regaring performs impact. I would suggest to have more than one ALSB instances to avoid any performance bottle necks. Few jargons definitions are below.

Project in ALSB.

A project is nothing but a group of proxy and business services. You may like to group proxy and business services of billing services in one project and a separate group for order management services.

Proxy Service:

You have guessed it! Proxy service is contacted by the client to eventually get response from target service. ALSB allows developer or ALSB administrator to modify any request data and metadata through Admin console configuration. For example, consider a case, where you have web service, is contacted by two different vendors (clients) with two different XML requests. You don't need to write code to understand both request formats. Instead, ALSB Proxy service could be configured using XQuery to convert the request XML to desirable / server understandable format. This means easy addition of new clients with any changes to Target service, which evetually facilitate easy integration of systems.

Using XQuery programming, literally, all request data or parameters could be modified. If also allows to write logics to decide which business services to contact in runtime. This is one of the key feature of an ESB.

Business Service:

Business service represents target service's URL. Business services are contacted by proxy service to route the requests to target service. If there are more than one target service (multiple instances of same service for scalability), all the URL can be added to single business service. ALSB also provides features for load-balancing of services. Business service also helps to fail-over services. Meaning, if one target service is down, Business service routes the requests to another service when there is an outage encountered by ALSB. It means high availability of your services.

Setting up ALSB:

• As mentioned earlier, business services (BS) and proxy services(PS) are grouped under ALSB projects. So, before we create PS, BS, lets create a new project.
• Open Admin console of ALSB. http://localhost:7001 or http://localhost:7021
• Go to "Project Explorer" on left panel.
• Once the Enter project name in text box in "Enter New Project Name:" and click on "Add Project". If the text box is not enabled, please click on "Edit" on top left panel.
• Project entry is added to the projects table just below that. Click on your project.
• Select a "Resource" type from drop down options. First we create PS, So, select Proxy service from the option. On selecting PS option, next page should be automatically loaded.
• Provide, proxy name and make sure you select "Any XML service option. Click on Next>>. As you can see, here, ALSB can support other services like Messaging, WSDL and SOAP services.
• Select protocol as "http" and leave rest of the options with default values. Click on Next>> to save the settings for proxy service.
• Now, you can new proxy serive is added to "Resources" table.
• Next, create a business service by selecting "Business Service" in the drop down options.
• Follow the setting wizard. Provide BS name and type. Please make sure you select same type as PS. Click on Next>>.
• On next page, we have to some important configuration for BS. Select protocol as http and Load balancing Algorithm as round-robin. Provide End point URL of Target service. In our case any webseite url. Since other options are not much important at this level, please go with default values and click on Next>>.
• Since, we are dealing with website, select HTTP request as "GET" and leave rest of the options with default values. Go to next page to save the settings.

Now that we have created both proxy service and business service, you might be wondering! "Wait a second, are we not supposed to hook up proxy service with business service???" Yes!! We should!. That's next.

I would recommend to save the configurations so far that we have done by hitting "Activate" button on left top panel. Please note, in ASLB console each time as you modify setting, they are created as sessions. Later, if you find that one of the setting needs to be reverted back to original setting. You can go to "View All Sessions" and clicking on appropriate icon under Options.

If you clicked on Activate button, Click on "Create" again to create a new session.

Configure Proxy Service:

• Click on your project under project explorer section. In Resources section, click on "Edit Message Flow" icon ( the one is activated) under "Actions" column.
• On the next page right click on proxy service cover and select "Add Route". Here, as you can see, you can "Add pipeline pair" to add request and response pipelines. Under request and response pipelines, you could "Add Stage" to perform logic operations.
• So, we have added new Route, Right click on RouteNode1 to select "Edit Route" to modify setting. You can also modify the name of this node to a meaning ful name by selecting "Edit name and Comments".
• On Edit Route page, click on "Add an Action" --> Communication --> Routing
• Now, you get new node added to this page. Click on "*" to select which business service should be contacted by this PS.
• You can also add other Request and Response Actions like modifying header or to perform some logic operations or Log the event to server log file for reporting purposes.
• "Save All" this settings and Activate the session to be effective.

Here, PS is hooked up with BS. You can contact PS through your browser by typing http://localhost:7001/proxyServiceName

The response should be eventually from the target website URL. Hope you are getting a proper response. You can further explorer to add stage (as mentioned in step 2) and modify request data and headers.

You can also refer reference documents from BEA site for more information.

Best Practices with Aqualogic Service Bus

I have been developing a lot many enterprise integration applications using Aqualogic Service Bus from BEA. It has been quite a nice experience first evaluating and then developing live. I must say i have been both amused and thrilled using the ALSB stack. But it has also been too frustrating at times. Here in this document i would talk about the various best practices one must adopt while using ALSB to both save time and write classy codes.

I hope the document would be useful. For any comments, suggestions or inquiries you can leave a message here or PM me.

Find the document here

Best Practices with BEA Products

1. In normal conditions use DSP for read services and WLI for database writes for WLI Server versions till 9.2. For cases where we have weblogic 10.3 app servers we can use DSP as the data services layer for both reads and writes.
2. Avoid 8.1, 9.2 and 10.3 Interoperability Issues by generating the WSDL in 10.3 if you are using all three platforms. In cases of any mixed usage always create the wsdl with the higer version of the platform being used.
3. Use BARE annotation in your web services so that you do not need the WSDL for types jar and only the XSD files.
4. Do not use DBControl as a DataSource. Close connections if you do a getConnection. Furthermore we shouldnt write java type code for accessing databases through WLI. Use the inbuilt DB Controls for the purpose which supports almost all databases.
5. All WSDLs should have a return parameter that indicates the SUCCESS or FAILURE of the call. Failure messages should be displayed on the user screen.
6. Data not found should not be displayed if system errors have occurred in reading or writing data.
7. While using portals or pageflows remove any empty catch blocks in the Front end.
8. In BPM, please do not write much code in commit action or the prepare action. Keep it as minimal as possible.
9. Exception occurred is not enough of a message. Please log the actual exception on the screen in a pretty format.
10. Redirect 500, 404 to meaningful pages.
11. Use XA datasources if your transaction spawns multiple datasources. XA is there for a reason.

Optimizing XQueries

Well can we imagine programming in the SOA world without knowledge of XML Technologies. As a matter of fact if we are working on ALSB and ALDSP then the knowledge of XPaths and XQueries is of prime importance. Here i would be discussing the optimal practice of writing XQueries.

Explanations of each of the following tips can be found at the end of the article.

DON'TS

Here are a few things that we must seek to avoid:

1. Don't use eval ()
2. Don't evaluate expressions several times over, and avoid redundant expressions.
3. Don't use //
4. Don't query constructed document fragments

DOS

Here are some recommendations for optimization:

1. Minimize the execution of queries based on a given search expression. Try instead to use navigation paths based on the parent, children and siblings of a node which has already been retrieved
2. Make appropriate use of indexes adapted to your search criteria.
3. Code Quality

TODO

1. Put $Id$ inside a comment at the top internal documentation of HTTP parameters
2. Document in Xquery the argument types and the return type
3. Use meaningful names for variables and functions, without abbreviations, and avoid ambiguous terms
4. Use Javadoc-style tags as in XQDOC ( http://www.xqdoc.org/ ) : @param, @return
5. Keep data retrieval separate from result construction

_______________________________________________________________________________________________

EXPLANATIONS

Don't use eval ()

The snag is, the arguments to the eval () function can't be cached. Beyond that, using eval () leads to a style of programming that's hard to read and to debug. And eval () can always be replaced by a standard expression.

Don't evaluate expressions several times over and avoid redundant expressions

Xquery doesn't perform any analysis or optimization of queries akin to what a Java compiler does. So no refactoring of repeatedly-evaluated expressions, no elimination of code that won't be executed, etc. Pay particular attention to repeatedly evaluated expressions, they should be evaluated once only and the result placed into a variable, which also makes for more readable code.

Don't use //

$a//b causes a complete traversal of all nodes of which $a is the root in search of an element b. In most cases the location of b is fairly precisely known, and so would be better to specify it.

Don't query constructed document fragments

A typical example (to avoid):

let $e := content (: $e is a constructed document fragment :)

let result := $e/b/text()

Minimize the execution of queries based on a given search expression.

A query like

res := collection("/db/projects") /a/b [ id = $val ]

causes a complete scan of an entire collection. Admittedly, queries like this are at the heart of an XQuery (and account for most of its execution time). But once the result $res has been retrieved, it can be efficiently used as a starting point for navigation to its parent, siblings and children:

$a: = $res / parent::a

$next-sibling: = $a / next-sibling:a

Make appropriate use of indexes adapted to your search criteria.

There are currently three types of user-configurable indexes in Xquery. All require pre-indexation either of the base collection or of specified node-sets in sub-collections.

• The fulltext index, which indexes lexical tokens ("words" in Western scripts). Indexation can be configured to include or exclude nodes specified using a limited subset of XPath
• Typed indexes over nodes specified by a limited subset of XPath (called "range indexes" because they permit queries referring to a range of numerical values)
• Indexes by tag name ("Qname index") http://wiki.exist-db.org/space/jmvanel/New+index+by+QName

Index 2. is slower than 3., but has two advantages

The request code doesn't have to be changed in order to use the index with 2, there is no danger of getting wrong results if the indexation hasn't been done.

Index 3 lacks these advantages, but is almost as fast as a relational database. Such an index cannot be constrained by an XPath, but only by a tag name. Both index and and index 2 are typed (integers or strings), and allow matching by criteria of equality or inequality (comparison).

Document in XQuery the argument and return types

Don't write :

declare function local:add($n, $m) {

$n + $m

};

This is more explicit and auto-documenting. And for the same price you get run-time arguments checking. If you know for sure the types you manipulate, declare them !

declare function local:add($n as xs:integer, $m as xs:integer)

as element(result) {

$n + $m

};

Also Keep data retrieval separate from result construction. It is good to create variables of child element nodes that are used in result construction rather than retrieving them every time from the root.

Open Source SOA Implementation

Hardly would there be anybody who hasnt heard or been a part of the SOA hype over the couple of years. It seems as if an entire industry is converging towards this amazing concept of sharing, interoperability and plug-ability. For long software architects for medium and large businesses have always had to go through a nightmare while designing systems that were heterogeneous. Well this is not a blog to present the advantages and disadvantages of SOA, its various pros and cons and the ideas revolving around its concepts.

Neverthless there has been a host of vendors that have leaped to embrace SOA and have created an entire product suite/stack for implementing it. For instance there is BEA with its entire Weblogic and Aqualogic suite, IBM with its Websphere Development Studio, Tibco, WebMethods and a host of other proprietary vendors.

I have however tried to make a presentation on how to implement using open source tools and technologies. Here is a slide that evaluates the various options available for an open source implementation of SOA.

Find it here.

Creating Weblogic 9.2 Cluster (With Remote Managed Servers) with WLST

Overview

This section describes how to setup a WebLogic 9.2 cluster on Linux servers using the BEA WebLogic Scripting Tool (WLST). WLST is a scripting tool installed with WebLogic that allows for command-line configuration of the WebLogic server. The set of scripts provided here along with these instructions will allow you to setup a WebLogic cluster in your environment with ease.

Clustering Scripts Setup

In preparation for running the WLST scripts to create a cluster, you will need to download the WLST Clustering Scripts (ZIP, 7KB) and unzip it to a directory. After that, configure the environment variables in all of the scripts to match your environment. A description of the variables that will need to be set can be found in the section Environment Variables to Configure in the WLST Scripts. After that, copy the files to the servers that you will be using in the cluster. You will then need to navigate to this directory on a server to execute any of the scripts on that server.

Steps to Create the WebLogic Domain and Setup the Cluster

On the Server hosting the AdminServer:

Create a domain with AdminServer and a cluster of two managed servers

	WL_HOME/common/bin/wlst.sh createcluster.py DOMAIN_NAME CLUSTER_NAME
Start the node manager and AdminServer	WL_HOME/common/bin/wlst.sh startadminserver.py DOMAIN_NAME
Setup JDBC Data Source	WL_HOME/common/bin/wlst.sh createjdbc.py t3://AdminServerIP:AdminServerHttpPort CLUSTER_NAME
Pack the domain	WL_HOME/common/bin/pack.sh -managed=true -domain=DOMAIN_PATH -template=DOMAIN_TEMPLATE -template_name=DOMAIN_TEMPLATE_NAME

On all servers not hosting the AdminServer:

Unpack the domain	Copy DOMAIN_TEMPLATE from the server hosting the AdminServer to the same directory on all other servers that are apart of the cluster and then execute the following command on the servers to create the base domain: WL_HOME/common/bin/unpack.sh -domain=DOMAIN_PATH -template=DOMAIN_TEMPLATE
Enroll the node manager with AdminServer and then start it	Execute the following command on the servers to setup the node managers and start them: WL_HOME/common/bin/wlst.sh enrollnodemanager.py t3://AdminServerIP:AdminServerHttpPort DOMAIN_NAME

On the Server hosting the AdminServer:

Deploy the Elastic Path code to the cluster	WL_HOME/common/bin/wlst.sh deploy.py t3://AdminServerIP:AdminServerHttpPort DEPLOYMENT_NAME APPLICATION_PATH CLUSTER_NAME
Start the cluster	WL_HOME/common/bin/wlst.sh startcluster.py t3://AdminServerIP:AdminServerHttpPort CLUSTER_NAME

Additional Useful Scripts

To create another server in the cluster, execute this command after step 1	WL_HOME/common/bin/wlst.sh createmanagedserver.py DOMAIN_NAME CLUSTER_NAME
To shut down server ServerName	WL_HOME/common/bin/wlst.sh stopserver.py DOMAIN_NAME ServerName
To remove a deployment	WL_HOME/common/bin/wlst.sh undeploy.py t3://AdminServerIP:AdminServerHttpPort DEPLOYMENT_NAME

Descriptions of Constants Used in Setup Steps

BEA_HOME	The BEA Home directory where files common to all BEA products are stored (eg. /opt/bea/)
WL_HOME	The WebLogic Server product installation directory (eg. /opt/bea/weblogic92/)
AdminServerIP	The IP address of the server that the AdminServer is on
AdminServerHttpPort	The port for the AdminServer to listen to http requests on
DOMAIN_NAME	The name of the clustered domain to be created (eg. epclusterdomain)
DOMAIN_PATH	BEA_HOME/user_projects/domains/DOMAIN_NAME (the path to the clustered domain)
DOMAIN_TEMPLATE	The filename of the domain template to be created or accessed (eg. BEA_HOME/user_templates/epclusterdomain_managed.jar)
DOMAIN_TEMPLATE_NAME	The descriptive name of the domain template to be created (eg. "EP Clustered Domain")
CLUSTER_NAME	The name of the cluster to create (eg. wlsCluster)
DEPLOYMENT_NAME	The name for the deployment of Elastic Path application code (eg. epsf_cluster_deployment)
APPLICATION_PATH	The path to where the Elastic Path application has been setup (eg. /home/build/ep_weblogic/com.elasticpath.sf/)

Environment Variables to Configure in the WLST Scripts

BEA_HOME	The BEA Home directory where files common to all BEA products are stored (eg. /opt/bea/)
WL_HOME	The WebLogic Server product installation directory (eg. /opt/bea/weblogic92/)
JAVA_HOME	The root directory of the Java JDK install that is used to run WebLogic (eg. /opt/j2sdk)
AdminServerIP	The IP address of the server that the AdminServer is on
AdminServerHttpPort	The port for the AdminServer to listen to http requests on
AdminServerHttpsPort	The port for the AdminServer to listen to https requests on
AdminServerPassword	The password used to connect to the AdminServer as default user weblogic
Machine1IP	The IP address of the server hosting the first managed server in the cluster
Machine1Name	The machine name of the server hosting the first managed server in the cluster
Server1HttpPort	The port for the first managed server to listen to http requests on
Server1HttpsPort	The port for the first managed server to listen to https requests on
Server1Name	The name to use for the first managed server in the cluster (eg. epServer1)
Machine2IP	The IP address of the server hosting the second managed server in the cluster
Machine2Name	The machine name of the server hosting the second managed server in the cluster
Server2HttpPort	The port for the second managed server to listen to http requests on
Server2HttpsPort	The port for the second managed server to listen to https requests on
Server2Name	The name to use for the second managed server in the cluster (eg. epServer2)
JdbcName	The descriptive name of the JDBC data source (eg. EP)
JndiName	The JNDI name of the JDBC data source (eg. jdbc/epjndi)
Url	The JDBC connection URL (eg. jdbc:oracle:thin:@11.11.1.111:1111:ep)
JdbcDriverName	The name of the JDBC driver (eg. oracle.jdbc.OracleDriver)
DbUserName	The username for accessing the database
DbUserPassword	The password for accessing the databas

Stripping XML Namespaces using XQuery

A lot of the times I had found people complaining about unneccessary namespaces that flood their xml. Offcourse namespaces are very important but during certain times when we have to process complex and large xmls using XPath it becomes necessary to strip the namespaces. Here is a demonstration of a small XQuery code that is used to strip any samespaces from an XML input.

(:

=====================================
Description: xquery to remove namespaces from an xml
@author Arun Pareek
=====================================
$Resource/XQUERY/stripNamespace.xq  $

:)

declare variable $inputRequest as element() external;
declare function strip-namespace($inputRequest  as element()) as element()
{
element {xs:QName(local-name($inputRequest ))}
{
for $child in $inputRequest /(@*,node())
return
if ($child instance of element())
then strip-namespace($child)
else $child
}
};

document
{ strip-namespace($inputRequest )}

Before explaining the code let me give a small example of how this piece of code works.

Input XML with Namespaces

<TestXML xmlns:sp-en="sch.soap.org">
<ZBAPI_COIB.ZBUR xmlns:sp-en="sch.soap.org">
<I_OUTPUT xmlns:sp-en="sch.soap.org"/>
<RETURN xmlns:sp-en="sch.soap.org">
<item xmlns:sp-en="sch.soap.org">
<TYPE xmlns:sp-en="sch.soap.org">S</TYPE>
<CODE xmlns:sp-en="sch.soap.org"/>
<MESSAGE xmlns:sp-en="sch.soap.org"/>
<LOG_NO xmlns:sp-en="sch.soap.org"/>
<MESSAGE_V1 xmlns:sp-en="sch.soap.org"/>
</item>
<item xmlns:sp-en="sch.soap.org">
<TYPE xmlns:sp-en="sch.soap.org">S</TYPE>
<CODE xmlns:sp-en="sch.soap.org"/>
<MESSAGE xmlns:sp-en="sch.soap.org"/>
<LOG_NO xmlns:sp-en="sch.soap.org"/>
<LOG_MSG_NO xmlns:sp-en="sch.soap.org">000000</LOG_MSG_NO>
<MESSAGE_V1 xmlns:sp-en="sch.soap.org"/>
</item>
</RETURN>
</ZBAPI_COIB.ZBUR>
</TestXML>

And the Output of the Xquery is

<TestXML>
<ZBAPI_COIB.ZBUR>
<I_OUTPUT/>
<RETURN>
<item>
<TYPE>S</TYPE>
<CODE/>
<MESSAGE/>
<LOG_NO/>
<MESSAGE_V1/>
</item>
<item>
<TYPE>S</TYPE>
<CODE/>
<MESSAGE/>
<LOG_NO/>
<LOG_MSG_NO>000000</LOG_MSG_NO>
<MESSAGE_V1/>
</item>
</RETURN>
</ZBAPI_COIB.ZBUR>
</TestXML>

As you can see the code is very much self-explanatory. The Xquery snippet expects an xml input to it. The Xpath entity crawls through each of the elements in the input and returns just the child elemnt and discards the namespace. A very much similar to recursive fucntions in C/C++.

Enjoi :-)