Eliminating Duplicate Element set using XSLT and XPath

In my previous blogpost here I had described how to remove duplicate element sets from a XML collection using XQuery and XPath constructs.

Well in case you would need to use XSLT this can be achieved relatively much easier.

Here is the XSLT that can be used to remove all duplicate elements from an xml collection for the same example.

<xsl:stylesheet version="1.0" xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform" exclude-result-prefixes="xsl xsd">
<xsl:template match="/">
<FaultLogCollection>
<xsl:for-each select="/FaultLogCollection/FaultLog[not(faultCode=following::faultCode)]">
<xsl:sort select="./faultCode" order="ascending"/>
<FaultLog>
<xsl:copy-of select="./faultCode"/>
<xsl:copy-of select="./faultText"/>
<xsl:copy-of select="./faultSeverity"/>
<xsl:copy-of select="./faultingServiceName"/>
<xsl:copy-of select="./faultLogId"/>
</FaultLog>
</xsl:for-each>
</FaultLogCollection>
</xsl:template>
</xsl:stylesheet>

You can create an XSLT resource in Eclipse and copy paste the above to see this example running. See below to see how





Use the same input XML for FaultCollection

<FaultLogCollection>
<FaultLog>
<faultCode>001</faultCode>
<faultText>SystemFault</faultText>
<faultSeverity>High</faultSeverity>
<faultingServiceName>OrderProcessingPipeline</faultingServiceName>
<faultLogId>23</faultLogId>
</FaultLog>
<FaultLog>
<faultCode>001</faultCode>
<faultText>SystemFault</faultText>
<faultSeverity>High</faultSeverity>
<faultingServiceName>OrderProcessingPipeline</faultingServiceName>
<faultLogId>23</faultLogId>
</FaultLog>
<FaultLog>
<faultCode>002</faultCode>
<faultText>SystemFault</faultText>
<faultSeverity>High</faultSeverity>
<faultingServiceName>OrderProcessingPipeline</faultingServiceName>
<faultLogId>23</faultLogId>
</FaultLog>
<FaultLog>
<faultCode>002</faultCode>
<faultText>SystemFault</faultText>
<faultSeverity>High</faultSeverity>
<faultingServiceName>OrderProcessingPipeline</faultingServiceName>
<faultLogId>23</faultLogId>
</FaultLog>
</FaultLogCollection>

And here is the output after running the XSL.



Pretty easy. Does the same work as the XQuery in the previous example.

Tracking SOA Suite Composite Instances with Business Indicators

 

Here are the two ways I have explored to add Business Field Names to Composite instances so that they can be searched through EM.

Approach 1

Use ora:setCompositeInstanceTitle() Xpath expression to assign/copy any value of business field to the composite Instance title.

Doing this will ensure that the value that we set here will appear in the EM under the column ‘Name’ for the composite instance.

Instances can then be searched using this ‘Name’ field in the EM console

The only limitation to this approach is that we can use only a single business indicator to the composite instance title. If we have more than once business indicator then we will have to probably concat all of them and then assign to compositeInstanceTitle value.

Overcoming Approach 1

The limitation above can be solved using Composite Sensors. We can add composite Sensors to the composite and assign the values of business fields to them. This way from the EM console the sensors can act as search keys that can be used to search process instances based on business fields.

And then from EM we can use these sensors as search keys to search instances

Also the values of these sensors are stored in COMPOSITE_SENSOR_VALUES table in the SOAINFRA schema. Hence we can even write an SQL to search on this table based on the search keys.

Integrating Oracle BAM 11g with BPEL JMS Sensor Action

First of all I am blogging about this thing because i recently had a requirement in one of my work to create variable sensors in my BPEL project that can be used to create real time reporting dashboards in BAM.

Seems quite an easy task in Oracle SOA Suite 11g as we can create a BAM Adapter in our composite and create a sensor action to post data to BAM ADC data objects. This blog by Lucas Jellema perfectly explains this

http://technology.amis.nl/blog/3098/integrating-bpel-and-bam-in-oracle-11g-soa-suite

However this was ruled out because we had a PUB SUB scenario and hence we wanted to create a JMS sensor action to  publish data to a Weblogic JMS topic and use BAM Enterprise Messaging Source to create and populate BAM data objects.

BAM Architect allows us to setup Enterprise Message Sources that can be used to link BAM Data Objects with JMS Topics or Queues. In such an Enterprise Message Source definition, we indicate such a JMS source of messages and specify how to transform the contents of the JMS messages into BAM Data Objects. Message queuing on a JMS queue is more or less a standard operation in many tools and technologies and gives us a means to have various technologies send their data (events) to Oracle BAM 11g.

Prerequisites

• SOA Suite 11gPS2 or PS3 is installed in your local machine.
• A SOA Domain that has BAM capabilities (either an all in one or standard) is created in your local machine.
• JDeveloper is installed in your local machine.
• An Oracle Database (11g preferably) is accessible from the SOA domain.

The Scenario Explained

We will be creating a very simple Order Creation Process in Oracle BPEL that takes a Sales Order Request. The BPEL process has a variable sensor that pushes sensor data to a JMS Queue.

We would then create a BAM Data object with key indicators we would need to monitor.

Using BAM Architect view we define a BAM Enterprise Message Source to create a hook to this queue, flatter the JMS sensor message to extract only the required indicators and transform the flattened message to map it to the data object we created.

Not that complex.

The Solution

Create an composite with BPEL Project in JDeveloper. While creating the BPEL project base it on the SalesOrder schema given below

<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:sal="http://beatechnologies.wordpress.com/bpel/sensor" targetNamespace="http://beatechnologies.wordpress.com/bpel/sensor" elementFormDefault="qualified">
<xs:element name="CreateSalesOrder">
<xs:complexType>
<xs:sequence>
<xs:element name="OrderID" type="xs:string"/>
<xs:element name="OrderDate" type="xs:dateTime"/>
<xs:element name="CustomerID" type="xs:string"/>
<xs:element name="ContactID" type="xs:string"/>
<xs:element name="SalesPerson" type="xs:string"/>
<xs:element name="OrderAllowance" type="sal:SalesOrderAllowance"/>
<xs:element name="OrderCharges" type="sal:SalesOrderCharges"/>
<xs:element name="OrderStatus" type="xs:string"/>
<xs:element name="OrderAmount" type="xs:string"/>
<xs:element name="OrderItem" maxOccurs="unbounded">
<xs:complexType>
<xs:sequence>
<xs:element name="OrderID" type="xs:string"/>
<xs:element name="ModelId" type="xs:string"/>
<xs:element name="Quantity" type="xs:string"/>
<xs:element name="Price" type="xs:string"/>
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:element name="PromisedShipDateTime" type="xs:dateTime"/>
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:element name="CreateSalesOrderResponse">
<xs:complexType>
<xs:sequence>
<xs:element name="orderBookingStatus" type="xs:string"/>
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:complexType name="SalesOrderAllowance">
<xs:sequence>
<xs:element name="OrderDiscount" type="xs:string"/>
<xs:element name="QuantityDiscount" type="xs:string"/>
<xs:element name="CustomerDiscount" type="xs:string"/>
</xs:sequence>
</xs:complexType>
<xs:complexType name="SalesOrderCharges">
<xs:sequence>
<xs:element name="Tax" type="xs:string"/>
<xs:element name="Excise" type="xs:string"/>
</xs:sequence>
</xs:complexType>
</xs:schema>

Similarly choose the Output type as CreateSalesOrderResponse.

Drag an Assign Action between the receiveSalesOrder and End salesOrderResponse and assign a dummy status to the output variable.

Click on the Monitor button in the BPEL pane to be able to add Sensor information for the process.

This will open up the BPEL Structure view where we can add monitors/sensors and sensor actions. If this view doent open up then Click on View->Structure to see this.

Under Sensors right click on Variable and click Create. This opens up a pop up to create variable sensor for the bpel process. Create a sensor as shown under.

Now we have to define a sensor Action. Basically we have created a sensor to publish the whole incoming message to the queue. You can selectively use your desired part in the XML.

Before creating a sensor Action go to the Admin console of your SOA domain and create a Connection Factory and Queue in the SOAJMSModule. Target them to the SOASubDeployment on the SOAJMSServer for the purpose of this demonstration.

Now in the Create Variable Sensor pop up define a sensor action as shown below

Now deploy this process to the soa_server1 Managed server and use either the EM console or any Web service Testing tool to invoke this process with a sample request.

I use the following request to invoke my process

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:sen="http://beatechnologies.wordpress.com/bpel/sensor">
<soapenv:Header/>
<soapenv:Body>
<sen:CreateSalesOrder>
<sen:OrderID>200100102</sen:OrderID>
<sen:OrderDate>2011-05-18T09:36:12</sen:OrderDate>
<sen:CustomerID>5001020</sen:CustomerID>
<sen:ContactID>161921021</sen:ContactID>
<sen:SalesPerson>Arun Pareek</sen:SalesPerson>
<sen:OrderAllowance>
<sen:OrderDiscount>4200</sen:OrderDiscount>
<sen:QuantityDiscount>200</sen:QuantityDiscount>
<sen:CustomerDiscount>100</sen:CustomerDiscount>
</sen:OrderAllowance>
<sen:OrderCharges>
<sen:Tax>300</sen:Tax>
<sen:Excise>200</sen:Excise>
</sen:OrderCharges>
<sen:OrderStatus>Create</sen:OrderStatus>
<sen:OrderAmount>18900</sen:OrderAmount>
<sen:OrderItem>
<sen:OrderID>1</sen:OrderID>
<sen:ModelId>1728192</sen:ModelId>
<sen:Quantity>2</sen:Quantity>
<sen:Price>6000</sen:Price>
</sen:OrderItem>
<sen:OrderItem>
<sen:OrderID>2</sen:OrderID>
<sen:ModelId>1728193</sen:ModelId>
<sen:Quantity>2</sen:Quantity>
<sen:Price>6000</sen:Price>
</sen:OrderItem>
<sen:OrderItem>
<sen:OrderID>3</sen:OrderID>
<sen:ModelId>1728198</sen:ModelId>
<sen:Quantity>4</sen:Quantity>
<sen:Price>6900</sen:Price>
</sen:OrderItem>
<sen:PromisedShipDateTime>2011-05-23T09:36:12</sen:PromisedShipDateTime>
</sen:CreateSalesOrder>
</soapenv:Body>
</soapenv:Envelope>

You can also see that the sensor data now lies in the JMS queue that we configured

Well now we are all good with the BPEL part. All that we have to do is create a Data Object in BAM Architect view and then an EMS source to consume the message from this queue and populate the BAM data object.

Open up BAM Architect View.

Create a folder named Order. Now create a Data Object under this folder called SalesOrder as under

Create this data Object. Now Select Enterprise Message Sources from the BAM Architect console and click on create to create a new EMS.

The first part of the configuration is rather simple. We have to give the Server JNDI, Queue name and Connection Factory name in the appropriate places to let this EMS link to our Queue.

JMS Message Type should be Text as this is the type selected while publishing the message to the queue.

Select the SalesOrder data object under Order folder for the Data Object Name field selection.

Specify the other settings as seen above.

Now the other part of configuring the EMS is involved in mapping the message in the queue to the Data Object variables.

For this what i feel appropriate is to flatten the Queue XML message to a rather simple XML message i.e remove all element hierarchies.

Under XML Formatting click on Pre Processing and then on Advanced Formatting options to open an XSLT editor.

Use the following XSLT to flatten the sensor data in the queue to a simple OrderInformationMetric xml fragment

<xsl:stylesheet xmlns:sal="http://beatechnologies.wordpress.com/bpel/sensor" xmlns:tns="http://xmlns.oracle.com/bpel/sensor" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:xsd="http://www.w3.org/2001/XMLSchema" exclude-result-prefixes="xsl sal xsd tns">
<xsl:template match="/">
<sal:OrderInformationMetric>
<sal:salesOrderID>
<xsl:value-of select="/tns:actionData/tns:payload/tns:variableData/tns:data/sal:CreateSalesOrder/sal:OrderID"/>
</sal:salesOrderID>
<sal:orderDate>
<xsl:value-of select="/tns:actionData/tns:payload/tns:variableData/tns:data/sal:CreateSalesOrder/sal:OrderDate"/>
</sal:orderDate>
<sal:customerNumber>
<xsl:value-of select="/tns:actionData/tns:payload/tns:variableData/tns:data/sal:CreateSalesOrder/sal:CustomerID"/>
</sal:customerNumber>
<sal:discount>
<xsl:value-of select="sum(/tns:actionData/tns:payload/tns:variableData/tns:data/sal:CreateSalesOrder/sal:OrderAllowance/*)"/>
</sal:discount>
<sal:orderTotal>
<xsl:value-of select="sum(/tns:actionData/tns:payload/tns:variableData/tns:data/sal:CreateSalesOrder/sal:OrderCharges/*)+
/tns:actionData/tns:payload/tns:variableData/tns:data/sal:CreateSalesOrder/sal:OrderAmount"/>
</sal:orderTotal>
<sal:orderLines>
<xsl:value-of select="count(/tns:actionData/tns:payload/tns:variableData/tns:data/sal:CreateSalesOrder/sal:OrderItem)"/>
</sal:orderLines>
<sal:promisedShipDate>
<xsl:value-of select="/tns:actionData/tns:payload/tns:variableData/tns:data/sal:CreateSalesOrder/sal:PromisedShipDateTime"/>
</sal:promisedShipDate>
</sal:OrderInformationMetric>
</xsl:template>
</xsl:stylesheet>

You can also verify and test your XSLT against the Sensor message.

Here is a the output of the XSLT transformation.

<sal:OrderInformationMetric xmlns:sal="http://beatechnologies.wordpress.com/bpel/sensor">
<sal:salesOrderID>200100102</sal:salesOrderID>
<sal:orderDate>2011-05-18T09:36:12</sal:orderDate>
<sal:customerNumber>5001020</sal:customerNumber>
<sal:discount>4500</sal:discount>
<sal:orderTotal>19400</sal:orderTotal>
<sal:orderLines>3</sal:orderLines>
<sal:promisedShipDate>2011-05-23T09:36:12</sal:promisedShipDate>
</sal:OrderInformationMetric>

The rest is pretty simple. Follow the screen shot below

Here Message Element Name should be the root element of the Transformed Message. Also give the correct namespace under the Namespace URI field.

Under Source to Data Object Field Mapping add all the fields in the Data Object one by one and in the  Tag/Attr name column simply assign the element name in the transformed message whose data you wish to map to the data object field. Save the EMS configuration.

The last thing would be to Start the EMS so that it can listen to the configured Queue.  After the status of the EMS is ‘Started’ click on ‘Metrics’.

The count of Total Message Received will now be shown as 1.

Now go to the Data Objects view in the BAM Architect and Sales Order data object. The Row Count is shown as 1. Click on Contents to see the data object content. This contains the sensor values for the sales order.

Well that's it. Now go ahead and create BAM dashboard/reports from the gathered data in the BAM active cache.

Try invoking the service again with more requests and see how the cache content refreshes.

Important Note

In case you need to create an EMS that can handle message batches configure the EMS with Batching enabled and specify the Message Batching Element [This will essentially be the repeating node in the XML]

Process Simulation in Oracle BPA Suite 11g

A Foreword about Oracle BPA Suite

Oracle Business Process Analysis Suite allows the business & IT community (process owners, business analysts and architects) to perform process modeling & analysis, simulation and publishing of business processes. It provides an integrated and comprehensive toolset, powered by the industry leading ARIS platform.

Many organizations use Oracle BPA Suite to model their organization’s process from the highest level (strategy) down to the operational process level.

Mark Nelson writes that

“There are a lot of benefits to organizations in using this structured modeling approach in terms of visibility into their processes, optimization/improvement of processes, especially through approaches like LEAN or Six Sigma, and documentation and compliance, just to name a few.”

He has also covered how a BPA blueprint can be exported as a BPMN process and imported in JDeveloper Studio in his post.

Simulating a Process in Oracle BPA Suite

What I would be covering in this blog post is how you can run simulations on a BPMN process modeled using Oracle BPA Suite.

Here is a step by step detail of how it can be done.

For the purpose of simulation i have created our favorite Sales Quote Approval Process (a rather simple version of this in Oracle BPA).

Create/Model a process similar to the diagram below or simply create a one of your own.

 

Right click on the Input Sales Quote Data human task and go to Select-> Select all of this type

Press F8 to open the Attributes pane. Alternatively Click on the tab Edit->Attributes

In the Attributes pane we will try to put simulation data for Time and Cost for each activity in the business process workflow. Expand Times and Costs in the pane.

You can enter simulation data based on predetermined process SLA’s.

You can also enter more precise time settings by entering Orientation and Wait time for each activities as well. 

Similarly enter the Total Costs associated with each activity. Here i have removed the columns that are served by Automatic activities in the business process. However still you can keep them and associate a cost for these activities as well.

Instead of entering Total costs you can even drill down to an even lower level of cost settings. Save the changes and close the Attributes Pane.

Having done this we are good to simulate the process. Click on the icon to start simulating the process that we created.

Wait till all models are loaded for simulation. The BPA suite converts all activities into corresponding Models.

Now for the most important part we have to define the number of maximum/available participants that can be allocated for the simulation have any significance.

Open the InputSalesQuoteData Model and click on the sequence flow from Single Approver to Sales Rep. Click F8 to open the attribute pane and click on the Simulation Attribute.

Enter a value for Number of employees and Number of required employees. Save and close.

Similarly open Model for each of the Task activity and enter the required number of resources required/available for them. If the Task is a Management/Group task you will have more than one participant type for them. Make sure that for you enter the number of participants for each type.

Similarly add the same for Contract Terms Approval and Freeze Contract.

Now we have to add simulation data (probability) for sequence flows from Gateways. This is really important as this will determine how many times the quote is done and redone. Every time a quote is redone the business process executes again leading to significant and redundant repeated costs.

Go to the SalesQuoteApprovalWorkflow Model and click on the sequence flow marked as No from the gateway Business Group Approval. Click F8 to open the Attribute Pane. Click on Simulation and enter a value for probability.

Add a probability factor for other sequence flow as well in a similar way.

For the purpose of this demonstration here I have added .7 probability factor for every Yes/Approved transition.

Now click on the Start event in SalesQuoteApprovalWorkflow and open the Attributes Pane. Go to Frequency and set the daily value of Quote Creation as 50.

Finally go the the Period Tab in the Simulation pallete and define a Start and End Time  to set the duration of simulation. After this click on the icon to start your business process simulation.

The simulation is run and we would get the simulation animation (that include the count of quote flow at every activity) and also simulation statistics. The simulation was run for 4 days.

Click on the Activities (cumul.) tab in the statistics pane and select all values under the Processing time sum column. Right click and Generate a Column chart.

You can see that the Business Group Review activity takes the maximum amount of time. This is probably because this is a Management Task which requires a multi step approval to complete.

Also see how the process fared in terms of cost. Click on the Activity Costs tab, select all values under the Avg. total costs and right click to generate a column chart.

The Input Sales Quote Data task consumes the bulk of Quote processing cost as this task is redone at every point of non approval.

Additionally you can also save the simulation statistics as .xml or csv data.

That concludes this article that deals with introductory steps towards simulation in Oracle BPA Suite.