Eliminating Duplicate Element set using XQuery and XPath

Well i am not to sure as how often one would encounter a scenario wherein we have to select only the distinct elements when it is defined as unbounded and there are duplicate values.

However if we ever encounter this scenario while using BPEL or OSB we can use a small xquery construct to eliminate the duplicate elements.

Consider that you have a collection of Fault coming as

<FaultLogCollection>
<FaultLog>
<faultCode>001</faultCode>
<faultText>SystemFault</faultText>
<faultSeverity>High</faultSeverity>
<faultingServiceName>OrderProcessingPipeline</faultingServiceName>
<faultLogId>23</faultLogId>
</FaultLog>
<FaultLog>
<faultCode>001</faultCode>
<faultText>SystemFault</faultText>
<faultSeverity>High</faultSeverity>
<faultingServiceName>OrderProcessingPipeline</faultingServiceName>
<faultLogId>23</faultLogId>
</FaultLog>
</FaultLogCollection>

This may very well be a scenario in case we are reading from a DB/File which has duplicate values. Consider that faultCode is supposed to uniquely determine the FaultLog element.

I would use the .xq resource in Eclipse to show how this can be achieved

The schema for to be used for creating the .xq resource can be copied from under

<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element name="FaultLogCollection" type="FaultLogCollection"/>
<xs:complexType name="FaultLogCollection">
<xs:sequence>
<xs:element name="FaultLog" type="FaultLog" minOccurs="0" maxOccurs="unbounded"/>
</xs:sequence>
</xs:complexType>
<xs:complexType name="FaultLog">
<xs:sequence>
<xs:element name="faultCode" type="xs:string" minOccurs="0"/>
<xs:element name="faultText" type="xs:string" minOccurs="0"/>
<xs:element name="faultSeverity" type="xs:string" minOccurs="0"/>
<xs:element name="faultingServiceName" type="xs:string" minOccurs="0"/>
<xs:element name="faultLogId" type="xs:decimal" minOccurs="0"/>
</xs:sequence>
</xs:complexType>
</xs:schema>

• Open Eclipse with OEPE and create a OSB Configuration Project. 
• Also create an OSB project a shown below

• Right click on the XQProject and create two new folder under it. 
• Name them as xsd and xq respectively.
• Right click on the xsd folder and then click New to create a new XML schema. 
• Name it as FaultCollection. 
• Copy the inline xsd content into FaultCollection.

• Right click on the xq folder and then click New to create a XQuery Transformation resource as under

• Click on the Source view

• Copy and paste the following code snippet in the source editor

xquery version "1.0" encoding "Cp1252";
(:: pragma bea:global-element-parameter parameter="$faultLogCollection" element="FaultLogCollection" location="../xsd/FaultCollection.xsd" ::)
(:: pragma bea:global-element-return element="FaultLogCollection" location="../xsd/FaultCollection.xsd" ::)
declare namespace xf = "http://tempuri.org/XQProject/xq/DistinctFaultCollection/";
declare namespace custxf = "http://tempuri.org/XQProject/CustomXq/DistinctFaultCollection";

declare function custxf:distinct-deep ( $nodes as node()* )  as node()*
{
for $seq in (1 to count($nodes))
return $nodes[$seq][not(custxf:is-node-in-sequence-deep-equal(.,$nodes[position() < $seq]))]
} ;
declare function custxf:is-node-in-sequence-deep-equal($node as node()? ,$seq as node()* )
as xs:boolean {
some $nodeInSeq in $seq satisfies deep-equal($nodeInSeq,$node)
} ;

declare function xf:DistinctFaultCollection($faultLogCollection as element(FaultLogCollection))
as element(FaultLogCollection) {
<FaultLogCollection>
{
let $distinctFaultLogCol := custxf:distinct-deep($faultLogCollection/FaultLog)
for $FaultLog in $distinctFaultLogCol
return
<FaultLog>
<faultCode>{data($FaultLog/faultCode)}</faultCode>
<faultText>{data($FaultLog/faultText)}</faultText>
<faultSeverity>{data($FaultLog/faultSeverity)}</faultSeverity>
<faultingServiceName>{data($FaultLog/faultingServiceName)}</faultingServiceName>
<faultLogId>{data($FaultLog/faultLogId)}</faultLogId>
</FaultLog>
}
</FaultLogCollection>
};
declare variable $faultLogCollection as element(FaultLogCollection) external;
xf:DistinctFaultCollection($faultLogCollection)

• Having done this, now we are all set to test the XQuery function in Eclipse. 
• Click on the the Test tab.
• Use the sample FaultLogCollection xml posted above to test the function and see that the output of the function has eliminated the duplicate node.

• The output collection filters the FaultLogCollection and removes the duplicates.
• Try with yet another request and see the result data again. 
• Here is the XML that can be used

<FaultLogCollection>
<FaultLog>
<faultCode>001</faultCode>
<faultText>SystemFault</faultText>
<faultSeverity>High</faultSeverity>
<faultingServiceName>OrderProcessingPipeline</faultingServiceName>
<faultLogId>23</faultLogId>
</FaultLog>
<FaultLog>
<faultCode>001</faultCode>
<faultText>SystemFault</faultText>
<faultSeverity>High</faultSeverity>
<faultingServiceName>OrderProcessingPipeline</faultingServiceName>
<faultLogId>23</faultLogId>
</FaultLog>
<FaultLog>
<faultCode>002</faultCode>
<faultText>SystemFault</faultText>
<faultSeverity>High</faultSeverity>
<faultingServiceName>OrderProcessingPipeline</faultingServiceName>
<faultLogId>23</faultLogId>
</FaultLog>
<FaultLog>
<faultCode>002</faultCode>
<faultText>SystemFault</faultText>
<faultSeverity>High</faultSeverity>
<faultingServiceName>OrderProcessingPipeline</faultingServiceName>
<faultLogId>23</faultLogId>
</FaultLog>
</FaultLogCollection>

The OSB Eclipse XQ project used in the example can be downloaded from here.

Unzip to get the OSB jar.

Using and Testing Complex Business Rules in Oracle BPM 11g

A Slight Briefing

Oracle Business Rules is a high performance and lightweight business rules product that is part of the Oracle Fusion Middleware Suite that can be used in both SOA and BPM suite.

To have a business process more agile and coherent with the changing demands of Business, Oracle Business rules is a must for any design. Also it should act as a central component where all process rules are located.

With OBR 11g one added advantage of business rules is that they can be exposed as any other web service. This makes it an instant hit as it becomes hot pluggable.

Here in this example blog i would show how to create a complex rule in JDeveloper and test it out through multiple ways. This is intended to be a zero lecture hands on so i would skip the talk.

Prerequisites

• SOA Suite 11gPS2 or PS3 is installed in your local machine.
• A SOA Domain (either an all in one or standard) is created in your local machine.
  http://wiki.oracle.com/page/Oracle+SOA+Suite+11g%3A+Hown+To+Create+All+In+One+AdminServer
• JDeveloper is installed in your local machine.
• An Oracle Database (11g preferably) is accessible from the SOA domain.

The Scenario

A high school needs a web service implemented as a rule in Oracle that calculates the grades of students.

The service would take some basic candidate information and an array of subjects/marks that the candidate has obtained. The rules engine will have to allocate the candidate Grades on the basis of the following logic.

Average Marks	Grade Allotted
<40	FAIL
40-60	FAIR
60-80	GOOD
80-90	VERY GOOD
90-100	OUTSTANDING

For the sake of this demonstration we would use an XSD definition for CandidateInformation and CandidateGrade.

<?xml version="1.0" encoding="UTF-8"?>
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns="http://www.example.org" targetNamespace="http://www.example.org" elementFormDefault="qualified">
<xsd:element name="CandidateInformation" type="CandidateInformationType"/>
<xsd:element name="CandidateGrade" type="CandidateGradeType"/>
<xsd:complexType name="CandidateInformationType">
<xsd:sequence>
<xsd:element name="name" type="xsd:string"/>
<xsd:element name="rollNumber" type="xsd:string"/>
<xsd:element name="class" type="xsd:string"/>
<xsd:element name="section" type="xsd:string"/>
<xsd:element name="remarks" type="xsd:string"/>
<xsd:element name="subject" type="SubjectType" maxOccurs="unbounded"/>
</xsd:sequence>
</xsd:complexType>
<xsd:complexType name="CandidateGradeType">
<xsd:sequence>
<xsd:element name="overallGrade" type="xsd:string"/>
</xsd:sequence>
</xsd:complexType>
<xsd:complexType name="SubjectType">
<xsd:sequence>
<xsd:element name="subjectName" type="xsd:string"/>
<xsd:element name="subjectCode" type="xsd:string"/>
<xsd:element name="subjectMark" type="xsd:int"/>
</xsd:sequence>
</xsd:complexType>
</xsd:schema>

Save it somewhere as GradesAllocation.xsd

The Solution

• Open JDeveloper and Create a New SOA Application.

• Name the application as BusinessRulesApplication and click on Next.

• Name the project as BusinessRulesProject and again click Next. Make sure ‘SOA’ is selected under ‘Project Technologies’

• Choose ‘Composite with Business Rule’ and click on ‘Finish’.
• You would see that a window pops out to create a rule and specify the inputs and output for the rule.
• Name the rule as ‘GradeAllocationRule’ and click on the ‘+’ icon  to specify the Input and Output types for the rule. Select ‘CandidateInformation’ from the GradeAllocation.xsd as the input and CandidateGrade as the output.

• Wait for the wizard to create the rule definition. Click on the Ruleset at the top and rename it to ‘GradeAllocation’.

• Click on ‘Bucketsets’ link in the left most panel and Add a ‘List of Ranges’.

• Name the bucketset as ‘markRange’ and create a list of ranges as under.

• Click OK to save changes to the bucketset and click on GradeAllocation ruleset.
• Select ‘Create Decision Table’ from the two options. Remember that we can either create an ‘if-then-else’ rule or a decision table. A decision table is implicitly evaluated as an ‘of-then-else’ rule only but gives a better manageability to rule definitions.

• Name the decision function as ‘decideGrade’ and check ‘Advanced Mode’ to be true.

• Click on ‘Insert Pattern’ in the workspace below.

• Right Click on the ‘variable’ block and click on ‘Surround with’ and click on ‘Pattern Block’

• Click on the auto generated expression and from the dropdown select ‘aggregate’

• Now click on variabe and define a variable named ‘averageMarks’, Click on ‘functions’ to select ‘average’ from the dropdown. Click on ‘fact type’ to select ‘SubjectType’ element as the fact. Name this as ‘subjectType’. Click on ‘expression’ now to select ‘subjectType.subjectMark’ as we are interested in the average marks across all subjects. The overall construct should look like below :

• Click on ‘insert condition’ in the panel below and then click ‘edit condition’ to select ‘averageMarks’ from the option. Choose the bucketset ‘marksRange’ from the dropdown ‘Local List of Ranges’.

• Click on the ‘+’ icon adjacent to the range dropdown and keep adding a rule for each of the range. Remember you have to add six distinct rule. Select a distinct value defined in the bucketset each time. Here is how to define the rules.

• Now go to the ‘Actions’ panel and click on ‘insert action’. Select ‘Assert new’. Double click on the action and click on ‘CandidateGradeType’ under Facts. Check the option ‘Parameterized’ for the property ‘overallGrade’

• Now click on each of the option box in the Grid to define a value for the outcome. Since overallGrade is a String type, assign a grade in string for each of the condition as under

• Save all projects and files in JDeveloper. With this we are done with the Rules creation part. So here is a summary of what we did.
• Created a Bucketset for a list of mark ranges and grade type associated with them.
• Created a decision table for a set of rules. Initialized a variable for ‘averageMarks’ and using a pattern block assigned it as an average of all subject marks.
• Now for each condition for the rule asserted the outcome for ‘CandidateGrade.overallGrade’ with the grade that has to be assigned.
• Pretty simple. Isn't it?

Testing the Rules

Creating business rules isn't just enough. There has to be a mechanism to test them. Remember Oracle Rules engine is a inference based rules engine i.e rules are all evaluated at runtime. For more information on how rules are evaluated refer to Oracle Business rules architecture. Here I would show how rules can be tested using three ways.

Testing Rules by Creating a Debug RL function

• Click on the ‘Functions’ link and add a new function and name it to ‘DebugRule’. Select boolean under both Return Type and Bucketset.
• For the body part of the function we would right an RL construct to initialize CandidateInformationType (input to the rule) and pass some dummy values to it.
• Check the screen snap below and create a body exactly like the one below

• Now if you are familiar with any programming language understanding the above construct should be like a cakewalk.
• You would now see that the ‘Test’ link for the function becomes enabled. Click on it to test the rule output.

• Here you go. You can see that the output grade is ‘OUTSTANDING’. Has to be since the dummy value of marks assigned were 100 and 100. You can now change the marks in the subjects or add a new subject type to test the rule again for a different output.

• You can create as many Rulesets as you may for evaluating more complex conditions and add them to the Decision Functions in the order you would need their evaluation to come up with complex business scenarios.

Testing Rules from EM console

• Deploy the BusinessRulesProject to a domain server extended with soa suite. Boot up the em console and browse to the project composite.

• Click on the ‘Test’ icon for the composite to launch the EM test wizard for the composite.

• You would see a Tree View for the request message for the composite wherein you can input sample values

• Fill in any random values for the type bpelInstance. The only important value would be the attribute ‘NCName’. Make sure you put the name of your Decision Function there.
• Fill the request wizard with CandidateInformation as under

• Click on Test Web Service button on top left of the page to test the Rules decision service.

• Expand the CandidateGrade in the Response tab to see the overallGrade for the student.

• You can click on ‘Launch Flow Trace’ to view the execution trace for the Decision Service.

• You can see how easily we can test out our Business rules from the EM console.

Testing Rules from SOAP UI

More than often in real life scenarios we would like to create some kind of a unit testing suite for out business rules. Ag you might have already made note that in SOA suite 11g Business rules are exposed as standard web services that can be invoked from anywhere. See the demonstration below to see how Business rules can be externalized as web service and invoked through third parties even.

• Go to the BusinessRulesProject composite in the EM console.
• Click on the ‘Service Icon’ to copy the WSDL endpoint for the rules service.

• Create a SOAPUI project based on this WSDL.

• Fill the mock service request with actual values

• Run the test and you should see the outcome from the Rules decision function.

Now you can build a test suite to create mock requests for various scenarios and assert the responses.

The JDeveloper project used in this example can be downloaded from here.

Importing a BPMN Visio Diagram as a BPM Process in Oracle BPM 11g

The Case

For many of you who have wondered how can a design created in Visio be converted into a BPM process in Oracle 11g this article will show a step by step guide in doing it.

I am sure not all Business Analyst/Process Analyst would like to install JDeveloper for modeling their business process. They would certain be scared to open a developer IDE to create a business process. There are a however a lot of other ways in which a process can be modeled outside JDeveloper and imported as a BPM process.

Prerequisites

1. JDeveloper 11g PS2 or JDeveloper 11g PS3
2. SOA and BPM Extensions for JDeveloper
3. Microsoft Visio (2010 or 2007)
4. Jdeveloper BPC Extension (Business Process Converter Extension)

The extension can be downloaded from the BPM download link from Oracle by expanding the Prerequisites & Recommended Install Process

Install JDeveloper with the SOA and BPM extensions. Unzip the Oracle Business Process Converter downloaded extension. It contains an extension zip (tutor_bpm_integrator.zip) for JDeveloper. Install this  extension to JDeveloper and restart it.

A little about the Process

We have to design a very simple basic process for Automated Credit Card application approval. The process have very little or no touch points.  A credit card application can be made either through an online customer self service portal or through a Customer Phone line. Applications can even be mailed to the Credit Card Division. The CC division converts all email applications into a batch.

The process checks the eligibility for an applicant to receive a card (credit check, background check etc). If the the applicant is eligible his Card type is determined else if it is rejected an email notification for the same is sent to him. In case the eligibility cannot be determined automatically the applicant is put on hold for a manual review.  If the manual reviewer approves the application then his card type is determined and the applicant is sent a notification about the same.

All outcomes must be delivered as an event to an external process that can act on the outcome of the process.

Not very informative but i guess this is enough for us to get started.

Creating a BPMN Process in Visio

• Open Visio 2007 or 2010 (Start->Run->Visio->Enter).
• Visio 2010 has already a BPMN 2.0 stencil which can be used to create a BPMN model.
• Go to File->New->BPMN Diagram.

• Upon clicking this you would see the BPMN Shapes on the left and an empty visio diagram on the right.
• Scroll down on the Shapes to locate the shape marked as ‘Pool /Lane’ and drag it onto the diagram.

• Double click on this and name this Pool as Customer . The Pool/Lane is equivalent to a swimlane in a BPM process. The Pool Name will be converted into a BPM Process Role.
• Create two more Pool /Lane below the first and name them as ‘Approver’ and ‘Admin’ respectively.

• Now from the BPMN Basic Shapes palette drag a Start Event and drop it to the Customer lane.  Name this as ‘Customer Rep Line’.
• Drag two more Start events to the Customer lane and name them as ‘Customer Request’ and ‘Batch’ respectively.
• Right click on the ‘Customer Request’ start event and change ‘Trigger’ type to Message.
• Do the same for the ‘Batch’ start activity as well.

• Now drag a Task activity to the customer lane and name it ‘Initialize Process’. Drag another Task activity adjacent to ‘Initialize Process’ activity and name it as ‘Credit Card Request’.
• Right click on ‘Initialize Process’ and change ‘Task Type’ to ‘Script’. Similarly change the ‘Credit Card Request’ task type to ‘User’.

• Create a Message flow from Customer Rep Line –> Initialize Process –>  Credit Card Request.
• Now create a BPMN diagram in the Visio as per the diagram below.
• Remember you can drag a Task to the diagram and change the Type to any of the below
  
  
  • Service
  • Recieve
  • Send
  • User
  • Script
  • Reference
  
  
• Similarly a Gateway can be of of the following type
  
  
  • Exclusive Data
  • Exclusive Data (with Marker)
  • Exclusive Event
  • Inclusive
  • Parallel
  • Complex
  
  
• It doesn't matter in which lane the Automatic Events or Tasks are dropped to. Only the user tasks have to be in the correct lane. Also remember to neatly label each event, activity, transition and gateway.

• Now save the Visio diagram as ‘CardApprovalProcess.vdx’. The one that i finally created can be downloaded from here.
  
• Now open JDeveloper and create a New SOA Application ‘CreditApprovalApplication’ and a new Project called ‘CreditApprovalProcess’ with SOA and BPM technologies.

• Go to the Application Navigator and right click on ‘CreditApprovalProcess’ project and click on ‘Import Model(s)’.

• Navigate to the directory where you saved your .vdx file and click on ‘Open’.

• JDeveloper will prompt for a window for options to create a single process for all ‘Pools’ in the .vdx or create separate models for each pool.
• Select ‘Merge pools into one model’ and click ‘Ok’.

• JDeveloper will now start the conversion process.

• Upon completion you would see that your BPMN process in .vdx has been converted into an Oracle 11g BPM process. Of course the layout looks horrible but it would take a small effort to rewire the layout.

• A little layout tweak and here is how the process would look. Works like a charm.

• Now an IT Analyst can implement the Business process using his set of IT services.

Here is the sample BPMN design in Visio.

And here is the zipped JDeveloper Application.