Wednesday, 24 May 2017

WebLogic Integration(WLI) to Oracle SOA Suite Migration

The Objective of this prototype is to analyze and assess the effort, and process involved in migrating the existing applications which are on WLI/ALDSP to Oracle SOA Suite 11G.

IDEXX’s WLI applications had too much of hand-written custom java code and WLI extended support by Oracle is soon coming to an end. So, IDEXX is looking for options to migrate their WLI/ALDSP codebase.

Custom written Java code and stability of the existing WLI/ALDSP code are major challenges for this WLI to SOA migration approach.

There were three options considered initially for the Migration. Each of these options explained below.

Option 1: Migrating to SOA Suite 11G
The developer Workshop (BEA-branded Eclipse Editor for WLI and ALDSP) allows converting the JPDs (Java Process Definitions) to BPEL automatically. However, this has limitations. It may work for simple processes but for complex systems, a manual migration is necessary. A per component based migration fits this scenario. This would be manual, but all the WLI controls have an equivalent adapter in SOA Suite 11g. The advantages are obvious -- as this option gets to leverage a range of full capabilities of SOA and Oracle SOA Suite 11g, which is a robust and SOA based middleware platform. Their final architecture state is much more sustainable, agile and requires less maintenance. Developers who have worked on WLI will often have a small learning curve to start working with Oracle SOA Suite 11g, this is because it’s relatively easy and straightforward. The same applies to other development teams too. The current licensing costs of WLI+DSP compared to Oracle SOA Suite 11g to be factored (if licensing is a deterrent for the customer).

Option 2: Migrating to JAX-WS/JPA or Other Java Based Frameworks
This could look easy and promising initially but get multiple frameworks to work together can be a bit challenging. This will also mean a lot of code based re-writes as pre-existing WLI controls available with the tool cannot be used. However, option 1 is much lesser code and migrating to a standard technology.

Option 3: Migrating to Oracle Service Bus
This would be a solid anti-pattern to what OSB is designed for and we should certainly not recommend it. This business logic is medium to medium-complex, and OSB will not stand as the right candidate to migrate them to, although a lot of transports in OSB may be available as equivalents of the WLI controls.

Selected Approach
The reasons for choosing SOA over the other migration approaches are explained below.
 . Robust and SOA based middleware platform
 . Agility and less maintenance
 . Working on SOA 11g is relatively easy and straightforward for developers from WLI background.
 . Exceptional handling Policy based Fault Handling and Management (Notifications, Audit Trails, Retry from point of failure)
 . Standard based integration platform.
 . Minimum or zero coding in the migrated prototype.
 . No Java coding competency is required unlike the WLI, where the source code editing requires java skills

 Tools/Technologies Used
 . WLI Eclipse based Workshop for 9.2
 . JDeveloper 11.7
 . soa-jdev extension Plugin

 Migration Plan
. Install the WLI workshop and important the WLI project into eclipse start analyzing the WLI code. The analysis of the BillingReprocessErrors jpd process can be found as below.
. The BillingReprocessErrors.jpd is a complex type project which has 5 Perform nodes, one DSP control call (getBillingHeaderById). Each of the perform nodes has java code, which is basically xmlbeans code to retrieve and initialize values to/from the xml elements.
BillingReprocessErrors.jpd also calls a utility java class, which has calls to DSP namely getCustomerHeaderById, getActivePriceGroupsByCustomerId, getLaboratoryHeaderByLabId, getOrderHeaderById, getBillingErrorById.
The Below steps explains the BPEL process activities corresponding to the WLI JPD
1. The first obtainBillingErrorRequestInfo Perform node logic in the WLI process is represented using switch case in BPEL to check if billingErrorArg is not null.
2. If the billingErrorArg is not null, loop through the BillingErrors using a While loop.
3. The Second perform node obtainBillingError Info retrieves the values of billing HeaderId and Owning LabId is represented using the Assign activity (InitializeVariables). Also, the other constant values used in the process are defined in this InitializeVariables Assign activity.
4. The condition  (BillingHeaderId is not Null) is represented using the switch activity. If BillingHeaderId is not null, the JPD has a call to retrieve BillingHeader by passing HeaderId and other parameters. This is represented using the Invoke Activity (InvokeDSPgetBillingHeaderById) in BPEL. The Invoke activity calls the DSP service, which is exposed as a Web service to the BPEL process.
5. The Third perform node reconcileDemographicsForBillingHeaderAndItems has call to the java utility class, which calls the above mentioned of the DSP control. Each of this is represented with a series of Switch and Invoke activities in the BPEL process.
6. The fourth (updateDemographicsForBillingHeadersAndItems) and fifth (updateBillingErrors) perform node have the update DSP calls for writing the BillingHeader and BillingError data to Database. This corresponds to the Invoke activities of the BPEL process.
Design :-
The below table shows the activities in the BPEL process corresponding to the nodes in the WLI JPD Process.

WLI Node
BPEL Activity
Client send and Receive node
Synchronous Receive and Reply
If Condition node
Switch activity
While Do node
While activity
XMLBeans java code
Simple Assign/Transform activities
DSP control calls
Webservice adapter calls to Oracle Service Bus
Email Control, File Control, Web Service Control and  XMLCacheControl
Email Adapter, File Adapter, WS-Adapter, and Domain Value Maps/Cross References respectively

.   The Migration approach for ALDSP is as below.
Migrating DML operations to SOA Suite 11G.
Migrating Data Query operations to OSB.
Using Coherence as an optional Layer for Data Layer Caching 

The logical and physical services have unfortunately no direct migration option on OSB or Oracle SOA Suite 11g. However, both OSB and Oracle SOA Suite 11g have an excellent support for a communicating with heterogeneous data sources using standard JCA-based database adapters. The ALDSP platform can be used to generate query plans for the operations defined in the logical services. These query plans can be used AS-IS in the JCA-based adapters providing exact same operational functionalities to be executed from within OSB and SOA Suite 11g.

. Minimal Coding in the Migrated Prototype
Existing Query Plans generated from ALDSP used to create DB adapters
Oracle Service Bus supports result can be used query based operations.
Seamless integration with Oracle Coherence, resulting in high volume transactions by using data tier.
. Caching


Development :- 

During the process, we developed WSDL files Laborator.wsdl, Order.wsdl, Pricing.wsdl, Customer.wsdl from the scratch following a bottom-up approach.
Developed SOA composite application with the project named BillingReprocessErrors, the composite application has one BPEL process named BillingReprocessErrorsprocess.bpel.
Developed OSB project with JCA DB adapter based proxy and business services.
Testing :-
Testing is planned to be executed in two phases.
Phase 1 – Sanity check is performed to ensure the web service adapters call to OSB.
Phase 2 - Actual execution of test case to test the business functionality is currently out of scope due to the in availability of connectivity to IDEXX environment, end systems, and test data. 

Deployment Process:-

Deployment of SOA applications will be done through WLST scripts or ANT scripts. Currently, the deployment is done from jdeveloper and by exporting the SCA composite jar and deploying from em console.

Effort Spent:- 
The effort spent for the Migration of the artifacts from WLI/ALDSP to SOA/OSB is summarized as below.

Lessons Learnt:-
If WLI applications have less of custom java code or custom controls, the migration could be automated and manual effort can be reduced to an extent. However, the BillingReprocessErrors.jpd uses quite a bit of java code without any transform activities. The JPD has java code comprised in perform node, hence the best fit for migration is to do a manual migration from WLI to SOA/OSB.
The logical and physical services have unfortunately no direct migration option on OSB or Oracle SOA Suite 11g. Both OSB and Oracle SOA Suite 11g have an excellent support for a communicating with heterogeneous data sources using standard JCA-based database adapters.

Acronyms and Glossary:-

Weblogic Integration
Aqua Logic Data Services Platform
Service Oriented Architecture
Web Service Definition Language
Java Process Definition
Business Process Execution Language
Oracle Service Bus

Please touch base with us if you need any help in migrating the WLI to Oracle SOA suite. We do automation of the smooth upgrade, code quality, auto deployments and continuous integrations.

Tuesday, 18 April 2017

Oracle SOA Suite 12c features- Composites Lazy loading & Lazy Deployment

Intro: While I was working with the lower version of SOA suite every time the SOA server is taking much time to come up and the composites are very delay loading as all the artifacts will be loaded at
the server startup. To get rid of this Oracle has introduced the lazy loading feature in SOA 12C.

We can implement this in SOA two ways.
1. Through SOA code by adding the property in Composite.xml
2. SOA server level by making the property true in system MBean browser.

I believe this would be the needed feature in Oracle SOA as and when it requires the connection to artifacts then it will look out for those artifacts. This will definitely improve the EM performance and server steadiness. We can enable/disable this feature on the fly in EM, without redeploying the code. There is a feasibility to adding this feature in specific composites while development. With this feature enabling in EM, we don’t see any server delay startup issues and less downtime in the maintenance window.

Note: There will be another property as CompositeLazyDeployment this will be “false” by default. It would applicable/control the composite when it’s redeployed. By default, the lazy loading would be enabled for speedy server startups.

 Enable/Disable composite lazy loading in code.

  1. Open Composite.xml in SOA project

  2. Add the property lazyLoading=”false”  à to Disable “true” à To Enable this feature

Enable/Disable composite lazy loading in EM.

Lazy Loading property in runtime:

If you want to see the property workability you can test the composite first time it would take X amount of time when you retest this composite it should take 30 times lesser than first instance time. You can observe this by looking at the instance time taking or open the server logs and see the each instance time to complete.
By enabling this feature you would see the drastic change in server startup time and speed of the EM. But still, you observe the latency in a response of the composite it would depend on the composites running on the server and a huge number of composites deployed on the server without this lazy loading feature enabled.

Happy Learning...!!!!!!!!!!!! Fun Sharing.........!!!!!!!!!


Friday, 10 March 2017

Oracle SOA suite 12.2.1- Composite Instance Patching

Oracle SOA Suite 12c (12.2.1) supports Composite Instance Patching, which enables you to patch running instances of a composite and recover faulted instances after patching the runtime. You can only include those fixes in the patch that are compatible with Composite Instance Patching. Use the SOA Patch Developer role in Oracle JDeveloper to make the fixes and create the patch.
Composite Instance Patching enables you to deliver urgent composite fixes that can be picked up by long running instances. You can make compatible/allowed changes without aborting in-flight instances. If a patched running instance comes across a business process that has been fixed by the patch, say a BPEL transformation, then it picks up the fixes applied to the business process.

When designing the patch, the SOA Patch Developer mode in JDeveloper automatically disables changes that cannot be made to the patch. Some of the compatible changes that you can make include:
·         Non-schema related XSLT changes, changes to fault policy, sensor data, and analytics data.
·         Compatible BPEL changes such as transformation activity, assign operations, etc.
·         JCA Adapter configuration properties.
You do not specify any composite version during deployment. The composite revision that you create the patch for, in Oracle JDeveloper, is the composite revision to which the patch is deployed.

You can validate the patch before deploying.

SOA Patch Developer Mode in JDeveloper

Use the SOA Patch Developer mode in Oracle JDeveloper to create a patch, containing fixes, for your deployed composite. The patch created in this mode can be applied to the currently deployed composite without changing the version number of the deployed composite. You can apply the patch to runtime even if the composite has running instances.
To use the SOA Patch Developer mode in JDeveloper:
  • If you already have your project open in JDeveloper, you need to switch to the SOA Patch Developer mode. Select Tools > Switch Roles > SOA Patch Developer from the Oracle JDeveloper menu bar

Click OK to restart JDeveloper.
If you do not have Oracle JDeveloper open, start JDeveloper and select the SOA Patch Developer role in the Select Role dialog.

After JDeveloper starts in the SOA Patch Developer mode, you’d notice that the composite editor has the SOA Patch mode label. This reminds you that you can only make edits that are compatible with the patch mode.

Also, when you are editing a BPEL component, for example, the BPEL editor has the Patch mode label.

Only certain activities in the BPEL process are available for editing, the rest of them appear in gray. Also, notice that the Components window shows only those components that are available for use in the SOA Patch Developer mode. A number of properties appear in read-only mode.

Happy Learning...!!!!!!!!!!!! Fun Sharing.........!!!!!!!!!

Oracle SOA suite 12.2.1- Integration Workload Statistics (IWS)

Integration Workload Statistics (IWS) Reports provide SOA system-wide reports that can help you analyze utilization, identify potential bottlenecks and backlogs, and perform top-down analysis of your integration system.
So, for instance, if there are stressed components or endpoints in your SOA system that are slowing down the system, IWS reports can help you narrow down on these. For example, a slow FTP or database adapter reference endpoint can be identified in the reports. Likewise, a BPEL process running slower than usual can also be identified. You can look at internal queue backlogs, like BPEL queues and EDN queues. SOA composite-wise summaries are also available.

IWS reports can include metrics like system resource usage, composite statistics, statistics for internal system queues, statistics for synchronous and asynchronous business processes, and endpoint statistics. The components supported in this release include BPEL Service Engine, EDN, Web Service Binding, File Adapter, JMS Adapter, FTP Adapter, DB Adapter, AQ Adapter, and MQ adapter.

Statistics Included in an IWS Report

An Integration Workload Statistics (IWS) report contains various statistics, depending on the data collection level that you have set. In addition to system-wide resource usage data, the report can include service and reference endpoint statistics, BPEL and EDN backup queue statistics, and BPEL instance statistics. Statistics on BPEL activities may also be included.
The IWS report contains the following broad sections when the data collection level is set to finest:

·         System Resource Usage: Statistics include Java Virtual Machine (JVM) statistics like CPU utilization and memory utilization (for JVM heap and non-heap memory), SOA Data Source statistics that show active connections and connection pool details, and SOA Work Manager statistics that include details on threads.
       ·         Composite (Rollup) Statistics: Aggregate composite-wise statistics that indicate flow rate          (throughput/transactions per second) and latency (in milliseconds) for the composite endpoints and internal   backup queues (EDN and BPEL queue).
       ·         Slowest Composite Endpoints: Aggregate composite-wise statistics that indicate the latency (in milliseconds) and flow rate (throughput) for the slowest endpoints.
       ·         Backups in Internal Queues: Aggregate statistics for the backups in internal system queues (BPEL queue and EDN queue).
      ·         Longest Running Business Processes: Aggregate statistics for top asynchronous and synchronous business (BPEL) process instances based on execution time
      ·         Most Time-Consuming Business Process Activities: Aggregate statistics for top business process activities (BPEL activities like Receive, Invoke, etc) based on execution time.

Enabling and Configuring IWS

Integration Workload Statistics (IWS) snapshot data is collected at periodic intervals. You can enable snapshot data collection, configure snapshot interval, and the granularity of data collected.
 Use the following steps to enable and configure IWS using Enterprise Manager Fusion Middleware Control:
1. Select Monitoring > IWS Reports from the SOA Infrastructure menu.
                The IWS Reports page appears.
2. Click Configure near the top right corner of the page.
               The Configure IWS Data Collection dialog appears.
3. Select a Snapshot Interval in minutes.
               The snapshot interval is the periodic interval at which data snapshots are collected.
4. Select a Data Collection Level. The level selected determines the metrics that are collected.
                The default level is OFF, which in effect disables IWS data collection. Use the Minimum level to collect only system-wide resource usage data. The Basic level additionally includes service and reference endpoint statistics, BPEL and EDN backup queue statistics, and BPEL instance statistics. If you choose Normal, it includes additional statistics on BPEL activities like Receive, Invoke, Pick, and onMessage. The Finest level additionally includes data on all BPEL activities.
5. Click OK to save your configuration changes.
               You have now configured the IWS data collection settings.
Generating an IWS Report
Use the IWS Reports page to create SOA-wide reports that help you identify bottlenecks and backlogs in the system. Integration Workload Statistics (IWS) include metrics like system resource usage, composite statistics, statistics for internal system queues, statistics for synchronous and asynchronous business processes, and endpoint statistics.
Use the following steps to create an IWS report in Enterprise Manager Fusion Middleware Control:
You must have already configured IWS data collection and set a snapshot interval.

1. Select Monitoring > IWS Reports from the SOA Infrastructure menu.
The IWS Reports page appears.
2. Select the period for which you wish to generate a report. Select timestamps for Start Date and End Date.
Ensure that the time period does not span server restarts, or periods where you have disabled IWS by setting Data Collection Level to OFF.
3. Select the SOA Server Name.
You can either accept the default server, or choose a different node in cases of multi-cluster environments. For clusters, you can also choose the cluster name to generate a consolidated report for all nodes in the cluster.
(Optional) Enter the result of the step here.
4. Optionally choose a partition name if you are using composite partitions and wish to limit your report to a particular partition.
The Select Composites field appears. This option enables you to select from all composites in the selected partition.
5. Under Select Composites, optionally choose one or more composite names to restrict your report to the specified composite applications.
6. Optionally change the number of results that are displayed.
So, if you choose the default of 10, the 10 slowest endpoints, the 10 longest running business processes, and so on, are selected.
7. Click the appropriate report format near the top right of the window to generate and download the report.
You can choose between CSV (comma-separated values), HTML, and XML formats.

The IWS report is generated and downloaded.

Happy Learning...!!!!!!!!!!!! Fun Sharing.........!!!!!!!!!

Oracle SOA suite 12.2.1- Resiliency or Circuit Breaker

I feel its a great feature to add in Oracle SOA suite. Here it is..

Whenever your end system URL is not working you will not until you test that composite. If you know that by running the service while business is in critical. Resiliency feature will showcase what URL's are UP and how many or not actually down before running those composites meanwhile we can fix the issues and get the business up to speed.

Resiliency or Circuit Breaker enables you to configure the system to automatically suspend upstream endpoints when a downstream endpoint is down in a SOA composite. This prevents fault buildup in the server and relieves you from having to bulk-recover faulted instances. The upstream endpoints are automatically resumed after the downstream endpoint comes back.
Enable Resiliency (Circuit Breaker) globally by configuring it at the SOA Infrastructure level. Once enabled, all downstream endpoints are monitored in all composites. If a downstream endpoint experiences errors that exceed the threshold, specified by you in the Resiliency configuration settings, then the upstream endpoints for that downstream endpoint are automatically suspended. So, for example, if a Reference file adapter fails to write to the directory, the upstream web service can be automatically suspended. The system will periodically check if the downstream file adapter is back, and re-enable the web service when the adapter comes back.
The following types of upstream endpoints can be automatically suspended:
·         Web Service: Incoming requests are rejected for the duration that the Web service is suspended.
·         Adapters: JMS, AQ, DB, File and FTP adapters can be automatically suspended in this release.
·         EDN Subscribers: The EDN subscriber closest to the downstream endpoint gets suspended.
Viewing and Resuming Suspended Services
·         Services suspended as a result of Resiliency kicking in appear on the SOA Infrastructure Dashboard page, under the Resiliency — Suspended Services section.

The Resiliency settings take care of automatically resuming any suspended service when the downstream endpoint comes back up.

There are additional steps to configure the Resiliency in Oracle EM/Console in Oracle SOA suite 12.2.1

Please refer Oracle documentation for more information.

Happy Learning...!!!!!!!!!!!! Fun Sharing.........!!!!!!!!!

What’s NEW in oracle SOA suite 12.2.1

New Certifications for supported platforms
Look & Feel of Enterprise Manager- It’s all new EM
JCA adapter for Siebel -  Explained in another blog post please Click here

Operational enhancements
            Resiliency: Circuit Breaker- More info Click here
            Integration Workload Statistics (IWS). Watch the IWS Video for more information.
            Composite Instance Patching - More info Click here
            In-Memory SOA - More info Click here
            Automatic Service Migration (ASM): a service fails over from an unavailable managed server to an already executing managed server.

End-to-End Native REST and JavaScript in SOA composites and Service Bus pipelines
            JavaScript activity in BPEL
            JavaScript action in Service Bus
            Handle/route any REST content type
            Access XML elements easily
            Native REST/JSON support for connecting JSON to JSON
            JavaScript used for expressions/conditions
            Converted, Typed and Un-typed REST Supported
            XSLT Debugger - More info Click here
            Conditional Breakpoints- More info Click here
            Exception Breakpoints
            New Oracle Standard Charts
            Custom Function Support for Alerts & KPIs
            SOA/BPM Process Analytics Dashboards v2
            Parameter Support in Custom Functions
            Performance Enhancements
Oracle Managed File Transfer - More info Click here
            New/improved endpoints for WebCenter, OPC Storage Cloud Service and Oracle Data Integrator (ODI)
            New APIs such as transfer creations, bulk resubmit, status, REST/SOAP run job events.
            Transfer Priorities
            Additional PGP encryption cipher options.
            Transfer life cycle management configuration plans.
Oracle B2B
            New features including PGP support, endpoint cloning and FIPS compliance.
            New T2P tools for automated cloning and promotion.
            A number of performance and resiliency improvements including features such as endpoint throttling.

Oracle SOA Suite for Healthcare Integration
            New features including support for PGP, performance and scale enhancements.
            Additional messaging patterns such as sync request-reply over MLLP.
            Custom acknowledgments for non HL7 messaging over MLLP.
            End point cloning.

            Several enhancements in the area of life cycle management such as start up, recovery, resubmit and retry.

Happy Learning...!!!!!!!!!!!! Fun Sharing.........!!!!!!!!!