Oracle Service Bus 11g and DB Adapter – Part II: The video

The following video demonstrates how the use case described in my previous blog article has been developed.

You can again download the original video if you like to follow it in better quality!

The completed solution can be downloaded from here.

Oracle Service Bus 11g and DB Adapter – Part II: Using an Inbound Database Adapter

Update 15.8.2010: Just uploaded the video for this blog article.

In Part 1 of this blog article series I presented how to use the Database Adapter with Oracle SOA Suite 11g in an outbound scenario. I showed a way to keep the JDeveloper project required to define the Database Adapter wrapped inside the Eclipse OSB project. That will become handy when extending the use case as presented in Part 2 now.

Extended Use Case

In this article I will extend the use case from Part 1 by an Inbound Database Adapter, which should poll the database for changes.  The extended scenario is shown in the image below using the notation from the Integration Blueprint book. The elements shown in blue are the new ones added to the use case from part 1.

 

The Database Adapter will be configured to listen on the PERSON_CHG_T helper table for new records. This table is filled by a trigger on the PERSON_T table and will hold one row for every change to the PERSON_T table.
For each new row in PERSON_CHG_T I want an new OSB service to be called. This new service will use the data from the inbound request and enrich it by re-using the PersonService proxy service we have built in Part 1.

Prerequisites

The prerequisites for the 2nd part are obviously the same as in Part 1. The following software needs to be installed and available:

• JDeveloper 11g with SOA extension
• Eclipse 3.5.2 with Oracle Enterprise Pack for Eclipse (OEPE) 11.1.1.5.0
• Oracle Service Bus 11.1.1.3
• Oracle Database (XE is good enough)

Additionally you need the completed OSB project from Part 1. The solution can be downloaded from here.

Project Setup

The project setup has been done in Part 1. We will reuse the same Eclipse OSB project with the nested JDeveloper SOA Project and just continue where we have left in Part 1.

Create the Inbound Database Adapter

First let’s create a new Database Adapter.

For that we don’t need a new JDeveloper project, we can reuse the same project we created in Part 1, wrapped inside the adapter folder. I think it’s a good practice to keep all the adapters necessary for one OSB project in only one JDeveloper project.

Let’s go to JDeveloper an open the composite.xml to show the SCA composite view.

1. Drag a new Database Adapter into the SCA composite. Because it’s an Inbound Adapter, we will use the left hand swimmlane named “Exposed Services” for that. This is not strictly necessary when using the OSB but I think it’s a good mnemonic trick to do so (organizing inbound adapters on the left and outbound adapters on the right, as discussed in part 1).
   
2. Give the adapter service a good and meaningful name:
   
3. For the connection we reuse the settings already their from part 1, so we can move forward to the the Operation Type selection. This time we want to use the Database Adapter to “Poll for New or Changed Records in a Table”.
   
4. We want to poll the PERSON_CHG_T table, so let’s select it.
5. We can see that the table only holds an ID and a timestamp. So that’s all we get in the inbound message, whenever a row is inserted into PERSON_CHG_T. This is the reason why we later want to enrich the message with more information in a second step.
    
6. Next we need to define the strategy to use for signaling that a row has been read and successfully processed by the adapter. Because PERSON_CHG_T is a helper table no one else is using, it’s fine to just delete the row.
    
7. Next the Polling Options can be specified. Among others you can specify the polling frequency, which is set to 5 seconds by default, meaning that the Adapter will do the SQL operation shown on the right every 5 seconds. For our sample that’s fine, but in real world you should of course set it to a value matching your requirements.
   
8. Last but not least the Database Adapter allows for setting a selection criteria. We don’t use it this time, as we want to read all the rows which are added to the PERSON_CHG_T.

This finishes the creation of the Inbound Database Adapter and our work in JDeveloper. We can see the Adapter on the right hand swimmlane.

 

The adapter is now prepared to poll the PERSON_CHG_T table for new records every 5 seconds. Each row being read will be send to the service linked to the adapter. So let’s switch to the OSB project in Eclipse and create a new service to handle these messages.

Creating the OSB Service and linking it to the Inbound Database Adapter

When working with Inbound Adapters, an OSB proxy service needs to be used. The adapter will invoke the proxy service whenever a new message “is created” by the adapter.

1. In order to be able to create/generate the proxy service, we need the new adapter artifacts in Eclipse. Just do a refresh on the adapter folder and they will show up.
2. No we can choose Generate Service on the JCA configuration file (PollingPersonService_db.jca) to create the necessary OSB service.
   
3. Based on the JCA settings, OSB knows that it is an Inbound Adapter and will generate a JCA Proxy Service automatically. All we need to specify is the right folder:
4. The proxy settings, created for you, show that a WSDL is used which has been generated as well:
   
5. The transport setting show the usage of the JCA Transport:
   
6. All we need to do is specify what should happen with the message, by defining a meaningful Message Flow. For a start add a Pipeline Pair Node with a nested Stage Node and a Log action to show the message in the OSB log on the console. Make sure to specify a Severity level in the Log action which is shown in the log. If you are unsure what to choose, then “Error” will be fine for that sample and shown by default.
   
7. Now let’s deploy the OSB project and test if the Inbound Adapter works. For that let’s open SQL PLus, connect to SOA_SAMPLE and do an UPDATE on the PERSON_T table. By that the trigger on that table will fire and signal the change by adding a row to the PERSON_CHG_T table. Make sure to commit the change!
8. After a maximum of 5 seconds (remember the polling frequency specified in the Database Adapter wizard) the log should show up on the OSB console window.

 

We can see that the polling Database Adapter worked. A message has been sent to the OSB proxy service holding the ID of the changed PERSON_T row and a timestamp!

In a real world scenario you would now want to do something more meaningful with this information than just logging it to the console, i.e. you want to inform another system about the change. In order to do that, you might need to send more information than just the ID of the person. The system to inform maybe require the person information, similar to the information returned by the PersonService we developed in Part 1. So let’s reuse that proxy service to enrich our message, implementing the Content Enricher design pattern.

Adding the Content Enricher

To enrich our message, we want to call the PersonService proxy service from the Message Flow of the PollingPersonServiceDB proxy service.

1. Let’s add a Service Callout action and rename the stage to EnrichmentStage. It’s always a good idea to meaningfully name the different nodes used to structure the message flow. This helps you to better understand and document your message flow at development time but also helps in case of errors at runtime, to easier identify the place where the error occurred.
    
2. Configure the Service Callout action to call the PersonService proxy and to invoke the findPerson operation. For the request and response message we define two variables and specify to use a Soap Body. The Service Callout action allows to use separate variables for the request and response message. By that the content of the $body variable from the request to the proxy service stays untouched during the service callout. This is important if you want to merge the response from the service callout with the original request. This is not necessary in our simple example, all we will use is the response directly from the service callout. But usually you will need to merge the two when implementing the Content Enricher pattern in OSB.
    
3. Next we implement the Assign action to set the requestBody variable. 
    
4. Because we specified “Configure Soap Body” in the Service Callout properties, we need to setup the <findPersonRequest> message wrapped in a <soap-env:Body> element. The value of the <personId> element can be retrieved from the $body variable by dragging it into the Expression view and defining the XPath expression shown in the image below
   

    
   

5. Last but not least you need to add v1 as a custom namespace:
   
6. In the Response Action of the Service Callout we will also use an Assign action, this time to copy the value of the $responseBody variable to the $body variable.
    
7. Let’s change the Annotation of the Log action from before to state the fact that we now log the content of the $body variable after the service callout has been made.
   
8. Let’s test it in the same way as before. Just re-execute the UPDATE on PERSON_T and this time a longer log message with a complete Person instance should be shown.
   

 

The Content Enrichment worked an the complete and up-to-date person information could now be sent to any system interested.

Conclusion

This finishes the 2nd part of this blog article series.

We have added an Inbound Adapter to the use case to get informed whenever the information changes in the PERSON_T table. By re-using the PersonService from a Service Callout in the Message Flow we were able to enrich the incoming message to a more meaningful “change message”, which could now be used to inform potential external systems of changes happening on the PERSON_T table.

We have used the OSB to implement parts of a typical integration scenario. Similar to one of the scenarios documented in our Integration Blueprint book!

The implementation of a dynamic publish-subscribe mechanism on the OSB, in order to inform the systems interested could be a topic of a next blog article.

The source code for the solution can be downloaded from here. I will again provide a video showing how this extension of the use case has been developed.

Will be speaking at ….

The next couple of months will be interesting with lots of exiting events.

Here a list of events I will attend together with the sessions I will give:

• Oracle Open World – San Francisco (USA)
  • Using a State Engine as an SCA component in SOA Suite 11g – 23rd September
• SOA and Cloud Symposium – Berlin (Germany)
  • Best Practices for Testing SOA Based Systems – 5th October
  • Using Domain Specific Language(s) to simplify creating SOA artifacts -  5th October
• DOAG SIG SOA, Köln, Germany
  • Service-Oriented Architecture: An Integration Blueprint – 20th October
• DOAG Conference 2010 – Nürnberg (Germany)
  • Normalisierung von CDC Events mit Complex Event Processing (CEP) – 16th November
  • State Engine als SCA Komponente in Oracle SOA Suite 11g – 17th November

Some interesting topics to talk about! Preparing the different topics will definitely keep me busy for the next few weeks.

Would be nice to meet at one of these events!

 

Oracle Service Bus 11g and DB Adapter: a more integrated approach! – the video tutorial

The following video demonstrates how the use case described in my previous blog article has been developed.

Sorry for the quality of the video. The original is of much better quality but VideoPress seem to have “optimized” it. You can download the original video if you like to follow it in better quality!

Please let me know if you find this kind of video tutorial helpful.

Oracle Service Bus 11g and DB Adapter: a more integrated approach!

Update 9.8.2010: Just uploaded a video showing how the use case described in this blog has been developed.
Update 15.8.2010: Part II: Using an Inbound Database Adapter has been published today.

The JCA adapter framework we know from SOA Suite is supported by the Oracle Service Bus (OSB) since 10.3.1. The Database Adapter fills one gap of the Oracle Service Bus: there is no OSB transport for accessing a database and accessing the database was previously only possible from an XPath function in read-only mode.

Many blog articles have already been published about using the JCA adapters with Oracle Service Bus. There are two good blog articles from Edwin Biemond and from James Taylor about how to use the Database Adapter with Oracle 10g and 11g. Additionally the Oracle Service Bus Samples page holds a viewlet that demonstrates the usage of the DB adapter with 10.3.1. So why another blog article?

First the Database Adapter is a feature, which deserves many blog articles and second when I went through the samples mentioned above, I’ve found a way to better integrate the definition of the JCA adapter with the OSB proxy service and business service development, which makes the handling much easier.

One of the difficulties when using the JCA adapter framework with the OSB is the two different IDE’s being necessary. The adapter wizards are only available in JDeveloper and therefore for the definition of the adapters JDeveloper needs to be used. After that only the artifacts generated by the adapter wizard (WSDL, XSD, JCA config, toplink mappings, ..) are necessary.

The approaches described by the sources mentioned above show how to create a JDeveloper project first, create the adapters and then copy the necessary files into the OSB projects. What I don’t like about that is the copying of the resources. Of course this can be automated, but when you have to go back an forth between the adapters and the OSB project during development, because you need to change the settings of the adapters multiple times, its just a matter of time until you for once work with an non-actualized version of some files. So how can we avoid that?

Of course we can not change the fact that we have to work with Eclipse and JDeveloper in parallel, until Oracle has moved the whole OSB development environment to JDeveloper, probably with 11R2.

The approach I present here is actually quite simple. Instead of having to separate projects, I just create the JDeveloper project embedded inside the OSB project in a special folder (adapter) as shown in the image below.

By that, all the adapter for one OSB project can be placed in one single JDeveloper SOA project and by that all the generated artifacts are always local to the OSB project. This way they can be used to generate a proxy or a business service directly.

If an adapter needs to be changed, then a refresh on the adapter folder is good enough to pick up the new version of the adapter files. No more copying of files between the two projects is necessary!

Some of the older sample available on the Web show how to use the OSB console to import the artifacts generated by the adapter. This is also no longer necessary! Everything can be done directly in Eclipse in 11g.

Use Case

The use case I will demonstrate is rather simple. The idea is to make the data of 3 tables in an Oracle DB accessible as a web service in a contract-first approach.

I will use the Database Adapter to access the data, wrap it by a business service and use a proxy service with two XQuery transformation to publish it as a SOAP WebService with its own WSDL and XSD. By that the data is immediately available to any SOAP WebService consumer. I will use SoapUI to demonstrate that.

The scenario is shown in the image below. I’m using the notation from our Integration Blueprint book.

Prerequisites

In order to follow the tutorial below, the following software has to be available:

• JDeveloper 11g with SOA extension
• Eclipse 3.5.2 with Oracle Enterprise Pack for Eclipse (OEPE) 11.1.1.5.0
• Oracle Service Bus 11.1.1.3
• Oracle Database (XE is good enough)

On the Oracle Database you have to install the SOA_SAMPLE schema available in download here. Just execute the cr_obj.sql located inside the database folder.

The Web Service interface (WSDL and XSD) to be published by the proxy service are available in the misc folder in the download. If you follow the tutorial then it’s assumed that this two files are available in c:\temp.

Project setup

Let’s first create the Oracle Service Bus project and inside in the adapter folder the nested JDeveloper SOA project

1. First create a new Oracle Service Bus Project and create a folder structure for the different artifacts created later.
2. The adapter folder is the place where we will embed the JDeveloper project. Check and copy the name of the folder to be used when creating the JDeveloper SOA project.
   
3. Now let’s switch to JDeveloper for a while and create the new SOA Project (inside the adapter folder of the OSB project), which will define and hold the adapter artifacts.
    
4. Choose Empty Composite for the Project template. We will only use the SCA composite to place a Database Adapter and we won’t use any of the components like BPEL or Mediator.
   
5. The empty composite window shows up. You can think of the Components section as the place where your OSB proxy and business services are located, although that’s not true before probably 11gR2. Using that mnemonic trick you can place the adapters in the same way as you are used from SOA Suite. Inbound adapters (file polling, database polling, de-queuing) should be placed on the Exposed Services swimmlane and outbound adapters (file write, database read/write, enqueueing, …) should be placed on the External References swimmlane.
    

All the JCA adapters used by one OSB project can be defined in the same SOA project.

Create the Database Adapter

With the project setup in place, let’s now configure the Database Adapter by going through the adapter wizard.

1. We will need an outbound Database Adapter, so we drag it to the right hand side swimmlane.
   
    
2. Give the adapter service a meaningful name
   
3. Create a connection to be used only at development time during the wizard and specify a JNDI Name to be used to retrieve the database connection factory at runtime. The Connection Factory object need to be setup on WebLogic before deploying the OSB project.
   
4. For the Operation we choose Select, we only want to read from the database.
   
5. In the next step the tables to SELECT from are specified. We want to read from PERSON_T, ADDRESS_T and COUNTRY_T all together; the PERSON_T should be the root table to start the query from.
   
6. The next step allows for creating the necessary relationships between the tables. PERSON_T has a 1:m relationship to ADDRESS_T which has a 1.1 relationship to COUNTRY_T
   
7. In the Attribute Filtering step all the attributes returned from the tables are shown and you can uncheck the attributes you don’t want the service to return. Here we specify that we don’t want to return the ISO Country number. You can also see that the hierarchical structure resembles the relationships defined above.
   
8. In the next step we define the restriction to be applied by the service. By default all the rows in PERSON_T would be returned. Our service should only return a given person defined by it’s primary key. So we define a personId parameter and add it in a WHERE clause.
   
9. By that the adapter is defined an we can click finish on the next page. The adapter wizard now generates the necessary artifacts like WSDL, XSD, JCA configuration and toplink mapping files.
   

This finishes the work in JDeveloper. Let’s switch back to Eclipse and the OSB project created before.

Create the Business Service

In order to use the Database Adapter from OSB, we either need a business or proxy services configured to use JCA transport. For outbound adapters, a business service is necessary, whereas for inbound adapters, a proxy service is used.

1. First let’s make the artifacts from the JDeveloper project visible in the OSB project by doing a refresh on the adapter folder. We can see the structure of our SOA project nested in the OSB project.
              
2. Now let’s create the business service, which will wrap our outbound Database Adapter defined above. We can do that directly in Eclipse, there is no longer a need to use the OSB console for that. Just right-click on the jca configuration file (RetrievePersonService_db.jdca), select Oracle Service Bus | Generate Service and specify the name and the folder where the business service and the WSDL should be created (folder business-service).     
3. The transport configuration is automatically done for us, nothing needs to be done here:

By that the business service is created and ready to be used. It could already be tested from the OSB console, but we want the service to be reachable from outside via a SOAP Web Service. So let’s create the proxy services doing exactly that.

Create the Proxy Service

When creating the proxy service it’s good to first think about the service interface it should provide. A SOAP based WebService interface is what we want, but what format do we use? Can’t we just use the WSDL generated by the Database Adapter also for the proxy service? It’s so easy, isn’t it?

It would be possible, but by that, we would expose information from the database to the outside and by that create a much stronger coupling between the service consumer and the database than necessary. We would use a contract-last approach, where the contract is just generated based on some artifacts already available! A change on the database (table name, column name, data type) would almost for sure have an impact on the interface, something we definitely want to avoid when using the service in a larger context in a Service-Oriented Architecture (SOA).

What we want to use is a contract-first approach, where we can independently define the service contract first. Fortunately that’s well supported by OSB and easily achieved by defining a new WSDL, using it when defining the proxy service and creating two transformation operation to translate to/from the new format.

The WSDL and XSD forming the service contract PersonService is available in the download. It uses a canonical format of a person and its addresses which is somehow different to the format used on the database and independent of any backend service.

1. First import the WSDL and XSD files into the wsdl folder of the OSB project
   
2. Now let’s create the proxy service 
    
3. Select the WSDL PersonService-1.0.wsdl for the interface of the proxy service
   
4. Add a Route Node with a Routing action to the empty message flow of the proxy service
    
5. Select the business service just created as the service to call by the route action
   

Create the two transformations

For the transformation of the request and of the response we need one XQuery transformation each.

1. First we create the XQuery transformation for the request, which is very easy, all we need to map is the personId query parameter. With the graphical mapper feature provided by the OSB Eclipse plugin it’s even easier!
   
2. Next we create the XQuery transformation of the response. This is a bit harder, as more items need to be mapped, but with the build in graphical mapper it’s again not a lot of work!
   

Add transformation to the message flow of the proxy service

Now the only thing left to do is inserting the two transformations into the message flow.

1. First we use the request transformation in the Request Action of the Routing action. By using a Replace action the already existing body with the <soap-env:Body> tag is reused and only the content is replaced by the result of the XQuery.
   
2. The response is handled similar to the request by another Replace action
   
3. In the parameter binding to the XQuery we manually have to specify the PersonTCollection element which holds the response from the DB adapter.
   
4. Additionally we also have to add a user-defined namespace
   

Create DataSource and Connection Factory in WebLogic

Before we can deploy and use the OSB service we need to create the necessary objects in WebLogic.

1. First we create the DataSource object with the JNDI alias jdbc/SoaSampleDataSource

2. Create the DB Adapter Connection Factory object
3. and configure the DataSource jdbc/soaSampleDataSource created above
   

Don’t forget to update the DB Adapter in order to activate the configuration changes.

Now it’s time to deploy and test the OSB service.

Deployment and Testing with soapUI

Deploy the OSB service to the OSB server and then start soapUI.

1. Create a new soapUI project and specify the WSDL the proxy service on the OSB exposes. On my machine this is  http://localhost:7001/DbAdapterOSBProject/PersonService?wsdl.
   
2. Now let’s use the generated request and send a message with personId = 1.
   
3. We should get a successful response like the one shown in the image below. This is the information from the database in the canonical format.

Conclusion

This finishes the tutorial of using the Database Adapter with the Oracle Service Bus.

I hope I was able to show how easy it is to integrate the JCA adapter framework with Oracle Service Bus 11g. Although there are two IDE’s involved, the strategy of embedding the JDeveloper SOA project inside the OSB project helps in keeping the OSB project in sync with the SOA project. By that it’s much easier to maintain the adapter, just change the settings by restarting the adapter wizard and after refreshing the OSB project everything is in sync.

In a next blog article I will show how to use the Database Adapter in an inbound scenario, where the adapter will trigger an OSB proxy service whenever a new row is added to the database.

The source code with the implementation of this use case can downloaded from here.