The Server Labs Blog Rotating Header Image

SOA

ALSB/OSB customization using WLST

One of the primary tasks in release management is environment promotion. From development to test or from test to production, environment promotion is a step which should be as much automated as possible.

We can use the service bus MBeans in WLST scripts to automate promotion of AquaLogic/Oracle Service Bus configurations from development environments through testing, staging, and finally to production environments.

Each environment has particularities which may need changes in configuration of the software. These are usually centralized in property files, database tables, environment variables or any other place to facilitate environment promotion.

In AquaLogic/Oracle Service Bus there is the concept of environment values:

Environment values are certain predefined fields in the configuration data whose values are very likely to change when you move your configuration from one domain to another (for example, from test to production). Environment values represent entities such as URLs, URIs, file and directory names, server names, e-mails, and such. Also, environment values can be found in alert destinations, proxy services, business services, SMTP Server and JNDI Provider resources, and UDDI Registry entries.

For these environment values, we have different standard operations

  • Finding and Replacing Environment Values
  • Creating Customization Files
  • Executing Customization Files

However, these operations are limited to the ‘predefined fields whose values are very likely to change’… and what happens if we need to modify one of the considered ‘not very likely’? A different story is whether to consider SAP client connection parameters ‘not very likely’ to change in a environment promotion from test to production…

In order to automate these necessary changes, one option is to modify directly the exported configuration prior to importing it to the destination environment but in our case, we want to maintain the philosophy of the customization after the importing, keeping the exported package untouched. We will try to use a WLST script instead of a customization file, as the later doesn’t satisfy our needs.

The first thing we have to do for using WLST is to add several service bus jar files to the WLST classpath. For example, if we have a Windows platform we add the following at the beginning of wlst.cmd file (I’m sure *nix people will know how to proceed in their case)

For Aqualogic Service Bus 3.0:

SET ALSB_HOME=c:\bea\alsb_3.0
SET CLASSPATH=%CLASSPATH%;%ALSB_HOME%\lib\sb-kernel-api.jar
SET CLASSPATH=%CLASSPATH%;%ALSB_HOME%\lib\sb-kernel-common.jar
SET CLASSPATH=%CLASSPATH%;%ALSB_HOME%\lib\sb-kernel-resources.jar
SET CLASSPATH=%CLASSPATH%;%ALSB_HOME%\lib\sb-kernel-impl.jar
SET CLASSPATH=%CLASSPATH%;%ALSB_HOME%\..\modules\com.bea.common.configfwk_1.1.0.0.jar
SET CLASSPATH=%CLASSPATH%;%ALSB_HOME%\..\modules\com.bea.alsb.statistics_1.0.0.0.jar

For Oracle Service Bus 10gR3:

SET ALSB_HOME=c:\bea\osb_10.3
SET CLASSPATH=%CLASSPATH%;%ALSB_HOME%\lib\sb-kernel-api.jar
SET CLASSPATH=%CLASSPATH%;%ALSB_HOME%\lib\sb-kernel-common.jar
SET CLASSPATH=%CLASSPATH%;%ALSB_HOME%\lib\sb-kernel-resources.jar
SET CLASSPATH=%CLASSPATH%;%ALSB_HOME%\lib\sb-kernel-impl.jar
SET CLASSPATH=%CLASSPATH%;%ALSB_HOME%\..\modules\com.bea.common.configfwk_1.2.1.0.jar
SET CLASSPATH=%CLASSPATH%;%ALSB_HOME%\..\modules\com.bea.alsb.statistics_1.0.1.0.jar

In our example, we will try to change the HTTP timeout in the normalLoanProcessor business service present in ALSB/OSB examples server.

normalLoanProcessor

normalLoanProcessor configuration

For that, we will first connect to the bus from WLST and open a session using SessionManagementMBean

from com.bea.wli.sb.management.configuration import SessionManagementMBean
connect("weblogic", "weblogic", "t3://localhost:7021")
domainRuntime()
sessionMBean = findService(SessionManagementMBean.NAME, SessionManagementMBean.TYPE)
sessionName = "mysession"
sessionMBean.createSession(sessionName)
mysession

mysession shown in sbconsole

Nothing new until now. Next thing we need is a reference to the component you want to modify. We chose to use a BusinessServiceQuery like:

from com.bea.wli.sb.management.query import BusinessServiceQuery
from com.bea.wli.sb.management.configuration import ALSBConfigurationMBean
bsQuery = BusinessServiceQuery()
bsQuery.setLocalName("normalLoanProcessor") 
bsQuery.setPath("MortgageBroker/BusinessServices")
alsbSession = findService(ALSBConfigurationMBean.NAME + "." + sessionName, ALSBConfigurationMBean.TYPE)
refs = alsbSession.getRefs(bsQuery)
bsRef = refs.iterator().next()

After this we have a reference to the business service we want to modify. Now is when fun begins.

There is an undocumented service bus ServiceConfigurationMBean (not to be confused with old com.bea.p13n.management.ServiceConfigurationMBean) whose description is ‘MBean for configuring Services’.

ServiceConfiguration.mysession as shown in jconsole

Among the different methods, we find one with an interesting name: getServiceDefinition

getServiceDefinition as shown in jconsole

It looks that we can use the getServiceDefinition method with our previous reference to the business service for obtaining exactly what its name states.

from com.bea.wli.sb.management.configuration import ServiceConfigurationMBean
servConfMBean = findService(ServiceConfigurationMBean.NAME + "." + sessionName, ServiceConfigurationMBean.TYPE)
serviceDefinition = servConfMBean.getServiceDefinition(bsRef)

This is the result of printing serviceDefinition variable:


  
    
    
      
      
        NormalLoanApprovalServiceSoapBinding
        http://example.org
      
    
    
      5
    
    
      normal
    
    
      wsdl-policy-attachments
    
  
  
    http
    false
    
      http://localhost:7021/njws_basic_ejb/NormalSimpleBean
    
    
      none
      0
      30
      true
    
    
      
        POST
        0
      
    
  

Surprised? It’s exactly the same definition written in .BusinessService XML files. In fact, the service definition implements XMLObject.

Now it’s time to update the business service definition with our new timeout value (let’s say 5000 milliseconds) using XPath and XMLBeans. We must also take care of defining namespaces in XPath the same way that are defined in .BusinessService XML files.

nsEnv = "declare namespace env='http://www.bea.com/wli/config/env' "
nsSer = "declare namespace ser='http://www.bea.com/wli/sb/services' "
nsTran = "declare namespace tran='http://www.bea.com/wli/sb/transports' "
nsHttp = "declare namespace http='http://www.bea.com/wli/sb/transports/http' "
nsIWay = "declare namespace iway='http://www.iwaysoftware.com/alsb/transports' "
confPath = "ser:endpointConfig/tran:provider-specific/http:outbound-properties/http:timeout"
confValue = "5000"
confElem = serviceDefinition.selectPath(nsSer + nsTran + nsHttp + confPath)[0]
confElem.setStringValue(confValue)

We are almost there. First we update the service.

servConfMBean.updateService(bsRef, serviceDefinition)

Modified mysession shown in sbconsole

And finally, we activate the session (see NOTE) like we would do in bus console.

sessionMBean.activateSession(sessionName, "Comments")

mysession changes shown in sbconsole

Task details of mysession

Updated normalLoanProcessor configuration

With this approach, it could be possible to build a framework that allows to customize ALL fields as needed.

NOTE:
If you get the exception below when activating changes, please update your WebLogic Server configuration as described in Deploy to Oracle Service Bus does not work

Traceback (innermost last):
  File "", line 1, in ?
com.bea.wli.config.deployment.server.ServerLockException: Failed to obtain WLS Edit lock; it is currently held by user weblogic. This indicates that you have either started a WLS change and forgotten to activate it, or another user is performing WLS changes which have yet to be activated. The WLS Edit lock can be released by logging into WLS console and either releasing the lock or activating the pending WLS changes.
        at com.bea.wli.config.deployment.server.ServerDeploymentInitiator.__serverCommit(Unknown Source)
        at com.bea.wli.config.deployment.server.ServerDeploymentInitiator.access$200(Unknown Source)
        at com.bea.wli.config.deployment.server.ServerDeploymentInitiator$1.run(Unknown Source)
        at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:363)
        at weblogic.security.service.SecurityManager.runAs(Unknown Source)
        at com.bea.wli.config.deployment.server.ServerDeploymentInitiator.serverCommit(Unknown Source)
        at com.bea.wli.config.deployment.server.ServerDeploymentInitiator.execute(Unknown Source)
        at com.bea.wli.config.session.SessionManager.commitSessionUnlocked(SessionManager.java:420)
        at com.bea.wli.config.session.SessionManager.commitSession(SessionManager.java:339)
        at com.bea.wli.config.session.SessionManager.commitSession(SessionManager.java:297)
        at com.bea.wli.config.session.SessionManager.commitSession(SessionManager.java:306)
        at com.bea.wli.sb.management.configuration.SessionManagementMBeanImpl.activateSession(SessionManagementMBeanImpl.java:47)
[...]

SCA Async/Conversational services Part 2: Non-SCA Web Service client

Following my previous post on the internals of asynchronous and conversational services in Tuscany SCA, which options are available for consuming these services when you cannot use a Tuscany SCA runtime on the client?.

Depending on the transport binding used you would expect to find a certain level of standarisation on conversational/asynchronous services implementation, allowing interoperable solutions. However, it is difficult, if not impossible, to find interoperable solutions for these types of services, even for highly standarised bindings such as SOAP Web Services. We have seen in the previous post how Tuscany SCA handles these services for WS and JMS bindings. We saw that at least for Web Services, some WS-* standards were used (i.e. WS-Addressing), but there are still solution-specific details that do not allow interoperability. This reflects how difficult it is for the standarisation community to define detailed standards and mechanisms to enable interoperable solutions for these kind of services.

In this post I present an option for this situation, using an standard JAX-WS client, attaching the appropriate WS-Addressing headers using JAXB. For that, I extend the pure Tuscany SCA client-server sample of the previous post with a non-SCA client.

You can find the extended source code of the maven projects here.

Overview

The two main pieces to setup on a JAX-WS client are:

  1. Proper WS-Addressing headers containing the Tuscany’s specific information (i.e callbackLocation, conversationID and optionally callbackID).
  2. A web service implementing the callback interface, listening in callbackLocation URI.

JAX-WS WS-Addressing headers setup

As for a normal WSDL-first client development, the first step is to generate the JAX-WS client classes from the Counter Service WSDL. This is done using the maven jaxws-maven-plugin on the wsdl file generated for the service (i.e. http://localhost:8086/CounterServiceComponent?wsdl).

To setup the SOAP WS-Addressing headers, I follow the recommendations from the JAX-WS guide and use WSBindingProvider interface, which offers a much better control on how headers can be added.

Therefore, in the constructor of CounterServiceClient.java, the WS-Addressing “TuscanyHeader” is added:

     public CounterServiceClient() {
        super();
        
        // Initialise JAX-WS Web Service Client.
        service = new CounterServiceService();
        servicePort = service.getCounterServicePort();
  
        // Set the WS-Addressing headers to use by the client.
        WSBindingProvider bp = (WSBindingProvider) servicePort;
        // Generate UUID for Tuscany headers
        conversationId = UUID.randomUUID().toString();
        callbackId = UUID.randomUUID().toString();
        ReferenceParameters rp = new ReferenceParameters(conversationId);
        rp.setCallbackId(callbackId);
        TuscanyHeader header = new TuscanyHeader(CALLBACKURI, rp);        
        bp.setOutboundHeaders(Headers.create((JAXBRIContext) jaxbContext,
                header));
        
        ...
    }        

The ReferenceParameters and TuscanyHeader classes are JAXB classes with the required information to map the TuscanyHeader Java object to the WS-Addressing XML header. For instance, the ReferenceParameters class, which includes the Tuscany SCA parameters, has the following JAXB definitions:

public class ReferenceParameters {
    private static final String TUSCANYSCA_NS = "http://tuscany.apache.org/xmlns/sca/1.0";
    /** The conversation id. */
    private String conversationId = "";
    /** The callback id. */
    private String callbackId = null;

    ...
    public void setCallbackId(String callbackId) {
        this.callbackId = callbackId;
    }

    public void setConversationId(String conversationId) {
        this.conversationId = conversationId;
    }

    @XmlElement(name = "CallbackID", namespace = TUSCANYSCA_NS, required = false)
    public String getCallbackId() {
        return callbackId;
    }

    @XmlElement(name = "ConversationID", namespace = TUSCANYSCA_NS, required = true)
    public String getConversationId() {
        return conversationId;
    }
}

JAX-WS callback service setup

The callback service needs to be created on the client side. For that we define a simple web service (CounterServiceCallbackImpl.java) with the CounterServiceCallback as the contract.

/**
 * The CounterServiceCallback implementation.
 * The Web service namespace must be the same as the one defined in the Server Side.
 */
@WebService(targetNamespace = "http://server.demo.tuscany.sca.tsl.com/")
public class CounterServiceCallbackImpl implements CounterServiceCallback {
	private int count;

	@Override
	public void receiveCount(int count, int end) {
		System.out.println("CounterServiceCallback --- Received Count: " + count + " out of " + end);
		this.count = count;
	}
        ...
}

In this sample project, the web service is also started during the instantiation of the JAX-WS Client. We use the embedded Sun httpserver to publish the web service instead of relying in other web application servers as Jetty or Tomcat:

     public CounterServiceClient() {
        super();
        ...
        
        // Setup Receiver Callback Web Service
        System.out.println("Starting Callback service in " + CALLBACKURI);
        callback = new CounterServiceCallbackImpl();
        callbackServiceEp = Endpoint.create(callback);
        callbackServiceEp.publish(CALLBACKURI);
    }        

Testing the client

As in the previous post, in order to run the client tests you need to:

  • Run the server side, executing the run-server.sh under the counter-callback-ws-service directory. The server component should start and wait for client requests.
  • Go to the client project, counter-callback-ws-client-SCA and execute the tests with:
    mvn clean test
    

    This runs both the SCA and non-SCA client tests.

  • In case you want to run two clients at the same time, you need to define two separate CallbackURIs. For that purpose the property sca.callbackuri has been defined to configure the URI to use. Therefore to run two clients in parallel execute
    mvn  test -Dsca.callbackuri="http://localhost:1999/callback" -Dtest=com.tsl.sca.tuscany.demo.client.WebServiceNonSCAClientTest 
    and another client with 
    mvn  test -Dsca.callbackuri="http://localhost:2999/callback" -Dtest=com.tsl.sca.tuscany.demo.client.WebServiceNonSCAClientTest 
    

    Once running you should see the different service calls and conversations IDs interleaved in the server side console:

    setCounterServiceCallback on thread Thread[Thread-2,5,main] 
    ConversationID = 81e257cd-af3b-4ea6-ae69-4b8e0d9db8a9. Starting count 
    Sleeping ...
    setCounterServiceCallback on thread Thread[Thread-5,5,main]                 
    ConversationID = 70e86dcf-436c-4df1-ab6b-165b3f9070a4. Starting count   
    Sleeping ...
    Sleeping ...
    Sleeping ...
    Sleeping ...
    Sleeping ...
    Sleeping ...
    Sleeping ...
    Sleeping ...
    Sleeping ...
    ConversationID = 81e257cd-af3b-4ea6-ae69-4b8e0d9db8a9. Stopping count
    Sleeping ...
    ConversationID = 70e86dcf-436c-4df1-ab6b-165b3f9070a4. Stopping count
    Sleeping ...
    Sleeping ...
    Sleeping ...
    Sleeping ...
    Sleeping ...
    Sleeping ...
    Sleeping ...
    Sleeping ...
    Sleeping ...
    ConversationID = 81e257cd-af3b-4ea6-ae69-4b8e0d9db8a9. Restarting count
    Sleeping ...
    ConversationID = 70e86dcf-436c-4df1-ab6b-165b3f9070a4. Restarting count
    Sleeping ...
    Sleeping ...
    Sleeping ...
    Sleeping ...
    Sleeping ...
    Sleeping ...
    Sleeping ...
    Sleeping ...
    Sleeping ...
    Thread interrupted.
    Thread interrupted.
    

Conclusions

I hope that these posts have shed some light on how SCA and conversational/async services are implemented taking Tuscany as SCA runtime reference. I also believe that it is important to know the available options for consuming a SCA service without an SCA runtime, and how we can do it in a simple manner.

SCA Async/Conversational services Part 1: Internals of Tuscany SCA

Sometime ago I wrote about developing applications with SCA on a JBI-based infrastructure, using simple SCA services for that.

I’m coming back again with two separate SCA blog posts discussing the usage of more complex services, asynchronous and conversational services:

  • In this post, I provide an example client-server project that implements a conversational and asynchronous service in Tuscany SCA, digging into the internals of the implementation for handling these services using Web Services and JMS bindings. I found this information quite difficult to find.
  • The second post looks into the situation where the usage of a Tuscany SCA runtime is not possible in the client side, but we still need a way to make use of these type of services from a more standard web service client. For the specific case of a web service binding, and knowing the internals of Tuscany I will use a standard jax-ws client to consume a Tuscany SCA conversational and asynchronous service.

Both posts use a common example project, a Counter service that notifies to the client each time the count is updated (every second) via an asynchronous callback. The service is conversational and allows clients to stop the count and restart it through separate independent service calls. Also, multiple clients can be run in parallel, each one with its own counter service instance. Version 1.6 of Tuscany SCA is used in this sample.

The source code of the sample counter service for this post can be found here.

SCA asynchronous and conversational service definition

Let’s refresh how to define a service in Tuscany SCA that is both conversational and defines a callback interface for asynchronous calls to the client. Good references for this are the SCA specifications, the Tuscany SCA documentation and the sample projects included in the distribution. However, while I could find several callback projects on the Tuscany SCA samples there are none that exercise conversations.

Below, you find the interface definition of the SCA service in Java for the CounterService (counter-callback-ws-service maven project).

I use the @Callback and @Conversational annotations to define our service. For the @Callback annotation we need also to define the callback interface class. Therefore, for our counter service we have two interfaces, one for the service which implementation is done in the server side, and another one for the callback which implementation is done in the client side.

/**
 * The remote service that will be invoked by the client
 */
@Remotable
@Callback(CounterServiceCallback.class)
@Conversational
public interface CounterService {

    @OneWay
    void startCounting(int maxcount);
    
    @OneWay
    void stopCounting();

    @OneWay
    void restartCounting();    
    
    @OneWay
    void shutdownCounting();     
}

The Callback interface is as simple as:

/**
 * The callback interface for {@link CounterService}.
 */
@Remotable
public interface CounterServiceCallback {

    void receiveCount(int count, int end);
}

The server side implementation of the service looks like:

/**
 * This class implements CounterService and uses a callback.
 */
@Service(CounterService.class)
@Scope("CONVERSATION")
public class CounterServiceImpl implements CounterService {
	@ConversationID
	protected String conversationID;

	/**
	 * The setter used by the runtime to set the callback reference
	 * 
	 * @param counterServiceCallback
	 */
	@Callback
	public void setMyServiceCallback(CounterServiceCallback counterServiceCallback) {
		System.out.println("setCounterServiceCallback on thread "
				+ Thread.currentThread());
		this.counterServiceCallback = counterServiceCallback;
	}

On the implementation, the @Service SCA annotation is used to identify the SCA service being implemented and the @Scope which specifies a visibility and lifecycle contract an implementation has with the SCA runtime. Possible values of @Scope are STATELESS, REQUEST, CONVERSATIONAL and COMPOSITE. For this specific service we need to use CONVERSATIONAL, instructing Tuscany that conversation state must be kept in order to correlate interactions between a client and the service. For a description of the rest of scopes, refer to the SCA Java Annotations and API reference specification.

Last, we use the @Callback annotation to instruct Tuscany SCA where to inject the Callback reference to be used by the service.

The server side SCA composite is very similar than the Tuscany SCA official callback-ws-client and you can find it under the “src/main/resources/counterws.composite“. The binding used for the sample is a Web Service binding.

Tuscany SCA Counter Service Client

On the client side (counter-callback-ws-client-SCA maven project) we need to define a client interface that makes use of the counter service. To make it simple, the interface has a method that returns the reference to the Counter Service and an additional helper method to get the current count recorded in the client based on the service callbacks. As in the server side, we have to make sure to define the interface as @Conversational and the implementation with @Scope(“CONVERSATION”) so Tuscany populates the service call with conversation information.

Also, to simplify the sample code, the shared interfaces between client and server (i.e. CounterService and CounterServiceCallback) are included in both projects separately. Ideally, these interfaces would be on a separate interfaces library used by both projects.

/**
 * The client component interface.
 */
@Conversational
public interface CounterServiceClient {
    
    public CounterService getCounterService();
    
    public int getCount();
    
}

The client composite is as listed below. It defines the reference to the counter Service and the associated callback. The url defined in the callback is used by Tuscany SCA in the client side to start the listening callback web service:



    
        
        
	        
	        
	        
	            
            
        
    

For the testing, a unit test is setup (WebServiceSCAClientTest.java), exercising the counter service, callbacks and conversations.

Running the counter service sample project

To run the counter service sample, extract the source code and:

  • Run the server side, executing the run-server.sh script in the counter-callback-ws-service directory. The server component should start and wait for client requests.
    ...
    Nov 8, 2010 4:11:23 PM org.apache.coyote.http11.Http11Protocol start
    INFO: Starting Coyote HTTP/1.1 on http-8086
    Callback server started (press enter to shutdown)
    Nov 8, 2010 4:11:23 PM org.apache.tuscany.sca.http.tomcat.TomcatServer addServletMapping
    INFO: Added Servlet mapping: http://localhost:8086/CounterServiceComponent
    
  • Go to the client project, counter-callback-ws-client-SCA and execute the tests with:
    mvn clean test
    
  • On the client side you should see how the client starts the counting service, receives the callbacks and how the counting is stopped and later restarted, continuing the count where it was left.
    ...
    Nov 8, 2010 4:11:28 PM org.apache.tuscany.sca.http.tomcat.TomcatServer addServletMapping
    INFO: Added Servlet mapping: http://localhost:1999/callback
    Starting Count and waiting 5 seconds for counts...
    CounterServiceCallback --- Received Count: 0 out of 30
    CounterServiceCallback --- Received Count: 1 out of 30
    CounterServiceCallback --- Received Count: 2 out of 30
    CounterServiceCallback --- Received Count: 3 out of 30
    CounterServiceCallback --- Received Count: 4 out of 30
    Stopping Count and waiting 5 seconds for no counts...
    Restarting Count and waiting 5 seconds for counts...
    CounterServiceCallback --- Received Count: 5 out of 30
    CounterServiceCallback --- Received Count: 6 out of 30
    CounterServiceCallback --- Received Count: 7 out of 30
    CounterServiceCallback --- Received Count: 8 out of 30
    Stopping the Client Node
    Nov 8, 2010 4:11:44 PM org.apache.tuscany.sca.node.impl.NodeImpl stop
    ...
    
  • On the server side, you should see the client call and the associated conversationID.
    ...
    INFO: Added Servlet mapping: http://jmatute-laptop:8086/CounterServiceComponent
    setMyServiceCallback on thread Thread[Thread-2,5,main]
    ConversationID = d8a05f47-064b-4832-9cca-7ad035bd36ee. Starting count
    Sleeping ...
    Sleeping ...
    Sleeping ...
    Sleeping ...
    Sleeping ...
    ConversationID = d8a05f47-064b-4832-9cca-7ad035bd36ee. Stopping count
    Sleeping ...
    Sleeping ...
    Sleeping ...
    Sleeping ...
    Sleeping ...
    ConversationID = d8a05f47-064b-4832-9cca-7ad035bd36ee. Restarting count
    Sleeping ...
    Sleeping ...
    Sleeping ...
    Sleeping ...
    Thread interrupted.
    

Having reached this point, let’s look into how Tuscany SCA is handling the conversations and callbacks and how the required information is included in the underling transport bindings (WS or JMS).

Internals of conversational and asynchronous services in Tuscany SCA

Tuscany SCA conversational services (i.e. marked with @Conversational) make use of conversation IDs in order to keep track and correctly map multiple invocations associated with a certain conversation.

SCA and Tuscany asynchronous services are similar to those defined, for instance, in BPEL where the forward and callback service calls are two complete separate service invocation calls. Therefore, Tuscany also requires additional information to know where the callback needs to be send and optionally provide a application correlation id for the callback call.

Summarising, the below table contains the information used by Tuscany SCA for conversation and callback services:

Tuscany information Description
conversationID UUID identifying the current conversation. This is used by Tuscany SCA to associate accordingly the service instance associated to that conversation.
callbackLocation The location of the callback service. This, depending on the binding used might be a URI for Web Service binding or a Queue name for JMS.
callbackID Application-specified callback ID, that is used by the application to correlate callbacks with the related application state.

The above information needs to be mapped to the specific binding and here is where no standarisation exists, making difficult to have interoperable solutions:

  • WebService Binding : for the Web Service binding, WS-Addressing headers are used in order to store the Tuscany SCA conversational/async required information. Below it is shown an example of a Tuscany Web Service conversational and asynchronous/bidrectional invocation:
    • The WS-Addressing Address contains the Callback URI to be used for the callback invocation. This is normally setup by the client to notify the server where to send the callbacks.
    • The WS-Addressing ReferenceParameters contains the other two information fields under specific Tuscany Namespace, the CallbackID and the ConversationID.
    
    
        
            
                
    http://localhost:1999/callback
    45a963da-2074-4bb2-b9ee-f721e2ec753b 309b8322-1dc2-4c51-a4db-73d65edae391
    30
  • JMS Binding : For JMS, Tuscany SCA uses JMS Message properties to store the required information. A screenshot (see below) of ActiveMQ console shows an Tuscany SCA JMS Message for conversational and async/bidrectional service.
    1. CallbackID, scaConversationID and scaCallbackQueue, Tuscany SCA propietary JMS message properties to hold the information.

    Tuscany SCA JMS Message

    Tuscany SCA JMS Message

Conclusion

In this first post I have presented a Tuscany SCA example that covers both conversational and asynchronous scenarios, not currently available in the official Tuscany SCA samples, and looked into the internals of Tuscany SCA used to handle these services. This provides the basis for the next post, where I will be using this information to extend the project code with a non-SCA JAXWS-based web service client as an approach to consume these services without a Tuscany SCA runtime.

Dynamic LDAP-based authentication and authorization in Servicemix 3

Recently, we have been working quite extensively with Apache Servicemix, a JBI-compliant ESB.
One of areas we have been looking into is securing services in general and how to perform ldap-based authentication and authorization for those services in particular.

A good starting point to understand Servicemix (SMX) security features can be found here. I will give a brief overview of the security features of JBI and SMX .

JBI does not very concisely define authentication and authorization mechanisms, but it does define the following:

  • JBI relies on JAAS definitions for security features (i.e. security Subject).
  • The Binding Components (e.g. servicemix-cxfbc) are expected to transform protocol-specific (e.g. WS-Security) credentials into a normalized format, a security Subject, and inject them into the normalized message.
  • The ESB will then apply the authentication and authorization mechanism based on Subject information.

In particular for SMX we have:

  • For authentication it relies on JAAS, being able to use custom Login modules to integrate different technologies (e.g. LDAP) and implement specific authentication logic. The SMX configuration for JAAS is located under $SERVICEMIX_HOME/conf/login.properties.
    • SMX comes by default with a file based authentication mechanism, which is not suitable for use in many environments since it requires an SMX restart after any change.
  • For authorization, SMX does not use JAAS, due to the different nature of the items to be authorized (i.e. service endpoints instead of Java code). It defines an specific ACL AuthorizationMap, used by the NormalizedRouter (SecureBroker) to allow or deny the invocation. The configuration file for this is located in $SERVICEMIX_HOME/conf/security.xml
    • Also in this case, SMX comes by default with a file based ACL definition, included in the security.xml file. Any change in the ACLs requires an SMX restart.

So the idea behind this post is to provide a dynamic authentication/authorization mechanism in SMX that does not require an ESB restart after changes whilst also making use of LDAP.

In this post I will go through a complete example that requires the following:

  1. Apache ServiceMix 3.3.2.
  2. OpenDS 2.2 as a test LDAP Server. However you can use any other LDAP server you prefer.
  3. Maven 2.0.9 or higher to build the projects.

The complete source code of this post can be found here. It contains:

  1. A LDAP ldif file with the sample directory structure used.
  2. As a test service, a secured version of the SMX’s cxf-wsdl-first, now called cxf-wsdl-first-secure.
  3. A smx-ldap maven project with the classes we need to deploy in SMX in order to enable LDAP.
  4. A smxConfig directory with the required SMX configuration.

With that in mind, let’s go through the details:

Install Servicemix and OpenDS

You can skip the OpenDS section if you have your own LDAP server, just make sure the relevant LDAP configuration parameters are used in the coming sections.

  1. Download and install Servicemix. No additional modifications are required at this stage.
  2. Download and install OpenDS. If you want to use the sample LDAP ldif file, setup the directory server with the following configuration:
    • Select as “Directory Base DN”: dc=theserverlabs,dc=com.
    • Select import data from LDIF and choose the file included in the source package.
    • Complete installation.

The users and passwords defined in the sample ldif are:

  • reader:reader
  • smx:smx
  • testuser:testuser

Create a test service that requires authentication

To show the authentication and authorization of a service deployed in SMX I took the “cxf-wsdl-first” example that is included in the SMX distribution and enabled WS-Security authentication headers. The result project is included in the source files as “cxf-wsdl-first-secure”.

For that I just added the following interceptors configuration into the xbean.xml file of the CXFBC service unit “wsdl-first-cxfbc-secure-su”:



     
             
             
      




     
         
		
		
		
		
	
    

In this case as you see the required passwordCallbackClass just defines a DummyCallbackHandler (also included in the cxfbc su) that does nothing at all, delegating the whole authentication/authorization to SMX.
This is the simplest case and obviously you could have a class that performs additional checks or needs to deal with keystore passwords in case encryption is used. I will not get into the details here about WSS4J as this can be a subject worth a whole entry by itself.

Also, for a very good background reading of WS-Security and SMX check this blog post by Torsten Mielke.

To install the example project into SMX, just build the maven project and copy the service assembly to the hotdeploy directory of SMX:

cd Source/cxf-wsdl-first-secure/
mvn clean install
cp  wsdl-first-cxf-secure-sa/target/wsdl-first-cxf-secure-sa-3.3.2.zip $SERVICEMIX_HOME/hotdeploy
 

An example soap request with the WS-Security headers would be:



   
       
            
                   testuser
                   testuser
            
       
   
   
      
         world
      
   

 

Setup the required LDAP classes for JAAS and SMX AuthorizationMap

The required JAAS LDAP Login Module and the SMX AuthorizationMap contains many LDAP connectivity logic that are common and can be found in several open-source implementations.
I took as a reference the ActiveMQ classes and modify them to fit the SMX needs.

You can find the two classes in the smx-ldap maven project:

  • LdapLoginModule: This class implements the JAAS LoginModule interface and it is responsible to perform the authentication (login method) based on the passed credentials. It is also responsible to extract the user’s roles once it is authenticated and inject them into the Subject as additional principals. The provided class is basically the same implementation used for ActiveMQ, no changes are required.
  • LdapAuthorizationMap: This class implements the AuthorizationMap interface offered by SMX. The interface has just one method:
    Set getAcls(endpoint, operation)
    

    The class simply provides a set of Principals allowed to invoke the passed endpoint and operation. Also in this case, I took as a baseline the LdapAuthorizationMap include in ActiveMQ. We need to adapt it as ActiveMQ maps work on queues and topics, not endpoints.

Having reached this point I needed to define the level of granularity and also the format of the entries in LDAP defining the ACLs. I provide one implementation possibility here and it is definitely not perfect. Depending on your needs you can take this as a baseline an implement a more complex logic, add caching of endpoints ACLs, etc…

So this is the format I defined:

  • Endpoint ACLs are located under “ou=endpoints,ou=smx” in the LDAP tree.
  • The entries defining the ACLs will be “GroupOfUniqueNames”, having an entry per allowed group.
  • The “cn” of the ACLs has the form:
    cn=::
    

    In the defined schema you can also use wildcards on Service and Operation. Examples of ACLs definition for the cxf-wsdl-first-secure project would be:

    cn=http://servicemix.apache.org/samples/wsdl-first:*:*   
           Grant permissions to all services on that ns.    
    
    cn=http://servicemix.apache.org/samples/wsdl-first:PersonService:*   
           Grant permissions to all operations of PersonService.    
    
    cn=http://servicemix.apache.org/samples/wsdl-first:*:GetPerson   
           Grant permissions to GetPerson operations of all services in that ns.    
    

    Another example in case you use dynamic SMX ftp endpoints would be:

    cn=urn:servicemix:ftp:* 
           Grant permissions to all dynamic endpoints of the servicemix-ftp BC. 
    

This logic is implemented in the LdapAuthorizationMap class.

Both ActiveMQ classes are highly configurable using properties as we will see how to setup their usage in SMX.

To install the classes, build the maven project and copy the jar file to the lib directory of SMX:

cd Source/smx-ldap/
mvn clean install
cp  cp target/smx-ldap-1.0-SNAPSHOT.jar $SERVICEMIX_HOME/lib


 

Configure SMX

To make use of the LDAP authentication and authorization two SMX files need to be modified:

  • $SERVICEMIX_HOME/conf/login.properties: This is the JAAS authentication configuration and must define the LdapLoginModule. The parameters are for locally configured OpenDS LDAP, modify them with your configuration if required:
    servicemix-domain {
        com.tsl.jbi.smx.security.login.LDAPLoginModule REQUIRED
            debug=true
            initialContextFactory=com.sun.jndi.ldap.LdapCtxFactory
            connectionURL="ldap://localhost:389"
            connectionUsername="cn=reader,ou=people,dc=theserverlabs,dc=com"
            connectionPassword=reader
            connectionProtocol=s
            authentication=simple
            userBase="ou=people,dc=theserverlabs,dc=com"
            userRoleName=dummyUserRoleName
            userSearchMatching="(uid={0})"
            userSearchSubtree=false
            roleBase="ou=groups,dc=theserverlabs,dc=com"
            roleName=cn
            roleSearchMatching="(member={0})"
            roleSearchSubtree=true
            ;
    };
    
     

    You can see that there are two parameters, userSearchMatching and roleSearchMatching that allow us to define the user/roles search filter that fit our environment.

  • $SERVICEMIX_HOME/conf/security.xml. This file contains general SMX security configuration and here we will define the bean for the LdapAuthorizationMap:
      
      
      
      
      
     
      
      
        
        
        
        
        
        
        
        
      
     

    Also in this case, we have an endpointRoleFilterFormat that defines the ACL’s search filter. This can be modified to match whatever other LDAP directory setup you want to define for ACLs.

  • $SERVICEMIX_HOME/conf/servicemix.xml. We need to modify the authorizationMap used by the secured broker and point to the new ldap-based map.
        
          
    
           .....
        
    


Restart ServiceMix to apply the changes and you are ready to go!

Test the service and play around

You can test the configuration with SoapUI (example project included in sources) or make use of the client.html (thanks SMX guys for this!!!) included in the cxf-wsdl-first-secure project.

The simplest one is the client.html, just load it in your web browser and fire away.

Test Web Service Client

Test Web Service Client

The test should run successfully and return the Person information.

Now, it’s time to start playing around with the LDAP and see dynamic authentication/authorization:

  • For instance, you can change the ACL group in the endpoint LDAP to SMX-GROUP2 and see how you get a “Endpoint is not authorized for this user” exception.
  • Create a more fine grained endpoint ACL that only assigns SMX-GROUP1 to:
    cn="http://servicemix.apache.org/samples/wsdl-first:PersonService:GetPerson"
    
  • Change the password used for authentication of testuser in the SOAP request to get an LDAP login error.

Conclusions

Dynamic authentication/authorization configuration in an ESB with mechanisms like LDAP is mandatory in any real deployment scenario.

You have seen that adding LDAP-based authentication/authorization to SMX is not difficult but to have a robust model some implementation work is required. The implementation shown in this post can be taken as a good starting point but there are areas of improvement both in terms of performance (e.g. cache, etc…) and in the authorization model itself (e.g. precedence of ACLs, etc…).

Also, the SMX authentication/authorization mechanisms described are for version 3.x. They should be applicable also for SMX 4 with its new architecture based on OSGi, as the same type of NMR is sitting on top of the OSGi runtime.

Developing applications with SCA and a JBI-Based supporting infrastructure

We have been working with SOA technologies and solutions in the commercial and open-source arena for some years now and I would like to start a new series with this post covering the developments of two mayor standardisation efforts in this area, SCA (Service Component Architecture) and JBI (Java Business Integration).

While for some time SCA and JBI were presented and considered competitors, it is now a quite accepted idea in the industry that these standards cover different standardisation areas. They can be used separately but also used together to get the best of breed solutions.

SCA main benefit is that provides a technology-agnostic generic programming model that decouples the components implementation from their communication, allowing high level of reuse. Applications developed following the SCA model should be able to be deployed without changes in different SCA vendor platforms and following different integration and deployments patterns, depending on project needs. This would help to clearly separate application concerns, allowing developers to focus on services business logic while integration and deployment issues can be handled by architects and integrators.

In the other hand, JBI standarises a Java-based integration infrastructure where components of different vendors can interact in a standard fashion. In many ways, this standard is currently used to implement standarised ESBs and can provide the integration platform where SCA applications can run.

I was especially interested in solutions implementing the mix, offering SCA to provide the level of standardisation at application composition level while using JBI to provide the standard integration and runtime infrastructure, in the form of an Enterprise Service Bus (ESB). Examples of JBI implementations of ESBs are Apache ServiceMix, OpenESB or OW2 PEtALS.

In this area, we can find several efforts, mainly the Eclipse Swordfish project and OW2 PEtALS.
Eclipse Swordfish looks a very promising project, mixing JBI and OSGI to implement a fully distributed Enterprise Service Bus infrastructure where SCA-based applications can be run. However, at this moment SCA support is quite limited. OW2 PEtALS offers also a distributed ESB solution based on JBI only, and has a SCA service engine to run SCA composed applications based on OW2 Frascati. To know more about how this is implemented, have a look to this presentation from PEtALS guys.

So I decided to try and use OW2 PEtALS to run a simple SCA calculator, similar to the Apache Tuscany calculator sample. My objective was to verify the value of developing a SOA solution with SCA, using the integration features of JBI as a mediator ESB, exploring the possibilities of distributed ESB features and the extensibility via new JBI components, as JBI4Corba.

In order to follow this post, it will be helpful that you are familiar with the JBI and SCA concepts.

To support the development I used the latest Eclipse 3.5 Galileo fully loaded with SOA Tools, that include the SCA Tools. These tools provide a nice graphical environment to develop SCA composites as we will see.
Additionally, PEtALS offers a series of Eclipse plugins that make developer´s life easier for creating JBI Service Units (SU) and JBI Service Assemblies (SA). I was nicely surprised to see that they give the user the possibility to setup simple projects or maven projects. Also, it is nice to see the use of maven archetypes all over the place.
As you can imagine, I decided to go the maven way, making my life much easier. They offer a quite good developer manual providing all the information to setup the development environment.

So, the complete list of the required gear is:

Overview

We want to deploy a SCA Calculator implemented as Java components and deploy it in the PEtALS ESB (as depicted below), using a SOAP Binding Component to expose the application as a WS to the external world. We will use SOAPUI to test the application.

SCA Calculator exposed as SOAP WS in PEtALS.

SCA Calculator exposed as SOAP WS in PEtALS.

In order to deploy this we need to configure and develop the following artifacts:

  1. Install necessary PEtALS components into the ESB, the SOAP BC and the SCA SE.
  2. Create a JBI Service Unit (SU Provide) containing the SCA composite to be deployed against the SCA SE.
  3. Create a JBI Service Unit (SU Consume) to expose the SCA composite via SOAP WS.
  4. Create a JBI Service Assembly (SA) containing the two SU, ready to be deployed into the ESB.

The complete sources of the article can be found here (Maven Projects).

Install necessary PEtALS components into the ESB

Starting with the quickstart PEtALS distribution makes everything really simple. You need to simply start the bus with the command:

/bin/startup.sh -C (Will start in console mode, very handy)

If you are in Windows there is a typo in the documentation and needs to be started with a lowercase -c.

To install the SOAP BC and the SCA SE is as simple as copying the two zip files for the components into the “install” directory of the PETALS_HOME. The components will be automatically deployed into the bus.
You can also deploy any BC, SE or SA using the terminal console or via the web console.

Make sure you read the limitations (e.g. redeployment of SU) and apply the required changes that are currently needed for the SCA Service Engine (i.e. isolatedClassLoader) as described in the documentation of the component

The webconsole will be available in http://localhost:7878/.

Let´s start with the difficult bit creating the SCA SU. The others, the SOAP SU and SA, will be very simple.

Create a JBI Service Unit (SU Provide) containing the SCA composite

Generate Maven Project
We can generate an empty maven Service Unit project in several ways, using maven archetypes from the command line of via the PEtALS Eclipse plugins. I found the Eclipse plugins more complete (see picture below) as they are able to create empty projects already targeted to work as consume/provide SU for existing PEtALS components.

PEtALS Eclipse plugins

PEtALS Eclipse plugins

For this SU, we need to define a new project that deploys into the SCA SE, therefore we need to select “Use PEtALS technical service -> Use SCA“.
You will have to fill some fields like name of the Composite (e.g. “Calculator”), the target namespace (e.g. “http://demo.sca.theserverlabs.com”).

This will create a maven project (you need to select it), let´s call the project “su-SCA-Calculator-provide”. You will need to fill the specific pom.xml entries related to the SU (i.e. groupId, modelVersion and version). Also, remove the parent definitions added by default.

Make sure to have a “description” on the pom.xml as it will be used later by the Service assembly to populate some fields.

Create SCA Composite
The wizard has already created the SCA composite, called “Calculator.composite” under “/src/main/jbi” directory. The composites must be defined in this directory so the files will be kept at root level of the SU when building the project, and not included in the jar file created with the SCA artifact code.
To associate a SCA Composite diagram to the xml configuration file, right click in the SCA composite file and select “SCA->Initialize SCA Composite Diagram file”. This will create a “.composite_diagram” file that will be always on sync with the xml configuration and viceversa.

Once the composite is created we can modify it via xml or via the designer. The SCA Composite for the Calculator would look something like this:

Calculator SCA Composite

Calculator SCA Composite

As you can see our composite is very simple and has a CalculatorService which interface is exposed as a service out of the composite and has four references to other composite components, providing the add, subtract, divide and multiply implementations.

The SCA composite configuration file, would look like this:



	
		
		
	
	
		
		
			
		
		
			
		
		
			
		
		
			
		
		
			
		
	
	
		
		
	
	
		
		
	
	
		
		
	
	
		
		
	

Note that at the beginning of the file, the CalculatorService interface is promoted as a composite service. The SCA binding defined for this is the frascati JBI binding, which will register the service in the PEtALS bus as an internal JBI endpoint. The CalculatorService references are wired with the rest of the components via “target”.

Create required WSDL files for promoted SCA services

JBI message exchange Model is based on WSDL and as described in the SCA SE documentation, the SU package must contain a WSDL describing for each composite the promoted services. In our case, we need to provide a WSDL for the CalculatorService.
Currently, the WSDL must be provided in document/literal wrapped style and it is not automatically generated. However, there are tools like Java2WSDL of Apache axis that allow us to create it in a simple way.

The example apache axis command to generate the WSDL would be:

java org.apache.axis.wsdl.Java2WSDL  -y WRAPPED -u LITERAL  -l localhost   calculator.CalculatorService

This will generate a WSDL that should be copied in the “/src/main/jbi” directory under the name provided in the composite file, section “frascati:binding.jbi”, in our case “calculator.wsdl”.

We need to do a few changes in the generated WSDL to make it to work:

  1. Change the request message definitions from for instance “addRequest” to “add”, so it complies to wsdl “wrapper” style, having the same name for the input wrapper as the operation name.
  2. Align this change for the operations port type and binding sections.

I don´t include the final WSDL file as it is a bit long. You can find it in the sources package.

Configure jbi.xml file

Everything is ready and we just need to define which services are provided or consumed by this SU. In this case, the SU provides Services, so the jbi.xml file would look like:




	

		

			calculator.wsdl

			Calculator.composite
		
	

For each provided service we need to define the interface-name, service-name and endpoint-name. It is important to match the calculator.wsdl binding definitions with this information. Therefore, interface-name must match the wsdl portype name and the endpoint-name must match the wsdl port..
We also need to inform PEtALS of which is the wsdl file that describes the service provided, and specifically for the SCA component, which is the composite file.

Build the project

The last step to do is to build the project with a simple:

mvn install

Make sure you have included the PEtALS maven repository into your maven configuration, so it will find the required artifacts. This is well described in their development guide.

Create a JBI Service Unit (SU Consume) to expose the SCA composite via SOAP WS

This SU is much simpler. The component just needs the information for the JBI internal service that must consume and it will do the rest for us.

Using again the PEtALS Eclipse plugins, we create a new maven project (“New->Other->Petals->Expose Service from PEtALS->use SOAP”) called “su-SOAP-calculatorService-consume”. The wizard will allow you to define the Service you want to expose, so if you select the SCA SU Eclipse project, the rest of the fields will be automatically populated.

SOAP SU Consume Wizard

SOAP SU Consume Wizard

As previously done in the other SU, you will need to define the specific fields of the pom.xml related to this project, and don´t forget to define a “description” field.

This SU only contains the jbi.xml, defining which service must be exposed (consumed). We need to make sure that the consume section contains the references to the previously defined SCA SU. This would be the required jbi.xml:




	
	
	
	
		
		
	
			
			
		
				
			CalculatorService
			false
			SOAP
			soapbc
		
	

All fields are defined at the moment of project creation. The only important thing to make sure is to provide in the “consumes” section, the proper the interface-name, service-name and endpoint-name. These must much the ones those defined in the SCA SU, so the can talk to each other. If you selected the SCA SU Eclipse project in the SOAP SU creation Wizard, all these fields should be already correctly filled.

Once this is done you can build the project with a maven install.

Create a JBI Service Assembly (SA) for deployment

The deployment in the JBI ESB is performed via Service Assemblies, which can contain many SUs, each one bounded to different components (BC, SE, etc…). In our case we have a the SCA SU bounded to the SCA SE Component and the SOAP SU bounded to the SOAP BC Component. This is defined in the jbi.xml of the Service Assembly.

As before, we can use the PEtALS Eclipse plugin to create an empty SA maven project (“New->Other-PEtALS->SA Maven Project”), called “sa-SCA-Calculator”. The wizard will allow as to add the SUs we want into the SA so no more configuration is needed.

Once created, as on previous artifacts we need to be defined in the pom.xml the specific project parameters. Make sure you defined a description.

The SA project defines the SUs to be included via standard maven dependencies. That´s the only configuration step to perform (automatically done by the wizard) and the jbi.xml will be automatically created. The pom.xml of the SA would then look like:



	
	sa-SCA-Calculator
	A description of sa-SCA-Calculator
	1.0-SNAPSHOT
	jbi-service-assembly

	sa-SCA-Calculator
	com.tsl.sca
	4.0.0
	
	
		
			su-SCA-Calculator-provide
			com.tsl.sca
			1.0-SNAPSHOT
			jbi-service-unit
		
		
			su-SOAP-calculatorService-consume
			com.tsl.sca
			1.0-SNAPSHOT
			jbi-service-unit
		
	
	
		
			
				org.ow2.petals
				maven-petals-plugin
				
				true
		
	

Run a maven install to build the SA.

Deploy SA into PEtALS

The generated zip file, “sa-SCA-Calculator-1.0-SNAPSHOT.zip”, just needs to be copied to the PETALS_HOME/install directory and the SA will be automatically installed and started.

The PEtALS console should show information regarding the compilation and creation of required classed for the SCA and SOAP SUs:

SCA Calculator deployed in PEtALS

SCA Calculator deployed in PEtALS

Test the Service

The only thing left is to test that everything works. For that I used SOAUP, loading the WSDL from the SOAP WS services page (provided by the PEtALS component in http://localhost:8084/petals/services/CalculatorService?wsdl).

Testing SCA Calculator with SOAPUI

Testing SCA Calculator with SOAPUI

Future Posts

In future posts I will explore transparent deployment of SCA applications across a distributed ESB (such as PEtALS) and the usage of external references to services using Corba (e.g. replacing the add service by a external CORBA based service).
The later will also explore the usage of third party JBI components (as JBI4Corba) into PEtALS, as there is no PEtALS native JBI BC Component for Corba.