WS-Security, Weblogic 10.2, Service Controls, and legacy namespaces

I was working on a Weblogic Portal application that needed to get information from a secure (WS-Security) third party web service over https. This web service used a legacy (pre-OASIS) WS-Security namespace and modern clients (like the one bundled with Weblogic server) don't create messages that conform to the old specification. Here's what I did to work around this restriction...

Disclaimer

This is a pretty lengthy read. Attached to this article are all of the work products that I created to solve this problem. You should be able to simply download the jars and the java source and get things working. It is still going to be worth your while to download the wss4j 1.1 source; it has example source files that will show you how to sign and encrypt messages using the old API.

Weblogic Web Service Client

In Weblogic Workshop it is pretty easy to consume a web service. Once you have a WSDL file, you can automagically create a ServiceControl by right clicking on the WSDL --> Web Services --> Generate Service Control and follow the wizard. This process will:

1. create objects to wrap the request and response objects (JAX-RPC or Apache XmlBeans)
2. create an invocation object (ServiceControl) that you can use to actually call the service

So in about 10 seconds you can consume an insecure web service and use it in your application.

What if the service uses SSL?

No real big deal here either. If the wsdl service address is http, you can either change that to https and re-generate the ServiceControl or modify the Location in the previously generated ServiceControl. e.g.

@ServiceControl.Location(urls = {"https://mattfleming.com/WebServiceEndPoint"})

You will also need to add the public key/certificate from the server that is hosting the web service to the trust store used by Weblogic. This isn't hard to do, and if you get hung up there are resources on the web to help (not a ton though).

What if the service uses message level security?

This is where things can start to get challenging. There isn't a lot of documentation to help you out with this but there are some Weblogic specific ones that can help.

After messing around with the annotations in the aforementioned document, I was able to get the ServiceControl to add the security headers to the SOAP request. This took me a couple of days to get working though. In the end, I could actually see the signatures and binary security tokens in the SOAP XML but the only response I would get back from the server was that the security headers were missing

Legacy namespaces

After going back and forth with the web service provider, we realized that the WS-Security namespace that their server was expecting was the OASIS 06/2003 namespace; the namespace that the ServiceControl sends is OASIS 2004. This namespace mismatch was causing the server to completely ignore the security markup.

My first question to them was, Can you upgrade the security spec used by the server? Nope, they sure couldn't because the application server they were using was Websphere 5. Next question: When are you going to upgrade your server to say WebSphere 6, when IBM officially added WS-Security 1.0 support? They said they may get to it this year, but I wasn't going to hold my breath-- WebSphere 6 has been out since late 2004.

Next on the thought process was Can I get the ServiceControl to use a legacy namespace? I dug around in the documents and couldn't find an answer (which is not usually a good omen). Ultimately I asked BEA support if this were possible and got a resounding no. Maybe a BEA server from that time period would have but their latest and greatest only supported the official WS-Security 1.0 and 1.1 specifications.

Ok now what

Since I couldn't:

1. change the 3rd party service to disable WS-Security
2. upgrade the web service provider server to support an approved specification
3. reduce the scope of the application until either of the previous happened

I had to get a little more creative. So I started to research writing my own web service client that implements WS-Security pre 1.0 specification from scratch. The first thing I always do is look around for any existing libraries that may help me in my endeavor.

WSS4J

All right, now we're getting somewhere. An open source library that implements WS-Security in java... perfect! This find was diminished though by the very next section:

Here we go again, what about specifications prior to OASIS 2004? After digging around some more, I found out that WSS4J 1.1 was the last version which supported multiple configurable namespaces prior to OASIS 2004. Nice!

New game plan

1. Build WSS4J 1.1
2. Build a security proxy service. This service needed to accepts a SOAP request, perform WS-Security and forward the resulting request to the true endpoint. It also needs to transform the reply from the true endpoint into a message that the original requestor can understand.
3. Use the existing ServiceControl in an insecure mode to call the intermediate security proxy service.

I'm sure there is some fancy shmancy design pattern that describes this, but the idea that comes to my mind is the slingshot around the moon approach that NASA used in the Apollo 13 mission. The ServiceControl slingshots the message through the local secure proxy to the remote destination and back.

WSS4J 1.1

You'll need to grab the 1.1 branch of the source code from the svn repository and run the build. Of course when I ran the ant build script, some of the unit tests failed (way to label a broken branch Apache). So I commented out the tests and built the package.

The WSSecurityWrapperProxy

For normal J2EE servers, a servlet will suffice nicely for this class. The actual implementation I used for Weblogic Portal was a JavaPageFlow but any kind of class that gives you access to the request's InputStream could be used. The full source code is attached to this post at the bottom. Here's a break down of the source:

private static final Crypto crypto = CryptoFactory.getInstance();

Such an innocuous line. In order for this to work you'll have to make sure several things are in place:

1. A crypto.properties file in the classpath.
2. A password protected java keystore in the classpath. In the sample below, the keystore is located in the classpath at keys/keystore.jks. The password for that keystore is key_store_password.
3. A password protected private key used to sign messages with the alias specified in the crypto.properties file. In the sample below, we use the alias key_for_signing with the password key_for_signing_password. The alias and password will need to match the credentials passed to the signer when invoking setUserInfo(alias, password).
4. If you are going to be connecting to the true endpoint via SSL, this keystore is a good place to put the public key of endpoint. However, you'll have to configure apache httpclient to use this keystore for https. Check out the apache httpclient documentation for information on how to do that.

Here's a sample crypto.properties:

org.apache.ws.security.crypto.provider=com.cvty.ws.security.components.crypto.Merlin
org.apache.ws.security.crypto.merlin.keystore.type=jks
org.apache.ws.security.crypto.merlin.file=keys/keystore.jks
org.apache.ws.security.crypto.merlin.keystore.password=key_store_password
org.apache.ws.security.crypto.merlin.keystore.alias=key_for_signing
org.apache.ws.security.crypto.merlin.alias.password=key_for_signing_password

I'm not going to write up how to use java's keytool to build the keystore and import the private key. There are numerous articles out in the wild that can help with this task. Back to the java source...

WSSConfig config = WSSConfig.getNewInstance();
config.setWsseNS(WSConstants.WSSE_NS_OASIS_2003_06);
config.setWsuNS(WSConstants.WSU_NS_OASIS_2003_06);
config.setBSTAttributesQualified(false);
config.setBSTValuesPrefixed(true);
signer = new WSSignEnvelope(config, null, true);

This section sets up the WSS4J engine so that it uses the OASIS 06/2003 specification and some attributes about the Binary Security Token that I noticed in the sample message that worked for WebSphere 5.x. This is of course not necessarily required (but it was in my case).

String alias = "key_for_signing";
String password = "key_for_signing_password";
signer.setUserInfo(alias, password);
signer.setKeyIdentifierType(WSConstants.BST_DIRECT_REFERENCE);

Pick the alias in the trust store which should be used to sign the message. This alias must match what is in crypto.properties. Also make sure that the BST is included in the message directly.

Message axisMessage = new Message(request.getInputStream());
axisMessage.setMessageContext(msgContext);
Document signedDoc = signer.build(axisMessage.getSOAPEnvelope().getAsDocument(), crypto);
Writer sw = new StringWriter();
XMLUtils.DocumentToWriter(signedDoc, sw);
String signedString = sw.toString();

Sign the body of the request using AXIS. The XML message sent from the ServiceControl is POSTed as the body.

Then there is a section of code that uses commons-httpclient or whatever they call it these days to hit the true destination of the SOAP request.

} catch (Exception e) {
	StringWriter sw = new StringWriter();
	e.printStackTrace(new PrintWriter(sw));
	response.getWriter().print(
			MessageFormat.format(ERROR_SOAP, new Object[] {
					e.getMessage(), sw.toString() }));
	}
}

private static final String ERROR_SOAP = "<?xml version=\"1.0\" encoding=\"utf-8\"?><soapenv:Envelope xmlns:soapenv=\"http://schemas.xmlsoap.org/soap/envelope/\"><soapenv:Body><soapenv:Fault><faultcode>soapenv:Client</faultcode><faultstring>{0}</faultstring>><detail><stackTrace>{1}</stackTrace></detail></soapenv:Fault></soapenv:Body></soapenv:Envelope>";
}

This is how the proxy returns a valid SOAP error message when something fails. It takes any exception and wraps it up in a Client exception. Notice the faultcode.

Not quite out of the woods (but really close)

I'm not sure how many times I've been burned by application servers' "bundling" of open source jars but it feels like a lot... probably because it is so painful to fix. Here's the problem this little software stack we built will have when you try to run it inside a bea container:

java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory
	at org.apache.xml.security.Init.<clinit>(Unknown Source)
	at org.apache.ws.security.WSSConfig.<init>(WSSConfig.java:66)
	at org.apache.ws.security.WSSConfig.getNewInstance(WSSConfig.java:90)
	at org.apache.ws.security.WSSConfig.<clinit>(WSSConfig.java:45)

Here's why you'd get it :

1. BEA has bundled the org.apache.xml.security classes into a module called com.bea.core.apache.xml.security_1.3.0.jar
2. When my classes use the apache xml security classes, it pulls the classes from that module instead of the local jar in WEB-INF/lib. I confirmed this by removing the package (and everything was fine because it used the WEB-INF/lib package).
3. commons-logging has several known issues when running inside multiple classloaders. The problem here is that the security classes are loaded using a different classloader than the web app. Since the parent classloader is loading the security jars and the child is loading the other jars, commons-logging blows up.

In short welcome to commons-logging hell.

Solve commons-logging hell by repackaging jars with jarjar

The easiest (IMHO) way to solve this dilemma is to repackage the xml security jars and have wss4j reference those classes instead of the org.apache.xml.security.* classes it finds in the BEA module.

After you download jarjar, you will need to make a build script and pass in the jars you want to combine and repackage.

The script below repackages all of the xml security classes into the com.mattfleming package hierarchy. It also updates all of the wss4j classes so that they point to the com.mattfleming classes instead of the org.apache classes. Once you put this jar into the classpath (provided there isn't another wss4j somewhere prior) the com.mattfleming xml security classes will be used.

<project default="jar">
	<taskdef name="jarjar" classname="com.tonicsystems.jarjar.JarJarTask" classpath="lib/jarjar-1.0rc7.jar" />
	<property name="lib.dir" value="to_refactor" />
	<property name="build.dir" value="build" />
	<property name="replace.string" value="com.mattfleming" />
	<property name="replace.folder" value="com/mattfleming"/>
	<property name="jar.filename" value="${replace.string}.security.jar" />

	<target name="jar">
		<jarjar jarfile="${jar.filename}" update="false">
			<zipfileset src="${lib.dir}/wss4j.jar" />
			<zipfileset src="${lib.dir}/xmlsec-1.2.1.jar" />
			<rule pattern="org.apache.xml.security.**" result="${replace.string}.xml.security.@1" />
		</jarjar>
		<mkdir dir="${build.dir}" />
		<unjar dest="${build.dir}" src="${jar.filename}" />
		<delete file="${jar.filename}"/>
		<replace file="${build.dir}/${replace.folder}/xml/security/resource/config.xml" token="org.apache.xml.security" value="${replace.string}.xml.security" />
		<jar jarfile="${jar.filename}">
			<fileset dir="${build.dir}"/>
		</jar>
		<delete dir="${build.dir}" failonerror="false" />
	</target>
</project>

Conclusion

At the end of this process, I had created:

1. A WSSecurityWrapperProxy class that added WS-Security using a legacy namespace to a passed SOAP message.
2. A custom packaged wss4j-1.1 and xmlsecurity-1.2.1 jar (com.mattfleming.security.jar) necessary for use within application servers which already have a pre-bundled xmlsecurity.jar.
3. A java keystore containing a private signing key which was used by our custom security jar via the crypto.properties file.

There are many enhancements that can be made to the prototype class. You'll probably want to cut down upon the amount of object instantiation used in the method, add logging, add SSL support for the true endpoint, allow for multi-threaded connection access, proxy support and timeouts.