This is a first-hand experience report for an undocumented bug found in bea Weblogic 6.1 SP3, along with a workaround for the bug


Before you go any further:

Lets clear up some vocabulary.

There are two ways the word "server" can be interpreted here. It can be taken to mean "The squarish box that is plugged into the wall and hums and clicks from time to time" or it can mean "an instance of Weblogic". One physical "server" (the metal kind) can run any number of weblogic server (the software kind) instances, since Weblogic is just a piece of software.

So to avoid this confusion I will use the word "host" when referring to a humming metal box, and the word "instance" when referring to Weblogic.

Additionally, there may be some confusion relating to names. Each host has a name (probably charmingly named after reindeer, or dwarves or something of the like). As well, each instance of Weblogic has it's own name. For my purposes, I will be referring to instance names only. Host names have no bearing on this bug.

You may now continue.


This bug manifests itself in two ways.

Manifestation #1: Connection problems.

The setting:

Two instances of Weblogic (instance A, instance B) are running. They can run either on separate hosts, or on the same host. Instance A has a JMS topic that is properly created, and bound to the JNDI tree. Instance B has a Message Driven Beans which attempts to connect to the JMS topic on instance A in order to receive it's messages. The MDB is correctly written, compiled and has well-formed and proper deployment descriptors.

The symptoms:

When deploying the MDB, instance B throws ConnectionException and\or JMSException errors. The error messages state that the MDB was unable to connect to the remote host, and\or that the JMS topic could not be found.

This will probably inspire you to many hours of in-depth digging around in JNDI space, instance logs and so forth, verifying that the instance can in fact be reached, and the JMS topic is in fact available.

Manifestation #2: No Messages.

The Setting:

Three instances of Weblogic (instance A, instance B, instance C) are running. They can run either on separate hosts, or on the same host. Instance A has a JMS topic that is properly created, and bound to the JNDI tree. Instances B and C both have an MDB that is designed to connect to instance A's JMS topic in order to receive it's messages. The MDBs are correctly written, compiled and have well-formed and proper deployment descriptors.

The Symptoms:

The MDB's on instances B and C are able to connect to the topic on instance A. However, only one of the two is able to receive the messages from the topic. The other mysteriously receives no messages.


Conclusion?

The cause:

This problem is likely due to the naming of the Weblogic instances. Each Weblogic instance runs in a domain, and has a name. For the sake of simplifying configuration management, it is common to have multiple instances running with the same name.

Manifestation #1 occurs when instance A and instance B have the same name. If you change the name of either instance, it should work fine.

Manifestation #2 occurs when instances B and C share a name. If you change the name of either instance, you should begin seeing the messages arriving in both locations.

Further complications:

This workaround makes things rather difficult. Since it requires a unique name for each Weblogic instance, it also requires a separate config.xml file for each instance. This makes it impossible to have guaranteed identical configurations across instances through the use of the same config file.

Imagine a situation where an organization has three tiers of applications (development, test, production) with two hosts in each tier and five instances on each host. Not an unheard-of level of complexity for an enterprise. In fact, not even a very high level of complexity for an enterprise, but this requires the management of 30 config files, when only three are actually needed (one configuration per tier). I won't even go into the frustration of ensuring each config file for each tier is identical in every way, except for the references to it's instance name (which are numerous).

The real cause (probably):

I'm no super-hacker, but you don't have to be a wizard of reverse engineering to see that Weblogic seems to identify incoming t3 connections by the name of the connecting instance. This may sound safe, but think about it. Would it really have been that much more difficult to use the port number in addition to the instance name? Since every instance requires its own port, that would uniquely identify each incoming connection.

The harsh reality:

Unless Weblogic 7 addresses this issue, or there's a forthcoming service pack 4, looks like we're just hopelessly stuck neck-deep in config.xml files.

Hope this helps someone. Or at least entertained someone with amusing links