Chances are, if you have ever connected to a shared computer, you have used some form of a dropbox. File servers, FTP servers, email servers even occasionally web servers can all serve as dropbox servers. If a coworker requests that you save a report to particular location on a server, you and your coworker are using a similar 'dropbox' mechanism and a simple form of messaging. DropboxMQ uses a very specific form of dropbox and a very specific protocol for communicating with other clients through shared dropboxes. Each dropbox is a unique JMS destination.

The Root

The DropboxMQ Dropbox

A Dropbox Message Producer

Dropbox Message Consumers

Message Filenames

The Root

DropboxMQ groups a set of dropboxes into a single directory called the root. More than one root can be configured, but any given DropboxMQ ConnectionFactory can only connect to one root at any given time. Each individual dropbox has its own directory directly in the root directory. The name of dropbox (which is the name of the Queue or Topic) is the name of the dropbox's directory.

.../root/
    queue1/...
    queue2/...
    topic-root/
        topic1/...
        topic2/...

The DropboxMQ Dropbox

Each queue dropbox has the same directory structure shown below. On all mainsteam operating systems and network operating systems, moving a file from one directory to another is an atomic operation (provided both directories are on the same logical disk). DropboxMQ relies on this atomicity to dole out work to message consumers and to resolve contention between message consumers.

.../root/
    queue1/
        incoming/
            working/...
            target/...
            processing/...
            processed/...
            routing/...
            expired/...
            error/...

Topic dropboxes have a slightly different structure to allow for multiple subscribers, but they work with the same basic principal:

.../root/
    topic-root/
        topic1
            .incoming/
                working/...
                target/...
                processing/...
                processed/...
                routing/...
                expired/...
                error/...
            .subscriptions/
                sub1.connection1/
                    working/...
                    target/...
                    processing/...
                    processed/...
                    routing/...
                    expired/...
                    error/...
                sub2.connection2/
                    working/...
                    target/...
                    processing/...
                    processed/...
                    routing/...
                    expired/...
                    error/...

A Dropbox Message Producer

A dropbox message producer creates a unique file in the working directory of the dropbox it is sending to and writes the message contents to the file. While the message file remains in the working directory, the message has not actually been sent and the producer can take as long as it needs to complete the message.

Once the producer is done writing the file and ready to commit the message, the producer moves the file to the target directory. The producer must move (not copy) the file to assure that a consumer doesn't attempt to read a message before it is completely written. Once the message file is in the target directory, the producer has relinquished control of the message and it is consumable.

NOTE: It is very important that producers of any type (shell scripts, etc.) follow the protocol of first writing the message to the working directory, then moving (not copying) the files to the target directory. Even when sending very small messages, it is possible a message consumer could attempt to read a message before it is complete if it is created directly in the target directory.

Dropbox Message Consumers

Several message consumers may be scanning the target at the same time. When a consumer finds a message file, it attempts to move the file to the processing directory. Only one consumer will succeed. The consumers that fail to move the message file (or fail to notice the message file) quietly go back to scanning the target directory for the next message file.

The single consumer that succeeded to move the file then begins the process of consuming the message. If the message is consumed and acknowledged, the consumer moves the message file to the processed directory. If the message is not successfully consumed and is recovered, the consumer moves the message file to the error directory.

Message Filenames

DropboxMQ encodes all message metadata (priority, message identifier, reply-to, etc.) in the message's filename. This allows the contents of the message file to contain just the message body which integrates more easily with dropbox clients that don't use DropboxMQ. Message producers are only required to create message files with unique names.

A message producing client that doesn't use DropboxMQ but wishes to have tighter integration with DropboxMQ, can choose to create message filenames that are compatible with the default DropboxMQ transcoder. By default, DropboxMQ message filenames are broken into several dot (.) separated fields. Trailing dots are required if the remaining fields are undefined. Of the following fields, only the first three (JMS Priority, JMS Message Id and Message Class) are required to permit integration with a DropboxMQ consumer.

4.1140429201295000-9262574723.T.1140429211295.corr1283.TestQueue1.XYZType.prop1S=hello

4.1140429201295000-9262574723.T

Each field is described below: