Chances are, if you have ever connected to a shared computer, you have used some form of a dropbox.
File servers, FTP servers, email servers even occasionally web servers can all serve as dropbox
servers. If a coworker requests that you save a report to particular location on a server, you and
your coworker are using a similar 'dropbox' mechanism and a simple form of messaging. DropboxMQ
uses a very specific form of dropbox and a very specific protocol for communicating with other
clients through shared dropboxes. Each dropbox is a unique JMS destination.
The Root
The DropboxMQ Dropbox
A Dropbox Message Producer
Dropbox Message Consumers
Message Filenames
DropboxMQ groups a set of dropboxes into a single directory called the root. More than one root can
be configured, but any given DropboxMQ ConnectionFactory can only connect to one root
at any given time. Each individual dropbox has its own directory directly in the root directory.
The name of dropbox (which is the name of the Queue or Topic ) is the name
of the dropbox's directory.
.../root/
queue1/...
queue2/...
topic-root/
topic1/...
topic2/...
|
Each queue dropbox has the same directory structure shown below. On all mainsteam operating systems
and network operating systems, moving a file from one directory to another is an atomic operation
(provided both directories are on the same logical disk). DropboxMQ relies on this atomicity to
dole out work to message consumers and to resolve contention between message consumers.
.../root/
queue1/
incoming/
working/...
target/...
processing/...
processed/...
routing/...
expired/...
error/...
|
Topic dropboxes have a slightly different structure to allow for multiple subscribers, but they
work with the same basic principal:
.../root/
topic-root/
topic1
.incoming/
working/...
target/...
processing/...
processed/...
routing/...
expired/...
error/...
.subscriptions/
sub1.connection1/
working/...
target/...
processing/...
processed/...
routing/...
expired/...
error/...
sub2.connection2/
working/...
target/...
processing/...
processed/...
routing/...
expired/...
error/...
|
A dropbox message producer creates a unique file in the working directory of the
dropbox it is sending to and writes the message contents to the file. While the message file
remains in the working directory, the message has not actually been sent and the
producer can take as long as it needs to complete the message.
Once the producer is done writing the file and ready to commit the message, the producer moves the
file to the target directory. The producer must move (not copy) the file to assure
that a consumer doesn't attempt to read a message before it is completely written. Once the
message file is in the target directory, the producer has relinquished control of the
message and it is consumable.
NOTE: It is very important that producers of any type (shell scripts, etc.) follow the
protocol of first writing the message to the working directory, then
moving (not copying) the files to the target directory. Even when
sending very small messages, it is possible a message consumer could attempt to read a message
before it is complete if it is created directly in the target directory.
Several message consumers may be scanning the target at the same time. When a consumer
finds a message file, it attempts to move the file to the processing directory. Only
one consumer will succeed. The consumers that fail to move the message file (or fail to notice the
message file) quietly go back to scanning the target directory for the next message
file.
The single consumer that succeeded to move the file then begins the process of consuming the
message. If the message is consumed and acknowledged, the consumer moves the message file to the
processed directory. If the message is not successfully consumed and is recovered,
the consumer moves the message file to the error directory.
DropboxMQ encodes all message metadata (priority, message identifier, reply-to, etc.) in the
message's filename. This allows the contents of the message file to contain just the message body
which integrates more easily with dropbox clients that don't use DropboxMQ. Message producers are
only required to create message files with unique names.
A message producing client that doesn't use DropboxMQ but wishes to have tighter integration with
DropboxMQ, can choose to create message filenames that are compatible with the
default DropboxMQ transcoder. By default, DropboxMQ message filenames are broken into several
dot (. ) separated fields. Trailing dots are required if the remaining fields are
undefined. Of the following fields, only the first three (JMS Priority, JMS Message Id and Message
Class) are required to permit integration with a DropboxMQ consumer.
Example 1: All fields defined
4.1140429201295000-9262574723.T.1140429211295.corr1283.TestQueue1.XYZType.prop1S=hello
Example 2: Minimum fields defined
4.1140429201295000-9262574723.T
|
Each field is described below:
Position |
Name |
Default |
|
1. |
JMS Priority |
Message.DEFAULT_PRIORITY (4) |
DropboxMQ supports priority values from -231 to 231-1. |
|
2. |
JMS Message Id |
None |
If no message identifier can be parsed, the entire message filename is used for
the message identifier. If DropboxMQ is used to produce a message, JMS Timestamp is also
encoded into the JMS Message Id. |
|
3. |
Message Class |
'B' (BytesMessage ) |
'M' = Message
'B' = BytesMessage
'S' = StreamMessage
'P' = MapMessage
'O' = ObjectMessage
'T' = TextMessage
|
|
4. |
JMS Expiration |
No expiration (0 ) |
The date in milliseconds past midnight, January 1, 1970 UTC at which time the
message will expire. |
|
5. |
JMS Correlation Id |
None, no correlation id |
|
6. |
JMS Reply To |
None, no reply-to |
|
7. |
JMS Type |
None, no type |
|
8. |
Properties |
None, no properties defined |
Properties are define in name-value pairs with each pair separated by an
ampersand (& ). Names and values are separated by an equals sign
(= ). Names and values are URL encoded to prevent odd characters from causing
problems with the parser. The last character of the name determines the property type.
Valid types are listed below. |
'B' = boolean
'Y' = byte
'H' = short
'I' = int
'L' = long
'F' = float
'D' = double
'S' = String
|
|
|