HornetQ JMS on Jetty

Over the past few days I've been working on getting HornetQ (http://www.hornetq.org) to work on Jetty (http://www.mortbay.org/jetty/) -- specifically the embedded Jetty that Gradle (http://www.gradle.org) uses. I would first like to thank those on the HornetQ irc channel (irc://freenode.net:6667/hornetq) for helping me out and offering advice. Specifically I would like to thank Clebert Suconic and Andy Taylor for looking at some of my code and making suggestions that ultimately led me to this solution. It should be noted that the two solutions I will be discussing here are probably not the only roads to success. I'm quite confident the same could be achieved programmatically by setting up HornetQ in a bean that's loaded at application deploy time or similar fashion.

HornetQ setup using jetty-env.xml

Both of my two solutions are using jetty-env.xml to call the needed setup functions for HornetQ. The same could be achieved with jetty.xml if deploying to a standard install of Jetty. More information about these configuration files can be found at http://docs.codehaus.org/display/JETTY/Configuring+Jetty. In short these files act as an XML means of working with Java objects.  Objects can be instantiated, methods called, etc. All that's being done here is calling the methods in the HornetQ API to startup HornetQ in Embedded mode. Below is the jetty-env.xml I used:

<?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Mort Bay Consulting//DTD Configure//EN" "http://jetty.mortbay.org/configure.dtd">

<Configure id="wac" class="org.mortbay.jetty.webapp.WebAppContext">
  <New id="jmsServerManager" class="org.mortbay.jetty.plus.naming.Resource">
    <Arg><Ref id="wac"/></Arg>
    <Arg>jms/serverManager</Arg>
    <Arg>
      <New class="org.hornetq.jms.server.impl.JMSServerManagerImpl">
        <Arg>
          <Call class="org.hornetq.core.server.HornetQ" name="newHornetQServer">
            <Arg>
              <New class="org.hornetq.core.config.impl.ConfigurationImpl">
                <Set name="persistenceEnabled">false</Set>
                <Set name="securityEnabled">false</Set>
                <Get name="AcceptorConfigurations">
                  <Call name="add">
                    <Arg>
                      <New class="org.hornetq.core.config.TransportConfiguration">
                        <Arg>org.hornetq.core.remoting.impl.invm.InVMAcceptorFactory</Arg>
                      </New>
                    </Arg>
                  </Call>
                </Get>
              </New>
            </Arg>
          </Call>
        </Arg>
        <Call name="start" />
        <Call name="createConnectionFactory">
          <Arg>HornetQConnectionFactory</Arg>
          <Arg>
            <New class="org.hornetq.core.config.TransportConfiguration">
              <Arg>org.hornetq.core.remoting.impl.invm.InVMConnectorFactory</Arg>
            </New>
          </Arg>
          <Arg>
            <New class="java.util.ArrayList">
              <Call name="add">
                <Arg>java:comp/env/jms/connectionFactory</Arg>
              </Call>
            </New>
          </Arg>
        </Call>
        <Set name="context">
          <New class="javax.naming.InitialContext">
          </New>
        </Set>
        <Call name="createQueue">
          <Arg>testQueue</Arg>
          <Arg>java:comp/env/jms/queues/testQueue</Arg>
          <Arg></Arg>
          <Arg type="boolean">false</Arg>
        </Call>
      </New>
    </Arg>
  </New>
</Configure>

An instance of JMSServerManager is being created and the needed configuration objects are being created inline in this example. It's very important to make the start call on the JMSServerManager before any other calls on the object to create queues, topics or connection factories are made, otherwise you'll get exceptions in your log saying the server hasn't been started yet. It's also important to make sure the full JNDI names are used. jetty-env.xml is used to bind elements to JNDI for you but because the JMSServerManager will be doing the binding for us I found that the full JNDI name must be used. If programmatic access to the JMSServerManager is needed this is also bound to JNDI for us by Jetty under jms/serverManager -- which doesn't use the full JNDI name because Jetty is handling the binding, a slight bit confusing I know. Work with JMS can now proceed using standard JNDI lookups and JMS code to send and receive messages. All of this code is available at my GitHub repo: http://github.com/LightGuard/Research---Development--JEE-/tree/hornetq feel free to check it out, fork it, whatever :)

HornetQ setup using jetty-env.xml and HornetQ configuration files

This setup is very similar to one above, only using the HornetQ configuration files. Code is located at http://github.com/LightGuard/Research---Development--JEE-/tree/hornetq-with-config-file. Here's the jetty-env.xml file:

<?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Mort Bay Consulting//DTD Configure//EN" "http://jetty.mortbay.org/configure.dtd">

<Configure id="wac" class="org.mortbay.jetty.webapp.WebAppContext">
  <New id="jmsServerManager" class="org.mortbay.jetty.plus.naming.Resource">
    <Arg><Ref id="wac"/></Arg>
    <Arg>jms/serverManager</Arg>
    <Arg>
      <New class="org.hornetq.jms.server.impl.JMSServerManagerImpl">
        <Arg>
          <Call class="org.hornetq.core.server.HornetQ" name="newHornetQServer">
            <Arg>
              <New class="org.hornetq.core.config.impl.FileConfiguration">
                <Set name="configurationUrl">hornetq-configuration.xml</Set>
                <Call name="start" />
              </New>
            </Arg>
          </Call>
        </Arg>
        <Arg>hornetq-jms.xml</Arg>
        <Call name="start" />
        <Set name="context">
          <New class="javax.naming.InitialContext">
          </New>
        </Set>
      </New>
    </Arg>
  </New> 
</Configure>

It's much smaller this time as most of the configuration has been moved into the hornetq-configuration.xml and hornetq-jms.xml files (both of which are on GitHub). That's really all there is to it! My next steps will be to get logging of hornetq (and hopefully Jetty) using log4j and same log file as the rest of the application.

Dependency Primer

Every software project has dependencies: your own resources, your own APIs you've created, third party APIs and projects, etc. We've all had to deal with them at some point in our career. Recently while playing with Gradle (www.gradle.org) I've come to a realization about how they should be handled correctly. In this post, I'll talk about the two most prominent kinds of dependencies, their scopes, and where they fit into an application. I will not be discussing packaging or how the different scopes must be resolved to create a distributable / deployable artifact. Being a Java developer this is seen from within the Java space, but the concepts are universal to all software projects.

Dependency Types

There are basically two types or categories of dependencies: first level and transitive. First level dependencies are those resources that your application directly relies upon. Examples of first level dependencies include the language in which the project is written, entities your project directly uses such as an XML parser, images, and classes from a third party project. Transitive dependencies are dependencies of your first level dependencies. An example from the Java world could be commons-logging which is dependent on some logging implementation, commonly log4j or JDK logging. For commons logging the log implementation is a first level dependency, but for your project it's a transitive dependency. Another example may be an SAX XML parser (or any other XML parsing API). The SAX API is used directly by your project and is therefore a first level dependency, but it requires an implementation, possibly Xerces, which would be a transitive dependency of your project. This definition of dependencies has really been ingrained in me while using and trying Gradle (a build system [yes, another one] written in Groovy). In the past I've used Maven or Ivy (basically as a Maven alternative, but it does much more in the world of dependencies). Maven introduced me to the concept of transitive dependencies and how they fit into it's life cycle and therefore your project's life cycle. I owe a great deal to Maven for introducing me to the concepts, but Maven has a few blemishes (at least I think so) in the way it handles these different types of dependencies. Maven will by default (I don't think you can change this) include all of the transitive dependencies of the same scope in your classpath, which in my opinion, is incorrect. Here's an example taken from JSFUnit: pom.xml:

   <dependency>
      <groupId>net.sourceforge.cssparser</groupId>
      <artifactId>cssparser</artifactId>
      <scope>compile</scope>
   </dependency>
   <dependency>
      <groupId>net.sourceforge.nekohtml</groupId>
      <artifactId>nekohtml</artifactId>
      <scope>compile</scope>
   </dependency>
   <dependency>
      <groupId>xalan</groupId>
      <artifactId>xalan</artifactId>
      <scope>compile</scope>
   </dependency>

Dependency Tree:

[INFO] +- net.sourceforge.cssparser:cssparser:jar:0.9.5:compile
[INFO] |  \- org.w3c.css:sac:jar:1.3:compile
[INFO] +- net.sourceforge.nekohtml:nekohtml:jar:1.9.9:compile
[INFO] |  \- xerces:xercesImpl:jar:2.8.1:compile
[INFO] +- xalan:xalan:jar:2.7.0:compile
[INFO] |  \- xml-apis:xml-apis:jar:1.0.b2:compile

For this project cssparser, nekohtml and xalan have been configured as first level dependencies, but the effective classpath contains their compile (first level) dependencies as well. If the project relies on these libraries it should state them explicitly and not rely on the crutch of having them as transitive dependencies. Ivy usage can fall into the same problem, but this is not the case in Gradle (at least not without changing the default), which I believe is the correct way of handling first level dependencies. With Gradle the compile scoped (more in the next section) is not resolved transitively, so you'll have compile time errors if you have not declared a needed dependency.

Life cycle Scopes and Dependencies

The build life cycle for a software project can be distilled into three phases: compile (if needed), test, and runtime. Test is a little special because it contains two phases itself: testCompile and testRuntime, which are extensions of compile and runtime. So where do the different types of dependencies come into play? Your first level dependencies become your compile dependencies and runtime dependencies are pretty much your compile dependencies with transitive dependencies and a few other things that may be provided for you like container provided dependencies (though, those are arguably transitive dependencies of any third party dependency) such as a messaging provider, an HTTP implementation, transaction support, etc. The test dependencies extend compile and possibly runtime, and add their own dependencies for testing: a testing framework, mocking framework, possibly a slimed down server, and others, which of course would be first level dependencies for your tests.

Summary

To recap, there are two different kinds of dependencies: first level and transitive. First level are dependencies needed to build an run your project. Transitive dependencies are those dependencies of your dependencies. A software build life cycle essentially has three phases: compile, test, run. The compile phase should only use your first level dependencies. Runtime extends compile and is resolved transitively. Test extends both compile and runtime (though at different times) and uses its own dependencies as well. I hope this has been informative and helped others understand the relationship and distinction of dependencies and a software project.

JPA Annotation Locations with Hibernate

Just ran into this one at work. If you're using Hibernate as your JPA provider (not sure if this is true for others, please comment) all of your annotations on the main entity, mapped super class, embedded classes, etc. must be in the same location (either all on the properties or all on the methods). The reason behind it is at StackOverflow. Basically Hibernate expects the annotations to be in the same location as the first @Id it comes across. I experienced weird errors where it was picking up the property which was mapped to a different column on the table so when Hibernate when to validate the schema it blew up. Hope this helps someone.

Ivy Configurations when pulling from a Maven Repository Part I

I heard about Ivy (http://ant.apache.org/ivy) some time ago, but never really took the time to look into it. After all, I had Maven, and that's what we were using at work. So I really had no incentive to look into it. As I'm sure many of you have found there are some issues with Maven. With all the things it does well, there are a few things where it really falls flat on it's face. How about transitive dependencies for example? Bane of my Maven experience. The standard project layout is very nice, but at the same time it is a hindrance if, for whatever reason, you need to go against it. As most of my readers have seen I'm pretty well entrenched in the Seam camp. Seam does not play well with Maven, or maybe it's Maven that doesn't play well with Seam (Embedded JBoss to be specific, but others have found ways around this [http://www.google.com/search?q=seamtest+maven&hl=en, http://www.seamframework.org/Community/SeamTestCoverageMavencobertura, https://jira.jboss.org/jira/browse/JBSEAM-2371, http://www.seamframework.org/Documentation/SeamWithMavenOverview to name a few]). For those that have been using Seam with Maven are familiar with not being able to run their Seam tests easily with Maven, unless you know to put your test scoped dependencies first in the pom. There are some other issues I have with Maven, but this is not a post about how much Maven sucks. You can google for those, there are a lot of them; back to Ivy. A few months ago my friend Dan Allen blogged about dependancy management in a seam-gen project with Ivy (http://in.relation.to/Bloggers/ManagingTheDependenciesOfASeamgenProjectWithIvy), see his post for a decent intro to Ivy. In his code download he was unable to setup the dependencies needed for testing his project. In this post I'm going to explain why Dan ran into problems, the relationship between Maven scopes and Ivy configurations, as well as provide an updated version of his Ivy-ized seam-gen download.

IVY CONFIGURATIONS

I believe a little background information about Ivy configurations may be in order. If you're coming from the Maven world they are somewhat similar to dependency scopes and profiles. Because Ant really has no concept of a build life cycle the way Maven does (one of the things I do like about Maven) Ivy doesn't either. So if Ivy configurations aren't really Maven Dependency scopes, what exactly are they? The official Ivy site calls them "views on your module" (http://ant.apache.org/ivy/history/trunk/tutorial/conf.html). Personally I still find that concept difficult to wrap my head around. The definition I have come up with is this: An Ivy configuration is a labeled grouping of a project's publications and that grouping's dependencies. Perhaps that's similar to the Ivy site's definition, but it helped me understand what was going on, and how to create my own configurations. Unlike Maven scope names, Ivy configuration names are completely arbitrary, which as you guessed is both a good and a bad thing. It's a bad thing when you go to share your application, module, whatever with someone else and they use it as a dependency. They'll have to see the ivy.xml you created to determine the correct configurations to use. With Maven, we were given the scopes and we couldn't change them. I would suggest defining a company wide set of configurations or at least list the public ones in a README or something if you are distributing your project. You could also use the makepom ant task, and use that to upload to a Maven Repository but that's a different post :) As I mentioned above, an Ivy configuration may also be used in mapping and tying together dependencies. A full discussion with examples is available at http://ant.apache.org/ivy/history/latest-milestone/ivyfile/dependency.html under the Configurations Mapping section (sorry, they didn't include an anchor for that section). In it's most basic form it looks like this: conf="my_conf->other_conf which translates to my_conf depends on other_conf. There are some special wild cards and other kinds of mappings you can do, which are in the above link. You can also specify multiple mappings within the same attribute by separating them with a semi-colon. Very handy for say depending on the module itself and also the source. I know it sounds a little odd to depend on the source of a module, but that's how Ivy sees it.

MAPPING MAVEN SCOPES TO IVY CONFIGURATIONS

Thanks for humoring me through that long block of text to finally get to the point of this post. As Dan mentioned in his blog post, Ivy can read from Maven repos (very good move on Ivy's part I believe), but in order to do this they have to convert the Maven POM to an Ivy file. When you setup a Maven Repository as an Ivy Resolver within an Ivy settings file there are a couple of attributes which affect the resulting Ivy file (http://ant.apache.org/ivy/history/latest-milestone/resolver/ibiblio.html). The first one is m2compatible, which if you're using a Maven 2 repository is always going to be true. The second attribute, which is true by default if m2compatible is true is usepoms. I can understand why you would select false to conserve bandwidth (although minor), and reduce network traffic (one less call to make) but you do lose some things when it is set to false when Ivy creates the ivy.xml file for the dependency. Below are the same ivy.xml files converted from a Maven repository. The first one is not using the pom:

<ivy-module version="1.0">
 <info organisation="org.testng" module="testng" revision="5.6"
               status="release" publication="20081031232755" default="true">
     <configurations>
      <conf name="default" visibility="public">
     </conf></configurations>
     <publications>
          <artifact name="testng" type="jar" ext="jar" conf="default"></artifact>
           </publications>
       </info>
</ivy-module>

<ivy-module version="1.0" m="http://ant.apache.org/ivy/Maven">
 <info organisation="org.testng" module="testng" revision="5.6" status="release" publication="20071116012303">
  <license name="Apache License, Version 2.0" url="http://apache.org/licenses/LICENSE-2.0">
  <description homepage="http://testng.org">
  TestNG is a unit testing framework.
  </description>
  <m:maven.plugins>org.codehaus.mojo__dependency-Maven-plugin__null|org.apache.Maven.plugins__Maven-clean-plugin__null|org.apache.Maven.plugins__Maven-jar-plugin__null|org.apache.Maven.plugins__Maven-source-plugin__null</m:maven.plugins>
 </license></info>
 <configurations>
  <conf name="default" visibility="public" description="runtime dependencies and master artifact can be used with this conf" extends="runtime,master"/>
  <conf name="master" visibility="public" description="contains only the artifact published by this module itself, with no transitive dependencies"/>
  <conf name="compile" visibility="public" description="this is the default scope, used if none is specified. Compile dependencies are available in all classpaths."/>
  <conf name="provided" visibility="public" description="this is much like compile, but indicates you expect the JDK or a container to provide it. It is only available on the compilation classpath, and is not transitive."/>
  <conf name="runtime" visibility="public" description="this scope indicates that the dependency is not required for compilation, but is for execution. It is in the runtime and test classpaths, but not the compile classpath." extends="compile"/>
  <conf name="test" visibility="private" description="this scope indicates that the dependency is not required for normal use of the application, and is only available for the test compilation and execution phases." extends="runtime"/>
  <conf name="system" visibility="public" description="this scope is similar to provided except that you have to provide the JAR which contains it explicitly. The artifact is always available and is not looked up in a repository."/>
  <conf name="sources" visibility="public" description="this configuration contains the source artifact of this module, if any."/>
  <conf name="javadoc" visibility="public" description="this configuration contains the javadoc artifact of this module, if any."/>
  <conf name="optional" visibility="public" description="contains all optional dependencies">
 </configurations>
 <publications>
  <artifact name="testng" type="jar" ext="jar" conf="master">
  <artifact name="testng" type="source" ext="jar" conf="sources" classifier="sources">
 </artifact></artifact></publications>
 <dependencies>
  <dependency org="ant" name="ant" rev="1.6.5" force="true" conf="">compile(*),master(*)">
  <dependency org="junit" name="junit" rev="3.8.1" force="true" conf="">compile(*),master(*);runtime->runtime(*)">
  <dependency org="qdox" name="qdox" rev="1.6.1" force="true" conf="">compile(*),provided(*),runtime(*),master(*)">
  <dependency org="org.beanshell" name="bsh" rev="2.0b4" force="true" conf="">compile(*),provided(*),runtime(*),master(*)">
 </dependency></dependency></dependency></dependency></dependencies>
</ivy-module>

The ivy.xml generated with the pom contains much more information, notably more configurations, publications, and dependencies. The configurations that are in the last file are configurations that Ivy places in every ivy.xml which is converted from a Maven POM. You can always rely on them being there if usepoms is set to true. This part, as well as really not understanding Ivy configurations (yes, they are difficult to understand as the official documentation isn't that great for configurations) is where I believe Dan had problems. I suggest always enabling usepoms, because you get everything you would normally if you were using Maven, and it easier to craft your own configurations with dependencies. In the download (a modified version of what is in Seam 2.1.2) you'll see how powerful this can be for your builds and dependency management with Ivy. If you find / know of a better way to accomplish what I've done, please post a comment and I will make corrections as needed. I hope this helped at least one person better understand Ivy configurations. I spent about a week diving through documentation, forum postings and the Ivy source code to figure this out, I hope you don't have to go through the same experience :) In the next part I will demonstrate how to use Ivy to manage your dependencies without needing to add them to your source control repository.

Stay At The Hospital

I know I said I would blog about Ivy first but we had some other stuff that happened after that last post. My wife started to have really bad pain on her mid-section on Thursday night around 10pm or so that kept intensifying. We were worried it might be her appendix so we went into the ER. Neither of us really like hospitals but sometimes it's just best to go. Anyway we sat there in the ER room, not the lobby, for about 30 minutes until someone to came see us. I could be wrong, but I don't think they were that busy. The whole point of this post is just to rant about being in an emergency and not being able to get any help. Very, very annoying. We found out that she has gall stones and will need to have her hall bladder out :( Bad way to end a week, but it's been a bad couple of weeks. Here's to hoping for a better one next week.

Posted with LifeCast

Future Topic

I'm sure my readers have noticed I don't follow any sort of schedule or topic list in my postings, after all I did call it Random Bytes :) One of the postings I plan on doing Real Soon™ is about Ivy and what they call configurations. Unfortunately the documentation really isn't all that clear nor easy to grok. It took me almost a week of looking through examples and reading the docs to really understand how it all works. I'm hoping that by blogging about it someone will be helped. That will probably be one of the next posts I do. I would like to take a look at Gradle, a build system, yes another one, for Java/Groovy. I saw it before and glanced at it for a sec and was interested in learning more but I haven't had the time yet. My good friend Dan Allen (if you're reading this Dan, I hope you don't mind) had a tweet about it a couple of days ago from when he was at TJSS, that just made me more interested. I'm still working on the SeamTest man pages, and they're coming along nicely, maybe a few more weeks until I'll have a first draft ready to go. I'm also trying to fit in some coding on DBunit (dbunit.sourceforge.net) to allow it to handle schema and table creates and drops, something that it is sorely lacking. Judging by the talk on the dev list this will probably be something that goes into a 3.0 release along with some major refactorimg of the code base. There you have it, a rough outline of things to come on my blog.

Posted with LifeCast

New Blogging App

I'm trying Lifecast to keep my blog up to date with the simple updates. Larger posts will probably be done with the regular web interface or via Google Docs. If you can't tell I'm very happy with my new iPhone!

Posted with LifeCast

Proud iPhone owner

We bought iPhones the other day, so both my wife and I become some of the newest members of the iPhone family. I've played with the iPhone before and was very impressed with how easy it was to use. I really enjoy having it. I think the thing I like the most is having the internet with me everywhere! I love it! Now 3G may not be as speedy as I'm used to at work or home, but you couple that with (hopefully) mobile device aware sites and it's really not that bad. Like any good geek I downloaded the Lightsaber application. It's a very, very worthless application, but it's fun :) I also downloaded a couple of Twitter (www.twitter.com/lightguard) applications and it's pretty easy to use so I'm happy there. I am looking for a blogging app (free preferably, yes I'm cheap) that I can use to update this blog, maybe I'll post a little more frequently.

Web performance benchmarks or the rule of Fast Enough

I recently saw a blog post that listed out performance benchmarks of two presentation layer technologies for Java.  The posting probably wasn't as non-biased as it could have been, but that's besides the point, and not the issue of this post.  The performance stats were measured in milliseconds from the time a request hit the server to the time it left the server to go back to the user.  A stat worth keeping an eye on, right?  We want our applications to be fast, of course we need to be focused on how long that request sits on the server processing, don't we?

As developers most of us want to squeeze out the last bit of speed we can get from our applications and technologies.  We want ours to be the fastest, naturally; and most developers have a tendency to be perfectionists to some degree or another.  But is it really necessary?

Back to the post I read.  As I stated the results were measured in milliseconds.  Most of recorded data (even in  multi-user scenarios, the author did up to 25 concurrent users) points for both technologies were sub second.  That's less than one second for a request sitting on the server processing, sounds pretty darn good to me.  The author's conclusion (assuming I read it correctly) was essentially technology X should be used instead of technology A because it's faster.  Now to the heart of this post: do your users really care about that much of a speed difference??

When we're dealing with web applications we have to remember that the users are not running this application on their local machines.  Each request has to go over through the Interet, which is definitely slower than seeing something on our local development machines, or even on a local network.  We also have to keep in mind the user's perception of "slow."  That's really what this boils down to.  If our users feel the application is sluggish then we should probably look into it.  Users of the web don't expect requests to come back instantaneously.  Most users are happy with a 5 - 10 second return of their request.  If a couple seconds of that is spent processing the request on the server, you're probably good.  There are also other considerations on the client's side for perceived web performance, namely the browser being used.  How fast does it render HTML, does it wait to display the full page or does it load incrementally, etc ?

Basically there are a lot of variables that make up the perception of a slow web site.  Your best source of knowing if something is slow is your user base, and I'm not just talking about one or two but a sizable chunk of users.  If they're saying it's slow, go look into it.  If you don't hear any rumblings, don't worry about it.  Your site is "fast enough."  Also remember web front end technologies that are mainstream are all "fast enough" otherwise no one would use them.

Documenting SeamTest

I've decided to start documenting SeamTest both to help out at work, and to contribute to the project. Watch for it on www.seamframework.org I have a working draft of AbstractSeamTest that people can view. Please comment as you see fit. I'm also interested in refactoring SeamTest for easy use with Groovy. If anyone has ideas, please post them up as comments and we'll see what we can get going.