Ivy Configurations when pulling from a Maven Repository Part I

I heard about Ivy (http://ant.apache.org/ivy) some time ago, but never really took the time to look into it. After all, I had Maven, and that's what we were using at work. So I really had no incentive to look into it. As I'm sure many of you have found there are some issues with Maven. With all the things it does well, there are a few things where it really falls flat on it's face. How about transitive dependencies for example? Bane of my Maven experience. The standard project layout is very nice, but at the same time it is a hindrance if, for whatever reason, you need to go against it. As most of my readers have seen I'm pretty well entrenched in the Seam camp. Seam does not play well with Maven, or maybe it's Maven that doesn't play well with Seam (Embedded JBoss to be specific, but others have found ways around this [http://www.google.com/search?q=seamtest+maven&hl=en, http://www.seamframework.org/Community/SeamTestCoverageMavencobertura, https://jira.jboss.org/jira/browse/JBSEAM-2371, http://www.seamframework.org/Documentation/SeamWithMavenOverview to name a few]). For those that have been using Seam with Maven are familiar with not being able to run their Seam tests easily with Maven, unless you know to put your test scoped dependencies first in the pom. There are some other issues I have with Maven, but this is not a post about how much Maven sucks. You can google for those, there are a lot of them; back to Ivy. A few months ago my friend Dan Allen blogged about dependancy management in a seam-gen project with Ivy (http://in.relation.to/Bloggers/ManagingTheDependenciesOfASeamgenProjectWithIvy), see his post for a decent intro to Ivy. In his code download he was unable to setup the dependencies needed for testing his project. In this post I'm going to explain why Dan ran into problems, the relationship between Maven scopes and Ivy configurations, as well as provide an updated version of his Ivy-ized seam-gen download.

IVY CONFIGURATIONS

I believe a little background information about Ivy configurations may be in order. If you're coming from the Maven world they are somewhat similar to dependency scopes and profiles. Because Ant really has no concept of a build life cycle the way Maven does (one of the things I do like about Maven) Ivy doesn't either. So if Ivy configurations aren't really Maven Dependency scopes, what exactly are they? The official Ivy site calls them "views on your module" (http://ant.apache.org/ivy/history/trunk/tutorial/conf.html). Personally I still find that concept difficult to wrap my head around. The definition I have come up with is this: An Ivy configuration is a labeled grouping of a project's publications and that grouping's dependencies. Perhaps that's similar to the Ivy site's definition, but it helped me understand what was going on, and how to create my own configurations. Unlike Maven scope names, Ivy configuration names are completely arbitrary, which as you guessed is both a good and a bad thing. It's a bad thing when you go to share your application, module, whatever with someone else and they use it as a dependency. They'll have to see the ivy.xml you created to determine the correct configurations to use. With Maven, we were given the scopes and we couldn't change them. I would suggest defining a company wide set of configurations or at least list the public ones in a README or something if you are distributing your project. You could also use the makepom ant task, and use that to upload to a Maven Repository but that's a different post :) As I mentioned above, an Ivy configuration may also be used in mapping and tying together dependencies. A full discussion with examples is available at http://ant.apache.org/ivy/history/latest-milestone/ivyfile/dependency.html under the Configurations Mapping section (sorry, they didn't include an anchor for that section). In it's most basic form it looks like this: conf="my_conf->other_conf which translates to my_conf depends on other_conf. There are some special wild cards and other kinds of mappings you can do, which are in the above link. You can also specify multiple mappings within the same attribute by separating them with a semi-colon. Very handy for say depending on the module itself and also the source. I know it sounds a little odd to depend on the source of a module, but that's how Ivy sees it.

MAPPING MAVEN SCOPES TO IVY CONFIGURATIONS

Thanks for humoring me through that long block of text to finally get to the point of this post. As Dan mentioned in his blog post, Ivy can read from Maven repos (very good move on Ivy's part I believe), but in order to do this they have to convert the Maven POM to an Ivy file. When you setup a Maven Repository as an Ivy Resolver within an Ivy settings file there are a couple of attributes which affect the resulting Ivy file (http://ant.apache.org/ivy/history/latest-milestone/resolver/ibiblio.html). The first one is m2compatible, which if you're using a Maven 2 repository is always going to be true. The second attribute, which is true by default if m2compatible is true is usepoms. I can understand why you would select false to conserve bandwidth (although minor), and reduce network traffic (one less call to make) but you do lose some things when it is set to false when Ivy creates the ivy.xml file for the dependency. Below are the same ivy.xml files converted from a Maven repository. The first one is not using the pom:

<ivy-module version="1.0">
 <info organisation="org.testng" module="testng" revision="5.6"
               status="release" publication="20081031232755" default="true">
     <configurations>
      <conf name="default" visibility="public">
     </conf></configurations>
     <publications>
          <artifact name="testng" type="jar" ext="jar" conf="default"></artifact>
           </publications>
       </info>
</ivy-module>

<ivy-module version="1.0" m="http://ant.apache.org/ivy/Maven">
 <info organisation="org.testng" module="testng" revision="5.6" status="release" publication="20071116012303">
  <license name="Apache License, Version 2.0" url="http://apache.org/licenses/LICENSE-2.0">
  <description homepage="http://testng.org">
  TestNG is a unit testing framework.
  </description>
  <m:maven.plugins>org.codehaus.mojo__dependency-Maven-plugin__null|org.apache.Maven.plugins__Maven-clean-plugin__null|org.apache.Maven.plugins__Maven-jar-plugin__null|org.apache.Maven.plugins__Maven-source-plugin__null</m:maven.plugins>
 </license></info>
 <configurations>
  <conf name="default" visibility="public" description="runtime dependencies and master artifact can be used with this conf" extends="runtime,master"/>
  <conf name="master" visibility="public" description="contains only the artifact published by this module itself, with no transitive dependencies"/>
  <conf name="compile" visibility="public" description="this is the default scope, used if none is specified. Compile dependencies are available in all classpaths."/>
  <conf name="provided" visibility="public" description="this is much like compile, but indicates you expect the JDK or a container to provide it. It is only available on the compilation classpath, and is not transitive."/>
  <conf name="runtime" visibility="public" description="this scope indicates that the dependency is not required for compilation, but is for execution. It is in the runtime and test classpaths, but not the compile classpath." extends="compile"/>
  <conf name="test" visibility="private" description="this scope indicates that the dependency is not required for normal use of the application, and is only available for the test compilation and execution phases." extends="runtime"/>
  <conf name="system" visibility="public" description="this scope is similar to provided except that you have to provide the JAR which contains it explicitly. The artifact is always available and is not looked up in a repository."/>
  <conf name="sources" visibility="public" description="this configuration contains the source artifact of this module, if any."/>
  <conf name="javadoc" visibility="public" description="this configuration contains the javadoc artifact of this module, if any."/>
  <conf name="optional" visibility="public" description="contains all optional dependencies">
 </configurations>
 <publications>
  <artifact name="testng" type="jar" ext="jar" conf="master">
  <artifact name="testng" type="source" ext="jar" conf="sources" classifier="sources">
 </artifact></artifact></publications>
 <dependencies>
  <dependency org="ant" name="ant" rev="1.6.5" force="true" conf="">compile(*),master(*)">
  <dependency org="junit" name="junit" rev="3.8.1" force="true" conf="">compile(*),master(*);runtime->runtime(*)">
  <dependency org="qdox" name="qdox" rev="1.6.1" force="true" conf="">compile(*),provided(*),runtime(*),master(*)">
  <dependency org="org.beanshell" name="bsh" rev="2.0b4" force="true" conf="">compile(*),provided(*),runtime(*),master(*)">
 </dependency></dependency></dependency></dependency></dependencies>
</ivy-module>

The ivy.xml generated with the pom contains much more information, notably more configurations, publications, and dependencies. The configurations that are in the last file are configurations that Ivy places in every ivy.xml which is converted from a Maven POM. You can always rely on them being there if usepoms is set to true. This part, as well as really not understanding Ivy configurations (yes, they are difficult to understand as the official documentation isn't that great for configurations) is where I believe Dan had problems. I suggest always enabling usepoms, because you get everything you would normally if you were using Maven, and it easier to craft your own configurations with dependencies. In the download (a modified version of what is in Seam 2.1.2) you'll see how powerful this can be for your builds and dependency management with Ivy. If you find / know of a better way to accomplish what I've done, please post a comment and I will make corrections as needed. I hope this helped at least one person better understand Ivy configurations. I spent about a week diving through documentation, forum postings and the Ivy source code to figure this out, I hope you don't have to go through the same experience :) In the next part I will demonstrate how to use Ivy to manage your dependencies without needing to add them to your source control repository.

Stay At The Hospital

I know I said I would blog about Ivy first but we had some other stuff that happened after that last post. My wife started to have really bad pain on her mid-section on Thursday night around 10pm or so that kept intensifying. We were worried it might be her appendix so we went into the ER. Neither of us really like hospitals but sometimes it's just best to go. Anyway we sat there in the ER room, not the lobby, for about 30 minutes until someone to came see us. I could be wrong, but I don't think they were that busy. The whole point of this post is just to rant about being in an emergency and not being able to get any help. Very, very annoying. We found out that she has gall stones and will need to have her hall bladder out :( Bad way to end a week, but it's been a bad couple of weeks. Here's to hoping for a better one next week.

Posted with LifeCast

Future Topic

I'm sure my readers have noticed I don't follow any sort of schedule or topic list in my postings, after all I did call it Random Bytes :) One of the postings I plan on doing Real Soon™ is about Ivy and what they call configurations. Unfortunately the documentation really isn't all that clear nor easy to grok. It took me almost a week of looking through examples and reading the docs to really understand how it all works. I'm hoping that by blogging about it someone will be helped. That will probably be one of the next posts I do. I would like to take a look at Gradle, a build system, yes another one, for Java/Groovy. I saw it before and glanced at it for a sec and was interested in learning more but I haven't had the time yet. My good friend Dan Allen (if you're reading this Dan, I hope you don't mind) had a tweet about it a couple of days ago from when he was at TJSS, that just made me more interested. I'm still working on the SeamTest man pages, and they're coming along nicely, maybe a few more weeks until I'll have a first draft ready to go. I'm also trying to fit in some coding on DBunit (dbunit.sourceforge.net) to allow it to handle schema and table creates and drops, something that it is sorely lacking. Judging by the talk on the dev list this will probably be something that goes into a 3.0 release along with some major refactorimg of the code base. There you have it, a rough outline of things to come on my blog.

Posted with LifeCast

New Blogging App

I'm trying Lifecast to keep my blog up to date with the simple updates. Larger posts will probably be done with the regular web interface or via Google Docs. If you can't tell I'm very happy with my new iPhone!

Posted with LifeCast

Proud iPhone owner

We bought iPhones the other day, so both my wife and I become some of the newest members of the iPhone family. I've played with the iPhone before and was very impressed with how easy it was to use. I really enjoy having it. I think the thing I like the most is having the internet with me everywhere! I love it! Now 3G may not be as speedy as I'm used to at work or home, but you couple that with (hopefully) mobile device aware sites and it's really not that bad. Like any good geek I downloaded the Lightsaber application. It's a very, very worthless application, but it's fun :) I also downloaded a couple of Twitter (www.twitter.com/lightguard) applications and it's pretty easy to use so I'm happy there. I am looking for a blogging app (free preferably, yes I'm cheap) that I can use to update this blog, maybe I'll post a little more frequently.

Web performance benchmarks or the rule of Fast Enough

I recently saw a blog post that listed out performance benchmarks of two presentation layer technologies for Java.  The posting probably wasn't as non-biased as it could have been, but that's besides the point, and not the issue of this post.  The performance stats were measured in milliseconds from the time a request hit the server to the time it left the server to go back to the user.  A stat worth keeping an eye on, right?  We want our applications to be fast, of course we need to be focused on how long that request sits on the server processing, don't we?

As developers most of us want to squeeze out the last bit of speed we can get from our applications and technologies.  We want ours to be the fastest, naturally; and most developers have a tendency to be perfectionists to some degree or another.  But is it really necessary?

Back to the post I read.  As I stated the results were measured in milliseconds.  Most of recorded data (even in  multi-user scenarios, the author did up to 25 concurrent users) points for both technologies were sub second.  That's less than one second for a request sitting on the server processing, sounds pretty darn good to me.  The author's conclusion (assuming I read it correctly) was essentially technology X should be used instead of technology A because it's faster.  Now to the heart of this post: do your users really care about that much of a speed difference??

When we're dealing with web applications we have to remember that the users are not running this application on their local machines.  Each request has to go over through the Interet, which is definitely slower than seeing something on our local development machines, or even on a local network.  We also have to keep in mind the user's perception of "slow."  That's really what this boils down to.  If our users feel the application is sluggish then we should probably look into it.  Users of the web don't expect requests to come back instantaneously.  Most users are happy with a 5 - 10 second return of their request.  If a couple seconds of that is spent processing the request on the server, you're probably good.  There are also other considerations on the client's side for perceived web performance, namely the browser being used.  How fast does it render HTML, does it wait to display the full page or does it load incrementally, etc ?

Basically there are a lot of variables that make up the perception of a slow web site.  Your best source of knowing if something is slow is your user base, and I'm not just talking about one or two but a sizable chunk of users.  If they're saying it's slow, go look into it.  If you don't hear any rumblings, don't worry about it.  Your site is "fast enough."  Also remember web front end technologies that are mainstream are all "fast enough" otherwise no one would use them.

Documenting SeamTest

I've decided to start documenting SeamTest both to help out at work, and to contribute to the project. Watch for it on www.seamframework.org I have a working draft of AbstractSeamTest that people can view. Please comment as you see fit. I'm also interested in refactoring SeamTest for easy use with Groovy. If anyone has ideas, please post them up as comments and we'll see what we can get going.