• Unicorn – Rack HTTP server
• Nassim Nicholas Taleb and his books Fooled by Randomness and The Black Swan as
  • Related discussed Benoit Mandelbrot
• A bunch of chat about Google Wave and how it would have been received if it were built by a small startup rather than Google
  • Led to discussion of Christensen’s The Innovator’s Dillema as well Moore’s Crossing the Chasm
• Comparison of work/life in NYC vs. Philly
• Professional poker playing

I’m sure there were more that I am forgetting now (shouldn’t have waited 48 hours to do it).  Very worth getting up early

Bookmark to:

Hide Sites

The Default Maven Install Run Configuration Fails

This issue can be seen when running a project that has multiple modules in it. In my case we have an ejb module and then an ear module that is dependent on the ejb module.  When we try to run the default maven install task for the project (see image to the right), we get an error:

The following mojo encountered an error while executing:

Group-Id: org.apache.maven.plugins

Artifact-Id: maven-ear-plugin

Version: 2.3.1

Mojo: ear

brought in via: packaging: ear

While building project:

Group-Id: edu.psu.gv

Artifact-Id: classtwo-ear

Version: 0.0.1-SNAPSHOT

From file: /Users/rdimarco/Documents/workspace/classtwo/ear/pom.xml

Reason: Cannot copy a directory: /Users/rdimarco/Documents/workspace/classtwo/ejb/target/classes; Did you package/install active project artifact:

artifact = edu.psu.gv:classtwo-ejb:ejb:0.0.1-SNAPSHOT:compile;

project: MavenProject: edu.psu.gv:classtwo-ejb:0.0.1-SNAPSHOT @ /Users/rdimarco/Documents/workspace/classtwo/ejb/pom.xml?

The problem is that there is a bug when the "Resolve Workspace Artifacts" button is checked.  To resolve the problem, we need to create a custom run configuration, set the goals to install, and make sure the "Resolve Workspace Artifacts" is unchecked.

The build will now proceed correctly

Unable to Locate the Javac Compiler

Many of my students saw the following error: 

Reactor Summary:

[INFO] ------------------------------------------------------------------------

[INFO] Class Two ............................................. SUCCESS [1.483s]

[INFO] ClassTwo EJBs ......................................... FAILED [2.388s]

[INFO] ClassTwo EAR .......................................... NOT BUILT

[INFO] ------------------------------------------------------------------------

[ERROR]

 
Mojo: org.apache.maven.plugins:maven-compiler-plugin:2.0.2:compile
 
FAILED for project: edu.psu.gv:classtwo-ejb:ejb:0.0.1-SNAPSHOT

Reason:Unable to locate the Javac Compiler in:  C:\Program Files\Java\jre6\..\lib\tools.jarPlease ensure you are using JDK 1.4 or above and not a JRE (the com.sun.tools.javac.Main class is required).In most cases you can change the location of your Javainstallation by setting the JAVA_HOME environment variable.

As the error message suggests, this issue occurs when you are trying to run Maven and your the runtime JRE used to run the Maven executable is from a JRE and not a JDK.  To fix this, you need to do the following things:

1. Install a valid JDK on to your computer.  You will need to download the JDK from Sun and then install it on your computer.
2. Tell Eclipse about the JDK.  To do this, you will
   1. Go to the Preferences. 
      • On Windows, choose the top menu "Window" and then select sub-menu "Preferences"
      • On a Mac, choose the top menu "Eclipse" and then select the sub-menu "Preferences"
   2. Click on >Java and then on > Installed JREs
   3. You will see a box listing one or more JREs.  Click on the button to the right of this box "Add"
   4. You will be given a list of types of JREs to use.  Choose Standard VM
   5. You will then get a screen asking for the JDK directory.  Click on the Directory button and navigate to the newly installed JDK directory
      • On Windows, this will be something like C:\Program Files\Java\jdk1.6_xx
      • On Mac, this will be something like /System/Library/Frameworks/JavaVM.framework/Versions/1.6.xx/Home
   6. After choosing your directory, Eclipse will think for a bit and then fill in some additional fields on the form.  You do not need to make any changes, just hit Finish.
3. Now we will need to tell Maven to use this JDK when it runs
   1. Right click on your project and choose Run As -> Run Configurations (as you normally would to build)
   2. In the Run Configuration dialog, click on the tab JRE.
   3. Click on the radio button for Alternate JRE
   4. From the drop down list, choose your JDK

You will now be able to run Maven without the error.

Bookmark to:

Hide Sites

• Investment funding
• Finding customers
• Distribution channels
• Capital investment decisions

I’ve seen a lot of my startup clients make some of the same assumptions and mistakes that the Flying Fish people have.  It is always interesting to me how some problems domains.

Bookmark to:

Hide Sites

#1 Development Process Risk Mitigation

Spending $3-5k for travel will not kill a project, but bad communication can…  Going to Israel, our plan was to have a "mini-project", ship something in the week I was there.  Day 1 involved project planning with a goal of shipping Day 5.  Going through this process showed me what holes we had in communicating with each other.  By communicating, I don’t just mean language issues, I mean effectively explaining what needed to be done to plan, develop, and ship the project.  If we have problems when we are in the same room, we are definitely going to have problems when we are separated distance and time zones.  In my week there, we did not ship on time.  I learned that we had a problem being overly optimistic in some of our estimating techniques.  To me, it showed that some of our slippages in the past were not as much due to the outsourcing model, rather to poor project planning techniques.  At the end of the week, it was my conclusion that either I (or my other team lead) that we really needed to change certain parts of the process to be better at planning and execution of our development interactions.

#2 More People Should Visit

The country was beautiful.  I was turned off a little by the architecture in Tel Aviv at the beginning of my trip, but by the end, I found it to be quite beautiful, especially the older parts of Tel Aviv.  Also, it was awesome to be on the beach in shorts in January, reminded me a little of California.

#3 There Will Never Be Peace

One thing that is missing in most American news analysis, the conflicts around Israel are not as simple as Israel vs. Palestine.  There are so many conflicts, between the secular and the religious, between Israeli Jews and Israeli Arabs, between people of Eastern European/Russian lineage and other groups, between recent immigrants and those that have been there longer, between Fatah and Hamas, the list goes on.  There is no where near a unified front on how the ending should look and people are militant about it on all sides.  The conflict is so much deeper than just us vs. them; I don’t see how the people (especially in Jerusalem) will ever agree on zoning laws, let alone bigger issues.

#4 An International Midge of Mystery

Christina (my wife, aka Midge) came with me on the trip and got to be a tourist while I worked.  Being out of the Philadelphia area really agreed with us.  Philly is a great town, but so insular and stifling.  I think we need to move, just not sure where yet.  If I could find a 3-6 month stint overseas, we would leap at it.

Bookmark to:

Hide Sites

But they suck.  And they suck because they convey a level of certainty that is completely unfounded.  It’s pretty rare that we put a plan together where it turns out we actually understood:

• What needed to be done
• How long each task would take
• What other distractions (vacation, crucial issues, etc.) would distract resources from full concentration on the project.

My hatred of Gantt charts stems from the way they make the schedule seem black and white.  So what I want is a way to create a project plan that shows the levels of gray in the timing.  So a task would not take 3 days, it would perhaps have the highest probability of being complete in 3 days, but there are non-zero probabilities that it would take 4, 5, 6 days to complete.  I’d also say that the more tasks in the project and the more resources involved the higher the likelihood of missing tasks, pushing out the schedule even more.

The closest I have found to this is FogBugz’s Evidence Based Scheduling that shows a probability of a release shipping.  I like that it is based on data from developers, but I want to find a generalized way to represent this schedule outside of entering every task into Fogbugz. 

Maybe there’s a product development idea in here somewhere…

Bookmark to:

Hide Sites

I figured that I would have to use keytool to load the certificates.  What I did not realize is that you cannot load a self-signed key where you already have a certificate using keytool directly!  So I had to follow the following steps

1. Convert my certs from PEM format into DER format

> openssl pkcs8 -topk8 -nocrypt -in server.key \

-inform PEM -out key.der -outform DER > openssl x509 -in server.csr -inform PEM \

-out cert.der -outform DER

2. Use the Java code from this AgentBob post to create a keystore

> java -Dkeystore=mycerts ImportKey key.der cert.der

3. Now when running the TCPProxy, I had to add the following:

java -Djavax.net.debug=all -classpath $GRINDER_JAR \

net.grinder.TCPProxy -console -http \

-keystore mycerts -keyStorePassword importkey

4. To use the certs from my Agent process

from java.lang import Systemgrinder.SSLControl.setKeyStoreFile(System.getProperty("keystore"),System.getProperty("keypass"))

4. Finally, I needed to add this to the properties file for my Grinder agent process

grinder.jvm.arguments=-Dkeystore=mycerts -Dkeypass=importkey

And tada it works! One other tidbit: using -Djavax.net.debug=ssl was invaluable in debugging. You can use -Djavax.net.debug=help to find out all of the debug options.

Bookmark to:

Hide Sites

My student Guoqing who is from China is always full of stories and proverbs and good advice. When he was young he dreamed of being a writer and from what I can tell, he writes like crazy still, not just science but about many topics.  Then he became a dancer. But at some stage his father told him that it wasn’t good for him to be a dancer, because he is a boy and also, because he has to make a living.  So his father told him he should become a scientist.  At that stage, that very year, it was possible to take the national exam and switch fields between high school and college.  He did well on the exam, in composite, and had the opportunity to go to the top science school, USTC.  Of course, with little background and training, that was no easy task to try to catch up, fit in, and manage. At first, he said he failed quite miserably and got very bad grades and he had to study like crazy.  Some people would be bitter and angry about this I guess, but he said that his father has always given him good advice and so there was no reason not to listen to him then too. Of course, now Guoqing is a truly outstanding scientist, a natural in terms of creativity, drive and insight….But the reason I am telling you this story is this.  He said that at one stage, when things were trully horrible, in that adjustment process, he told his father…that it was horrible, he was depressed and things couldn’t be worse.  His father told him that was great news because then that meant it could only get better from there.  I was talking to Guoqing about difficulty today too, and he said to tell you about sine and the derivative cosine.  I said, what?!  He went to the board and drew it out, the waves, pointing out that when things are at their peak, in fact the change or derivative is zero. However when things are very low or at zero, then the potential for change is greatest.  I think he was trying to say that it’s not just about the highs and lows, but also the potential for change and also, there can be bigger meaning beyond…difficulty sometimes.  There was more to it when he explained this today, and I am not doing it justice.

I like the idea of life as the sine curve.  When things are good, stay grounded as you may be losing momentum.  When things are low, know that there is more potential now for growth than ever.

Bookmark to:

Hide Sites

The Problem

 The JVM defines a single heap for all objects created across all threads in the java process.  A consequence of this is that if one processing thread has a memory leak, any thread on the JVM may start suffering from OOM errors.  However, not all threads are equally important to my application.  I may not mind too much if a batch processing thread fails with an OOM but I may care very much if the lack of memory causes all of my Tomcat threads to no longer be able to process web requests.

Proposed Solution

What I would like is to be able to segment my heap so that I can dedicate portions of the heap (either by percentage of the total heap or by absolute number of bytes) to a specific set of work.  That way if an OOM occurs, I can contain its impact to a certain set of threads while other threads continue processing.  I would see this being configured as JVM runtime -X arguments, something like -XHeapSegment:Name=MyMemHeap,Size=25%,Thread=<ThreadNameRegex>

I could see the partitioning being relative (by percentage) or absolute (total bytes allocated).  I could also see linking it to ThreadGroups as opposed to just Threads.

Some Questions

• Do others see value in this proposal?
• I am not an expert at JVM internals.  Is there a fundamental reason this would not work?
• Any other suggestions?
• Should we allow this segmentation to be defined at compile-time as well through annotations?

Notes

• For the purposes of the problem I defined, I do not have a requirement that the young generations also be segmented.  However, if it is impractical to keep a single set of young generations and segmented older generations, I can deal with having segmentation in the young generations as well.
• Obviously, this would in no way address all OutOfMemoryErrors.  We can still look forward to running out of PermGenSpace

Bookmark to:

Hide Sites

Let’s consider a theory that ignores silent evidence.  I plan on voting for Barack Obama for president.  If you were to make conclusions from a poll that consisted ONLY of asking me who I would vote for, it would be logical for you to conclude that Obama is going to overwhelmingly win both Pennsylvania and the national popular vote. These predictions may very well prove to be accurate and you might be hailed as a tremendous prognosticaltor.  But most serious analysts would mock you.  They would be correct to do it.  Because of the miniscule sample size, your poll does not accurately reflect the size of the entire population that will be participating in the election.  The "silent evidence" are the opinions of all the voters who have NOT been polled.  A good pollster mitigates ignoring the silent evidence by trying to come up with a representative sample; the idea being that if you can get a large enough sample broken down by representative demographics, you can make a reasonable prediction.

So what’s the point?  Let’s consider another example.  In a famous essay, Paul Graham talks about the advantage of using functional programming languages when creating your startup.  If you haven’t read the post, I highly recommend it, it is well written and thought provoken.  However, it clearly ignores the silent evidence.  Graham takes his singular experience and makes a conclusion based off it.  But consider the silent evidence:

• How successful of functional language based software startups versus all software startups.

or a slightly different cohort

• How successful were functional language startups during 1995 (when ViaWeb was started) versus all software startups founded during that year.

With both these cohorts, we should probably include in the sample those kitchen table companies that started but never shipped a product.  Maybe many functional language startups fizzle quickly out and only the ones with really great ideas survive.

Of course, I could just as easily make the argument that statups that have two really, really smart co-founders who have impressive technical and business savvy and are starting their company in an exploding market have a much better probability of success than companies without those advantages.  My theory is no more right or wrong than the one posited above, we both are ignoring the silent evidence in positing our

My point is not to say Paul Graham is wrong about startups and functional languages.  He is a smart guy and has accomplished much more than I.  The Beating the Averages post was just the first one to popped into my mind while reading Black Swan, I’m sure I can find countless other technical articles.  But I do want to introduce some skepticism into the reading of his post and many other software legends.  When critically reading a book, article, blog posting, consider the silent evidence before making judgement on the theory.

Bookmark to:

Hide Sites

As i talked about in Effective Technology Teams, what someone knows is much less important than what they can learn.  For further reading, I highly recommend the book Mindset by Carol Dweck.  Fascinating book that helped make this concept clear to me.

Bookmark to:

Hide Sites