May 24, 2011

Oh Stacktrace, My Stacktrace...

Do you remember the first time you wrote a Wizard for Eclipse? Do you still know what went wrong the first time (and maybe a few times more)? Let's assume you wrote a wizard page like the one below which just contains a simple text widget inside the page's composite:


It's not necessarily a fancy UI nor complete but it doesn't look obviously wrong, right?
However, when you run the wizard it just fails without any error message.
A look into the error log reveals that there is a problem with JFace - but not your code:

Now, you might guess that this exception is caused by you, but frankly: When I wrote my first wizard page I was quite sure I didn't make any mistakes... God, yes. How wrong I've been...

After googleing for an explanation for a while, I found out that I was missing just one simple method call. Namely, calling setControl(..) on my wizard page. I figured this out thirty minutes later - what a great waste of time...

But as I've learned from my talk at Eclipse Summit, I'm actually not the only jerk doing this wrong.  I asked the audience to raise their hands if they know that error - and after a while they started grinning and one-by-one raised the hand to show that they experienced the same problem just some time ago. How about you? Do you know it too?

Let's see what Google returned me when I pasted the first line of my exception into its search box: ("null argument: has no page control" is part of the query but was cut off in the screenshot):


Google obviously does not a good job here. It still doesn't know the answer - even after years this error (better: wrong usage) exists.
So what can we do to get rid of this? Bug trackers like Jira and Bugzilla (with Mylyn extension) have shown the way. These tools have duplicate stacktrace detectors that, whenever they observe a stacktrace in a new report, query the bug database for bugs that contain similar stacktraces.
And guess what: They work - and not too bad! To evaluate that, we replayed the bug reports from Eclipse Bugzilla from 2001 to 2008 one by one and for each new bug report containing a stacktrace we tried to find the corresponding bug id this new report was marked as duplicate of). The amazing result: In ~70% of all cases we found the right bug id under the top 3 positions! Goodbye duplicates - if you submit a stacktrace...

Just another example. Maven had a neat feature that whenever an exception occurs during a build, it printed out a message that directs you to some wiki page which may contain further help about this error. Unfortunately, these pages have been mostly empty (at least for the cases I looked at them). However, the idea is great!
What if there would be a place where you can send your stacktrace to and get a list of resources in the web that discuss exactly your problem? With such a help it should be quite easy to nail down and fix your problem in just a few minutes, right? You may think of it as being similar to  stackoverflow.com - but just for stacktraces. Something like stacktraces.org - or stacktraces.eclipse.org - a repository which contains stacktraces of the users, error descriptions and solutions including pointers to resources in the web that deal with these problems.

What we have in mind is something like this: An Eclipse client that listens for exceptions printed on the console or written into the log:

which are parsed automatically by the tool Whenever it observes an exception it may show a pop-up (3) which starts a search dialog that shows your most recent stacktraces:
If you know click on "search" it will search stacktraces.org for similar stacktraces and returns pointers to resources in the web that deal with similar stacktraces - it may even include a discussion on this stacktrace with some explanation of what caused this exception and how to fix it. This summary could be displayed in your browser or directly in Eclipse as shown below:


Additional tabs may provide additional information like similar stacktraces, related web-resources etc.:

That's just a thought about how we could get rid of it. Best of it: in its minimum version there is no additional effort of a user required. We may just give pointers to existing resources or we may integrate the results into a forum etc.
What do you think? 
Would you use such a stacktrace search engine that leverages the community knowledge about common pitfalls and usage errors of your favorite open source project? How much time you could have saved if someone had pointed you directly to the forum post telling you that you simply forgot to call setControl.

Just click on the like/dislike button below - and if you have a few minutes, feel invited to drop a comment below.

Thanks, 
Marcel

Update #1:

If you like the idea you may also consider commenting on it at/for stackoverflow.com.


Stay in touch: Follow me on Twitter.

7 comments:

  1. Hi Marcel,

    Really liked your post! In fact, I had the same idea sometime ago and was trying to implement it. You can find my initial code in https://github.com/pasemes/buggenome/tree/devmatcher

    I have already implemented what I've called the "stacktrace parser" which identifies the stack lines and persists it in a mongodb instance. Currently, I'm working on the "matcher" which is responsible for identifying similar stacks. I'm planning to have a web interface that will work like a forum just like you have exposed. Also, I've thought about having IDE's plugins, like the Eclipse one you have described, but I was thinking to implement only a web interface for pasting stacktraces and a web API for communicating with the IDE's plugins later.

    I didn't know about the stacktrace detectors in the bugzilla and jira, maybe because I'm more used to Trac :-(, but I'll take a look.

    I would be really glad to participate or be involved in this initiative. I was implementing all alone, so I don't have much documentation (besides the tests). Also, I'm implementing it in Scala, I don't know if you would be interested. Anyway, we can talk about this if you want.

    ReplyDelete
  2. Hi Paulo Sérgio,

    a joint work on this would be great! We plan to build the stacktrace search engine as part of Code Recommenders which unfortunately has some implications on the licenses and software we can use.

    Should we continue the discussion at http://eclipse.org/forums/eclipse.recommenders ?

    ReplyDelete
  3. Sure, we can discuss there. The only problem is that I don't have an account. I've tried to register, but it seems to be disabled.

    Regarding the licensing I was still thinking about it, so this should not be a problem for me to contribute to the project. Also, I don't see any problems with using other software either.

    ReplyDelete
  4. The forum software upgrade seems to be challenging... Let's discuss this via developer mailing list then:

    http://wiki.eclipse.org/Recommenders/Contributor_Guide#Mailing_List

    ReplyDelete
  5. If nearly everybody hits this error wouldn't it be better to refactor your API or implementation to remove this common Pitfall instead of making an other tool nobody will know?

    ReplyDelete
  6. Great idea. Did you consider to approach the guys at stackoverflow.com? They already have an infrastructure to answer human questions. It should be pretty simple to build a "stacktrace query" engine on top of that.

    I imagine a Wiki with stack traces as "page names" and solutions/comments as page text.

    ReplyDelete
  7. Integrating such a stacktrace search engine into stackoverflow.com would be great and could - to some extent - solve the issue mentioned above. Yet there are no contacts to the guys from stackoverflow. But I think we should come up with a prototype which is capable to show its usefulness before.

    Regarding your idea of integration into existing platforms and wiki-style:
    You may check:

    http://wiki.eclipse.org/Recommenders/Proposal#Scope

    point 5 as well as one of the last slides here:

    http://www.slideshare.net/Microbiotic/eclipse-indigo-democamp-walldorf-2011

    which describe in little more detail what is planned. Also the discussion of next steps already begun and your comments are all welcome:

    http://dev.eclipse.org/mhonarc/lists/recommenders-dev/msg00056.html

    Best,
    Marcel

    ReplyDelete