"Clean Code" Method Sorter for Eclipse?

Have you ever chased your tail through a class, hopping from one function to the next, scrolling up and down the source file, trying to divine how the functions relate and operate, only to get lost in a rat’s nest of confusion? Have you ever hunted up the chain of inheritance for the definition of a variable or function? This is frustrating because you are trying to understand what the system does, but you are spending your time and mental energy on trying to locate and remember where the pieces are.

Concepts that are closely related should be kept vertically close to each other. We want to avoid forcing our readers to hop around through our source files and classes.

Robert C. Martin "Clean Code" pp. 80

If you have to read other people's code quite often you probably know this, right?

Typically, there are two types of code ordering. Alphabetically (+ modifiers) or completely hand-crafted (which in many cases means "without any particular order"). But is it that complicated to let the IDE sort a class' members for you? Reducing the need to hop from top to the bottom of the file and thus minimizing scrolling and confusion?

The slide below gives you an example of how a well-sorted piece of code might look like. As you see the number of hops required to read the whole code is minimal since there is a clear ordering between the code:



How complicated can it be?

In the end, there are just a few things to do:

1. Extracting the caller-callee relationships from code,
2. Sorting these relations using some topological sorting algorithm,
3. Dealing with some pat/remi situations where several elements may have the same ordering, and finally
4. A "physically"reordering these methods in your source file.

That's all. At least in theory. It's well known that "in theory, there is no difference between theory and practice", and thus, it should be straight-forward to implement :-P

I'm glad that we found two students who will implement this nice Clean Code Method Sorter during their Eclipse hands-on this semester. I'm really looking forward to their clean code!

Good Luck, Mateusz and Fabian!

Track progress of this feature via Bugzilla.

Stay in touch with Code Recommenders via Twitter.

Say "Hello Android!"

Android is all around you, isn't it? It's in your smartphone, in your tablet - and chances are not too bad that you already wrote your own app for your device.

Me myself is still using an old-school Nokia 6230 that is used more often as watch than as phone or even internet device.

This is nagging me. I'm looking for a new phone. A smarter phone. A smartphone.

I'd like to have this one.

A phone which I can use to explore the internet, write emails, and even to write apps for myself (and sometimes to call people).

Unfortunately, I've no knowledge about how to write apps with Android. How about you? Do you know how to write apps in Android? How smooth was your start with Android?

For beginners (and experts alike) Lars' blog vogella.de is an excellent resource to learn about Android and related technologies.

However, not all details are yet covered in tutorials and two weeks ago Lars asked me wether we couldn't run Code Recommenders on Android?

It took me a few hours to figure out but yes - Code Recommenders is capable to learn models for Android too!

The only disadvantage with Android is, that there are not too many open source projects available. That means, that is quite tricky to get a reasonable amount of data to train Code Recommenders. Thus, we have to come up with a purely community driven approach that allows developers to share their knowledge about how to use Android directly from their IDE and - in turn - generating new models from this usage data on a daily basis. With such a system in place we think it should be possible to create a continuously growing knowledge base for Android.

Since my spare time is currently very low, I'm glad that Andreas Frankenberger signed up to implement this upload-train-deliver life-cycle for Android in his bachelor thesis. I'm really looking forward to his results. Until now he has collected a set of 119 open source Android applications (taken from http://en.wikipedia.org/wiki/List_of_open_source_Android_applications) which he will use to train the initial Android recommendation models used as proof of concept before implementing the upload and train cycle.

If you like the idea let us know. We will soon create an early set of recommendation models for Android and we would be glad to get your feedback about it. In addition, if you know any other open source Android projects not listed above, please feel free to give us a pointer to them (preferably as Eclipse ADT projects). Andreas will be glad to add them to the initial training repository.

If you have any other comments or ideas on this idea just discuss them here on the blog or in our forum.


See you!
Marcel

Stay in touch with Code Recommenders via Twitter.

At Lunch with JDT Team Zurich...

Just a few weeks ago I had the pleasure to present Code Recommenders to Frank Tip from IBM Research - a nice guy who, as I learned during the demo, has implemented some of the refactorings we use day in, day out in Eclipse! Today he's not working on Eclipse anymore but manages the Program Analysis and Transformation Group at IBM Research which is developing (among many other things) the outstanding WALA analysis toolkit we use inside Code Recommenders - definitely a must-have when doing static analysis in Java/JS.

During a 30 minutes demo I presented him the whole collection of tools we worked on in the group - a quite compact session with lots of ideas and concepts in it. After I finished, Frank asked whether we already presented the project to the JDT team and (as I negated) just said "well I will tell Erich about it. I think he should see that." Wow. Is it actually that easy to get in contact?

Just a few days later I was introduced to Daniel Megert and Markus Keller from IBM Research Zurich, two guys who are working on Eclipse JDT from its first day (and even before :)). We arranged a meeting in Zurich last Monday where I got the opportunity to present Code Recommenders during a so called "lunch talk"... I had no idea what a lunch talk actually looks like - but hey! No worries :)

On Monday morning I arrived at IBM Rationale in Zurich downtown. I had a nice chat with Daniel and learned quite a few things what the JDT team keeps busy behind the scenes. Then at 12 o'clock everyone in the office moved out to grab some food from the surrounding restaurants and went back to the group's common room where they unpacked their lunch and started eating.

Except me. I started talking.

Okay, so that's a lunch talk: 10 people having lunch while one guy is sitting in front of the desk entertaining the audience... Well, this looks odd for moment but frankly: it was a great atmosphere! I only wonder how previous speakers managed to eat their food. I guess they didn't... :)

However, after one hour we finished the lunch talk and started a detailed discussion on how Code Recommenders uses JDT and where JDT's defensive APIs affect code recommenders performance. We identified several potential changes / extensions to JDT which I would like to summarize in this post.


More Chatty JavaCompletionContentAssistContext

So far, the java content assist context provides plug-in developers with just a small number of interesting facts:

1. The position in text where code completion has been triggered on,
2. Some completion prefix like 'setT' as in 'button.setT<^Space>',
3. The expected type the proposal should return if any,
4. The enclosing Java element, i.e., the method or class in which code completion was triggered.

But what to do if you need the information of which kind the receiver of a code completion event is, or which name the receiver has? In a completion event like 'b.setT' you are almost on your own to figure out information like type and name of the receiver, the exact type of completion event (CompletionOnMemberAccess, CompletionOnMessageSend...) by either parsing the source file or rerunning the completion parser as we do in Code Recommenders. Both ways are time-consuming and take between 10 to 50 ms in our cases. Fast in general but unnecessary given that JDT completion already computed (almost) all relevant information before.

During discussion we identified a set of potential changes how content assist context could be improved to help other plug-ins to leverage the analysis results of JDT and how to provide access to more structural information like variable names and types etc.

If you are interested in discussing advances to JDT JavaCompletionContentAssistContext, just add your comments to Bug 340945 - Extension Request of JavaContentAssistInvocationContext.

More flexible Content Assist Lists

Currently, JDT provides two types of content assist windows. The 'default' content assist that shows the proposals of several engines on the first ^Space cycle, followed by various content assists lists that present proposals per category one-by-one. For instance when you press ^Space twice you get to the 'Templates' proposals, pressing ^Space three times in a row you get to SWT Templates etc.

This works pretty well and is a neat way to group things if there are just a few completion engines available. But how to deal with cases where your Eclipse installation has, say, 10 different completion engines? Putting all of them on the first, default, content assist list is not necessarily the best choice - but having to press ^Space ten times in a row to get the proposals of the last category seems odd too, right?

Agreed. This scenario is a bit special to Code Recommenders because it adds five new code completion engines to Eclipse (Intelligent Call Completion, Overrides Completion, Dynamic Templates, Chain Completion, and Example Code Search). However, it shows a configuration limitation that is not necessarily required. Wouldn't it be great if you could decide completely yourself which proposals are displayed on which content assist list? Maybe by configuring something like "Show all template proposals on the second ^Space"? How do you feel about a drag'n drop UI to configure Code Completion as sketched here:

There is a feature request Bug 340876 - [preferences][content assist] Request for more flexible configuration of content assist lists on this. We'll come up with a first patch for this request - but if you are interested in discussing how to implement this feature we'd appreciate your comments!

Now, the final topic and probably the most exciting one for tool developers at Eclipse!

Fine-grained IElementChangedListener notification on content change that is not a structural change

If you ever tried to get notifications about AST changes inside a method body, you probably know that there was no way to figure out which method's body actually changed. The only information you received was "this AST has changed... somewhere." As a tool builder who relies on information in source code you now have to reanalyze the whole AST to figure out what has changed and rerun your analysis. There was no way to build something like an incremental analyzer - and thus these tools were unnecessarily slowed down. Significantly.

With Bug 100267 - Fine-grained IElementChangedListener notification on content change that is not a structural change the world of Eclipse tool builders changes. Drastically. JDT currently considers to notify listeners about fine-grained deltas on method level! That's incredibly great since this would allow tools like Findbugs, PMD, and any other code analyzer to drastically improve their performance by only reanalyzing those parts that actually changed - or are affected by these changes.

I can't say how exciting this feature is to me. If you are interested in discussing fine-grained AST deltas and want to add your requirements - please join the discussion and vote for this bug!

Code Recommenders Contributing Back to JDT

I'm also happy to announce that Code Recommenders Chain Completion will move to Eclipse JDT! Therefore, we'll develop a pure JDT-based implementation and contribute this engine to JDT when ready. 

That's thrilling for me too :) 

Sum Up

It was a very exciting event being at Erich Gamma's group in Zurich and presenting Code Recommenders to JDT experts. I'm very glad and thankful for the planned support - I'm looking forward to all these improvements!

Unfortunately, most things won't happen before Eclipse 3.8. Eclipse Release cycles for such changes are unfortunately extremely long - and thus most of the changes become effective next year earliest.  But that's the way it goes. Thus: If you have any comments on any of these features - tell us about them now (here or in bugzilla)!

All the best,

Marcel

Code Recommenders' Next Steps: Interviews at jaxenter.com and jaxenter.de

Two interviews about Code Recommenders' current state and next steps have been published at jaxenter.de  (German) and jaxenter.com (English). Comments appreciated - just send them to the forum here.

Interviews are published here:

• jaxenter.de: Das Eclipse der Zukunft: Intelligent, sozial, kollaborativ
• jaxenter.com: Changing the Way Developers Learn About New APIs

Best,
Marcel

Eclipse Code Recommenders 0.2!

We are happy to announce the second drop of Eclipse Code Recommenders today! Two weeks after the first drop two very exciting new features are ready to be released - and we are curios to hear what you think about them!

Feature #1: Java chain completion for Eclipse

How many times did you ask yourself "How do I get an instance of, say, IStatusLineManager" in scenarios like the one below?

And how much time did you then spent with shimming from one method call to another to find a call chain returning the type you were looking for - OR- reading Javadoc's Uses documentation to learn how to obtain such an instance?

If you have the feeling that this was quite too long, then this feature is for you:

Code Recommenders chain completion (described in more detail here) supports you in situations like the one above by automatically searching the API jungle for you, i.e., it takes every local variable, every field and every method it can invoke and follows its members until it finds a field or method returning an instance of the requested type.

For example when searching for an IStatusLineManager inside a ViewPart it returns:

Code Recommender chain completion also works on static classes:

And directly on method returns:

And even on method returns with type casts:

Many thanks to Andreas Kaluza and Gary Fritz who implemented this feature for Eclipse Code Recommenders!



Feature #2: Dynamic, example-driven templates engine

The second feature is something I'm getting really excited about.

Have you ever wondered how to deal with a framework's object you are not used to? While working with SWT you may stumbled across SWT Templates. SWT Templates offer common best practices of how to use SWT directly inside your IDE as code completion proposals. For instance, when triggered on the literal "Text" in your current editor, they propose a complete object creation pattern as shown below.

These templates are excellent for occasional users of SWT. However, these templates suffer from one major drawback: They are hand-crafted by experts and consequently only a small number of such templates exist - and only for SWT.

But what if you want to use JFace? Or core parts of Eclipse UI? Or whatever framework you use in your projects?

Let's stick with JFace to illustrate Code Recommenders' template completion engine. Assume that we want to use a JFace TableViewer inside our own view. When triggering SWT Templates on the literal "TableViewer" Eclipse tells you that there are no matching SWT templates (well, not a big surprise, eh?) :




But the question is: Couldn't we generate something similar to SWT Templates for every other framework too? Automatically? Maybe by just looking on how others used the APIs before?

Right, that's Code Recommenders' template completion engine! The idea behind our template engine is to identify common usage patterns from example code and to create something similar to SWT templates for every type we came across when analyzing code examples.

The result of this is shown below: The screenshots show the same situation as above, but just with our example-driven template completion engine and its proposals for JFace TableViewer:

Neat, isn't it? Well, as you might see, these templates are not always as perfect as hand-crafted examples but good enough to be valuable, right? What do you think? Do you like it? Let us know.

And as you may expect from call completion: Of course these templates adopt themselves based on what you have typed before! See this screencast for a detailed example of this:

Many thanks to Stefan Henß who did an amazing job to integrate Code Recommenders intelligent call completion engine into this pretty neat templates completion engine!

Where do I get it?
If you have Code Recommenders 0.1 installed already you can simply choose Help > Check for Updates in Eclipse. Otherwise you can get it from the Eclipse Marketplace or from our update site:
http://download.eclipse.org/recommenders/updates/dev/. You may also check the project homepage for more screencasts one code recommenders.

It's a drop! (not a release)
To avoid any confusions we had with our first drop, we want to explicitly state that we are currently publishing drops only but no releases as defined by the Eclipse Incubator guidelines. Before we can officially release new versions of Code Recommenders we have to pass an official release review by the Eclipse EMO (read more about that here). So far we didn't request a release review because some dependencies are under examination by the IP team - but should do so for later versions to avoid further confusion :-)

What's next?
For 0.3 the goal is to stabilize the current tool suite. Thus do no hesitate to file bugs against the project. The more the better!

Also some exciting news:
We will start collaboration with UC Irvine's Sourcer Research Group - a research group which has an excellent body of knowledge in how to create next generation of code search engines.  This will be an exciting project - we will blog about it separately when more details are out. But just to let them know that you like the idea: Click this link


Thanks,
Johannes & Marcel