With Eclipse Juno, Code Recommenders stripped off its incubation status and released v1.0.0 to the public! In this post, we give a quick overview on all the new shiny code completion engines we've built in the past months for Eclipse - and introduce those who haven't heard of Code Recommenders yet to the idea of mining source code and more to build intelligent IDEs. This article is a copy of the Java Tech Journal Article written for the Eclipse Juno Special Edition. Get a copy to read more about other great innovations in Eclipse 4.2!
Teams and technologies change. Embrace it.
Developers and CTOs know: frequently changing teams and the “liberal” use of the latest and greatest technologies make your burndown charts and project cost estimations burst just within a few weeks. To compensate for the brain-drain and to lower the entrance barrier, team leads plan a fair amount of their team’s time for documenting software, doing reviews, and working together in pairs in front of one screen to evenly spread the knowledge about how to use APIs in the entire team – and to newcomers in particular. If done right, this is the best a team lead can do to make the team effective in the long run. But Code Recommenders thinks there is more a team lead can do...
Leveraging the hidden gems in your source code
API designers typically have some expectations as to how developers should use their APIs, i.e. they expect their clients to call certain methods at certain points in time and in a particular order. The challenge to all newcomers is to learn about these implicit expectations and API usage rules fast enough to be of help to the team they joined.
Sure, the API documentation may contain the necessary information. Somewhere. But between you and me: How often do you read the API documentation of, say, JButton with its 381 methods to figure out how to use it? Don’t you rather use Google to find a code snippet that does what you need? Or, if Google can’t help because you are programming against an in-house library, don’t you prefer looking at your existing code base to see how your colleagues successfully used that API before? And sometimes you just scroll through the code completion proposal pop-up to see which proposals may sound best for what you are trying to achieve, right?
There is nothing bad about it; it’s just horribly ineffective and costly. And apparently API documentation as it is today is only of limited use for developers.
This is where Eclipse Code Recommenders comes in. Code Recommenders is an extension to Eclipse’s Java Development Tools that analyzes code of existing applications, extracts common patterns of how other developers have used and extended certain APIs before, and re-integrates this knowledge back into your IDE in form of (i) intelligent code completion, (ii) extended API documentation, (iii) sophisticated example code search, and even (iv) bug detection tools – all powered by the implicit knowledge of the programming masses. If you like, you can think of Code Recommenders as bringing the idea of Web 2.0 into your IDE – or as we call it sometimes: Code Recommenders is about creating the IDE 2.0.
The remainder of this article will give a short overview over Code Recommender’s completion engines coming up in Eclipse Juno – as part of the Eclipse Java Developer Package, the Eclipse RCP packaging. Alternatively, if you start with any other package, it can be installed from the Juno Release Train update site.
Recommenders’ Code Completion Engines
Code Recommenders 1.0 adds five new code completion engines to Eclipse:
- Intelligent Call Completion
- Intelligent Code Snippet Completion
- Intelligent Overrides Completion
- Call Chain Completion
- Subwords Completion
Intelligent Call Completion
The Intelligent Call Completion engine probably illustrates the idea of Code Recommenders best. When dealing with Framework APIs, developers frequently have to deal with complex APIs. For illustration, consider the public API of javax.swing.JButton which consists of 381 (!) public methods. This is a huge API of which a developer typically only has to know a small subset. The remaining, say, 360 methods unnecessarily bloat the API (from a API user’s viewpoint) and thus increase the complexity and burden of learning and using this API.
 |
| All potential completions on JButton |
This is where Code Recommenders’ Intelligent Call Completion comes in. It assists the developer by recommending only those methods that are actually relevant for the task at hand. For instance, given the fact a developer just created a text widget makes it obvious for Code Recommenders which methods that developer should want to use next – even if the developer doesn’t know it himself (yet):
 |
| Recommenders' intelligent Completion on SWT Text after calling 'new Text()' |
As of this writing (1.0.0), Code Recommenders partially supports the Java Standard Library, namely the main packages under java.* and some packages under javax.*. As the recommendation models are generated from the Eclipse Juno Release Train code base only, packages like java.awt or javax.swing are not yet supported, as no data was available at generation time.
Intelligent Code Templates
Code templates are helpful when code is needed to iterate over an array of objects or when creating a getter for a property of a class. But code templates really shine when developers have to use APIs they are not familiar with. Code templates then serve as additional documentation that quickly shows how to use an API, and thus can save developers a lot of time that would otherwise be needed for reading API documentation.
Eclipse maintains more than 70 of such Java code templates ranging from simple loops up to complex API usage patterns like creating an SWT Button or Composite. Unfortunately, developing templates for API usage patterns is an extremely costly and tedious job and consequently only few templates on how to use complex APIs of, say, JFace, Eclipse UI or even the Java Standard Library exist.
This is where Code Recommenders comes in again. In the previous section we showed how to recommend single methods to invoke on an object. Code Recommenders’ Templates Completion takes this to the next level by recommending not only single methods but complete sets of methods:
 |
| Intelligent template completion on JDT ASTParser |
As you probably have noticed in the example above, Recommenders' templates completion can be applied on existing usages but also works on type names as the example below shows:
 |
| Intelligent Template Completion on JFace TableViewer type name |
The final result after applying the template then looks as displayed below:
 |
| Resulting code snippet for JFace TableViewer |
Warning: The generated template proposals do not reflect method execution ordering constraints, i.e. the order of the proposed method may have to be changed manually after insertion.
Intelligent Overrides Completion
Similar to recommending method calls, one can also recommend which methods a developer should typically override. This is what Recommenders’ Intelligent Overrides Completion does. There is much more to say about how classes can be extended, but we’ll save this for another article about Code Recommenders’ Extended Documentation Platform – a platform that extracts valuable (extension) patterns in code and enriches existing API documentation with these patterns.
 |
| Recommenders' intelligent overrides completion on JFace Dialog |
Subwords Completion
Another noteworthy extension engine is Code Recommenders Subwords Completion. As an experienced Eclipse user you probably know JDT’s CamelCaseCompletion. This engine is nice but requires you to remember the exact uppercase letters of the completion proposal you want to insert.
Subwords makes this more convenient. The idea is simple enough: You do not have to type a name from the beginning to find a match in the content assist pop-up. It helps if a developer does not know if one has to “find” – or – “get” an element:
 |
| Recommenders' Subwords completion on JDT's CompilationUnit |
And it’s even sophisticated enough to understand a rough shorthand e. g. dclr for declaration or combinations of words such as 'ty + dclr' , which finds all proposals containing the words 'type' + 'declarations':
 |
| Recommenders' Subwords completion on JDT's AST |
Note: Subwords does not fall into the group of intelligent completion engines, i.e. it does not need any training data and thus works out of the box with any framework or API.
Also note that Subwords completion is a replacement of the JDT's standard Java content assist and cannot be used together with JDT's proposal computer. Visit Subwords preference page to enable Subwords for your installation.
See Deepak Azad's original blog post for more detailed description. Thanks for permission to reuse.
Chain Completion
The last engine I’d like to introduce is Recommenders’ Chain Completion. Sometimes you need to access objects that can be reached by invoking several method calls in a row – so-called call chains. Usually you have to find these call chains yourself by traversing the API call graph manually, and evaluating whether each potential chain may return an instance of the required type.
Code Recommenders’ call chain completion automates this for you. It quickly traverses the whole API call graph and finds all possible paths through the API that may return an appropriate object:
 |
| Recommenders' Chain completion on IStatusLineManager in a ViewPart |
Note: Templates and Chain completion don't run on the default content assist list and deactivate themselves if they figure out you put them on the default content assist. You can put them on the second or third content assist list and press ctrl+space twice (or more) to get to the subsequent content assist lists.
What’s coming next?
The Code Recommenders completion engines we introduced in this post are just a teaser. There are many more exciting features coming like extended (mined) documentation platform, personalized code search engines, code snippets miner, stacktrace search engines, and many tools more. Keep an eye on this project which has exciting things in delivery. Promised.
Thoughts and new ideas for Code Recommenders?
We are curious to hear how you like these new completion engines and what features you'd like to see for the next release. The Recommenders forum is a good place to discuss new ideas - or Twitter for really short ones.
