IDE Usability Resuts – Part II
Posted by dennisacosta on 2009/11/02
Maxime Caron recently posted results from the French version of our survey. Here are some of the findings from the English version and how I interpret them. (Note: Unfortunately, the server that was hosting the survey had a slight meltdown (literally!). We were able to recover some data but not everything.)
The first five questions were used to gain an understanding of the user and their previous programming experience. This allowed us to interpret the results in the proper context. Of particular interest are the questions about IDE preference:
42% of participants have used Eclipse
23% of participants prefer Eclipse
The drop in user retention can be attributed to many different reasons, but the results may suggest that users are finding the standard Eclipse environment too complicated. These results are comparable to the French version of the survey.
Questions 6-10 were used to determine whether a feature is commonly used by a beginner programmer. The features in question are currently up for debate within the Eclipse community. Here’s what we learned:
73% of participants use a Debugger
88% of participants find a line-number “ruler” useful
The results suggest that users aren’t having a hard time using a debugger as originally assumed. This is significant for us because we were considering simplifying the Eclipse Debug Perspective but this may no longer be necessary or a priority. Also, line-numbers in the editor are currently not available by default in both the standard Eclipse environment and Eclipse4Edu. This is something we may want to consider including.
Questions 11-13 were aimed at determining whether specific features (compiling, execution, and importing/exporting) were complex for beginners. The results show that all of these tasks are easily accomplished or learned and therefore need little changing.
The final questions were open asking for any difficulties encountered, likes/dislikes, and feature requests. Here is a sample of the feedback:
“creating a project in eclipse is confusing because you need to actually right click on the src folder when you want to create a new java class. While this sounds extremely simple now, it was a task that I was almost ready to give up on when I tried to create my first project.” –This is actually addressed in Eclipse4Edu.
“For Eclipse, I wish it was easier to attach the java source code to the program. As it stands I have to install the jdk in order to attach the source. This isnt very handy when I’m on a network machine where I’m unable to install anything. Also I wish the program would tell me about such features as the debugger, revision control system and refactoring features because I have no idea what those are and would like to get the most out of the program. Finally I wish it was easier to search in Eclipse for java code. Having to use the hierarchy pane is complicated, I would like to just open a search window and type in what Im looking for.”