Undergraduate Capstone Open Source Projects

A Lesson from “Coders at Work”

Posted by Greg Wilson on 2009/10/21

Over the last four or five years, there’s been a dramatic increase in the number of books about programming: not the specifics of this language or that framework, but the act of programming itself. Fogel’s Producing Open Source Software, Spinellis’s Code Reading, Ford’s The Productive Programmer, Doar’s Practical Development Environments, and multi-author collections like Beautiful Code and Beautiful Testing all discuss the things that separate people who can write code that compiles and runs from people who can build useful software.

The most recent entry in this category is Seibel’s Coders at Work, which collects his interviews with 15 great programmers. I haven’t finished it yet, but two things keep jumping out at me. The first is the importance of getting things into users’ hands as early as possible. It doesn’t have to be finished, and it certainly doesn’t have to be perfect; as long as it does something—anything—that they find useful, they’ll start giving you feedback. No matter how smart you are, you can only foresee a small fraction of what people will actually want to do, or how they’ll want to do it; letting them drive your software around is the best way to get the early course corrections that can save you days or weeks of wasted effort.

The second thing that jumps out at me is how focused these programmers are on avoiding waste—or rather, on delivering value. Will adding this feature or refactoring that module make the user’s life better in the near term? If not, file a ticket and move on, because there are plenty of other things that will. No, you shouldn’t let too much technical debt pile up, and yes, there are times when writing another abstraction layer will make development of the next dozen things so much faster that you’ll be ahead overall, but you have to know both your users and your code very well to be sure of that payoff. It’s all to easy to go from, “I want to fix this bug,” to, “Hm, I should refactor these classes first,” and then to, “Well, if I’m going to do that, I should upgrade this library,” and, “So first I’ll patch my OS,” until you find yourself three hours later saying, “OK, so once I’ve shaved the yak…”

So does this mean people should just hack together whatever works today with no thought for tomorrow? Of course not. Well-written code is its own reward, and one of your goals as a professional should be to make the right way of doing things a habit so that you won’t make silly mistakes when you’re tired, stressed, and up against a deadline. As an inveterate yak-shaver, I’ve found it helpful to always ask someone by email, “Should I be doing this?” before I start anything that’s going to take more than half an hour. At the very least, it keeps my teammates up to date with what I’m working on, and more often than you’d think, someone on the team will say, “Actually, wouldn’t it be more productive for you to do this other thing instead?”

Now, if you’ll excuse me, I need to go milk a whale…

9 Responses to “A Lesson from “Coders at Work””

  1. Bill Konrad said

    Our work on Basie possibly sheds a bit of light on when to shave and when not to, so to speak.

    People are always coming and going on the Basie team. We appear to retain ~1/5 of our members each term. This is not due to people WANTING to leave, they simply graduate, or need to fulfill a requirement and can’t afford the time. The upshot is that as Basie has grown it has become harder and harder to teach incoming developers enough about Basie to be productive. They only have about three months to catch up AND contribute.

    So, what’s the point? My thinking is that if we let a bunch of unpaid bills pile up (technical debt really is that granular) we make it harder and harder for our new contributers to catch up. They end up playing by a bunch of house rules, and potentially, before leaving, add a few new rules of their own.

    The other issue, to wrap up my thoughts, is that technical debt is best paid by the creator of that debt. He/she spent a long time figuring out why the current code should be refined, and while the intent is always to gracefully pass along knowledge, we all know that’s not easy.

    So, if you have a high contributer turnover rate, pay debt down early! Interest only piles up.

  2. Chani said

    over dinner tonight, I was whining to my roommates about robocup having no users. 😉 unlike a FOSS project, we don’t really have anyone to tell us what they want or what needs to be fixed. we just kinda have to make something that’s better at soccer than it was when we started. or something.

    this makes it even harder to avoid the “yak shaving” problem. for instance, we have a little debug gui that me and another group member made, so that we can get a better idea of what the heck the soccer players think they’re doing when they run in circles instead of kicking the ball. I could happily add features to that debug viewer for days… but it wouldn’t actually make them any smarter. on the other hand, the tool proved its usefulness last week when I was wondering what was going on – and then actually I found I hadn’t done *enough* work on the debug tool, and had to hack on it a bit to properly see the information I wanted.

    it’s a fine line at times, certainly. we have other areas of the project where there’s uncertainty about whether something’s worth implementing, too. it’s a good thing that we discuss at least the general idea of what we’re all working on and nobody’s off hacking on their own for long. 🙂
    as for the debug ui, I think it’s reached the Good Enough stage, now. I’ll keep improving it in my spare time, though; easy little tasks like that are fun when you’re too tired to do anything complicated.

    oh, and this can be important when you’re not getting money/grades too. in KDE there’s always a large element of people working on what they feel like working on (it’s one of the reasons we’re there, really) but at the same time, at least in core areas (libraries, workspace, etc), we have a practice of asking ourselves “how will this benefit the user?”. the abstraction layers written for KDE 4 all had a clear purpose and justification, and *have* paid off (we get to skip the polkit-1.0 porting pain that other applications are having now, for example 🙂

    I can go off and hack on some convoluted fancy doodad that’s really fun to hack on, but for it to be shipped in the ‘core’ kde packages (as opposed to extragear or playground), it has to have a reason for being there. plasma is especially strict about this; there are three main areas in subversion for plasma widgets based on their stability and value. features that don’t improve plasma as a whole, without causing undue trouble in other areas, don’t get into core areas of plasma. those of us who invest a lot of time into plasma want to see our work get in, so we tend to work on things that are worth getting in there. 🙂 I think this is one of our strengths, that we try hard to balance our desire to have fun hacking against our desire to write code that’s also clean and useful (and most of the time, useful code can be lots of fun too 🙂

  3. plorimer said

    I actually just read this article on Google for my Organizational Management class. It’s quite interesting how Google would rather you submit something that only partially works so that they can release it and build on it, rather than have one person sit and pain away at it until it works 100% (for them). It totally makes sense, because the minds of many will have many more ideas than the minds of one. I guess that’s the whole thing behind open source software. I thought that was cool about Google, they even get their Chefs to have ingenuity.

  4. Tara said

    I’ve definitely noticed the problem of “technical debt” in the work we’ve been doing on MarkUs so far this term – we have a LOT of tickets for writing (and fixing) functional and unit tests. In fact, three of the people working on the project are primarily working on testing. At the beginning of the term, when we would run tests, they would all fail! This is obviously coming from people working hard over the summer to get the functionality working so that they could deploy MarkUs at the beginning of September, but always, always leaving the testing off and then it finally ended up getting dealt with by some of us this term.

    I think that we are doing a better job of prioritizing our time this term. Yes, there is a long list of open tickets sitting in DrProject right now, but each of us have been tasked with a main priority of our analyzing, designing, implementing, and testing our feature for the term, with picking up some of the tickets on the side in the down time. A lot of these tickets are nice to haves, but not at all immediately necessary, but by prioritizing them as “high” it is easy to see which ones are much more relevant to be working on.

    I think that we need to be careful of not pushing out code we think is working, but without yet testing it, particularly in industry where it is very important that you stick to the crazy deadlines that your manager has set (or anywhere else that you find yourself in a similar situation). It is of the utmost importance that you strike the balance between getting your projects done on time and ensuring that they are fully tested prior to release.

  5. Holly Becker said

    Everything I’ve read says that having users is the best thing for a software project – from open source’s “release early, release often” philosophy, to comments like this. It’s far too easy to end up yak shaving without outside incentive to produce something actually useful. The only users of WikiDev currently are the developers, so you’d think that we’d end up developing things that are useful to us, but I’m not sure that’s true. Since I’m used to working with the semi-functional code, I don’t really mind it being broken in the same way that an actual user would. It seems like that would be different from having the incentive of someone asking for a feature and seeing their (hopefully) pleased reaction to its implementation.

    I wonder how technical debt relates to the initial choice of platform? On one hand, if you never pick a place to start, then nothing gets done. On the other hand, at what point does the realization that there’s a better way to do this make it worthwhile to switch systems? Probably depends a lot on how much has been done, and what can be ported. I guess it comes down to the users, and what would benefit them more, and how much you think they’d be willing to wait for the supposedly better version.

  6. With the Eclipse4Edu project I look at the importance of getting “things” into users’ hands as early as possible in two ways. The first is from the community perspective. Where the users are the members of the Eclipse development community. Getting patches/features/ideas into the community early and often is important because the dialog that follows, sets the future direction the project will move in. Of course the more obvious way to look at it is from the end user perspective. However, I’m finding it difficult to find a user base that is currently using Eclipse4Edu since it’s still in the incubation phase.

    I’m not sure how much of a problem technical debt is in the Eclipse4Edu project. On the one hand there are some activities such as compiler warnings that seem to take a backseat to larger tasks. But on the other hand committing code into the project can be long, arduous process because of the software standards imposed by the Eclipse Foundation.

  7. Louie said

    Reading this post reminds me of a Aaron Swartz wrote a while back. Like any life hacker, he analyzes his own procrastination habits and figures out the nature of procrastination and even offers a strategy for getting around it. He suggests that people HATE being told what to do, even by themselves. I have proven this fact many times in my feeble attempts at ordering myself to start homework. In every round between my pre-frontal cortex and reptilian subconscious, the animal in me wins every time. The more productive byproducts of this is probably what we call yak-shaving. Since the top order of work isn’t being accomplished, I naturally fall back to meta-work, which probably means beefing up my .vimrc and such.

    While working on ISPDB, I do run into times when I’m not sure what the best approach is to a given problem. In this situation I just write a bunch of code and post it for review. This is actually a great way to keep the “debt” at a manageable level because the reviewers have a much bigger picture and can see possible future issues. Right now I”m working on writing tests before coding to them; it’s a bit like that savings technique, if you pay yourself first (from each paycheque) then you’ll hardly notice the difference.

  8. Jack Carlson said

    It’s been hard sometimes with ElmCity to walk the line between paying down technical debts and yak shaving. Until recently, we’d been having some serious communication issues, so not everyone in the project was completely sure what they should be working on, and some of the issues that we felt were most important our clients and users did not. It’s often a fine line, and without good communication, it’s all too easy to end up shaving an entire field of yaks 😉

    Keeping an open issues list that is viewable to the public seems to have helped a great deal with walking this line though. This way, the programmers can add whatever issues they feel is needed, and is the clients / users doesn’t think it’s appropriate, they can say something regarding the issue before work progresses too far. Also, the users / clients can suggest new items that are more along the lines of what they want–and then the coders can review these suggestions and determine how actually relevant they are. The open issues list can also provide a rough roadmap for the challenges ahead, which can easily be lost sight of otherwise… And without that forward-looking goal, it’s once again too easy to get stuck working in the same state trying to re-factor everything as it is, without pressing forward–yak shaving.

  9. […] Oct 21: When is it “planning ahead” and when is it “shaving the yak”? […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: