A Rational Response to an Irrational Environment
Posted by Greg Wilson on 2009/10/21
Four years ago, I asked one of the best students I’ve ever had why she always did her projects in one big burst of activity starting at the last possible minute. She explained to me that (a) she was more productive if she focused on one thing at a time, rather than trying to timeslice on three or four assignments at once, and (b) the later she left it, the more stable the assignment spec would be, because the keeners that started early would have pestered the prof into correcting the inevitable omissions, ambiguities, and what-not. In short, doing things the “wrong” way was actually a very rational response to an irrational environment.
I’ve seen a different sort of “rational response” in project courses like this one. Other courses have assignments with set weights and fixed deadlines, so on any given Tuesday, a student working on Basie or WikiDev has to choose between putting hours into a 10% stats assignment that’s due on Friday, or putting those same hours into—well, into something that could probably be pushed back a day or two. Or three. Or into next week, because once stats is out of the way it’s time to catch up on operating systems, which is due on Monday. And then there’s the AI midterm, and the outline of the term paper for the psych course that turned out not to be as easy as everyone had said, and around and around it goes.
Now, if we (the mentors for the UCOSP projects) had been parceling things up into bite-sized chunks and setting specific due dates, I’m confident that almost everyone taking part would have done the work, and done it well, by our deadlines. But we’re not doing that, because one of the points of this course is to give you a chance to find out what it’s like to set and then meet your own goals. The net result is pretty clear at this point: in many cases, students are doing less per week on this course than they would on a more structured course that had exactly the same content. It isn’t that anyone is deliberately slacking; it’s just that the fixed deadlines in other course always seem to be looming up out of the fog, and it’s very hard to choose to put them out of your mind and work on something whose panic point is further off.
It’s hard for us as instructors to figure out how to address this (which is a prof’s way of saying, “I don’t know what to do.”) Micromanaging students the way they’re usually micromanaged defeats the purpose of this course, and almost certainly makes it more fun. Asking students to specify each week what they will have completed a week hence doesn’t work well either: what does the prof do when the student says, “I will have written some tests for the module I’m working on” three weeks in a row? If the prof says, “Enough: if they aren’t checked in by next Thursday, I’m taking 5% off your course grade,” then we’re right back into traditional assignment-based micromanagement (or even worse, since the “assignments” are being sprung on students at unpredictable intervals). We haven’t yet found something that works well; suggestions would be welcome.
UCOSP 
stroulia said
Indeed “It is hard for us as instructors to figure out how to address this” … I have been thinking exactly the same thing many times, especially when the meeting time comes and an issue that could have been resolved with a quick email rises as a todo for next week, thus further delaying whatever real progress could be made.
One thing that I have come to believe (not too strongly I must admit) is that in non-required courses, the student makes a choice to participate, and thus becomes more of a co-investor in the experience (and at the same time, a co-risk taker). It is a project, it is open ended and I hope that al students have gone into this with their eyes open and with the intent to make most of the experience, which also includes designing the deliverables and the deadlines. I see myself as an enabler: I know what would be nice to have (many alternatives in fact) and I know how to provide resources, including answers to technical and managerial questions and infrastructure. Beyond that, I have an idea bout deliverable chunks, but I cannot really enforce them, just present them.
Is that view a renouncement of my responsibility as an instructor?
Eric Burnett said
I would certainly agree with your student, and add to the list a third reason to do everything at the last minute:
(c) At the last minute, you can’t procrastinate any longer.
Sure, procrastination for the sake of procrastination isn’t a ‘rational’ response like the first two, but let’s face it, humans aren’t rational creatures. Trying to pretend otherwise is a recipe for disaster, so those of us that love to procrastinate need to find other ways to ensure things get done.
To convince myself to get tasks done I use two main motivators, both purely related to how I think about the task in question. Deadlines are the first and simplest: if it has a deadline, I will make sure it gets done by then, or nearly kill myself trying. The downside is deadlines need to be external – If I set one myself, I can just as easily rationalize it away when the time comes and goes. The other trick I use is debt. For example, when I’m on a work term I feel I owe a full days worth of effort every single day to the company I’m working for, and if I find myself slacking off I’ll stay later to make sure it gets put in. The problem I find with debt is similar to deadlines – I can’t owe the debt to myself. So the same motivation that works great at work is pretty useless when applied to school, since the only person impacted when I don’t do school work is myself.
Like Eleni, I think all students (or at least, students worth having) really do want to get things done, but may have trouble with procrastinating too far. So, to answer your question, I think you need to help students overcome this inherent desire. I’d recommend an approach that ties into both the motivators above and lets the student do the rest. For example, declare that
a) Every week, every student owes their project one assignment worth of effort (or 5 hours, or…), and
b) Every week a mini writeup is needed, to show where the effort went. So a list of deliverables accomplished, what was tried and what problems were encountered if for some reason nothing could get completed, etc, possibly with time spent on each.
Requiring status reports is certainly a good start, and I wouldn’t be surprised if it is enough for most people. As long as the focus is on what they did not what they intended to do.
onitony said
I’m not sure if I agree with that last point for weekly status writeups. It seems like it’s putting more effort into talking about getting work done, rather than just getting anything done. Can’t we just count Version Control commit messages as those writeups?
Alex said
It is talking about getting work done, rather than getting work done. That doesn’t mean there is no value to it. I believe that the key advantage of writing mini write-ups is that it forces you to examine what you’ve done and how much you’ve achieved, because you need do that examination as part of your writing process.
For a conscientious person, a realisation that they’re not working hard enough can then help ameliorate procrastination. Motivated people may become inspired to internalise this do-evalute feedback loop and become much more effective as a result.
So, while doing a reflective write-up may not contribute to your project directly, it can help you work better and faster on the next project. In my opinion that’s a worthwhile trade-off and I gladly exchange short-term productivity for long-term productivity in my job and personal life.
Ian Lienert said
The nice thing about university assignments is that you have a good sense of where you are standing at all times. In a project course of this calibre, what you are working towards in the long-term (i.e. your goal for the term or even just the week) is your only point of reference. If you happen to come across something major that you’ve overlooked in the short-term which requires immediate attention in order to further progress in the long-term, then you’ve branched out and now have two things to worry about!
I think a potential solution to this problem is to allow students to have loose goals at the beginning of term while tracking their ability to meet them and enforcing tighter ones further in, all the while maintaining a strict if-you-haven’t-spent-8-hours-on-this-we’ll-dock-you caveat. As the term progresses and students become more accustomed to the environment, they should be able to assess their abilities more accurately.
The hard part is getting students to keep track of the number of hours they put in. But, that’s exactly where it lifts the burden on the student side of the coin. If I have only contributed four hours of work in a week (I’ll come clean, I’m speaking from personal experience — cursed operating systems assignments!), it shows. That can make you feel just as guilty as leaving your assignment to the last minute or taking a three hour lunch break at work. But, if you maintain a weekly hour count, it actually gives you a reference point and some cold, indifferent math to quantify your short-term progress.
This is something I’ve picked up within the last few weeks, but it should have been obvious from when I jump-started my professional career in the dish pit at a local golf course. Time is money after all!
Tara said
I have also been tracking my hours, mostly to make sure that I don’t slack off. I found that it feels like I’m putting in way more hours than I actually am. For MarkUs, we have two week sprints set up with milestones in DrProject so that we say that we’re going to try to finish ticket #x in that two-week period.
What I’ve also been doing is after Friday’s status update, I make a new .txt file with the contents of the week’s “Next Steps” under a “To do” heading and I slowly move those things over to “Status” after I’ve completed them. This is something that I’ve found particularly useful both on school terms and on work terms – I always make a list of what I’m going to try to accomplish each day at the beginning of the week. I find that having a list of what I should accomplish helps me to actually get stuff done and otherwise I’m much more likely to get distracted.
onitony said
One item of interest that I find (it might be applicable more to the Basie team than others), is that the ReviewBoard often seems blocking.
Once some code goes up, one can either wait for someone to review it (typically just the few of the senior members that actually take it upon themselves to look at all/most review requests), or (under the current svn/diff/patch system) face a complicated version control mess of having overlapping and uncommited changesets. This is the point where that Combinatorial Analysis assignment starts to become less painful.
Another related point would be that because of the way Django isolates apps from each other, I feel that team members are similarly isolated. Sure, we are all working on “Basie”, but I don’t really have an idea of what’s going on with… lets say the Milestones UI, outside of a screenshot on the blog and that somebody is leaving comments on review requests. If I suddenly decide that jQuery is the best thing ever, and get an itch to hack some JavaScript… there’s this barrier (at least psychological) that I’d be committing effort to another sub-team at the expense of my own.
zuzelvp said
Once some code goes up: the other option is to review someone else code and get yours reviewed as payment. That option has the extra bonus of allowing you to not be so isolated if you peek someone that is working in another app.
Blake Winton said
You know, while you’re waiting, you could review someone else’s patch. I’m sure that would count as “work”, and if everyone did it, then you could all move much faster.
And it would let you see what the other team members are working on. 🙂
Later,
Blake.
fgarces said
Deadlines for this course are going to be very hard to set. The projects will change a lot from term to term and the problems that will be encountered will most probably be different from the ones that have already been solved. Tracking the current status of each student is a good estimate of how much work can it be done, but i don’t think it will give you with enough information for how to set deadlines for the next term. This will sound so snobbish but anyways, the MarkUs team have been doing a pretty good job on informing each member what the others are working on, in fact, my email is spammed with review requests each day from all the team (it is starting to get annoying, but what can i say). The blog posts are done to give more information on the ideas that are being coded and to receive some other ideas on how to approach the problem.
Status punchlines are also a good measure of how things are going for each student, and how much work each student is actually doing. For me those punchlines were the ones that made me feel guilty in the past week that for working a lot less than i was supposed to, thus this week i am catching up =).
Tara said
Gmail’s skip inbox filter feature is very useful 😉 I just check my MarkUs label at particular times so that I don’t feel inundated with review requests.
More on the evaluation process « UCOSP said
[…] students when thinking about creating their evaluation scheme. It is also related to Greg’s Rational Response post Possibly related posts: (automatically generated)10 key steps for developing a distributor […]
Evan Bowling said
I think it is a gross oversimplification to assume that there are only two options: micromanaging each individual student to death, and leaving them entirely to their own devices.
This completely ignores the fact that while we have managers for each team, we are a collective group of students. Right now there is nothing mentioned about students contacting each other outside of their teams to discuss problems and success. I think this makes it hard for students not to feel alone if they are having problems with communication in their group. The rest of the class should be a resource to everyone, not just their teammates.
While I agree that strict point assignment deters from the freedom that this class can offer, I think we could easily have agreed on a 5% grade on communication between all groups. This could be made by students keeping in touch with their peers from their respective schools, or by being active on the forums. It could also be useful to have summations of our experience for things that work and things that don’t in a project like this. While some posts have already been made that are related to this, the uniqueness of this environment warrants a re-examination.
pgornicz said
Interesting indeed.
I find myself performing a mixture of everything above, that is, some courses I
will leave to the last minutes while others I keenly approach earlier. This has
been working rather well for me over the years and I’m not likely going to
change.
The question is then what courses do I do early and what ones do I leave behind
to the last minute? This is mostly driven by interest. For example, would I
rather poke around C code in my operating systems assignment or draw a silly
box-plot for Stat? For me the obvious answer is poking around the C code.
As a side effect to my assignment habit, point (b) applies to me in an
interesting way. I tend to be the keener in courses that have inevitable
ambiguities and I tend to be the procrastinator in courses where the assignment
has barely changed over the past 3 offerings for the course.
As far as these projects go, I think the freedom works well. While I feel the
pain of having 3 other heavy courses I still find the project I’m working as
interesting and hence I can still find time to work on it. I think I real
strong point to make is that _we_ decided to join these project so _we_ likely
find them interesting and hence (hopefully) _we_ will put in the required work
towards the projects.
Well that’s my two cents,
Patrik
luispedro said
“”” the later she left it, the more stable the assignment spec would be, because the keeners that started early would have pestered the prof into correcting the inevitable omissions, ambiguities, and what-not. “””
I have found this to be true in more than 50% of the courses I took as a student. The problem is worse for the better students because they can pull it off at the last minute, so they get little benefit from starting early and a great deal of benefit from learning that question 3b as stated is actually unsolvable instead of spending 5 hours trying to prove a theorem that is false (or something to that effect).
Mike Dimmick said
This isn’t just a problem for making sure that students deliver on time – what you’re describing was called the ‘stealth bomber’ personality type in one software engineering text (was it Code Complete?) and applies just as much in industry. The ‘stealth bomber’ type is not seen or heard of for the entire project until returning on the due date reporting either ‘missed’ or ‘target destroyed’.
Managers in industry also need to keep track of what their employees are doing to know whether software is going to come in on time and on budget. If they decide to move into industry, your students will have to manage their time in this way as well. It’s a discussion, not an imposition, but being able to quote the cost and delivery date of a software project with reasonable accuracy is a necessary skill: the project will be assessed in terms of return on investment and whether it allows cost savings elsewhere, and therefore be worthwhile doing, depends on how much it will cost and when the users can start using it. Or, for a commercial software house developing speculatively, how much finance needs to be raised.
Michael Hamilton said
Something a professor tried once with a senior level game course I was taking worked well. That was to offer an assortment of assignments which built off of each-other.
You could equate this to project milestones, where at the end of each milestone you have a hand in and evaluation.
Theory AND Practice « UCOSP said
[…] our blog, he came across my post “A Rational Response to an Irrational Environment“. It starts by explaining that it’s rational for students to leave assignments to the […]
Maxime Caron said
Like greg said the main problem is the find a way to keep people motivated to a point that they feel the project is a priority but without having strict deadline.
To be honest i find that having to do status report each week prevent us from slacking or at least make us aware we are slacking.
An other factor that affect motivation is to know that someone is waiting for the result of your work. This is why if we work as a team and some one need you to finish your part so i can begin to work over it, you feel that if it take too much time the whole team is slowed down by you so you get more pressure.
A good way to make this happen is to make people review each other code or to write “unit test” for each other code.
An other good way for this to happen is if we all work on strongly related component that have to talk to each other. Because this way everybody need to be aware of the latest interfaces for all object so they can be put together and work as expected.
It also help to have a clear team goal and know what others expect from you.
But as i understand it, it’s all come to having feedback on our work and to really work as a team. If it where to be a individual project you would surely need a mentor that would give you continuous feedback on what he think about your progress.
Filips said
I would suggest doing something like SCRUM demonstration – a regular, frequent (once per two weeks) demonstration of what you have accomplished, followed by discussion what needs to be done next in what order of priorities.
No judges, no grades.
Make sure, though, students actually show (hence demonstration) the things they have done – that way it gets too hard to pull off “i was preparing the framework for general architecture” BS.
Uneasiness about having to come to demo empty handed should be enough to motivate most of the teams, imo.
Mark said
The project plan has to be part of the exercise. Don’t grade them based on achieving the plan, though; grade them on how well they react to new information that they learn while executing the plan.