Archive

Archive for December, 2009

Programmer Pitfalls I

December 29th, 2009 No comments

A quick list of the classic errors I’ve seen programmers make (and which I’ve made as well at some point in time!).

Reinvent the wheel
Most programming assignments are not there to provide you with a sandbox where you can play. While you’re putting together your super-optimized implementation of the linked-list or quick-sort algorithm, someone is paying for your time and waiting on you to reap the benefits of their investment. If you must insist on reinventing the wheel, consider an academic career, do it in your spare time or take part in the greatest adventure ever, become part of a new startup!

Reuse blindly
You cannot assume that an API available in the wild will solve your problem completely. Unless it is a mature component for a well understood problem (e.g. the Apache Web Server), you’ll probably exert a lot of effort to shape it to fit your need. Even if you’ve managed to do what you set out to, you’ll have a lot of functionality that is in your code which shouldn’t really be there. This excess functionality will not only slow down your system, it will make it difficult to transition other developers in (without a lot of hand-holding on your part) and can represent a serious security risk (in some instances). I should know, I just finished a project where we started by trying to leverage the Apache Nutch web crawler, and in the end, we made a decision to roll our own simpler crawler rather than modify core components of Nutch to server our needs. Prior to that, I was writing a scraper, and our first attempt was to piggy-back on a headless (i.e. without GUI) instance of the Mozilla XUL code-base. Even though it was theoretically possible to scrape through AJAX driven menus with that approach, in the end, we decided to employ a simpler regular expression driven logic via HTML.

Too much alphabet soup? The lesson here is simple; there is nothing wrong with trying to reuse, but develop a system to quickly map out the capabilities of the system, compare them against your requirements and have an idea of how much effort a simpler ‘re-implement the wheel’ strategy would involve.

If this flies in the face of the first rule-of-thumb, consider that you should not reinvent the wheel for mature, well understood component. However, when you have a core innovative piece in your system, and there is nothing that matches it immediately out there, you then have license to build a working component that does exactly what is necessary (and no more).

Estimate based on a deadline or a desired date
This is a classic recipe for death-march project. Your estimates should always be based on a completely list of steps required to get from the beginning of the project to the end of it, aggregated from the cost (in time and money) associated with each sub-task. It is basically dishonest to promise to hit deadlines that you do not have a plan for.

I recently lost a project due to this policy of mine. The clients had asked me for some documentation, and instead of agreeing blindly, I decided to map out the full effort required. The final project become a multi-month activity, and required a budget of 60K+. When the client saw the bill, they balked.

I am actually quite happy that we did not go forward with this contract, as the alternatives were not were appealing. Consider them:

Scenario A If I had billed by the hour (which the clients were willing to accept BTW), the clients would have received their project at the end of month X, but would have probably been very unhappy (as it would have cost them more than they expected).

Scenario B If I had given a fixed quote, I’d have been working for many weeks beyond what was reasonably expected for this project.

This way, the contract was not finalized, but neither I, nor the client, was unhappy :-)

Some people would argue that it is rational to have billed by the hour in this case (as in scenario A). That may be the case in a Machiavellian sort of ends-justify-the-means type of world where profits are supreme. After all, you’re shifting all the risk to the clients!

However, I am not a faceless multinational, and am not beholden to callous stockholders to keep my job in my firm. I have to live with myself, and to be happy with the person in the mirror, I’d rather not benefit at someone else’s expense.

Organizational Politics (Enron email dataset)

December 23rd, 2009 No comments

When I look back at my career, with the exception of two projects, I’ve almost exclusively worked for startups. Today, while examining the Enron email dataset and validating that my tools were working correctly, I came across a gem of an email that just brought everything into perspective, and I thanked my lucky stars.

How is this for a prime example of an insecure, pushy, crazies bit of communication?

I thought Dilbert was a bit of imaginative caricaturing, but it does appear that the spirit of cubicle-nation lives on in firms like Enron.

Jennifer, I’m not so territorial that fire hydrants and trees are on my
priority list, but I thought that I was doing my job OK on this. If I read
Crowder’s FWD to Kim in a certain “tone”, it appears to me that they feel I
am overstepping. I’m sure there is a better way for me to coordinate and
facilitate these meetings, and do so in such a “tone” that Jim doesn’t react
the way I sense he did, so I’ll be coming to you to discuss, OK?

On a lighter note (I hope), I hope your NY trip went well, and I’ll see you
tomorrow!

Thank you,

Jeff

For reference, this email was in maildir/arnold-j/all_documents/3.

Tags: