Posts Tagged ‘software engineering’

Promoting Code from R&D to Engineering: The 3 Decimal Place Barrier

August 23rd, 2011 No comments

One of the watershed events that marks a successful R&D project is the handover to engineering.

This step generally involves a fair amount of integration effort, and occasionally requires that the code be rewritten to employ databases, web services or conform to data contracts with other modules. Sometimes, this also requires that the logic be rewritten in a different programming languages, perhaps a translation from Python to C# or from Perl to Java.

I’ve noticed in most such projects, even after the most faithful reimplementation of the code, the output only matches to three decimal places. Beyond that, the results invariably vary (pun intended).

For most decision support application (which dominate the artificial intelligence field), this is close enough for all intents and purposes. However, for applications from the field of finance, certain life-sciences and space technologies, this would be considered a minor crisis!

Technical Debt — Defined

February 7th, 2011 No comments

Designing software that is meant to be used requires that you put the user experience front-and-center when coming up with the design.

However, when  you have functionality that you need to add to your system you have two ways to do it,

  1. Quick and messy – you are sure that it will make further changes harder in the future. This involves actions like hardcoding parameters or bringing in libraries that you don’t completely understand.
  2. The other results in a cleaner design, but will take longer to put in place.

Ward Cunningham coined a wonderful metaphor (Technical Debt) to help us think about this problem.

In this metaphor, doing things the quick and dirty way sets us up with a technical debt, which is similar to a financial debt. Like a financial debt, the technical debt incurs interest payments, which come in the form of the extra effort that we have to do in future development because of the quick and dirty design choice. We can choose to continue paying the interest, or we can pay down the principal by refactoring the quick and dirty design into the better design. Although it costs to pay down the principal, we gain by reduced interest payments in the future.

The metaphor also explains why it may be sensible to do the quick and dirty approach. Just as a business incurs some debt to take advantage of a market opportunity developers may incur technical debt to hit an important deadline. The all too common problem is that development organizations let their debt get out of control and spend most of their future development effort paying crippling interest payments.

I’ve made some choices during my career where hard deadlines, or the limited maintenance nature of the project meant that the effort for very clean code and architecture was not justified. However, one practice that I would advocate is to keep a copy of Bugzilla around where you can log all the ‘todos’ required to refactor, clean-up and enhance robustness in your project.

When you have debt, you have to keep track of it so that you can pay it off. Any other alternative is reckless and irresponsible. The metaphor hold equally well in the domain of software engineering as it does in the field of personal (or corporate) finance.

Taming the ‘Wicked’ Problem of Creative Design

October 2nd, 2010 No comments

I absolutely love the definition of a ‘wicked problem’

It describes the majority of research problems that I work on. There is an objective that appears just out of reach, a real client-need, and limited resources. Formally, it is defined as:

“Wicked problem” is a phrase originally used in social planning to describe a problem that is difficult or impossible to solve because of incomplete, contradictory, and changing requirements that are often difficult to recognize. Moreover, because of complex interdependencies, the effort to solve one aspect of a wicked problem may reveal or create other problems.

Interestingly enough, much of the fog surrounding these problems can be dispelled by putting some effort into identifying the goals of the users, and creating task analysis models around the plans that are likely to be followed. Simple, non-development intensive efforts such as storyboarding and Wizard of Oz experiments.

Considering that most user experience is based on contextual issues, and their perceptions of their interactions with the GUI, it would be irresponsible to not put some extra effort into understanding how to support the user with your software (rather than building yet another expensive software project that is destined to not be used due to user rejection, and the subsequent price and time overruns).

The development of a user interface is an issue that needs a lot of attention. For example, Sutton and Sprague conducted a survey in IBM in 1978 [1], which revealed that 50% of the designer’s time was spent on designing the user interface of a system. The same result was also confirmed by a more recent survey by Myers and Rosson [2] in 1990s.

So before you spend that six months pouring your sweat, blood and tears into an effort that will not bear fruit, why not spend a week to work with the users to make sure that you’re building the right thing (and set up the validation criteria), and that you are building it right (and set up the verification criterian)?

Trust me, you’ll thank me if you do this. It saves grief, and keeps your professional reputation intact.

Read more…

Agile Development

March 11th, 2010 No comments

I’ll admit it up front. During my undergrad (which was many many years ago), I was taught the waterfall model. Indeed, there was some mention made of other paths, such as rapid application development (RAD) and joint application development (JAD), but there was always the implication that those models were used by individuals who smoked their vegetables rather than eating them.

There are so many mature components available nowadays that unless you’re developing air traffic control software or logic for the Mars mission, you’d be irresponsible to NOT go agile!

What I love about Agile is:

  • At the end of each sprint, you have production ready code. If you can manage that, work is a joy, as nothing stresses me out more than half-finished tasks!
  • You usually know what to do next, even if you can’t see the complete path to the end of the project. Dead reckoning worked for sailors who did not have the luxury of compasses and maps, and we have the luxury of not betting our lives on our priorities and decisions!
  • The solution takes shape almost by itself. It perfectly fits the thesis of Al-Khawazmi, who states as when you take a big problem and break it into smaller pieces, and you solve all the smaller pieces, the big problem is automatically solved! Ancient philosophies with modern utilities. This theory is the basis of the algorithm and also contemporary map-reduce

I am satisfied customer. Three sprints from beginning to end, and I have a product on-line. What better proof do I need?

– Shahzad