Web Site: http://skhanyz.myopenid.com/
Bio: Dr. Shahzad Khan studied computational linguistics at Cambridge University (UK) and has extensive experience in AI, machine learning, computational linguistics, web technologies and analytics. Shahzad is an accomplished educator, having taught for 3 years at Pakistan's National University of Sciences and Technology. Shahzad's industry experience has focused on computational modeling approaches to understanding human learning, language, and behavior, which he continues with his work on creating automated assessment. Shahzad is also interested in research to automate and streamline game content creation via AI based scalable programmatic methods. Shahzad welcomes opportunities to collaborate on assessment in serious games or AI based method for content creation. He is also deeply interest in sentiment analysis, user profiling techniques for online social media, and enterprise search technologies.
Posts by Shahzad:
I’ve spent seven years developing gnowit.com. It arose out of my PhD thesis, and employs the codebase and architecture I put together over the years.
Gnowit.com contains some very novel innovations around sentiment classification, auto-categorization of information, core-topic extraction and ‘noise-cancelling’ (let’s leave it at that for now).
As we’ve come out of stealth mode two weeks ago, I’ve found that it has become important to share details of these with potential partners, who may actually be in the same field, and would find them valuable.
The way to approach this is to either get an NDA (which is ineffective when the partner already has related IP in the same area) or to get patents. Thus, I am forced to take the second route.
There are some interesting wrinkles when it comes to software patents (and patents in general). They have to use a certain language, and for software in particular, you have to take pains to emphasize that it is not an abstract operation that is being carried out, but one where the ‘signals’ are being transformed. You actually need to write the patent as if you’re taking a lump of rubber and producing something from it!
There is an entire art-form related to the claims, where you try and ‘capture’ as wide a claim as possible, and yet, keep the claims attached to the concrete processing (so they do not get dismissed for being too vague or abstract).
It’s like writing a research paper, but with one hand tied behind your back, and without the need to prove anything (except that you can actually build something useful from this !).
I refactored my codebase to use the new Apache HttpClient 4 component (from the org.apache.http.package).
This broke Solrj, which is dependent on the Apache HttpClient 3.x components. Th e resulting error was the cryptic
The fix was to reintroduce the commons-httpclient-3.1jar file, and explicitly pass in the older httpclient when creating the connection to Solrj.
HttpClient httpClient = new HttpClient();
server = new CommonsHttpSolrServer(url, httpClient);
One of the watershed events that marks a successful R&D project is the handover to engineering.
This step generally involves a fair amount of integration effort, and occasionally requires that the code be rewritten to employ databases, web services or conform to data contracts with other modules. Sometimes, this also requires that the logic be rewritten in a different programming languages, perhaps a translation from Python to C# or from Perl to Java.
I’ve noticed in most such projects, even after the most faithful reimplementation of the code, the output only matches to three decimal places. Beyond that, the results invariably vary (pun intended).
For most decision support application (which dominate the artificial intelligence field), this is close enough for all intents and purposes. However, for applications from the field of finance, certain life-sciences and space technologies, this would be considered a minor crisis!
Whenever I feel a bit intimidated by the sheer scope of the work I’ve set out for myself, I pause and take a look at the inauspicious beginnings of the great IT giants of today.
Some of them are available here at The Telegraph
I especially love the page of Google, with the big exclamation mark. The exclamation mark (!) tells me that they wanted to be a Yahoo! me-too clone. Lucky for us all, their vision become much larger at some point. Otherwise we’d still be navigating subject directories!
You can find many more examples of these in the waybackmachine (for all you Rocky and Bullwinkle fans!).
My my, fortunes have changed significantly, haven’t they?
One of these, Twitter, is only 4 years old. Facebook is less than a decade old.
What will we see in 10 years from now?
My mother gave me the cure for procrastination when I was 6, and I could not start my homework. She encouraged me to ‘just start it’ (her exact words were, ‘just sit down’, I believe).
Today, I came across proof from the field of psychology that she was right!
Let’s go back a few years to the late 60s.
A prominent Russian psychologist, Bluma Zeigarnik , noticed an odd thing while sitting in a restaurant in Vienna. The waiters seemed only to remember orders which were in the process of being served. When completed, the orders evaporated from their memory.
Those who dable in cognitive sciences (like me for example!) refer to this phenomenon as ‘closure’. Closure effectively flushes your short-term memory. This is why early ATMs used to cause people to ‘forget their cash cards’; once people got their money, they walked away (without their card) as their ‘task’ (i.e. get the cash) is now complete. It wasn’t the people’s fault (they beat themselves up for forgetting their card!), but the designer’s fault for not being aware of this closure phenomenon. This is why nowadays, the ATM gives you the card, and refuses to give you the cash until you have taken the offered card.
So… the answer to procrastination is.. (wait for it !) …being in a state without closure. The easiest way to get there is to start something, as by definition, until it is complete you are not in a state of closure!
Zeigarnik proved this in the lab. She asked participants to do twenty or so simple little tasks in the lab, like solving puzzles and stringing beads (Zeigarnik, 1927).All standard stuff.. EXCEPT that some of the time they were interrupted half way through the task.
Afterwards she asked them which activities they remembered doing. People were about twice as likely to remember the tasks during which they’d been interrupted than those they completed.
When people manage to start something they’re more inclined to finish it. Procrastination bites worst when we’re faced with a large task that we’re trying to avoid starting. It might be because we don’t know how to start or even where to start. Figure out one piece that is mandatory for the final solution, and start chipping away, and before you know it, it’s all done.
Zeigarnik’s gift to us is the lesson that one weapon for beating procrastination is starting somewhere…anywhere. Carpe Diem. Get back to work now!
Designing software that is meant to be used requires that you put the user experience front-and-center when coming up with the design.
However, when you have functionality that you need to add to your system you have two ways to do it,
- Quick and messy – you are sure that it will make further changes harder in the future. This involves actions like hardcoding parameters or bringing in libraries that you don’t completely understand.
- The other results in a cleaner design, but will take longer to put in place.
Ward Cunningham coined a wonderful metaphor (Technical Debt) to help us think about this problem.
In this metaphor, doing things the quick and dirty way sets us up with a technical debt, which is similar to a financial debt. Like a financial debt, the technical debt incurs interest payments, which come in the form of the extra effort that we have to do in future development because of the quick and dirty design choice. We can choose to continue paying the interest, or we can pay down the principal by refactoring the quick and dirty design into the better design. Although it costs to pay down the principal, we gain by reduced interest payments in the future.
The metaphor also explains why it may be sensible to do the quick and dirty approach. Just as a business incurs some debt to take advantage of a market opportunity developers may incur technical debt to hit an important deadline. The all too common problem is that development organizations let their debt get out of control and spend most of their future development effort paying crippling interest payments.
I’ve made some choices during my career where hard deadlines, or the limited maintenance nature of the project meant that the effort for very clean code and architecture was not justified. However, one practice that I would advocate is to keep a copy of Bugzilla around where you can log all the ‘todos’ required to refactor, clean-up and enhance robustness in your project.
When you have debt, you have to keep track of it so that you can pay it off. Any other alternative is reckless and irresponsible. The metaphor hold equally well in the domain of software engineering as it does in the field of personal (or corporate) finance.
I wanted to have access to a machine that would fit in my backpack. Laptops were out, as I don’t want to lug around something heavy, and the iPad was a bit too underpowered for my needs and was lacking the keyboard. I was going to run Ubuntu Netbook remix on it anyways, so the Apple OS was not a great draw for me.
I decided on the Gateway LT25, which is essentially similar to an Acer Aspire One.
It comes with 250GB (more than enough space) and a celeron-like (but power-conserving) Intel Atom processor, and has a 6 cell battery, which means that it would last me most of a day’s work (even if I did not have access to power, or was moving around).
The only thing I did not like about it was the limited RAM. 1GB did not cut it for me. It was also a big sluggish due to the paging taking place when I ran my DB, webserver and firefox.
So I decided to do something about it, and ordered a 2 GB RAM stick.
The RAM I ordered was the Mushkin 2GB PC2-6400 DDR2 SODIMM for Notebooks. However, any PC2-6400 memory should work with the Atom N450 processor. More than 2GB probably will not be useful, and in any case, the CPU will be the bottleneck at that point.
Upgrading the Memory
To upgrade the memory, you have to open the netbook (essentially voiding the warranty). It’s a lot simpler than it appears when you start it. I’m writing this point up, with a few picture, to make it easier for the next person attempting this.
Please place a pillowcase or some other cloth under the laptop when carrying out the operation, otherwise you will scratch your laptop’s exterior casing. You may also want to either use a static-protection wrist-strap, or touch the metal on your sink to protect your system’s sensitive electronic components from electrostatic discharge damage.
Step 1: Remove the battery.
Flip the netbook over, and remove the battery.
It is held by two clips on the back. Move them towards the outside of the case, and the battery should be easy to pop out.
Step 2: Remove the keyboard
Turn the netbook over again (the right way) and open the lid so that you can see the keyboard. Using a credit card, push the little tabs in that are holding the keyboard in place. Don’t use a screwdriver as you may end up scratching your case (or damaging something or another). I’ve marked the location of the clip (tabs) in red circles in the picture below:
Step 3: Release the screws holding the back plate in place
Thre are four screws holding the back plate in place. You’ll need a jeweller’s screwdriver to remove these. This is smaller than your everyday screwdriver, and is used for things like jewelery and eyeglass frames.
Step 4: (Optional) Remove the keyboard data cable
I removed the keyboard data cable, which is probably something that you don’t need to do. You pull the little white plastic clip slightly out and gently slide out the data cable. This would allow you to remove the keyboard entirely.
Step 5: Remove back plate
Push a pen (or the screwdriver) through the hole to push out the back plate. The back plate should pop out, and you can then flip the netbook over and remove the back plate entirely.
Step 6:Remove the Existing RAM
The RAM is in the slot on the top left. Gently pull the tabs away from the RAM and the chip should pop out. Make sure that you have drained any electrostatic discharge before attempting this step (touch something grounded, like your bathroom sink’s tap’s metal part). You may want to keep track of the notch at the bottom of the chip, to ensure that you replace the new RAM in the same place (using the same orientation).
Step 7: Insert the new RAM
Gently slide in the new RAM unit, and pop it into place. The tabs should securely hold the RAM in place.
That’s it, you’re done. Put everything back in the reverse order, and boot up the machine.
Go into the bios (by typing F2 at startup) and the machine should have automatically picked up that it has 2 GB now.
As I’m getting close to the launch of my product, I’ve been struggling with the question of how much to charge. There are a number of models, with differentiated version-based subscription being the overall favorite for online services. However, other models do exist as well; you can charge based on percentage of spend, have a consultancy based component (which is usually more than you would charge for the product).
However, you do not want to charge too much (which would mean ‘no-deal’) and not too little (which would imply that you are leaving money on the table and which ultimately can threaten the viability of your concern).
A happy median does exist though. If you can figure out the value that your product provides to the customer, you can charge accordingly. Even if your product is not the most sophisticated, it is worth something (as long as it does not provide negative value).
Just start charging for what you have today, and work on improving the value you provide to to customer. There is a nice 45 degree slope that you can climb, whereas as long as you provide increasing value, you can charge a higher price.
Information technology solutions are very powerful. If properly planned out (with the goals of the customer paramount), they have the potential to save time, effort, expenses and grief.
I get a high when I see my customers using my product, especially as I know what they used to go through before the solution was in place. These moments of satisfaction are what keeps me happily chugging away through the my grueling, crazy-packed schedule.
A bit of extra effort and due diligence will ensure that my customer (and therefore I) am happier and more satisfied tomorrow. It’s well worth the effort and time put in.
.. and I like to be a happy person
I absolutely love the definition of a ‘wicked problem’ http://en.wikipedia.org/wiki/Wicked_problem
It describes the majority of research problems that I work on. There is an objective that appears just out of reach, a real client-need, and limited resources. Formally, it is defined as:
“Wicked problem” is a phrase originally used in social planning to describe a problem that is difficult or impossible to solve because of incomplete, contradictory, and changing requirements that are often difficult to recognize. Moreover, because of complex interdependencies, the effort to solve one aspect of a wicked problem may reveal or create other problems.
Interestingly enough, much of the fog surrounding these problems can be dispelled by putting some effort into identifying the goals of the users, and creating task analysis models around the plans that are likely to be followed. Simple, non-development intensive efforts such as storyboarding and Wizard of Oz experiments.
Considering that most user experience is based on contextual issues, and their perceptions of their interactions with the GUI, it would be irresponsible to not put some extra effort into understanding how to support the user with your software (rather than building yet another expensive software project that is destined to not be used due to user rejection, and the subsequent price and time overruns).
The development of a user interface is an issue that needs a lot of attention. For example, Sutton and Sprague conducted a survey in IBM in 1978 , which revealed that 50% of the designer’s time was spent on designing the user interface of a system. The same result was also confirmed by a more recent survey by Myers and Rosson  in 1990s.
So before you spend that six months pouring your sweat, blood and tears into an effort that will not bear fruit, why not spend a week to work with the users to make sure that you’re building the right thing (and set up the validation criteria), and that you are building it right (and set up the verification criterian)?
Trust me, you’ll thank me if you do this. It saves grief, and keeps your professional reputation intact.