Information Overload

George Kingsley Zipf stated the Principle of Least Effort: Each individual will adopt a course of action that will involve the expenditure of the least average of his work. Hence, when faced with too much information, we tend to acquire the easiest to access, rather than the best information for the task at hand. It is like looking for one’s lost keys wherever the light is good, not where they may be more likely to be found.

Therefore, too much information can easily be a bad thing.

Zipf is known for Zipf’s Law, which describes the frequency of words in text as following a power law. The frequency of the n-th ranked word in the English language is found to be proportional to , where ‘a’ is the power coefficient. ‘a’ is known to be close to 1. In the same vein, if ‘n’ is used to denote increasingly relevant sources of information, the frequency of good information drops off quickly. Hence, we need to be careful to spend more time on choosing good sources of information rather than acquiring any and all information.

In fact, collecting the easy-to-get information means that we may in fact collect too much of it, and thus end up spending more time and money on it, than if we were more selective.

This almost suggests a “toe-in-the-water” search model in two parts:

(1) Get only as much information as is needed to evaluate the quality of the source and no more, and then drop it if found wanting.

(2) Set a higher than average threshold for quality, so as to avoid waste.

I wrote about the emotional cost of information overload in a previous blog. That cost is distinct from the one being discussed here, which is one related to bad decision-making, not self-negativity. But the relevant point still is that bad information has a three-fold cost in terms of (a) emotion, (b) time, and (c) money.

Bad quality information breeds even more bad information. This is because too often information is simply served up in another form. The student term paper has become a special case of this form of abuse of information. There is an interesting negative dynamic to easy access to information. In the past, when researching a term paper, much effort was involved in gathering information. So before doing so, we thought a lot about what exactly we wanted to achieve, and then only did we begin information gathering. This process usually resulted in the “first access” of information being of reasonably high quality. Nowadays, because there is easy access to way too much information, the process of thought before retrieval is often skipped, resulting in low quality “first access”. In general, information gathering tends to follow a stopping rule which is just based on stopping some time after first access.

Mostly, once we have gathered a certain a certain quantity of information, we tend to stop. Hence, the quality of our data will almost always be a function of the quality level at initial access.

So, what can we do to avoid these pitfalls of information overload? There are many things I like to try.

(1) Avoid using the browser unless you absolutely have to. Even if it is work-related, determine if it can wait. The days I collect my web-based activity (including email) into one chunk of time are often the days I get quality work done.

(2) Do as much theorizing as possible before reaching out for data. I often find that I decide against pursuing an idea, and for good reason – after thinking a lot, it simply is not worth it. If I had instead rushed out and collected data, then it would be time spent badly, and, the much bigger problem is that once I have collected data, then I am much more committed to the project (sunk cost fallacy) and throw even more good time at it.

(3) If I am not sure about some information, I try not to waste time reading further to examine it. I simply save it away on my hard drive with a brief descriptor. (I have an excellent mnemonic system on my personal server for saving files, giving descriptive file names, syncing it with my other machines, and accessing the data from many places. This is in addition to the usual software for indexing my drive on my Mac and Linux machines). This way, if I need it, I can get it, but for now, I have saved time and effort in dealing with information “on the cusp”. I was reading the book on Fischer Black (by Perry Mehrling), and was struck by how close my system is to Black’s. He was clearly already grappling with the problem years ago, and had a detailed collection of manila folders where he stashed all his information and ideas. Were he alive today, I am sure he would have figured out something eons more sophisticated than what I have.

(4) Jeff Hawkins had this idea of the brain in his book “On Intelligence”. He likens the brain not to a fast CPU but to a vast memory bank. The brain solves problems and makes judgments exceedingly fast because all it does is fast retrieval, through pattern recognition of stored life scenarios. Every iota of learning is stored, to be dormant until recalled. This is similar to stashing information for probable future use.

(5) Pull at information, dont allow it to be pushed at you. Ignore Amazon’s recommendations. Given, that software is pretty smart, and often does very well in anticipating your latent needs, but then, it all comes at a cost.

(6) Never spend more time reading web sites in a day than you spend reading books or newspapers. You know why already.

Most of all, do not worry about being on top of all the information being thrown at you. If you want to live life like that in a reactionary fashion, then you are done for already.

Information is the food of the mind – eat well and do not overeat.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: