Wednesday, October 12, 2005

Great websearch techniques courtesy of Lifehacker

I'm certainly not posting much but I just had to publicize a link that I think is fantastic.

Lifehacker is a great blog devoted to computer and technology tips that can really can make a difference in how we work and play. This post that I came across belatedly is a gem.

Their rather apt mantra is:
"Computers make us more productive. Yeah, right. Lifehacker recommends the downloads, web sites and shortcuts that actually save time. Don't live to geek; geek to live."
I believe these folks are quite true to their word. I personally believe that the majority of modern high tech gadgetry and software is fairly useless. People develop new devices and software algorithms and post-spark-of-ingenuity try to craft a use for them. In general, this kind of reverse engineering adds little to the great cosmic noise.

No doubt occasionally someone comes along with a fresh look at previously conceived technology and with true inspiration finds a brilliant use for it. However, I think this is fairly rare. To me a real improvement occurs when someone comes across a novel, ingenious real-time solution to a problem that they have confronted in their own life for some period of time. But I digress.

The outstanding post I linked above discusses some extremely useful ways of performing searches of the "invisible" or "deep" web. The basic idea is that many databases are either inadvertently (or by design) impermeable to search engines. For example, when the Google "webcrawler" bot searches through web pages, it doesn't go into information and data that is located inside many large databases. Such databases are often found at university, corporate and government websites for example.

Therefore, this vast repository of information (estimated to be many times greater than the "visible" net) will not lead to hits when used by popular search engines.

The techniques described in the above linked article are really a goldmine as they can be used to access much of this information using conventional search engines. I'm sure they're old hat to librarians and other experts on information technology but I certainly never come across them before.

I hope they become more generally known.


Post a Comment

<< Home