I've been thinking a lot about dealing with information overload recently. WIth the ever-increasing hype for sites like
Twitter and
FriendFeed, neither of which I use, and a steady stream of Facebook invitations,
LinkedIn requests and invites to a continuously growing set of social networking sites and bizarre (to me) services that all add some form of social networking services, I'm more and more fatigued.
I hardly keep up with even my feeds and my e-mail, never mind my IM accounts - messages from people keep building up for days before I answer them.
And the thing is, I'm not actually connecting to large numbers of people. I'm fairly anti-social and notoriously bad at keeping in touch with people I've worked with in the past etc. (it's nothing personal folks - I'm happy to hear from people, I'm just rarely initiating contact myself because I'm always deep into focusing about something or other).
I need better tools to manage it.
Friendfeed aims to combine lots of streams of data into one huge river. The problem is that my challenge isn't to get a single view - it's to effectively manage whats there:
What should I read? What should I ignore? How do I find past information? What do I need to respond to? What should I keep and what should I discard completely?
In other words I need a personal search engine and my own personal recommendation or classification engine.
If I had time I'd build one. Building a small scale search engine for this data is trivial - there are tons of packages like
Sphinx that are "good enough", so the problem is mostly ranking (and that is by no means trivial for data like this, that for many elements can be as short as a single line, and that doesn't have enough internal linking for something like pagerank to work). Building a classifier is also fairly trivial - but the problem is similar: Small snippets of data + training. Again, that's not a trivial problem.
But even a badly flawed one would be vastly superior to nothing. Getting something like that to "product quality" is hard. Getting something that is more than marginally usable for personal use should be doable, and mostly limited by my continuous lack of time. So I guess I'll paraphrase
Abbie Hoffman:
-Steal this idea
Pretty please?
I want someone to build a service that'll do this for me, damn it. Even if I have to install a local client to capture some of the data. Yes, it will have a ton of privacy and security implications, and no, I wouldn't trust just anyone with it.
Let's summarize:
- Search my e-mail, feeds, pages I've liked (via StumbleUpon, DZone or others), my IM's, the streams from any social networking sites I'm on (any not covered by my RSS feeds), comments I've written on other blogs, and generally my whole "online footprint".
- Give me a filtered view that clearly shows me what I a) need to respond to, b) need to read, c) is likely to find interesting, it's source, and relevant/related items (replies/comments from other people, other blogs referring to the same thing, etc.)
- Oh, and if it could recommend new information sources and show me which ones I really shouldn't bother with(of the ones I'm already following) based on my pattern of usage, that'd be nice too.
Yeah, it's a tall order.