Tuesday, November 20, 2012

Update - Adding data from the cloud: What are your top priorities?

We're making rapid progress on Zekira in two areas: fixing the remaining bugs that our testers found in beta five, and adding more data sources.  In this note I'll update you on our status, and I also have some questions about which cloud information you'd like added to your index.

Let's start with the cloud data.  The current beta of Zekira indexes information stored on your computer, but we've always planned to reach out to the cloud to index the information you have there as well. We've started implementing that in the next Zekira beta.  Twitter is coming first, followed closely by gMail.  If all goes well, the next beta of Zekira will index your Tweets and gMail messages, and perhaps some other cloud information as well.

Here's a little peek at what Twitter integration looks like.  Tweets are included in your message index.  You can click to find addresses and hashtags the same as you would any other word. 


We'd like your help in prioritizing our cloud features and the way they work; there are three questions below.  We'd be very grateful if you could read through the questions and share your thoughts.  You can post a comment here, or send an e-mail to contactus (at) zekira (dot) com

1. Here's a list of web information sources we can add to Zekira.  Which ones are most important to you?
   Google calendar/contacts
   Google Docs
   IMAP e-mail
   Other web e-mail (please specify)
   Something else?

Some web services generate an enormous volume of information. Please help us think about what you'd want to index from those sources.  Do you want everything, or will that be overwhelming?  Specifically...

2. If you're interested in Facebook integration, which types of information do you most want added to your index?  Private messages?  Status updates?  Wall posts?  Comments?  Photos?  Chat sessions? Do you want to index only the things you send, or also things you receive from friends?

3. If you're interested in Twitter integration, which types of messages would you like indexed?  Direct messages?  Tweets that mention you?  All tweets you send?  All tweets from everyone you follow, or tweets only from specific accounts?

Thanks in advance for your feedback.

Status update.  We're working in parallel on two things: the web services integration I mentioned above, and fixing issues that people identified in the current beta.  We're no longer hearing many concerns about the speed of indexing; if you're testing Zekira and still have worries about that, let me know.  For now we're focusing on improving the speed at which Zekira loads (a big index can take a long time to start up) and fixing a variety of bugs and UI tweaks that our testers identified.  Probably the most common comment we've received is that people are confused by the Connections slider, so we're revising that.

We deeply appreciate your support; you're helping to make Zekira a better product.

Monday, September 24, 2012

Hidden in Plain Sight: Zekira and the Battle for Facebook

A long and complicated essay was posted last week by Aaron Greenspan, one of the people who claim that Mark Zuckerberg stole the idea for Facebook from him (link).  Most of the essay is focused on Greenspan's vast reservoir of dislike for Zuckerberg.  I didn't read the whole thing; it's about as entertaining as listening to me complain about the stuff Microsoft did to Apple in the 1990s.

But what did stand out to me was the start of the essay, where Greenspan reports that just this month he found copies of IM conversations he had with Zuckerberg when Facebook was conceived in 2004.  Greenspan writes:

"It turns out that my Other folder contained some of the most important legal evidence regarding the origins of Facebook, Inc. that I had been trying to find for years, but without much success. I had already scoured the folders I created for the Student Entrepreneurship Council and come up with a number of the AOL Instant Messenger conversations that I had with Mark, but not all of them, and not the most revealing ones. I had also checked my "Aaron" folder, which had another sub-folder for Instant Messenger conversations, as well as the large hierarchy of folders containing all of Think Computer Corporation's data, which had even more conversations, but none with Mark. The conversations I remembered having, but could not recall the exact details of, were nowhere to be found, at least until September 12, 2012."

The files stayed lost for eight years because they were buried in a flood of other documents Greenspan had saved.  Most people who save documents end up with this problem.  You can't know what you'll need in the future, so you save everything.  But the more information you save, the more difficult it is to find anything when you need it.  It sounds like Greenspan was unusually rigorous in the way he organized his files, but no amount of careful organization can help you find something if you don't remember where you put it.

This is one of the problems that a context manager should solve.  In addition to helping you recall the context around current events (meetings you're going to, people you know, etc), it should help you go back in time and recall your context around past issues.  I think that if Greenspan had been using Zekira, he could have found those files in under a minute.  And who knows, maybe the legal wars over Facebook would have turned out differently.

I'm not saying you'll end up in a legal fight over the origins of a leading web property, but you never know.  What I can promise is that the more successful you get, the more likely it is that you'll desperately need to find some files or messages you saved, and Zekira will be able to help you.

Announcing Zekira Beta Five

I'm happy to report that the fifth beta version of Zekira has now been completed.  If you sponsored us on Indiegogo at the $35 level or higher, you should have already received an e-mail with downloading instructions for the beta.  If you didn't receive your e-mail, please send us a message right away.

We took a lot of time on this beta because we wanted to thoroughly optimize the performance of Zekira while it indexes your information.  That was the #1 request from testers.  Our engineers did a huge amount of difficult low-level work on the innards of the app, running enormous in-house tests to identify bottlenecks, and carefully tuning the code.  Mike spent much of the summer as Mr. Tester so the engineers could focus on the hard work.  The whole process took longer than we wanted, but the result is a much snappier indexing process in our tests, and we look forward to hearing what the beta testers think.

We also simplified the UI, added support for more file types, and of course squished a lot of bugs.  A partial summary of the changes is below.

I've learned over the years not to try to predict a specific release date while waiting for beta feedback.  We originally hoped to ship in summer, but all I'm comfortable committing to at the moment is that we'll ship version 1.0 this year.  Thanks for your patience, and we'll update you as we progress.

Changes in the latest beta

Support for more document types.  We added support for Excel files, Outlook messages saved outside of a PST (.msg files), an additional form of web file (.mht format), and RSS feeds stored by your e-mail system.  The index also now lists every document Zekira finds, even ones that it doesn't know how to read.  Those documents are indexed by name, date, and folder location.

We eliminated the Context list.  Many of you told us that the Context list, displayed on the right side of the main window, was confusing and cluttered up the screen.  Since you can now search for any word or date by clicking on it, the list is redundant.  So we eliminated it. 

If you want to reduce the size of your index, or speed up the indexing process, you can specify a cutoff date for how far back in time you want information indexed.  Items older than that date will be ignored.

We made a huge number of bug fixes.  One highlight: Zekira now handles Outlook PST files more robustly.

These changes are in addition to the new features we added in beta four:

Everything's clickable.  When you're looking at a document, meeting record, or other item in Zekira, you can click on any word, name, or date in it to do an instant search for it.  This makes it much easier to follow the connections between items.

Narrow or broaden a search with a single click.  If you click on an item like a name, meeting, or document to see other things related to it, you'll sometimes find that you get either far too many matches, or too few.  You can use a new control, the Connections slider, to adjust your search.  Slide it to the left to show only the closest matches; slide it to the right to do a broader search.

Filter by folder.  We've added a new filter that lets you look for items that were saved in a particular folder or set of folders. 

We added a Query window.  This is a small window that you can leave open on your desktop to get info on your next meeting, and do quick queries on a word or name.

We simplified the setup process so you don't have to fuss with choosing folders to index or look for the location of your Outlook or AppleMail files.  

Add up all the changes, and this is a very different app from the one we previewed at the beginning of summer.  That means we need to go back and re-create the preview videos on the Zekira website.  It's a nice problem to have -- it means we're making progress.

Wednesday, August 8, 2012

Zekira Update: Can You Spare a PST?

We're continuing to make very good progress on Zekira, squashing bugs found by our beta testers and adding features you asked for.  We've now implemented the features I talked about in our last update, including click-anywhere searching and filtering by folder.  I think they make Zekira easier to use and more powerful. 

We're also making steady progress on increasing the speed at which we index your files.  Although the performance has improved a lot, it's not yet all the way to where we want it to be.  So this is our number one goal, and we're holding off on the next beta release until we meet it.  There are still a lot of things in indexing that we can tune, so I'm very confident that we'll hit our goal.

Meanwhile, we're learning some interesting things about file compatibility on personal computers.  Although there are many commercial and open source software projects that claim to be able to read various file types, in practice many of them are riddled with bugs.   Even something that you'd expect to be simple and well-defined, like reading a PDF file, can be a minefield of incompatibilities. 

Apparently many of the disk indexing products out there handle this problem by just skipping the files that they have trouble reading, and then not telling you about it.  This makes them look compatible, but you may not be able to find what you need, when you need it.  Unfortunately, there will always be cases of damaged files that can't be read and must be skipped, but in the Zekira project we're investing heavily in fixing as many incompatibilities as we can, and I think we'll have a very high level of compatibility as a result.

As part of this compatibility testing, we'd like to create a collection of Outlook .PST files that we can use in testing.  We have many of our own, but the more we can get, the better.  We know many people are not comfortable sharing .PST files because they contain personal information, and we respect that.  We've set up a procedure under which your file can be accessed by only one of us (Steven Glass), and he will make sure that the file never gets to anyone else.

If you have a .PST file or files that you're willing to share, please contact me at michael.mace@zekira.com .  I'll put you in touch with Steven so you can send the files to him directly.

Thanks for your support.

Thursday, June 21, 2012

Zekira Update: Successful Fundraising and New Features

Thanks again to all of you who contributed to our crowdfunding campaign.  We beat our goal, which is not an easy thing for a productivity app considering that crowdfunding is usually used for games and hardware accessories.  But most importantly, we found some really enthusiastic beta users who have already provided us with great feedback and helped us prioritize what we’ll improve first.

Features just added and available soon

In response to your requests, we’ve made a number of improvements.  We are running them though a test cycle now and will be posting Beta 4 with these changes in the next week or so.  We've made it much easier to quickly have Zekira do various sorts of sophisticated searches.  The changes include:

Folder-based filtering. We added a fifth filter to Zekira to help you when you remember that the thing you're looking for came from a specific place on your hard drive or a mail folder.

In addition to filtering the search by folder, you can now right-click on an item to search for everything else that was stored in the same folder as that item.  This is helpful when you suspect that something relevant is stored in the same place as an item you've found.

As in all the Zekira filters, you can combine this with other filters, such as date and time, keywords, etc.

Click anything to search. When you click on an item, we put information about the item and the item’s contents in the blue area at the top of the results window.  The main body of text scrolls so you can quickly see it all without opening the original.  Sometimes you'll see a name word that you'd like to use to start a new search, or add to the current one.

You can now do that immediately by clicking on the word.  Everything in the items displayed is clickable.  Exactly what happens will depend on what you click, and what other keys are held down when you click:

If you click on a word in the body of the item, Zekira initiates a new search for that word. 

If you shift click on a word, Zekira adds the word to the keyword filter with an OR. 

If you option click, we’ll add it with an AND.

Clicking on the metadata (any text outside the scrolling box) will also initiate new searches.  Shift clicking will add the search to the current collection of filters.

Click on... Action Action with shift
Title Initiates new search using word clicked on Adds word to keyword filter panel (joined by OR)
Author, Sender or Meeting Organizer Initiates new search for person Adds person to people filter
Date Initiates new search using date clicked Adds date clicked to date filter
Path Initiates new search for everything in the folder that contains the item Adds folder to the folder filter

Improved Connections slider.  The connections slider now has 11 settings.  The higher the setting, the more items Zekira will find.  The lower the setting, the fewer items Zekira will find. 
Easily Unapply a Filter.  You can now remove a filter from the current search by clicking its breadcrumb (the dark gray icon at the top of the window).  This makes it easier to try alternative scenarios.  The forward and back buttons let you quickly move back and forth between different choices.

Easily Unapply a Filter.  You can now remove a filter from the current search by clicking its breadcrumb (the dark gray icon at the top of the window).  This makes it easier to try alternative scenarios.  The forward and back buttons let you quickly move back and forth between different choices.

Features we're working on

Indexing performance.  Not surprisingly, many of you have some really large archives of information.  Our initial beta releases could not index it fast enough.  We found that we could handle about 40,000 items overnight but that’s just a drop in the bucket for some users.  We made some improvements in the Beta 2 and Beta 3 release, but these are not enough.  This is our number one priority and will remain so until we meet our performance goals.

PST challenges.  While we’ve been successful indexing the context of most PST files, a number of you have files that we’ve not been able to index.  Most of these are older PST files from older versions of outlook.  We found and fixed a couple of errors that stopped the indexing process, but there are still some files we can’t read.  We have more homework to do here as we try to figure out what’s going wrong.  In the meantime, as a work-around, you may try opening troublesome PST files in Outlook and copying their contents to a newly created PST.  We have yet to find a PST file that could not be fixed with this process. 

This problem is difficult to diagnose because we don't have any of these un-indexable files in our hands so that we can pick them apart.  If you have troublesome PST files that you're are willing to share, we’ll be glad to add them to our testing matrix and see if we can figure out how to index them.  We will take special processes to protect your personal info.  Send us a note at support@zekira.com to let us know if you can help.

European date formats.  In the spirit of getting the first product to market quickly, we took a few short-cuts around internationalization.  As such, our beta releases only support US date formats so far.  We’ve added support for European date formats to our “to do” list.

Indexing all files.  We got some feedback about files whose contents we do not yet read.  Currently we ignore those files, but several of you asked us to at least include the file in the index with its name and other metadata.  We have that working on a prototype now and will be releasing it in an upcoming beta.

Mac behavior.  We heard that you don’t like the “windows-style” menus in a window, when running Zekira on a Mac.  This was another one of those time to market trade-offs.  We’ve put it on our “to do” list.

Mac data types.  Our initial indexing support on the Mac includes Mac Mail the built-in calendar and the built-in contacts.  You’ve asked us to also support
•    Outlook on the Mac
•    Entourage on the Mac
•    Shared calendars on the Mac
These are all on our “to do” list.

Indexing your cloud data.   In the cloud era, you are more likely to have information stored at multiple locations and in multiple services.  We want Zekira to be the go to place for accessing all that information.  We’re looking to index your Google docs, Gmail, Google Calendar, Google Contacts, Google Drive, LinkedIn Account, Twitter and more.

More feedback welcome
The context engine is a new concept.  The Zekira team has spent a lot of time brainstorming ways it can and will be used.  But we don’t for a minute believe we’ve thought of everything.  We know that as more people use Zekira, they’ll think of other ways they want to try to get back to something they remember.  If you wish Zekira would let you do something it isn’t already doing, let us know.  We want your feedback.  You are helping to make Zekira better.

Wednesday, March 28, 2012

Hey Wall Street Journal, Saving Computer Files is Not a Disease

It's often a sign of trouble when a newspaper's health reporter writes about computing issues.  Usually you're going to hear about either carpal tunnel syndrome or some psychological disease supposedly caused by computer technology.  Yesterday's Wall Street Journal was a great example (link).  It delved into the woes of "digital hoarders," people who save large quantities of old files on their computers.

Quoting two psychologists, two nonprofit researchers, and a cognitive-behavioral therapist, the article doesn't directly say that people who save computer files are sick, but it kind of implies it:

"Digital hoarding can stem from the same psychological issues as other kinds of hoarding, experts say.... Hoarding is officially considered a form of obsessive-compulsive disorder, but some hoarders also suffer from attention-deficit-hyperactive disorder."

We're treated to the usual stereotype of sad loners who collect technology as "a way to feel good or fill an emptiness," but are soon dominated by their habit:

"Young men... spend so much time and money amassing collections of music or games or gadgets that they withdraw from the real world. 'They can't pay their rent or buy food because they have to have this latest piece of equipment to support their habit.' "

It's interesting how we've suddenly gone from people who save files to young men (probably underwashed and in badly-fitting t-shirts) who obsessively buy so much tech gear that they can't afford food.  The article implies it's all one big slippery slope. 

In other words, don't save that e-mail -- you might turn into a bag lady.

Baloney.  I'm sure there are people with obsessions that affect their computer use (just like those obsessions affect the rest of their lives).  But routinely saving all of your computer files isn't a disease, it's a natural reaction to the flood of information in our lives.  The people who save those files aren't compulsive, they're smart.

Think about it:  The WSJ article says that people use "only" about 20% of what they save.  But flip that around, and it means you'll need 20% of your files again in the future.  You don't know in advance which 20% you'll need, so if you throw out all your old computer files, it's 100% certain that you'll miss some of those files and regret it later. 

Besides, unlike someone who saves physical objects, there is almost no downside to saving digital data:

Downside of saving all your old physical goods:
-House eventually fills up
-Old pizza boxes attract rats
-Fire hazard
-Risk of being buried alive when piles of old magazines collapse on you
-Unfavorable publicity on 6 O'Clock News when Health Department raids your house

Downside of saving all your old computer files:

-You might need a bigger hard drive

The Journal article implies that people who save files need some sort of intervention.  It advises you delete old e-mails, unsubscribe to newsletters and mailing lists, and don't save web pages that interest you.  The cognitive-behavioral therapist says you might also need to "relearn basic social skills and find other enjoyable activities."

I'd like to suggest a different approach.  If you're such a compulsive neatnick that you feel good about yourself after you clean out your hard drive, by all means indulge yourself.  And after that, your paper clip collection needs sorting.

For the rest of us, my advice is to ignore the experts and save all your computer files.  Hard drives are cheap, and storage capacity continues to grow explosively.  Unless you're "hoarding" high-res movies, chances are drive capacity will grow faster than your archive.

The real problem isn't file saving, it's that we need a better way of finding old files when we need them.  Most of today's computer search tools are either utilities first designed for the world of 40-megabyte hard drives; or are derived from Internet keyword search engines, and good luck finding something if you don't remember its keyword.  Apple's Time Machine gets a lot of praise, and tools like Evernote and X1 can help.  But I think there's also a need for a different approach, a fundamental rethinking of how we manage our information archives.  The problem is not in ourselves, it is in our search tools.

Sunday, February 26, 2012

The Reality of Information Overload

Information overload is a feature of modern life for knowledge workers.  Our productivity depends on how well we can create and recall information, but we deal with far more of it than anyone can hold in their memory.  The result is often frustration and lost productivity.

In the tech industry, we talk about information overload a lot, but there's surprisingly little hard data on what drives it, who has it the most, and what you can do about it. 

Everyone's overloaded, but in different ways.  In January of 2012, we did a survey to explore computer users' information overload problems.  We found that information overload is widespread but extremely variable.  Although almost everyone experiences it sometimes, it's a much bigger problem for some people than it is for others. 

The causes of information overload also vary.  For some people it's e-mail overload, for others it's having more contacts and meetings than they can keep track of.  In many cases, the issue is managing the enormous archives of files that people have built up throughout their careers. Here are some statistics:

  • More than 40% of the respondents said they feel overwhelmed by the amount of information in their lives.
  • About half of the respondents experience information overload several times a week, and about 15% experience it several times a day.
  • 20% of the respondents receive more than 25,000 e-mails every year.
  • A quarter of the respondents receive more than 100 text messages a day.
  • A third of the respondents have saved more than 100 gigabytes of business files in their personal archives.

In this post I'll give you the details on the different forms of information overload, and what I think it all means.

Understanding information overload

Almost everyone has information overload, but some have it a lot more than others.

The first thing we wanted to check in the survey was how many people are experiencing information overload.  We asked about that in a couple of ways -- what info overload-related problems do they run into, and how do they feel in general about the flow of information through their lives.

Let's start with the problems.  We asked how often people experience four situations:

  • You can't find a document that you know you stored somewhere
  • You can't find something you saw on the web
  • You don't know the context for a meeting you're about to go into, and
  • You don't remember your context with a person

Everyone has those problems sometimes.  But about half of the people surveyed said they have one or more of those problems several times a week, and fifteen percent have at least one of the problems daily.

Are those big percentages or small ones?  This is one of those "glass is half full" situations where you could make an argument in either direction.  But from my perspective as a product guy, I think it means there is a large group of people who have a problem that needs to be solved.

How often do you encounter one of the four problems?

The chart below shows the frequency of each of the specific problems.  It's important to keep in mind that not everyone has the same information overload problems.  Some folks are more concerned about finding files, others about remembering people.  For example, I've had some folks tell me they can't believe that anyone could be about to go into a meeting without remembering the meeting's purpose.  But more than ten percent of the respondents said that happens to them several times a week.

How often do you encounter the following situations:

Feelings about information overload

To get a second test of the extent of information overload, we asked people to agree or disagree with a series of statements related to it.  More than 50% of the total sample said they agreed strongly with at least one of the statements.  The broadest agreement was with statements that talked about information overload in general: About half of the respondents said they feel overloaded by the information in their lives, and a third said they would be more productive if they could manage their information better.  More specific statements, like "I'm overloaded by my e-mail," had lower rates of agreement.

As we saw in the first question, it looks like different people have different info overload issues, rather than everyone sharing the same single problem.  That's why the general questions get higher agreement than the specific ones.  I think this also helps to explain why the problem persists even though there have been a lot of products aimed at solving it. I don't think any single product can solve all the issues that people have.  We'll talk about those products later.

How strongly do you agree or disagree with the statements below?
Percent who agree with a statement (those who answered 5 or 6 on a 1-6 scale, with 6=agree strongly, 5=agree moderately, and 1=disagree strongly).

Many of us are Information Savers

The single most popular statement above was "I hate throwing away information of any kind."  About half of the respondents agreed with it.  We did some followup questions about that, and it turned out that many of us are systematically saving our old files for future reference.

That is one of the sources of information overload: How do you make use of that big personal archive of documents and files? It's a problem that feeds on itself -- the more successful you become in your work, the more files you accumulate, and the harder it is to navigate the archive.

Some of the info archives that people are creating are truly huge.  About 40% of the respondents have saved between 10 and 100 gigabytes of information, a quarter have saved between 100 gigabytes and one terabyte, and almost 10% have saved more than a terabyte.  To give you a sense of how much information that is, a single gigabyte can hold the text of more than 2,000 English-language novels uncompressed, so a person with a one-terabyte archive has saved information equivalent to about two million novels.

Here's the breakdown of archive size:

About what quantity of work-related or school-related computer files (documents, e-mails, images, etc) do you have stored on your computer(s)? Please include files from both current and former jobs.

Of course not all that information is printed text.  Other types of files consume much more space (for example, if it were all video, a terabyte archive wouldn't be very big at all, depending on the resolution of the videos ).  But most archives aren't videos. The files saved are the same types of files people use in business.

The most broadly-saved document format is, no surprise, PDF files. About 95% of the respondents said they have saved old PDFs.  That is followed closely by Microsoft Office documents, and then image files.  Here's the chart:

Please select all the types of files you have saved
(The chart shows the percent of respondents who say they have saved a particular type of file in their archive)

I should add that there are several other file types that we didn't ask about directly, but were mentioned by some respondents.  They include source code, notes, sound files, and video.

How far does the archive go back?

The typical personal archive has files dating back about 10-15 years.  About a quarter of the respondents have saved files going back 20 years or more.  As you'd expect, age of the respondent correlates to age of the oldest files.  It looks like many people are saving files that date back to the start of their careers.

What is the creation date of the oldest file you've saved?

Why we save files

We asked people why they had saved the oldest file in their archive.  They gave a mix of practical and emotional reasons:

We also gave people an opportunity to add their own comments on why they save files.  Some of those responses stood out to me:

"Because storage is free, but my time is not."
"Save in case I can leverage in next job."
"Why not? Why delete it?"
"Record of work for use in professional portfolio."

And my personal favorite:

"It's like a little time capsule of my life that someday I will open again."

For many information savers, I think that nails the issue.

What causes information overload?

Information overload isn't just driven by your old files, of course.  Many people are also dealing with a river of incoming information every day.  To get a handle on how much info they're dealing with, we asked them about their volume of things like e-mails and text messages.  As you'd expect, many people get a lot, but once again there's a huge variation from person to person.  Here are the specifics:

E-mail.  The typical respondent has four e-mail accounts, and receives about 13,000 e-mail messages a year (in case you want the statistical details, I'm using the median here rather than the average because the heaviest e-mail users skew the average). 

Speaking of heavy e-mail users, the top 20% of respondents said they get more than 25,000 e-mails a year.

How many e-mail accounts do you personally have?

About how many e-mails (including both personal and work mail) do you receive on an average week day?

Text messaging is also a huge source of incoming information, although its impact varies tremendously from person to person.  The typical respondent receives about 4,000 text messages a year (including tweets, instant messages, and phone texts). But as you can see from the chart below, the numbers are all over the map.  Some people are very heavy text consumers while others hardly use them at all. About a quarter of the respondents said they receive four or fewer texts in an average week day, while another quarter said they receive more than 75.

About how many text messages do you receive on an average week day?

Number of contacts.  The typical respondent has about 200 contacts on their computer and phone, 100 friends on Facebook, and 100 connections on LinkedIn.  LinkedIn had a slightly higher rate of membership than Facebook, which I think reflects the ubiquity of LinkedIn in business, especially the tech industry.

Including information saved on both your computer and phone, about how many people are on your contact list(s)?

How many people are you connected to on Facebook and LinkedIn?

(Be careful reading this chart.  It makes it appear like there's a spike in the number of people with 250-499 contacts, but that's an illusion created by the change in segment sizes at that point.  In reality, this is a very extended longtail curve, in which the number of respondents goes down steadily as you go up in number of contacts.  I apologize for the poor question design here.)

Meetings.  This is another surprise.  I expected most people to have a lot of scheduled meetings (maybe because Silicon Valley is so meeting-happy).  But in the survey, about a third of the respondents said they average less than one meeting a day, and most people had just a couple.  On the other hand, the top 20% of respondents have more than five meetings every day. 

I think this helps to explain why some people have trouble remembering meeting context and others don't.

About how many scheduled meetings and appointments (including in-person meetings and scheduled calls) do you have on an average week day?

What to do about information overload

There are a number of computer products that help with parts of the information overload problem.  In the survey we asked which ones people use.  Evernote is the most common, followed closely by Google Desktop (which, ironically, Google just killed).

Please choose the answer that best describes your current use of each of the following applications / web services:

These products all work in different ways, and many of them solve different problems.  For example, Xobni helps you sort through email, while Google Desktop and X1 index your hard drive.  So really you're looking at a lot of different market segments here that were grouped together because we make the mistake of thinking of information overload as a single thing.

I think the one common theme in these responses is that a lot of people have tried different products for info overload, and haven't necessarily been satisfied.  Note the relatively large yellow bars for many products -- those are people who have tried a product and stopped using it.  You don't generally want the yellow bar to be bigger than the blue one.

Finally, we asked people if they would be interested in a new software product that attacked information overload.  The response was strong.  Usually in this sort of question, you'd be very happy to get a fifth of the respondents in the top two interest categories.  In this case, 50% of the respondents were in the top two. 

There are all sorts of caveats that need to be attached to that number.  People always respond better to a theoretical product idea than they do to an actual product.  And this survey wasn't based on a sampling of all computer users, it is weighted toward the tech-aware people who read my blog.  In other surveys I was involved in at Palm, we did talk to the general public, and we found that around 15% of the population responded very strongly to the idea of an info management product.  That's probably the real market size, at least in terms of who would be an early adopter.

Based on this survey, and the ones I did in the past, I think it's fair to say that a lot of people have an unmet hunger for better information management.  To use a Silicon Valley phrase, there really is a pony in the stable.*

I'll talk about the pony next time.

We're working on a product that will help people recall and re-create the context around any piece of information in their careers -- a name, event, message, word, etc.  How interested would you be in getting a product like this, on a 1-6 scale (with 6=extremely interested and 1=not at all interested)?


This was a voluntary survey of readers of the Mobile Opportunity weblog.  It is not a statistically meaningful sample of the general population, but I think it does reflect the opinions of tech-savvy knowledge workers.  When I worked at Palm and PalmSource, I did other surveys on information overload, using statistically balanced samples of web users in the US, Europe, and China.  The findings of those surveys were directionally consistent with these results.  (In other words, when the general public was surveyed, the percent interested was smaller but the findings were similar. Those survey results were released to the public, so I am comfortable talking about them here.) 

This survey was 30 questions, most multiple choice, and was hosted on SurveyMonkey.  There were 446 respondents with an 81% completion rate, a very good response considering that we didn't offer an incentive.

What do you think about the survey results?  Do you agree?  Disagree?  Where do you fit relative to the results?  And do you have anything to add?  This is a complex subject and I don't think we have all the answers, so we'd welcome your comments.  And thanks!


*It turns out that the pony joke isn't necessarily known worldwide.  There are a number of different versions, with slightly different lessons to them, but the version I prefer goes something like this:

A young boy was an eternal optimist, always seeing the bright side of everything. His parents decided he needed to learn that life isn't always fun, so they decided to expose him to some unpleasant situations. One day they took him down to a horse stable and showed him a stall packed to the roof with manure. He squealed with glee, grabbed a shovel, and started digging out the stall.  "What are you doing?" the boy's father asked.  The boy replied, "With all that manure, there's got to be a pony in here somewhere."

The phrase "there's got to be a pony in here somewhere" is used in Silicon Valley (and in much of American business) to refer to people who believe there are good prospects in a market just because everyone else is targeting it. The joke was reportedly a favorite of Ronald Reagan (link), and Kara Swisher famously used it to summarize the entire history of AOL (link).

In this case, I'm using the phrase to acknowledge that many of people have claimed they can solve information overload, without a lot of success.  But with the help of the survey, I think we know where the pony is.