Don Dodge on The Next Big Thing

Why Powerset is important and different

2008-07-02T14:35:00Z

Microsoft announced yesterday the acquisition of Powerset, a natural language search engine. Techmeme has lots of stories about the acquisition. Today the stories are all about Microsoft/Yahoo potential deals. But lets take a step back and look at how Powerset works and why it is an important development in search.

There are two key things to consider;

Powerset technology is more about indexing the content and understanding its meaning, than the query itself. This has enormous implications.
There are many lucrative markets for this technology...not just consumer web search.

How does natural language search work? There is a lot of linguistic rocket science to this but basically it breaks the search problem into two parts. First, understanding the intent of the user's query by using (NLP) natural language processing. Second, training their search index algorithm to parse the structure and context of individual web sites, and add meta data about the pages to the index.

Powerset is using linguistics and (NLP) to better understand the meaning and context of search queries. But the real power of Powerset is applied to the search index, not the query. The index of billions of web pages is indexed in the traditional way. The big difference is in the post processing of the index. They analyze the indexed pages for "semantics", context, meaning, similar words, and categories. They add all of this contextual meta data to the search index so that search queries can find better results.

Who is the best ballplayer of all time? Powerset breaks this query down very carefully using linguistic ontologies and all sorts of proprietary rules. For example, they know that "ballplayer" can mean Sports. Sports can be separated into categories that involve a "Ball". Things like baseball, basketball, soccer, and football. Note that soccer does not include the word ball, yet Powerset knows this is a sport that includes a ball.

Powerset knows that "ballplayer" can mean an individual player of a sport that includes a ball. They know that "best of all time" means history, not time in the clock sense.

Powerset understands the intent of the query, but more importantly, it understands the meaning and context of all the relevant web pages. Rather than just match keywords from the query, Powerset looks for "semantic" matches in its index of billions of web pages.

Why hasn't this been done before? Powerset uses all these rules and linguistic approaches to analyze billions of web pages, and adds "meta data" hooks into each word on each page. As you can imagine this is a huge scaling problem, that has been impossible to solve economically until now. With Moore's Law applied to constantly reducing the cost of computing, storage, and bandwidth, it is now possible to solve this problem, although it is still very expensive.

Where else could Powerset technology be used? Consumer web search is one obvious market, but there are many more.

Enterprise search, companies searching their own internal documents and information, is a huge market that would benefit from Powerset technology. Enterprise search is a multi-billion dollar market.
Advertising targeting. It is easy to target ads to search terms, but extremely difficult to target ads to general web content or User Generated Content. Today ad targeting technology does a poor job of understanding the context and meaning of a news article, a blog post, or magazine article. Powerset technology could be used to better understand this content and better target more relevant ads. This is a multi-billion dollar opportunity.
Vertical Search - News search, medical search, people search, resume search, and basic knowledge search could be dramatically improved with Powerset's "semantic" search indexing.

I am sure you can think of many other places Powerset's semantic technology could be used. The technology will improve and expand over time. There is enormous potential here, more than a small startup with limited funding could hope to address. This is why Powerset joining forces with Microsoft makes so much sense.

Microsoft acquires Powerset

2008-07-01T19:57:00Z

Microsoft has agreed to acquire Powerset, a Natural Language search engine, founded by Barney Pell. Satya Nadella, Senior Vice President, Search, Portal, and Advertising said on his blog;

Powerset will join our core Search Relevance team, remaining intact in San Francisco. Powerset brings with it natural language technology that nicely complements other natural language processing technologies we have in Microsoft Research.

More importantly, Powerset brings to Live Search a set of talented engineers and computational linguists in downtown San Francisco.

Powerset said on their company blog post;

"With any start-up, the challenge is to take the seeds of an idea and grow it into a viable company," he wrote. "At Powerset, we transformed our idea into a world-class semantic search platform, demonstrating the future of search with our Wikipedia search experience. But building a large-scale semantic search engine is expensive, requiring an engineering effort and computing resources beyond what most start-ups could ever imagine...We believe that this is the fastest way to bring our technology to market at a large scale."

Information security weakest link? Stolen laptops and cell phones

2008-07-01T15:55:08Z

Over 600,000 laptops are reported lost or stolen every year at US airports according to a ComputerWorld story. The story, based on a survey by the Ponemon Institute says that 65% of them are never recovered. Laptops are most often lost at security checkpoints. Hotels and taxi cabs are also prime locations for losing laptops and cell phones.

Close to 10,278 laptops are reported lost every week at 36 of the largest U.S. airports, and 65% of those laptops are not reclaimed, the survey said. Around 2,000 laptops are recorded lost at the medium-size airports, and 69% are not reclaimed. The institute conducted field surveys at 106 airports in 46 states and surveyed 864 business travelers.

Millions of dollars are spent each year on sophisticated information security products. Simple things like stolen laptops and lost memory sticks / thumb drives are a major security vulnerability.

The Washington Post reports that data breaches are up 69% this year.

The Identity Theft Resource Center in San Diego tracked 342 data breach reports from Jan. 1 to June 27. More than one-third of the reports came from businesses, a 27 percent increase over business breaches for all of 2007.

The center found that data breaches among health-care providers and banks also increased. They now account for 15 percent and 10 percent of the breaches, respectively. Breaches from educational institutions, government entities and the military declined for the third year in a row, the center found.

Hacking was the least-cited cause of data breaches in the first six months of this year. Instead, lost or stolen laptops and other digital storage media remain the most frequently cited cause of data breaches, accounting for more than 20 percent of all reported cases, the center found. The inadvertent posting of personal and financial data online prompted roughly 15 percent.

The Industry Standard reports that losing cell phones is even more common. And, the newer Smart Phones contain contact lists, pictures, documents, confidential emails, links to sensitive information, etc.

I didn't mean for this to be an alarming post about information security breaches. I just did a simple web search and these were the top three links in the search results. It looks like there is a lot of work yet to be done in the Information Security area.

Microsoft innovation in Boston/Cambridge

2008-06-30T18:23:00Z

Microsoft is making a big investment in innovation in Cambridge, Massachusetts. The Boston Globe has a front page story about Microsoft's new Concept Development Center led by Reed Sturtevant. Reed was recruited by Ray Ozzie and his brother Jack to build the new lab.

The Boston Globe story says;

Boston software legend Ray Ozzie replaced Bill Gates as Microsoft's chief software architect in 2006. Ozzie has been pushing for a transition from the desktop software that accounts for the bulk of its revenue to the Internet services that are the wave of the future.

"Microsoft is making a big investment in Massachusetts," said Reed Sturtevant, 51, the director of the Boston Concept Development Center, who worked with Ozzie in the 1980s at Lotus Development Corp. and joined Microsoft last fall. He's spent most of his time so far recruiting. "There's a huge amount of talent in Boston," he said, "and the question is, how do you bring new talent into Microsoft?"

Microsoft's presence in Massachusetts is growing with the acquisitions of Groove Networks, Softricity, and Danger Inc. Microsoft is also growing internal product groups in the Boston area and consolidating many of the groups at One Memorial Drive in Cambridge.

Ray Ozzie, creator of Lotus Notes, and founder of Groove Networks, has lived and worked in the Boston area most of his life and wants to see Microsoft build a much larger presence over time.

IPO market ups and downs - Q2 worst in 30 years

2008-06-28T18:17:00Z

Not one single VC backed company went public in the 2nd quarter of 2008 according to the New York Times. The last time that happened was 30 years ago. However, 2007 was one of the best years for IPOs since the dot com bubble of 1999. Such are the cycles of VC investing and IPO exits.

	Investments	Returns
Year	VCs	M&A	IPO
2001	$32.1	$16.8	$3.5
2002	$22.1	$7.9	$2.1
2003	$19.6	$7.7	$2.0
2004	$22.4	$15.4	$11.0
2005	$23.7	$16.0	$4.5
2006	$25.5	$17.1	$5.1
2007	$29.4	$25.4	$10.3

Totals	$174.8	$106.3	$38.5

Fear is temporary, greed is permanent. We are in a "fear" cycle now. There is fear in the public IPO markets due to slow growth, high oil prices, and recession fears.

VCs continue to raise funds and invest aggressively. Last year VCs invested nearly $30 Billion, the most since 2001. VCs have a long term view. They don't expect their investments to pay off for at least 5 years, sometimes much longer.

Fred Wilson has an interesting post today that includes this 10 year chart of the Dow Jones Industrial Average for the 1970s.

The Dow tanked in 1974 when OPEC put on an oil embargo that caused wide spread shortages and gas prices to sky rocket. The 70s were a dismal decade for business and investing. The Dow hovered around 900 and hit 1,000 just twice in the decade. It is hard to imagine now with the Dow around 12,000, but we have seen massive swings, up and down, in the 30 years since the last oil crisis.

Is this IPO market "goose egg" a one time thing? Will M&A transactions remain strong? What do you think?

Dan'l Lewin in San Francisco Chronicle

2008-06-27T17:20:00Z

Dan'l Lewin is Microsoft's Corporate VP for Strategic and Emerging Business, based in Silicon Valley. I work in Dan'l's organization so it was a treat to read a story in the Chronicle about Dan'l's work with VCs and startups around the world.

Dan'l is a legend in Silicon Valley where he was one of the early employees at Apple, and later started Next with Steve Jobs.

Dan'l (that is not a typo, that really is his name) was hired by Steve Ballmer almost 8 years ago to oversee Microsoft's global relationships with venture capitalists, startups and Microsoft technology partners as well as industry and community organizations in Silicon Valley.

Great quotes from Mike Moritz of Sequoia Capital "He ensures nothing gets lost in translation between here and Redmond" and Sam Altman, CEO of Loopt "Were it not for Dan'l - if we just knew Microsoft by reputation - I don't think we'd be working with them nearly as closely as we are."

The Chronicle story goes into how Microsoft works with VCs and startups, what areas are hot now, and how Microsoft handles acquisitions. You can read the whole story on SFgate, the Chronicle web site.

TechCrunch 50 Conference panel of experts

2008-06-27T13:59:06Z

Mike Arrington and Jason Calacanis asked me again this year to participate on the TechCrunch 50 panel of experts. TechCrunch announced yesterday that Mark Cuban, Marc Benioff, and I will be on the panel. I am humbled…and honored to be mentioned with those guys.

The complete list of “experts” is listed here. They include; Mark Andreessen, Marissa Mayer (Google), Jeff Weiner (formerly Yahoo), Chris DeWolfe (MySpace), Sheryl Sandberg (Facebook COO), Kevin Rose (Digg), and journalists Dan Farber and Om Malik.

TechCrunch 50 will be on September 8-10 in San Francisco. Companies can submit applications to participate here.

The panel of experts will sort through the hundreds of applications and narrow it down to 50 companies who will present at the conference. We will also judge the company presentations at the conference and select winners in various categories.

TechCrunch40 sold out quickly last year. Register here for TechCrunch 50.

Can Social Networks Generate $1M In Ad Revenue?

2008-06-25T21:36:00Z

Facebook, MySpace, LinkedIn, Beebo, Plaxo, Orkut, and MSN Spaces are the biggest well known social network spaces. But what about the 800 other sites scrambling for audiences in the social network space? Can they generate $1M in advertising revenue per month? Will they ever be profitable?

USA Today says "Social Network Sites Work To Turn Users Into Profits" and summarizes the problem with this quote;

"Short of striking it rich with online ads or creating a new revenue stream, how can so many sites leverage their vast audiences? In many respects, it is the same query that dogged portal companies in the mid-1990s and search engines in the early '90s. Some were sold. Some went public. Some went belly up.
The ongoing challenge is to concoct a potion — be it through banner ads, premium subscriptions or licensing agreements — that no one has perfected. Facebook, crown jewel of the field, is valued at $15 billion but barely turns a profit."

CPM versus CPC - Big audiences are great but how you monetize them is the key to financial success. Google and the search companies are able to sell Cost Per Click (CPC) ads and command very high rates. Content sites and social networks don't have a search term to key off so they charge Cost Per Thousand (Mil) or CPM rates. In some cases it can take 1,000 page views to generate the same revenue as one click on an ad.

A penny for your thoughts? I talked to a Facebook App developer at the ReMix conference a few weeks ago. He told me his app is generating 300 million page views per month. Wow! Then I asked what kind of CPM (Cost Per Thousand) ad rates he was getting. He shrugged and said somewhere between $0.02 and $0.05 per thousand. That pencils out to between $6K and $15K of advertising revenue per month for those 300 million page views.

How much traffic is needed to generate $1M in ad revenue? It all depends on how well you can target your audience and how much you can charge for CPM rates. But, based on a survey of social network sites let's assume an average CPM of $0.40. You would need 2.5 Billion page views per month to earn $1M in ad revenues. That is 2,500,000,000 page views...and how many sites can sell out all their page view inventory?

A New Revenue Model? - Google revolutionized the search business by banning display ads sold on a CPM basis, and instead offering text based ads where you only pay when someone clicks on the ad, what we now refer to as CPC.

It will probably take a new revenue approach to make many Social Networks profitable. From the USA Today story;

"Facebook's ambitious plan to reshape advertising — via a new approach to social marketing, called Beacon — was a bust. The idea was to inform friends whenever a Facebook member purchased something from online retailers. When consumers protested its invasion of privacy, Facebook CEO Mark Zuckerberg acknowledged the miscue and promptly apologized.
Even Google, as close to a money mint as anything online, has struggled. Google has a deal with Rupert Murdoch's News Corp. to place ads on MySpace, and owns Orkut, which flopped in the USA. Co-founder Sergey Brin recently admitted the "monetization work we were doing there didn't pan out as well as we had hoped."

Which new model will work? No one knows at this point, but there will be billions of dollars for whoever figures it out.

Beacon was innovative, but privacy concerns killed it. We are often influenced by what our friends buy, maybe just a slightly different approach will work.

Social recommendations are very powerful. Back in the early days of the web there were several attempts to consolidate buyers into groups to get better prices. Could social networks do something similar?

Businesses and advertisers are anxious to tap into the power of social networks. The social networks are building huge audiences but can't figure out how to monetize them. When they learn how to connect effectively the benefits will be amazing for everyone involved.

This is a business problem, not a technology problem. The answer will be simple and obvious. In fact, it has probably already been considered and rejected several times. Someone will come along and put a slightly different twist on it and...Eureka!!! Don't you just love business? How do you think this will play out?

11 Billion videos served - Profits ???

2008-06-18T01:04:45Z

Comscore says Americans watched 11 Billion videos in April. Nearly 75% of Americans with Internet access watched at least one video. The "average" user watched 82 videos. Average length was 3 minutes.

John Paczkowski at AllThingsD says most of those 11 billion videos were unmonetized.

Mark Cuban says Hulu is making more money at video than YouTube. Although Hulu is tiny by comparison, they are better able to monetize their traffic...and use YouTube as a traffic generator.

Google / YouTube streamed over 4 Billion videos. Hulu wasn't even in the top 10. This is a case where bigger isn't better. My guess is that 5th place Viacom made way more profit on their videos than YouTube made streaming 20 times as many videos. Google CEO Eric Schmidt admitted in a recent interview that they haven't yet figured out how to effectively monetize YouTube.

Video is incredibly expensive to host on servers and stream out to users. YouTube is loading 10 hours of video every minute of every day. Imagine how much storage and bandwidth it takes to handle that. The costs are staggering. The revenues? Not so much.

Andrew Chen asks "where are all the video startups on this list?" Simple answer Andrew, the online video business is way too expensive for any startup to compete on volume. Every company on this list is a huge media company because the costs of creating, hosting, streaming, and audience building is enormously expensive. Well beyond the means of any startup. However, there is a place for clever startups like Hip Mojo. Small is beautiful...and profitable.

Social Networks 1% rule

2008-06-01T19:43:00Z

What is the social network 1% rule? Generally in a group of 100 people online, one will create content, 10 will "interact" with it (commenting or adding to it) and the other 89 will just view it. But, everyone benefits from the activities of the whole group.

Bradley Horowitz, an old friend from my AltaVista days, and one of the smartest guys I know uses this simple illustration to convey the idea.

I have seen this natural hierarchy many times. My experience with SiliconInvestor, one of the first investment discussion boards on the web, matches these findings. The contributor to commenter to reader ratios were about the same. Later at Napster I saw a similar pattern. Very few people shared their music collections while millions downloaded.

Web 2.0 social network sites are finding the same thing. It takes a relatively small group of contributors to create the content. These contributors attract the commenters or editors, which in turn attracts the huge audience.

At Wikipedia about 50% of all article edits are done by 0.7% of users, and more than 70% of all articles have been written by just 1.8% of all users, according to the Church of the Customer blog (http://customerevangelists.typepad.com/blog/).

Fred Wilson says his vision for social media is; "every single human being posting their thoughts and experiences in any number of ways to the Internet." TechMeme has picked up Fred's post, contributors are sure to follow, and thousands of readers will enjoy the benefits.

Web 2.0 = Bubble 2.0?

2008-05-27T17:15:00Z

At TiECON East, Friday, May 30th Fred Wilson and I will be debating "Web 2.0: Viable Business Model or Bursting Bubble?" David Cancel (Lookery) will be moderating, and we will be joined by Brian Balfour (Viximo) and Nabeel Hyatt (Conduit Labs).

All booms eventually go bust. Web 2.0 will too, but, as I wrote last year, this time it will be different. The boom companies aren't publicly traded stocks like they were in 1999, they are VC backed companies. VCs are very experienced at handling risk and failure. No one is going to lose their college or retirement savings in this bust cycle.

The Financial Times has a story today "Web 2.0 Fails To Produce Cash:"

Many members of the Web 2.0 generation of Internet companies have so far produced little in the way of revenue, despite bringing about some significant changes in online behaviour, according to some of the entrepreneurs and financiers behind the movement.

The shortage of revenue among social networks, blogs and other “social media” sites that put user-generated content and communications at their core has persisted despite more than four years of experimentation aimed at turning such sites into money-makers. Together with the US economic downturn and a shortage of initial public offerings, the failure has damped the mood in internet start-up circles.

Bubble Cycle - Bubbles go through predictable cycles. Bubbles emerge from the ashes of despair. It takes a while to gain momentum but eventually greed overtakes fear and we are off on another bubble adventure. Stage one of a bubble is when most smart money declares we are NOT in a bubble...it is different this time. Stage Two is more dangerous. Many people agree that we are in a bubble, but it will last another year or two, and there is still money to be made. The third stage is when the bubble has burst but most people are in denial and think it is a temporary set back. The fourth stage is when everyone agrees the bubble has burst and life will never be the same. My guess is that we are now well into Stage Two of the bubble cycle.

Advertising Revenue Math - How much traffic is needed to generate $1M in ad revenue? It all depends on how well you can target your audience and how much you can charge for CPM rates. For social network sites let's assume an average CPM of $0.40. You would need 2.5 Billion page views per month to earn $1M in ad revenues. That is 2,500,000,000 page views...how many sites generate that traffic?

Extrapolating Success - Facebook, MySpace, and a few other social network sites do generate significant traffic where the advertising numbers do work out very well. Some investors make the leap of faith that the next "shiny object" idea will enjoy similar success. In the rush to do a deal, it seems that few of them stop to do the math on what it takes to get there and calculate the odds of success. This is another one of those cases where there will be one or two winners and a hundred broken hearts.

Who do you see as the winners...and losers?

TiECON East 2008 Communication, Collaboration, Convergence

2008-05-21T11:39:58Z

TiECON East 2008 is coming up next week, May 29th and 30th at the Westin Waltham. The agenda is packed with excellent sessions with a great line up of speakers including Craig Newmark, founder of craigslist, and Desh Deshpande, founder of Sycamore Networks.

Fred Wilson and I will be debating "Web 2.0: Viable Business Model or Bursting Bubble?". David Cancel (Lookery) will be moderating, and we will be joined by Brian Balfour (Viximo) and Nabeel Hyatt (Conduit Labs).

The first day of the conference is devoted to entrepreneurs with several workshops, sessions, an evening keynote, and networking reception.

Please stop by and say hello if you are attending.

ClearContext for Outlook announces public beta

2008-05-19T12:57:13Z

ClearContext – is an organizer for Microsoft Outlook email that analyzes how you handle email, who you respond to, what gets saved and what gets deleted, etc., then forms rules and priorities for how to deal with future email.

I have been following ClearContext and Xobni for more than a year now. I believe it was Om Malik who pointed me to them. Today they announced public beta for the personal version of ClearContext. Om Malik and TechCrunch have good reviews of ClearContext.

“Email applications are built around the concept of processing messages on a one-by-one basis, an approach that is no longer viable for many people due to the number of emails they receive and all of the different tasks now being done in email,” said Deva Hazarika, co-founder and CEO of ClearContext. “We provide the context to help people go beyond just dealing with individual messages, taking email productivity to a new level.”

ClearContext helps with project management: sorting and categorizing emails relating to a particular subject or project. The user can also separate emails according to their priority, set them to re-appear later, delegate them to a different user or file them into various project views.

The ClearContext Dashboard offers a consolidated view of e-mails, tasks, and appointments related to a project. Results can be pulled from multiple folders and data stores within and across Outlook and Exchange.

Try it out and let me know what you think.

Top 10 Tech Trends from Churchill Club

2008-05-15T17:17:51Z

Some of the best minds in technology gathered at for The Churchill Club's annual dinner. Eric Savitz of Barron's covered the event in detail. VentureBeat also has a nice summary including audience votes on the trends.

The highlights, excerpted from Eric Savitz article, were;

The rise of the “implicit” Internet. Today your permanent record exists; you create a trail of data exhaust, digital bread crumbs. Implicit data that exists in silence. Movie rentals, restaurant reservations, books purchased, Web sites visited, etc. All of this data existed in silence. No easy way until now to benefit from the data; but the silos are coming down. Google, Yahoo, Facebook, Mozilla collecting data.

I have seen so many companies, literally hundreds, that are building social, fun, aggregators, filters, recommendations, communities, and services that all boil down to one thing...building profiles from implicit data and explicit actions to better target advertising.

Betting on smart phones: The mobile device migration to smart phones from features phones will produce even greater disruption than PC industry moving from character mode to graphical interface.

This prediction was made by Roger McNamee who has a major investment in Palm, so he is definitely putting his money where his mouth is. There are more cell phones than desktop computers in the world today. Cell phones have been called the 'Third Screen", but they are quickly becoming the first screen for the younger generation.

Water tech will replace global warming as a global priority. The world is running our of usable water and will kill millions more in our lifetime than global warming.

I agree water already is a big priority in some parts of the world. At the NVCA conference last week one panel member said that 90% of the water we use in the USA is used for agriculture and irrigation. That is an astounding truth. We only drink something like 4% of all the water we produce. The rest is used for washing, flushing toilets, and irrigation. Dean Kamen has done some amazing work in the area of low cost water purification systems.

Fossilizing fossil energy. Oil and coal will have trouble competing with biofuels. 99% of discussion on the topic is completely irrelevant to the topic. In 4-5 years will have production proof that can sell biofuel at well below $2 a gallon at today’s tax structure and no subsidy.

Vinod Khosla made this prediction. I hope he is right but I have a hard time believing this. There have been numerous stories written on this subject that suggest it takes more energy to produce a gallon of biofuel than it is worth. Wind, Solar, and Mini-Hydro are proven sources of clean renewable energy where the technology and economics are viable at current energy prices. Green energy is certainly a hot area for investment and innovation. The question is can startups play a role here or will it be dominated by the big energy companies and utilities.

Read the complete list of top trends at Barron's Tech Trader Daily by Eric Savitz.

Xobni social networking for Outlook users

2008-05-13T19:19:00Z

Xobni - is an organizer for Microsoft Outlook email. The beta is now available to the public. I have been using Xobni for 6 months and love it. Xobni is inbox spelled backwards. Xobni compiles a summary for each email sender, extracts phone numbers, photos, organizes attachments in several different ways, and ranks each sender by how many emails they send to you, and how many you send to them. People connected to the email sender are listed, along with email threads from each of them. See all the attachments associated with these people. Search is an integral part of Xobni and can be done in several different ways.

Robert Scoble has a great video interview with Xobni founders Matt Brezina and Adam Smith. I knew the instant I first saw Xobni at the TechCrunch40 Conference that we should be partners. I talked to Adam Smith immediately after his presentation, and had dinner with Adam and his partner Matt Brezina that night. We sketched out how Microsoft could help Xobni, and started executing on the plan the very next day.

Xobni is now a Microsoft Startup Accelerator Program partner. There are lots of benefits to the program like free software, consulting help, press releases, introductions to Microsoft people, partners, and even VCs. But this is one benefit that we didn't anticipate...a demo by Bill Gates at the Microsoft Office Developers Conference. Check out this 2 minute video of Bill talking about Xobni.

Try out Xobni and let me know what you think. What additional features would you like to see? What other email productivity tools do you use?