Twitter Moving To Make Real-Time Search More Central

Twitter is starting to test ways to put its real-time search front and center. It is just bucket-testing the change right now with a few randomly selected users, so you might not see it. But you should expect it to be rolled out to everybody eventually. The search and trend features, which currently exist on a separate page, are being placed on the home page of the test accounts.

Twitter’s search technology comes from its acquisition of Summize, so its integration into Twitter is taking time. But it is a major plank of Twitter’s business strategy. Twitter co-founder Biz Stone writes:

Searching over Twitter messages is like a filter for what is happening right now—it’s an interesting look into the real-time thoughts of people and organizations around the world.

It is more than interesting. Real-time search is a potential game-changer. I tried to explain this in a post on Sunday, “Mining the Thought Stream”.

Google the next Antitrust target?

Christine A. Varney, nominated by President Barack Obama to be the U.S.’s next antitrust chief, has described Google Inc. as a monopolist that will dominate online computing services the way Microsoft Corp. ruled software.

“For me, Microsoft is so last century. They are not the problem,” Varney said at a June 19 panel discussion sponsored by the American Antitrust Institute. The U.S. economy will “continually see a problem -- potentially with Google” because it already “has acquired a monopoly in Internet online advertising,” she said.

While the remarks were made months before Obama picked her to head the Justice Department’s antitrust division, the comments signal her approach to the job if confirmed by the Senate. The Microsoft case, brought in 1998 by the Clinton administration, could have led to the breakup of the software giant and was a landmark in antitrust law.

In her remarks at the American Antitrust Institute, Varney advocated aggressive enforcement of antitrust laws to curb the conduct of individual companies that dominate an industry. She didn’t return a reporter’s telephone call seeking comment today.

Google Testing SearchWiki On AdWords

I am shocked to see this! Google seems to be testing part of SearchWiki, Google's way of promoting and removing results, to be spotted in the AdWords or sponsored ads section of the Google search results. Some people are noticing the X icon, which allows searchers to delete results from Google, in the sponsored listings.

Here is a picture from Rehan via WebmasterWorld:

SearchWiki AdWords

Why am I surprised to see this on the AdWords results? Well, one, I was surprised to see them on the main results for all Google users. Second reason, these are paid ads. But I guess they can use the feedback as part of the overall quality score.

Got that? Quality score metric. A new way to perform competitive sabotage on the Google ads. Well, not yet at least.

Twitter Fight Scandalizes Micro-Blogosphere - Switched


What happens when micro-blogging turns macro-psychotic? On February 11th, marketing consultant April Dunford posted an annoyed message to Twitter, the mircro-blogging site, about being condescendingly scolded by a newspaper reporter. Although she named no names, the reporter in question, David George-Cosh of the National Post, saw her 'tweet' and took offense in a big way. His venomous, profanity-laced responses started an embarrassing (to him) Twitter fight that played out in public for the whole world to see.

This was April Dunford's first post: "Reporter to me 'When the media calls you, you jump, OK!?' Why, when you called me and I'm not selling? Newspapers will get what they deserve"

You can check out Mr. George-Cosh's response -- and the subsequent war of words -- after the break.


Twitter makes all posts public -- each time a user writes an entry, all the user's 'followers' on the site can see it. Responses to posts are public as well. Why David George-Cosh elected not to use the system's private 'direct message' function is beyond us, and it's safe to say that this incident will undoubtedly have negative repercussions for both Mr. George-Cosh and his employer, the National Post. It serves as a potent reminder that the Internet is no longer the sanctuary of anonymity it once was -- with social networking now a fixture of mainstream society, how you behave online sends a powerful message. [From: MediaStyle]

Mining The Thought Stream

What if you could peer into the thoughts of millions of people as they were thinking those thoughts or shortly thereafter? And what if all of these thoughts were immediately available in a database that could be mined easily to tell you what people both individually and in aggregate are thinking right nowabout any imaginable subject or event? Well, then you’d have a different kind of search engine altogether. A real-time search engine. A what’s-happening-right-now search engine.

In fact, the crude beginnings of this “now” search engine already exists. It is called Twitter, and it is a big reason why new investors poured another $35 million into the two-year-old startup on Friday. Twitter is not the only company trying to solve this problem. Facebook, FriendFeed, and even Google are trying to crack it, but Twitter has a decided advantage in that it is capturing the vast majority of the real-time thought stream on the Web (because more people enter their thoughts directly into Twitter’s database than any other, and are doing so at an increasing rate).

What makes Google and other search engines so valuable is that they capture people’s intent—what they are looking for, what they desire, what they want to learn about. But they don’t do a great job at capturing what people are doing or what they are thinking about. For thoughts and events that are happening right now, searching Twitter increasingly brings up better results than searching Google.

Whether you want to know how people are mentally gearing up for this week’s Mobile World Congress in Barcelona or what they are thinking about today’s Ireland vs. Italy rugby match, searching Twitter will give you a pretty good smattering of sentiment and opinion. It is also a lot faster at getting out the essential details about breaking news, such as the Mumbai attacks or the plane that landed on the Hudson.

Twitter’s search engine is powered by Summize, a startup it acquired last July. But it also developed a feature called Track, currently disabled but coming back soon, that allowed people to follow the mention of specified keywords. John Borthwick, an investor in Summize (and thus now an investor in Twitter), explained in a blog post earlier this month ago why he thinks that “Twitter search changes everything.” Excerpt:

Imagine you are in line waiting for coffee and you hear people chattering about a plane landing on the Hudson. You go back to your desk and search Google for plane on the Hudson — today — weeks after the event, Google is replete with results — but the DAY of the incident there was nothing on the topic to be found on Google. Yet at http://search.twitter.com the conversations are right there in front of you. The same holds for any topical issues — lipstick on pig? — for real time questions, real time branding analysis, tracking a new product launch — on pretty much any subject if you want to know whats happening now, search.twitter.com will come up with a superior result set.

. . . How is real time search different? History isn’t that relevant — relevancy is driven mostly by time. . . . This reformulation of search as navigation is, I think, a step into a very new and different future. Google.com has suddenly become the source for pages — not conversations, not the real time web. What comes next? I think context is the next hurdle. Social context and page based context. . . . Twitter search today is crude — but so was Google.com once upon a not so long time ago.

Twitter may just be a collection of inane thoughts, but in aggregate that is a valuable thing. In aggregate, what you get is a direct view into consumer sentiment, political sentiment, any kind of sentiment. For companies trying to figure out what people are thinking about their brands, searching Twitter is a good place to start. To get a sense of what I’m talking about, try searching for “iPhone,” “Zune,” or “Volvo wagon”.

Why can’t Google simply index Twitter? It does, but its search results give more weight to links than to time. It could create a new search product along the lines of Blog Search or News search that is geared more towards Micro-messaging services such as Twitter, FriendFeed, and the rest. But what it really needs to go beyond simply indexing Twitter after the fact. IVP partner, and Twitter investor, Todd Chaffee, suggests:

If they were really smart they could partner with Twitter and make Twitter their real-time feed.

Doing that would require Google to “affirm Twitter’s dominance in this category and the importance of the Twitter data stream,” contends Borthwick. But so far, Google has pretty much flubbed this opportunity to open up real-time search. It bought Twitter competitor Jaiku, only to shut it down. And now it is hoping to create a counterweight to Twitter’s growing strength in real-time data by open-sourcing Jaiku. Good luck with that one.

Listening to Twitter’s investors gives a good sense of how they think Twitter can become a game-changer in real-time search. While it is instructive, it is also important to note that much of this vision has yet to materialize. Twitter’s current search is extremely crude, as Borthwick readily admits. It simply brings up the most recent Tweets with the keyword you are looking for. There is no ranking or clustering beyond that.

An undifferentiated thought stream of the masses at some point becomes unwieldy. In order to truly mine that data, Twitter needs to figure out how to extract the common sentiments from the noise (something which Summize was originally designed to do, by the way, but it was putting the cart before the horse—you need to be able to do simple searches before you start looking for patterns). But what is the best way to rank real-time search results—by number of followers, retweets, some other variable? It is not exactly clear. But if Twitter doesn’t solve this problem, someone else will and they will make a lot of money if they do it right.

The Death Of “Web 2.0″

I’m not going to discuss the economic meltdown and its devastating effect on technology companies and internet startups in this post, but rather something that crossed my mind earlier this morning: “Web 2.0″ seems to become more and more a void (and an avoided) term. Of course, that’s not necessarily a bad thing, but it is definitely apparent.

So why do I say it’s fading? For one, because the number of startups that contact us and include the term Web 2.0 in the subject line or message is visibly dropping (and that’s a good thing), and I hardly ever see it mentioned anymore on other technology blogs and news sites either. That’s not really tangible, so I took a look at the number of mentions of the phrase across the web, and they seem to be decreasing significantly, reflecting my feeling on this.

Judging by Google Trends, which shows how often a particular search term is entered relative to the total search volume across various regions of the world (and in various languages), the term started being used at the end of 2004 when Tim O’Reilly organized the first edition of the Web 2.0 Conference. Search queries for the term started picking up in the middle of 2005, when TechCrunch was started - with the tagline “Tracking Web 2.0″ by the way - and the number kept increasing until the end of 2007. After that, the trend is clearly downwards, falling back to the level it reached in early 2006 today. If the trend continues, there should only be a handful of people left who scour search engines for “Web 2.0″ by 2011.

Also noteworthy: take a look at the geographic regions that have generated the highest volumes of worldwide search traffic for the term over the years - it’s Asia, with the top 5 regions being India, Singapore, Hong Kong, Taiwan and Malaysia (in that order). Furthermore, Google Trends pegs the number one language in which people search for stuff related to the topic of Web 2.0 to be Russian before English.

And just in case you’re curious: “Web 3.0″ doesn’t seem to picking up much.
Let’s all rejoice.

Google’s “Insights for Search”, a beta service that analyzes a portion of worldwide Google web searches from all Google domains to compute how many searches have been done for the terms you’ve entered - relative to the total number of searches done on Google over time - gives an even better overview:

I’ve never had anything against the phrase “Web 2.0″, but I wouldn’t miss it a bit if it were never used again.

How about you?

MySpace Ad Revenues Closing In On AOL's

We just got off the phone with MySpace sales boss Jeff Berman, who also sent us this slide. Check it out; selling portal-like ads, MySpace revenues are quickly gaining on portals AOL and MSN.

Jeff says the success is all about MySpace being one of those rare places on the Web that's kind of a portal and kind of a social network. It can sell ads against everything from U2's latest single to photos from your backyard BBQ.

What impresses us is that MySpace is making all this money while shaking off a reputation "as a place for friends " that's also home to sleazy advertisers. Because they don't want their brands to share that reputation, Madison Avenue execs tell us they sometimes warn clients against buying MySpace ads.

But suddenly, there's McDonald's, Toyota, State Farm and Sony. They're all on MySpace.

Jeff says MySpace cleaned up its image by creating distinct areas of the site for advertisers to sponsor, but also by poaching top execs from rivals like MTV and eBay. From Yahoo, Jeff says he hired "Wenda Harris Millard's secret weapon."

 "We've made a very conscious effort to hire leaders from best in breed companies."

(It also helps that unlike Facebook, MySpace is willing to hand over huge tracts of its pages to sponsors.)

Anyway, here's more charts chronicling MySpace's rapid march on the traditional portals:

 

 

iTunes, we have a storage problem (and a potential fix) » VentureBeat

Late last year I bought an Apple TV. It’s a great device for playing digital content in the living room, but it has some limitations. Some of those I laid out recently in explaining why Apple TV is at a crossroads, but the single biggest problem going forward for such a device, and Apple’s expansion into video content overall, is storage. Now, a solution for that problem may be near.

The problem: It’s a matter of storage

The problem is that video files, especially high definition video files, take up a lot of space. Obviously, Apple and the movie and television studios that provide the content to serve over iTunes, want you to purchase and download as much as possible, but given the amount of storage space available not only on the Apple TV but on computers themselves, it’s impossible to have a huge collection. For example, say you bought just one season of ABC’s hit show Lost in HD on iTunes. That will take up over 30 gigabytes of hard drive space. And since you can’t rent television shows, it’s there for good on your hard drive. You could delete it after watching it, but why would you? You paid to own it.

Again, that is 30GB for one season of one show. When you consider that the Apple TV is now my primary means of television watching, and that I’ve bought about six different seasons of various shows so far, it should be no shock that I recently had to invest in a 1 terabyte (roughly 1,000GBs) external hard drive. Yes, just for iTunes.

Even if you opt to get the non HD version of shows, that same season of Lost is over 10GB in standard definition. And what about movies? Well luckily (for your hard drive), you don’t yet have to option to buy movies in HD (only rent them, and they are deleted after you watch them), but a standard definition movie still takes up between 1 and 2 gigabytes per film. When I say that I have over 30 films already in my iTunes collection, added to my TV shows, you can see the problem.

The situation is untenable. The more popular video on iTunes becomes, the more selective you’re likely going to have to be in buying something, not because of cost, but because of space. That’s a bad situation for everyone. Even the largest desktop hard drives usually top out at 500GB to 1TB — when HD movies start coming to iTunes to buy, that storage won’t last long either.

The solution: iTunes Replay in the cloud?

But there may be a solution. Apple is supposedly preparing something called “iTunes Replay,” AppleInsider reports today. The feature, which would be included in the next version of iTunes, would allow you to store your video files bought through iTunes remotely on Apple’s servers and stream them to your devices as you require. Think of it kind of like Netflix’s Watch Instantly service, except that apparently, you would own the actual content.

This distinction is important because it could allow you to download the films if you know you are going to be offline for something like a flight. Or if you wanted to download the film to your iPod or iPhone, it seems likely that it would be possible to do. Or if you simply have a poor Internet connection and need to download a movie for a better experience while watching, but want to delete the movie locally afterwards, that would probably be possible as well.

In short, it sounds like a great idea. And a solution to the problem I laid out above.

But what would it mean for Apple? Well, online storage is relatively cheap and getting cheaper all the time. The storage of the movies themselves probably wouldn’t be an issue, since Apple already does store them somewhere for users to download from the iTunes store right now. But it’s the streaming that could cause Apple some headaches. Streaming isn’t cheap, and if you’re Apple, attempting to become the go-to place for all movies and television shows online, the bills could mount for such a service, fast.

So would Apple pass the costs on to customers? AppleInsider isn’t sure if Apple would charge for such a service, but I bet that it would. Perhaps it would make the charge a flat yearly fee like its MobileMe cloud service, or maybe it would even tie it directly in to MobileMe, to make that service more useful to customers and expand its base.

Recent reports had Apple teaming up with Limelight, a digital content delivery network (CDN) to help it distribute both its iTunes and OS X content. Apple already has a deal with Akamai, and is probably using the two CDNs together now to offer the best possible experience for the customer, as AppleInsider notes. It’s certainly possible that Apple did this deal with Limelight ahead of this iTunes Replay service.

The iTunes lock-in

Some users will undoubtedly worry that if the movies they buy will stay on iTunes’ servers, that they’ll be locked in to iTunes. Well I’ve got news for you: You’re already locked into iTunes if you’re buying video content from there. Aside from some music videos, all of iTunes video content has DRM, even with the music side of the service getting rid of it. Get used to it. The movies studios, unlike the music labels, are not going to budge from that anytime soon.

The more interesting question for me is whether Apple will allow you to upload movies that you obtain through other means (maybe ripping them, maybe through something like Amazon’s video service) to their servers to hold your entire collection? While the immediate thought is “no,” remember, that’s basically what helped iTunes, the software, take off. It could house all your digital music, not just the stuff bought from iTunes.

There have been rumors of other solutions, such as a home server with a huge amount of storage that you would set up in your home to pipe your iTunes video content to all of your computers and devices. This online solution seems to make more sense, because you could access it from anywhere, probably over the Internet, without having to remotely connect to something in your home, which may be confusing for some users.

And if the service is entirely in the cloud, perhaps you could start buying movies and television shows on your iPhone and stream them to your device instantaneously. Right now, you can only buy music on your iPhone through iTunes.

I hope this iTunes Replay solution pans out. As storage is starting to shift from traditional hard drives to solid-state drives, that are generally faster and more stable, but are much more expensive, and thus come in smaller storage sizes for now, we’re simply not going to be able to keep up with the storage demands for HD TV shows and movies being downloaded. To the cloud we go!