Viacom won a ruling requiring Youtube to give all of its user data to Viacom.

Google will have to turn over every record of every video watched by YouTube users, including users' names and IP addresses, to Viacom, which is suing Google for allowing clips of its copyright videos to appear on YouTube, a judge ruled Wednesday.

Viacom wants the data to prove that infringing material is more popular than user-created videos, which could be used to increase Google's liability if it is found guilty of contributory infringement.

So I understand why Google has to give Viacom something as part of discovery -- Viacom is alleging something about popularity and traffic, and they get to look at the data underlying the numbers. But I don't understand why they need names especially, or even IP addresses.

This sort of thing makes me think that companies like Google or Yahoo! really ought to delete things like IP addresses, or trapdoor filter them so that they aren't recoverable. It's not that I worry about what Google's nefarious plans are; it's that if the data exists, the government or other agencies can force Google to turn it over. If it's not there, that's not a problem.

From Google's perspective, they can at least keep their source code private:

Viacom also requested YouTube's source code, the code for identifying repeat copyright infringement uploads, copies of all videos marked private, and Google's advertising database schema.

Those requests were denied in whole, except that Google will have to turn over data about how often each private video has been watched and by how many persons.

[Note: much of this post is talking about the pre-Panama world for Yahoo!. The whole point of Panama is that it will close this gap, but that it will take time. If my analysis is right, then Yahoo! is doing exactly what it needs to do to close the gap. I should have made that more clear in the original post.]

jgische asked the other day why Google is doing so much better than Yahoo!. The answer is a little subtle if you aren't in the search business, so here's my take on it. The main difference is that Google makes more money per search -- about 20-25 cents per query -- than Yahoo, which makes 10-11 cents per query. [These numbers came from here but are currently wrong, and may have been wrong at the time.]

The first thing that people think is that Google's search results are better, so that translates into more money. That might help to explain the market share difference -- Google has a bit over half the US search market; Yahoo has about 20% (Neilsen stats). But Google makes a ton more money per search than Yahoo! does, so the gap in market share doesn't explain the revenue difference. Also, no external measures of quality that I've seen find that Google's results are better than Yahoo's (the last two I read found Yahoo to be better, though both had some methodological problems). From the standpoint of the organic search results, Google and Yahoo are tied.

So what's going on? The difference is not in the search results; the difference is in the quality of the ads. Google's ads are way better than Yahoo's. To see why, we need a little bit of history.

Ads in search are based on keyword matches. If you want to place an ad, you write the ad, then you buy the keyword, and the ad will show up somewhere on the page when you run a search. (Incidentally, this was patented by Overture, which Yahoo! later bought. Google licensed the patent from Yahoo for 2.7 million shares of Google stock.)

In the old days, how well your ad placed depended on how much you paid. Willing to pay more per click? You get placed higher. Bidding was what mattered for ad placement. Both search engines did that for years.

However, some bright spark at Google realized that to maximize revenue, what matters isn't the price per click, but rather the price per click times the clickthrough rate. If you have an ad that gets twice the clickthrough at the top spot than some other ad, you can charge that buyer half as much and make the same amount of money. So that advertiser needs to spend much less to get the top spot. The other way to think about that is that Google offered advertisers a discount for quality. If you have a relevant site and a good ad, Google said, you can pay much less per click to get good placement on our page.

This turned out to have a huge impact on search revenue over time. What happened is that advertisers started to compete on ad quality, and over time, the ads became more and more relevant to the searches. In fact, after a while, users who were previously disinclined to click on ads learned that the ads could be useful after all, and they started to click more. So Google started to ramp up its monetization, pulling away from Yahoo, and in the process, making the search experience better for its users. Even if Google's results weren't any better, the overall experience was better because the ads were so much higher quality that they were like having extra useful results, whereas the old-style Yahoo ads were still more of an annoyance than an asset.

So where was Yahoo! in all of this? Well, they realized pretty quickly that they needed similar technology, and so they had Overture (remember them, the ad patent?) create a new system. This was a long project, planned for a year, because of the approach they took.

There are two ways to find out what the CTR*price number will come out to. One is to occasionally vary the order of the results, and see if the CTR changes. If the new ad gets a high CTR*price, then you can promote it. However, this isn't ideal: it takes a while for you to figure out which is the best ad, and advertisers change their ads; plus, since you're sampling, you can have all kinds of sampling errors or biases. You can also notice when the CTR profile of the ads is unusual -- if the second ad is getting many more clicks than you expected relative to the first ad, then you might promote it. I think this is how Google initially started their system, though it's not what they do now (or not only what they do now).

The other approach is to have a system that can estimate the CTR of an ad before it gets shown, based on the ad text, the query, and attributes of the page. If your estimate is good, then you just calculate the estimated CTR*price, and you have an ordering. This gives advertisers quick feed back on how their ads are predicted to do, and then you can adjust based on actual CTRs once the ads are out. (Here's a paper from WWW 2007 on this problem.)

Yahoo decided to skip the earlier order experimentation and CTR observation method and went straight to the predictive model. This was a big effort, an attempt to skip the early steps and jump straight to the end in an effort to leapfrog Google. More critically, they decided to do it along with a whole revamping of their entire advertising platform, revamping the whole way that advertisers bought any kind of ads, not just search ads, from Yahoo. As happens with big projects, it hit some problems and came out late, but with surprisingly high quality and functionality.

So has Yahoo caught up? Not yet. The reason is that marketplaces take time to work, and Google has had an ad quality marketplace for years, whereas Yahoo has only had one for a few months. Google's ads are therefore higher quality, and Google's users are used to high quality ads so are more likely to click on them. Yahoo will continue to catch up as time passes, but the gap is huge: several years in Silicon Valley is an eternity, and that is the head start that Google has in search monetization.

This monetization also translates directly into market share. Here's how. Google makes 20 cents or so per query, whereas Yahoo makes half that. So when AOL solicits bids to provide search technology, Google can bid twice what [more than] Yahoo can and still make a profit.

So there you have it. Google is making much more money than Yahoo not because of search result quality, but because of ad quality. Usually if you take care of your core business, serving good results, the rest will take care of itself. Google figured out how to turn advertising, formerly parasitic on the core business, into a way to actually help customers. Ads became part of the core business, and revenue skyrocketed.

Note: I work for Yahoo, but none of this is corporate opinion. I've used only publicly accessible information for this posting. Not that anyone will care, but you never know.