data mining

As the year comes to a close, it's time to look at the big events, memorable moments and emerging trends that captivated us in 2008. As it happens, studying the aggregation of the billions of search queries that people type into the Google search box gives us a glimpse into the zeitgeist--the spirit of the times. We've compiled some of the highlights from Google searches around the globe and hope you enjoy looking back as much as we do.

Recently my husband and I received nearly identical balance-transfer offers from our respective Bank of America cards. The offers were identical, that is, except for the rates we'd be given. He was enticed with a 0% rate. Mine was 2.99%.

We live at the same address and share the same income. We both have high credit scores (although his are, annoyingly, a few points higher than mine).

So are these different offers evidence of rampant sexism on BofA's part? Hardly. The pitches were the result of complex and largely secret scoring systems that most financial institutions use to boost profits while limiting losses.

You've heard by now of credit scores, the three-digit numbers lenders use to gauge your creditworthiness. Credit scores predict how likely you are to default on a credit account or loan; they're used to help set interest rates and terms.

What you may not know is that credit scores are just the start of the way financial institutions evaluate you, and they're not even the most commonly used scores -- far from it.

While a credit card issuer might check your credit scores once a month as part of its regular account review process, the same company probably checks other kinds of scores every time you pull out your plastic.

Last month VanityFair.com published an enlightening article that unfortunately got lost in the holiday shuffle and didn't get a lot of play. It's worth reading so I want to highlight it here.

The article examines a voter registration data broker named Aristotle, which buys voter registration lists from counties and states. It then combines that information with highly personal and detailed information about voters that it mines from various other sources before reselling the data to candidates, political operatives, and commercial entities.

The company has quietly become the largest voter registration data broker in the country and is the go-to source for people like political mastermind Karl Rove to learn everything about you, such as how much money you make, whether you own a gun and potentially even what medical procedures you've had done.

Unlike data brokers such as Choicepoint and Axciom, Aristotle has managed to remain under the radar for more than two decades. But it's just as powerful as these other brokers, if not more, because the information it sells has the ability to directly influence presidential elections and public policy decisions.

1. Get a friend of yours to download, using Facebook, the ten most popular books at every college (manually -- as not to violate Facebook's ToS). These ten books are indicative of the overall intellectual milieu of that college.
2. Download the average SAT/ACT score for students attending every college.
3. Presto! We have a correlation between books and dumbitude (smartitude too)!

Welcome to Wonkosphere, the best place to keep a finger on the pulse of the 2008 Presidential election. We use patented technology to scour the blogosphere and analyze what is being said, who is saying it, and whether they're ranting or raving. Updated every 4 hours.

I especially like their "conservative buzz", "liberal buzz", "independent buzz" groups. Be interesting to see how this site does over the next year. (And what they do with themselves, if successful, once the election is over.)

"It started in China," she went on, finally. "Once we moved our servers onto the mainland, they went under Chinese jurisdiction."

Greg sighed. He knew Google's reach all too well: Every time you visited a page with Google ads on it, or used Google maps or Google mail—even if you sent mail to a Gmail account—the company diligently collected your info. Recently, the site's search-optimization software had begun using the data to tailor Web searches to individual users. It proved to be a revolutionary tool for advertisers. An authoritarian government would have other purposes in mind.

Is there a link between donations given and bills passed? MAPLight.org aims to help you find out, giving you the ability to compare contributions with how legislators voted.

WikiScanner, developed by California Institute of Technology graduate student Virgil Griffith, traces the IP addresses that get stored when anonymous editors make their changes to Wikipedia articles and links them back to the organizations they came from. So, if U.S. Senate staff members were to repeat the editing efforts they undertook last year to remove information that made certain senators look bad, for example, everyone would know it.

This is an awesome way to data mine already available data (due to Wikipedia's open processes) and mash it up. This list of Fox's changes to Wikipedia is pretty interesting.