PNUTS - Platform for Nimble Universal Table Storage
Before joining Yahoo! Research, Raghu Ramakrishnan founded an Internet startup called Quiq. The technology was similar to Yahoo! Answers in that website visitors could post questions, and anyone on the planet could answer them.
Ramakrishnan found that conventional database technologies could not adequately handle the demands of a hosted Web application like Quiq. When he arrived at Yahoo! in 2006, he discovered that product groups at the company faced similar challenges.
During brainstorming sessions with colleagues Philip Bohannon and Brian Cooper, Ramakrishnan wondered whether they could design a simpler, hosted database platform that product groups could plug into on an as-needed basis—without having to stitch together a mosaic of backend database platforms or worry about custom engineering.
The group responsible for the design and development of PNUTS has grown rapidly. In Research, the group now includes core members Utkarsh Srivastava and Ramana Yerneni, as well as Minos Garofalakis and Ben Reed, who collaborate on specific issues.
Also, what began as a research project quickly drew the attention and support of other groups. Platform Engineering, for example, joined the collaborative effort with a team led by Mike Bigby that includes Marc Abramowicz, Bryan Call, Andy Feng, Pauli Wang, and Dan Weaver, with additional members set to join this summer.
Bigby's group was already acutely aware of the need for improved shared-service infrastructure in Yahoo! through their experience building and running UDB - the company's world-wide object data store that allows properties to store and retrieve information about Yahoo!'s users in real-time. The team was building a distributed hash-table platform (YDHT) that would significantly generalize UDB and reduce operations cost when it recognized the synergy with the PNUTS project.
"How quickly the company decided that PNUTS needed to go forward, and how it responded, is remarkable," says Bigby. "The project was put in co-development mode with Platform Engineering, a really novel arrangement, to ensure fast innovation. This demonstrates tremendous flexibility on the part of the company, and it's also a huge vote of confidence in Yahoo! Research."
The combined effort aims to develop a family of data hosting services that will allow application developers to choose the right kind of storage and indexing service for each of their data collections. Individual product groups will no longer have to buy their own database technology or manage an army of administrators. In effect, they can store their vital data by renting space on the hosted PNUTS/YDHT cluster, which ideally will be deployed across all Yahoo!
For instance, if a particular application turns out to be a smash hit, the group could simply access more resources on the cluster, rather than going out and buying 1,000 new machines and then struggling to deploy them. Not only does this bring the total cost of database management down dramatically, it could also offer faster performance, better information retrieval features and improved quality of service.
"We set about jointly developing a massively distributed, highly scalable data management infrastructure, upon which the company can build new applications very easily," Ramakrishnan explains. "A key goal is to sharply reduce the overhead of creating and maintaining new applications. We also envision future generations of some existing products migrating to this infrastructure."
But how did the project come to be named after a salty treat? "We were sitting at Brian’s desk wondering what to call the project, and Brian happened to have a jar of peanuts on his desk," Ramakrishnan says. "We didn't want to waste time thinking of a clever name, so we decided to call it PNUTS for the time being. And the name just stuck."
Later, to fill out the acronym, the team dubbed the project Platform for Nimble Universal Table Storage. Naturally, everyone still calls it PNUTS.
See what Amin Vahdat, Professor in Computer Science and Engineering at UC San Diego, had to say about PNUTS - http://idleprocess.wordpress.com/2009/08/09/yahoos-geo-replication-service-pnuts/.
