Amazon enhances S3 storage with next generation cloud technology
by Doc on Apr.01, 2011, under Science
Amazon’s Web Services group today announced a breakthrough in a new cloud storage technology that will allow for massive amounts of data to be stored in a global network, and gets right down to the molecular level.
“We’re literally encoding the data by combining molecules into a matrix,” said lead scientist Dr. Rane Scamall, whose team has been developing the technology in the company’s Dublin-based labs. “By chaining the molecules in a certain way, we are able to encode the bits in the smallest ever domain, allowing for near-infinite storage – certainly enough to store the entirety of human knowledge.”
The details of how it works are still vague (mastermind Jeff Bezos is trying to accelerate the patent filings), but the system is envisioned as a near-line medium in a hierarchical storage system. “Although the density of data can be quite high, there are limits to the overall density of the molecules themselves within the medium,” Dr. Scamall explained. “If we try to increase the density too much, we can have phase transition issues and it becomes harder to decode the data. So currently the data migration rates are quite slow. We have plenty of volume to work with, however.”
Data that isn’t needed rapidly eventually gets “floated” to new storage system. While in this state, it can migrate freely to other systems until it is time to be recalled via “currents”. When it is time to be recalled, an electrical field is applied, which sets the system into motion, the molecules combine and begin to precipitate out of the medium and into collectors. More traditional channels can then carry the data to where it is actually needed.
There are still issues to be worked out, however.
“One problem we are still trying to solve is that the data precipitation is chaotic. Various environmental factors can cause perturbations in the flow, and the data could in fact be received in the wrong collector – or miss it entirely.” Dr. Scamall does add, “We don’t technically lose any of the data, but it can be very hard to find. Eventually it will find it’s way back into the cloud where we can attempt to retrieve it again.
Currently the technology works with water molecules, which are already quite abundant on Earth, but it could extend to other more complex molecules – an important need if the technology is to extend beyond the bounds of our planet – or if there are issues on availability on this one. Dr. Scamall quipped, “Water is both plentiful and scarce on our world, so we wouldn’t want our technology to trump other needs. But in reality, this technology has been around for centuries – we are only now learning how to harness it more effectively. And eventually we’ll have it pefect, as right as rain.”