Now we’re getting to the real crux of the issue when it comes to data storage. I’m really excited about all of the responses from the last two editorials on data storage engines. They all show insight, experience and perception. Today I want to pull out the comment from online because it clarifies some of the concepts regarding what is being accomplished.
A reader using the handle meilenberger62 writes:
I don’t think the problem is that people are trying to press data into relational data storage because that is how it’s done. I think the obverse is true, that people are trying to press their data into object models because it’s more "modern" and solves a perceived problem.
I wouldn’t call the technology that Google uses a "hierarchical database". It’s not a database at all. It’s a method to process and store any type of data, structured or otherwise. So I don’t think we should conflate hierarchical data, object models, relational databases with the Google problem.
What Google did was implement a Map/Reduce technology to solve a data problem on a massive scale.
But as all things, Map/Reduce became passe because of scaling problems. It’s been replaced with a hyper-scale analytics system called Cloud Dataflow.
So anyone who has been chasing Google and implementing Map/Reduce processes because it was "modern" needs to throw it all away because it’s not modern or scalable anymore.
I agree that big table (the google storage engine) is not a hierarchical data store. The point I was trying to make, and it appears to have been effective, is that we haven’t come up with the best solution, for every situation, for all time, in the form of relational storage.
One thing for sure, I’ll be looking into Cloud Dataflow as that is a term new to me.
Thanks, everyone, for your contributions. There’s so much to know; so much I don’t know; and if you’re like me, I’m always thirsty for more.
Cheers,
Ben