Data Usage – A Great Analysis
I really liked the analogies and information in this post from Jill – good food for thought. Wanted to be sure to pass it along.
"You’ve just begun to scratch the surface of a huge issue that is not new at all. However, the problem is growing as data becomes more easily accessible. I disagree that the resolution lies within the DBA’s realm of responsibilities. To a large degree the responsibility for the appropriate use of data to create information belongs to the vendor creating solutions that use data or the Analyst creating reports and/or maps. They need to verify that the data is appropriate for the intended use and they should have training in the analytical practices and data standards for their industry. They should also document what they consider appropriate uses for the application/report/map and, hopefully, the limitations. Also, there is a responsibility for the end-user to have read the fine print and understand the implications.
I don’t know if you have run across the FGDC (http://www.fgdc.gov/), you should check it out. The FGDC is committed to the development, use and sharing of data, mainly geographic, but the principles can be applied to non-spatial data. A key component of this effort is documentation. Yes, metadata. We all hate filling it out, but it truly is indispensable. Metadata combined with professional knowledge on the appropriate use of data for analysis helps to minimize the creation of poor information generated from data.
Another contributing factor to poor information that we have seen in our shop, is the lack of understanding(or respect) by the general populace about the complexities of data analysis. There are now tools available for the desktop and web that make the application of analysis tools very easy. Many users are lulled by the ease of the interface into thinking that just because an error doesn’t come up and they get results, that the results are good. Unfortunately, these tools just perform the computations; they don’t make determinations about the suitability of the data or the analysis. Similarly, people think just because they have the data, that it’s all good. Well, if data is collected for the purpose of valuing land, that doesn’t mean it is good for determining environmental constraints. We have a tongue-in-cheek saying here: Best Available Data – BAD.
In short, I view the DBA role as the modern equivalent of the librarian. Amongst other things we store the data, protect it, and ensure it is documented and accessible to the correct people. We shouldn’t be the ones determining the best use of the data – as in many cases we do not know the data intimately enough to make that determination. It is the analyst or vendor that needs to be ethical and knowledgeable about how they use the data so as to present useful, accurate information. However, ultimately, the end-user, needs be skeptical about the results and if it is a critical application, do some independent research and become knowledgeable themselves. "
Featured White Paper(s)
EMC Solutions for Microsoft SQL Server 2005: EMC CLARiiON CX3 Series Physical iSCSI
With increasing demands and limited resources, today’s midsize enterprises face challenges similar to their larger counterpar… (read more)
Making Databases Explorable
This paper looks at common difficulties encountered when maintaining a large number of databases and how the products in the … (read more)