Dealing with Lots of Data
It’s not just making your database bigger, it’s more about architecture.
As I’ve been talking to different people about what they’re seeing with databases, it’s becoming clear that we can do a better job planning for more data to be managed in our systems. The intriguing challenge I’ve started to see is that, while data storage is clearly not cost-prohibitive, the impact of a huge amount of data is more than people plan on in many cases.
Performance is impacted, maintenance and recovery are impacted and much more… it’s something that takes some unique attention when it comes to building an architecture that supports the data loads we see and will continue to see going forward.
A few things to get you started:
– What are the plans for a prolonged recovery? If you’re faced with recovering/restoring a database that has grown substantially (don’t forget, it also means restoring differentials and transaction logs), are you prepared to take that time for the restore? Are your users?
– Do your applications support splitting your database up? You could archive things that are older, for example, saving the space allocated to them. Be sure you’ve investigated this with 3rd party applications too. This may impact your licensing as many applications consider an archive database a second installation – be sure to understand your licensing models.
– How long DOES a restore take? Don’t know? Find out. Do a restore. Test it. Time it. Make sure you know exactly what it’s going to take. That window of downtime can be substantial. There are options, but you need to know this baseline. Knowing this timeframe will help you determine if you need additional disaster recovery plans (perhaps a warm standby server or other options).
– Do you have information that is simply old and no longer useful at this point? It’s worth looking into. These days there are so many automated data bits flying around that may have far less value at a point in the future. Consider some archive routines so these don’t simply continue to add to the recovery planning.
What types of things are you doing as your databases grow? What has helped? What has been painful?
Let me know…
swynk@sswug.org