I found this article about personalization and marketing using big data to be really interesting. It was talking about some trends that are addressing why personalization just isn’t quite there yet. Some of the statements were really revealing in real-world terms for how big data projects are progressing. Here’s an example:
One reason marketing messages and offers are still so off-base is because companies are doing quasi-personalization rather than actual personalization. They’re using a group approach to marketing that targets individuals, but it inherently includes some false assumptions. Demographics help, personas help, and the combination of the two is better than either alone. Nevertheless, more companies are embracing sophisticated analytics and data science to improve the accuracy of their efforts even more.
This talks to one of the things I’ve seen a lot as people build out solutions that work with large datasets to make decisions. Many times it seems, people rely on buckets or modeling against the data. While this seems to provide a “handle” on using the information, it also potentially waters down the usefulness of the information.
A favorite thing of software designers (and marketers as well, as mentioned in the reference) is to create a persona. This persona is used to group and target behaviors. This might be a “typical” target user, customer or other person that interacts with your solutions.
It’s a good way to get your head around requirements, and it’s used a lot too to get a handle on marketing. It provides for some nice targeting buckets that make it easier to work with groupings of people rather than individuals.
But this is some of the gold in the big data information stores that we’re all managing. The gold is in the individualization. This might be to a specific person, but it’s also individualization to a specific situation, scenario or goal. When we group things to manage them more easily, we water down potential results by ignoring specifics, particularly unique specifics, in our use of the information.
I think one of the challenges going forward is going to be using the information in ways that take full advantage of the specificity of it all. To be able to apply the full breadth of information and gained knowledge to the target, whether it’s a customer, a problem to solve or other investigation you may be doing.
This sounds really great – being able to use specific information for a given situation without generalization. But it’s tough, at the same time.
It’s hard because it means being able to query against, report on and analyze very large chunks and data sets of information – often in real-time with queries, ad-hoc queries and more. It means architecting the data store to support really digging in and pulling specific information bits to the surface, rather than summaries.
I think this is the next clear direction – that specificity of information that is used in decisions. Rather than talking about sales trends even for a specific product, we may need to go down even deeper into combination scenarios – days of the week, months of the year, neighborhoods, types of customers or customers that do other types of business with us. By going to a detail level, rather than saying “customers that like bikes” and filtering results, I think we gain huge value out of the information.
But as data folk, it’s going to be very challenging to make this possible and make it perform well and protect the information in use. I’m starting to see this a lot with systems we’re working with – the expectation is to start at a summary level, but then support drilling down into data sets, re-relating data sets and so-on to explore relationships and draw conclusions. In many cases, these are some of the most expensive queries to support, processing-wise.
Be sure to keep an eye out for those times where you can architect support for a very detailed, very intricate access system for your data. I think it’s definitely something that we’ll all be working through quite a bit moving forward.