The growth in big data has been monumental, and continues to accelerate. We are awash in data, but often bereft of insight. It brings to mind the old adage, “Water, water, everywhere, but not a drop to drink.” Analytic tools are typically inadequate, and while visualization helps in the processing of data, true insight is often lacking.
The tasks faced by most companies are manifold – how to identify and capture data; how to classify and index it; how to handle data that ranges from structured and formatted to unstructured and unformatted; how to make this data accessible to the people that need it, when they need it, and so on. All of these issues typically relate to capture, storage, access, and retrieval – all of which are non-trivial but ultimately solvable. Horizontal scaling and vertical stacking are only two of the many techniques that financial institutions use in handling big data.
The challenge, however, lies in analyzing data. For this, most institutions need to develop a big brain, which is easier said than done. In fact, what is a big brain – is it just sheer size of a data engine? Is it speed? Is it complexity? Is it all of the above? How can a financial institution develop a big brain? The answer may lie in biological phenomenon such as compound eyes (many small eyes working to piece parts into a whole), in computing arrays of parallel processors, or even organizational structures of conglomerates.
We know that a company like GE is a conglomerate and has many different businesses. However, we think of companies like Amazon and Google as being single business companies. Amazon has multiple businesses within it, including a second-hand book company (Abebooks), a publisher (Kindle), a download company (Audible), a newspaper (Washington Post), a movie database company (IMDb), several shoe companies, and many others. To handle Big Brain needs of Big Data, Amazon has actually broken the problem into small parts, developed multiple data stores, and uses mining techniques both within and across these data stores to drive insight.
So the question then becomes not only “How can financial institutions develop a big brain?” but “What big brain is right for this financial institution?”, which may actually be the harder question.
Sandeep Vishnu is a Partner in Capco’s North American Finance, Risk, and Compliance practice. He focuses on enterprise risk management, financial analysis, data, risk analytics/modeling, business intelligence/reporting, compliance, capital, and operational/control issues.
The content and opinions posted on this blog and any corresponding comments are the personal opinions of the original authors, not those of Capco or FIS.