The Big Picture of Big Data: Why using it is imperative
by Patricia Rae Linn
Organizations have or have access to plenty of data that can give them extreme differentiation and competitive edge, but most companies, government agencies and not-for-profits struggle with turning data into insights that enable timely, actionable, effective and measurable strategy. Even the organizations that do optimize enterprise, machine, social and other data battle challenges that make the Big-Data-to-Intelligent-Enterprise (i.e.) translation and application undesirably slow and expensive.
By the year 2020, IDC (International Data Corporation) predicts Big Data will reach “40 zettabytes (ZB), which is 40 trillion GB of data, or 5,200 GB of data for every person on Earth.” Currently only 1% of Big Data is analyzed.[i]
The “Big Data Imperative,” for organizations is to master insight-based strategic planning and execution using whatever elements of the massive data available apply to do what they do faster and better than their competitors, and their success in doing so requires that they understand and respond to the four “Vs” of Big Data:
- Value — It’s imperative that organizations have the ability to identify what data has value to them, where to obtain it, what the value is, how to extract and analyze that data for actionable meaning, and how to implement its use in such a way as to create a data feedback loop that measures results.
- Volume — With billions of meaningful transactions, communications and results of processes being generated every second, anyone who intends to use Big Data must have the facility and capacity to manage the data from storage through prioritization to analysis. Speed to insight is the key to this weighty process giving the organization their desired differentiation outcomes.
- Variety — Variety is abundant in Big Data, in both source and content as well as multi-purposability; for example, any single data string may analyze out to indicate the resulting insights apply to more than one division, process, goal, etc. of a company. Consequently, variety occurs at both the input and output stages of the data-to-insight-to-action process, and organizations must have systems in place to complete the process successfully.
- Velocity — The speed at which data is created and comes available is nothing short of daunting to both humans and information processing machines. Those who manage data have to make fast and tough decisions about the size of data samples needed, when the data stream starts and is shut off, how much integrity it has and if it is worthy of analysis, and even whether it has been compromised in any way that might put an organization’s decision making or system health at risk.
Ultimately, every organization that takes the Big Data Imperative to heart and recognizes it is the cornerstone of future business success must become an Intelligent Enterprise (i.e.) that culminates the best practices in people, process and technology for competitive differentiation. As such, the data management professionals in successful organizations must understand, be capable of acting upon and manage fast and accurate systems that deal with the following:
Why the Imperative
Every industry and every business function within each industry is influenced daily by big data.[ii] Leaders in every sector will have to grapple with the implications of big data, not just a few data-oriented managers. [iii] (McKinsey Group)
Simply put, Big-Data-driven insight will be what drives business by 2020. For every organization the core, fundamental, four benefits of using Big Data analytics for strategy-driving insight are:
- Customer Growth and Retention: Data-driven insight improves organizations’ ability to identify the right customers, attract them, convert them, retain them by preempting defection, transform them into brand ambassadors that sell the organization voluntarily… all by allowing company teams to hear exactly what customers want. how they want it, and what they thing of their experience with the company and reacting to tat information on a timely basis.
- Profitability and Sustainability: By selling the right value to the right customer the right way, every time, organizations reduce the waste of returns, unwanted inventory and doing customer dis-service damage control, increase volume, increase margins and keep products and services fresh and in demand based on customer feedback… all these profoundly and sustain-ably impact the bottom line in a positive way.
- Process Improvement and Change Management: Intelligent, data-driven enterprise virtually eliminates the guesswork in innovation, sales and marketing and creation of the customer experience, thereby streamlining those and other processes into ones where the pathway to the anticipated outcomes is clear, concise and the outcome results are predictable. When outcomes are clear at the outset and have inspiring dividends, and team members know what pathway will deliver them, getting them to align their efforts with the goals and processes, thus changing the ways they do things, becomes easy.
- Product and Service Innovation and R&D: The customer is (almost) always right. While great innovators develop products and services customers want, those consumers create data that makes their needs and desires very clear, and their disappointments tangible… a solid, communicative customer base can virtually do away with the necessity to manage and compensate high-end innovation teams.
About Big Data
- Big Data comes from almost everything… Sensors used to gather climate information, posts to social media sites, digital pictures and videos, transaction records, cell phone GPS signals, and more.
- Each day, people create 2.5 quintillion bytes of data, both intentionally and accidently; for example by sending 1 bil. Tweets daily and 30 bil. messages on Facebook each month.
- 90% of all data is unstructured, has a lot of noise built in, is frequently corrupted, and almost always has value.[iv] Organizations will have to have app-driven analytic infrastructures or access to them as a service.
- Big Data Statistics[v]:
- By 2014, 90% of Big Data was created in 2012 and 2013
- In 2012 the Big Data market was valued at $10.2 bil.; it will be a $53 bil. market by 2017
- 70% of Big Data is generated by people not machines
- Organizations self-store about 80% of the data they create and use.
- The Internet of Things experiences the addition of 570 new websites — data sources and generators — every minute of every day.
- By 202, 33% of Big Data will be in the cloud.
- A 10% increase in data accessibility translates into an additional $66 mil. in net income for the average Fortune 1000 company.
- 65% of senior executives are basing management decisions on Big Data analytics insight.
Trends and Technologies
- Data Storage/theCloud — With annual increases of 60% in data growth, storage demand is increasing too rapidly for technology to keep up. Cloud technologies, the current “big hope” for storage and dissemination of big data offers performance well below 30-minute-transfer times for 1 terabyte of data (considered good performance for single-drive read).
- Ability to combine Big Data sets and still obtain insightful information via complex queries quickly because of increased processing capability and speed.
- Open-source data-integration tools that allow diverse data types and sources to be blended into analysis-ready data sets of any scope.
- “The emergence of the “industrial Internet” — GE’s term for the Internet of things — and big data platforms. The challenge, he said, is creating open platforms for ingesting and sharing high-scale data, and building applications while also ensuring security.” (Jim Fowler, CIO GE Water & Power)[vi]
- 2014 will be the year of: What can I do with big data? rather than: What is big data? [vii]
- Real time in-memory analytics, complex event processing and ETL will combine: Serial extract, transform, load (ETL) processes will largely go away in 2014. As the velocity of data increases, especially social data, theres more need to analyze data in real time as a stream.[viii]
- Master data management (MDM) single-definition data, now exclusively internal, will become external as well. This data will define situations or dimensions the solve big problems.
Not Only SQL (NoSQL), developed to facilitate data analysis without strict parameters and concrete schema, will further develop into a master core system (or a few), thus consolidating NoSQL access/services.
In Summary… Business has entered and Information Imperative Age that is rooted in the massive data generated by people and the machines they use. Analyze or Atrophy is fast becoming a mantra organizations need to follow to compete on every level of business.
Join us on April 23, 12 Noon ET.
The Digital Universe in 2020: Big Data, Bigger Digital Shadows, and Biggest Growth in the Far East by John Gantz and David Reinsel sponsored by EMC, http://www.emc.com/leadership/digital-universe/iview/executive-summary-a-universe-of.htm [ii] The Economist 2010, Data, data everywhere, A special report on managing information[iii]
[iv] IBM 2010, http://www.eiseverywhere.com/file_uploads/dbece17417f38de495ce73d330adf4bc_MON_1015am_Big_Data_Track_TUTORIAL_David_Barnes.pdf[v] Surprising Statistics About Big Data, Dennis McCafferty, 2014, http://www.baselinemag.com/analytics-big-data/slideshows/surprising-statistics-about-big-data.html [vi] 3 Trends Driving Big Data Breakthroughs: A CIO’s View, InformationWeek 2014,
http://www.informationweek.com/big-data/software-platforms/3-trends-driving-big-data-breakthroughs-a-cios-view/d/d-id/1174139[vii] Trends in Bg Data: A Forecast for 2014, Andy Walker, CSC,
http://www.csc.com/big_data/publications/91710/105057-trends_in_big_data_a_forecast_for_2014[viii] Trends in Bg Data: A Forecast for 2014, Andy Walker, CSC,
Connect with us in LinkedIn: