Suppose you wanted to access ‘big data’ stored in HDFS or HBase. What would you do? Well, for many people, the first step is to find out what we’re talking about. So, let’s start with big data – it’s data that’s so large and complex that it’s difficult to process using standard and familiar database management tools or applications.
According to Wikipedia, there are issues around data capture, curation, storage, search, sharing, analysis, and visualization. You’re probably thinking: why not go back to using smaller and manageable data? It seems that people want access to larger and larger amounts of data because additional information can be gained from it – allowing people to “spot business trends, determine quality of research, prevent diseases, link legal citations, combat crime, and determine real-time roadway traffic conditions”.
Now that’s clear, what are HDFS and HBase? HDFS stands for Hadoop Distributed File System. It’s a distributed, scalable, and portable file system written in Java for the Hadoop framework. HDFS stores large files across multiple machines, and replicates the data across multiple hosts. HBase is an open source, non-relational, distributed database and is also written in Java. It was developed as part of Apache Software Foundation’s Apache Hadoop project and runs on top of HDFS (Hadoop Distributed File System), providing a fault-tolerant way of storing large quantities of data.
Each node in a Hadoop instance typically has a single namenode; a cluster of datanodes form the HDFS cluster. So what’s needed is some way to access that cluster. At the moment, the choices are basically Hive, Impala, and Big SQL.
Again, a search on Wikipedia informs me that “Hive supports analysis of large datasets stored in Hadoop-compatible file systems such as Amazon S3 filesystem. It provides an SQL-like language called HiveQL while maintaining full support for map/reduce. To accelerate queries, it provides indexes, including bitmap indexes. By default, Hive stores metadata in an embedded Apache Derby database, and other client/server databases like MySQL can optionally be used. Currently, there are three file formats supported in Hive, which are TEXTFILE, SEQUENCEFILE, and RCFILE”
The Cloudera Impala project allows users to query data, whether stored in HDFS or HBase – including SELECT, JOIN, and aggregate functions – in real time. Furthermore, it uses the same metadata, SQL syntax (Hive SQL), ODBC driver, and user interface (Hue Beeswax) as Apache Hive. To avoid latency, Impala circumvents MapReduce to directly access the data through a specialized distributed query engine.
When you look up information about these things, names like Apache, Cloudera, Amazon, Facebook, Google crop up, but not IBM. You might think that’s a bit strange. Wouldn’t IBM be the organization you’d expect to have experience of big data? I mean just think of those massive IMS databases. So, why haven’t I mentioned IBM? The answer is because I haven’t got to Big SQL yet.
IBM claims that Big SQL provides robust SQL support for the Hadoop ecosystem:
- it has a scalable architecture;
- it supports SQL and data types available in SQL '92, plus it has some additional capabilities;
- it supports JDBC and ODBC client drivers;
- it has efficient handling of ‘point queries’;
- there are a wide variety of data sources and file formats for HDFS and HBase that it supports;
- And, although it isn’t open source, it does interoperate well with the open source ecosystem within Hadoop.
The really interesting thing about this is that all the information is available in one place – Big Data University (http://bigdatauniversity.com). I’m looking forward to taking the course. Big data isn’t going away any time soon.
What is a social business? Wikipedia tells me that a social business was first defined by Prof Muhammad Yunus as a non-loss, non-dividend company designed to address a social objective within the highly regulated marketplace of today. A wider definition includes any business having a social rather than financial objective. Or a social enterprise can be defined as a commercial activity by socially minded organizations. Perhaps a better definition these days is a business that has adopted social networking tools and practices for internal and external functions across their organization.
The IBM Web site tells us that: “Social business technologies help people connect, communicate, and share information. Becoming a leader in your marketplace means using social solutions to transform how business gets done – driving cost savings, increasing revenue, and cultivating competitive advantages.” That definitely sounds like a good thing.
In a report from Altimeter entitled The Evolution of Social Business: Six Stages of Social Business Transformation, it’s suggested that many social strategies are not linked to business goals, but are instead often driven by a “social for social’s sake” orientation. The report notes that, even when goals are aligned, social initiatives frequently must deal with a lack of defined strategy, governance, or funding. The report suggest six stages a company has to go through to become a social business:
- Planning – understand how customers use social channels and prioritize strategic goals where social can have the most impact.
- Presence – amplify existing marketing efforts and encourage sharing
- Engagement – drive considerations to purchase and provide direct support with internal employee engagement.
- Formalized – set governance for social, create discipline and process, and have strategic business goals.
- Strategic – scale across business units and move into HR, sales, finance, and supply chain.
- Converged – social drives transformation and integrates social philosophy into all aspects of the enterprise.
Becoming a social business is not something that can be left for tomorrow, organizations need to act now. They need to identify what it is they do well, and they need to get the message out there. And that last part comes with two health warnings! You need to be using the same social media as your clients and potential clients. If they don’t use the same media, they won’t see the message. Secondly, keep your eye on new media. Facebook, Twitter, Youtube, LinkedIn, Google plus may not always be kings of the heap. Pinterest is currently very popular. Watch out for trends – so you don’t lose that link with clients and potential clients. Maybe you should be looking at smartphone apps that will make it easy for people to come to you for whatever it is you supply.
Interestingly, there are products like Yammer (www.yammer.com), which Microsoft bought last year, Chatter, and others, that offer private social networks for organizations.
IBM VPs Jeff Schick and Sandy Carter were on stage at IBM Connect 2013 showing off the capabilities in IBM’s social business platform. It seems that companies are combining social technologies, the cloud, mobile, and analytics to create a flexible, intelligent framework for making the most of social connections. In an IBM survey of CEOs, 57% picked social business as a top priority and nearly three quarters (73%) are making significant investments in analysing data.
According to Jeff Schick: “A disconnect between how we live our lives – through digital connections and sharing – and how we work isn’t sustainable. Particularly since the new generation of workers who will fill companies’ ranks don’t know any other way of interacting and working than social media. The rate of productivity, innovation, and employee retention will increasingly depend on who can master this coming age of social business.”
For an organization to just have a social media strategy isn’t enough. The organization must transform itself into a social business to ensure its continued success.
