Tell me about NoSQL

NoSQL seems to be the buzzword of choice at the moment for people who want the flexibility to build and frequently alter their databases. But there are plenty of people who still aren’t quite sure what a NoSQL database is and why they should want to use it. So let’s take a brief overview of NoSQL.

The term, NoSQL, first saw the light of day in 1998 when Carlo Strozzi used it as the name of his lightweight, open-source, relational database because it didn’t expose the standard SQL interface. But the term gained its modern usage in 2009 when it was used as a generic label for non-relational, distributed, data stores. So, it refers to a whole family of databases, rather than a single type of database.

Developers like NoSQL because they can store and retrieve data without being locked into the tabular relationships used in relational databases. It makes scaling easier and they provide superior performance. They can store large volumes of structured, semi-structured, and unstructured data. They can handle agile sprints, quick iteration, and frequent code pushes. They use object-oriented programming that is easy to use and flexible. And they use efficient scale-out architecture instead of expensive monolithic architecture.

But, on the down side, NoSQL lacks ACID (Atomicity, Consistency, Isolation, Durability) transaction support. Atomicity means that each transaction is ‘all or nothing’, ie if one part of the transaction fails, the entire transaction fails, and the database state is left unchanged. Consistency ensures that any transaction brings the database from one valid state to another. Isolation means that the concurrent execution of transactions results in a system state that would be obtained if transactions were executed sequentially. Durability means that once a transaction has been committed, it will remain so, even in the event of power loss, crashes, or errors. And that’s the kind of reliability you want in a business-critical database.

NoSQL databases are typically used in Big Data and real-time Web applications. The different NoSQL database technologies were developed because of the increase in the volume of data that people needed to store, the frequency the data is accessed, and increased performance and processing needs.

There are estimated to be over 150 open source databases available. And there are many different types of NoSQL database, including some that allow the use of SQL-like languages – these are sometimes referred to as ‘Not only SQL’ databases. Classify NoSQL databases is quite a challenge, but they can be grouped, by the features they offer, into column, document, key-value, and graph types. Alternatively, they can be classified by data model into KV Cache, KV Store, KV Store - Eventually consistent, Data-structures server, KV Store – Ordered, Tuple Store, Object Database, Document Store, and Wide Columnar Store.

The good news for DB2 users is that IBM has provided a new API that supports multiple calls and a NoSQL software solution stack that ships with DB2. It’s free with DB2 on distributed platforms and with DB2 Connect. DB2 also offers a second type of NoSQL-like database – the XML data store. This can store the growing volume of Web-based data.

Rocket Software has a way of using MongoDB (an example of a NoSQL database) on a mainframe. Rocket can provide access to any System z database using any MongoDB client driver. DB2 supports MongoDB.

IBM recently announced Zdoop, Hadoop database software for Linux from Veristorm on System z mainframes, stating: “This will help clients to avoid staging and offloading mainframe data to maintain existing security and governance controls”.

Other NoSQL databases that you might want to look out for include Cassandra, CouchBase, Redis, and Riak.

Clearly, with the growth in Big Data, we’ll be hearing a lot more about NoSQL databases and how they can be integrated into mainframe technology. There are lots of them out there and they can be quite different from each other in terms of their features and data models used.

DB2 goes big and mobile

So where’s all the excitement in computing these days? If your answer is Big Data and mobile apps then you’ll be fascinated by the latest DB2-related news.

Let’s start at the big end. IBM announced DB2 Version 10.5 recently and included in it are a set of acceleration technologies code-named BLU – apparently standing for Big data, Lightning fast, and Ultra easy! BLU is a bundle of new techniques for columnar processing, data deduplication, parallel vector processing, and data compression – everything you’d need if you were working on Big Data in memory.

So, BLU enables databases to be “memory optimized” – which means that they will run in memory, but not everything has to be put in memory. BLU has also removed the need for hand-tuning SQL queries to optimize performance.

IBM is saying that this new version of DB2 can speed up data analysis by over 25 times. That means databases don’t need to be sized so they everything fits in memory, and there’s no need to purchase separate in-memory databases for fast data analysis and transaction processing jobs. IBM has been showing an example of a 32-core system using BLU technology executing a query against a 10TB data set in less than a second.

This kind of processing ability makes DB2 a better choice in some cases than using Hadoop. The data is compressed in the order in which it is stored, allowing predicate operations to be executed without decompressing the data set. The software also keeps a metadata table that lists the high and low key values for each data page or column of data. The advantage of this is that when a query is executed, the database can check whether any of the required values are on the data page.

IBM is using BLU in its DB2 SmartCloud IaaS (Infrastructure as a Service) to add power for data analysis and data reporting jobs.

Meanwhile, DB2 and MongoDB are getting together to announce a new standard to make it easier for organizations to implement data-intensive apps for the Web and mobile devices. MongoDB, you say, what’s that? MongoDB is owned by 10gen and utilizes NoSQL database technology. It’s used for lots of mobile and Web apps.

Developers will be able to use Eclipse tools with IBM Worklight Studio to integrate MongoDB APIs using the MongoDB query language. That allows developers to more easily query JSON (JavaScript Object Notation) documents in DB2. JSON documents are frequently used for storing Web-based data. A NoSQL database allows data to be added without a predefined schema and allows a wider range of choices when scaling up.

The plan is that later this year developers will be able to combine the WebSphere eXtreme Scale data grid platform with MongoDB, and they’ll be able to run MongoDB apps directly on DB2. Developers will be able to write apps using MongoDB’s query language to interact with data stored in DB2 and WebSphere, making the vast amount of data in IBM data stores available to modern application environments. IBM hopes to broaden the API and is already working on open source code for security, extended transaction support, and extended join support, among others.

So DB2 is growing at the big end of the database world and the little (mobile) end. Interesting!