Conversational Monitoring System

I do a lot of work with Web sites and people are often talking about using a CMS to make it easier to upload and place content. For them, CMS stands for Content Management System, but I thought it would be fun to revisit Conversational Monitor System – one of the first virtual machines that people really enjoyed using because of its flexibility and functionality.

Our story starts in the mid-1960s at IBM’s Cambridge Scientific Center, where CMS – then called Cambridge Monitor System – first saw the light of day running under CP-40, a VM-like control program, which then developed into CP-67. It provided a way of giving every user the appearance of working on their own computer system. So, every user had their own terminal (the system console for their system) as if they had their own processor, unit record device (what we’d call a printer, card reader, and card punch), and DASD (disk space). The control program looked after the real resources and allocated them, as required, to the users.

In 1972, when IBM released its 370-architecture machine, a new version of VM called VM/370 became available. Unlike CP/CMS, IBM supported VM/370 and CMS retained its initials, but was now known as Conversational Monitor System – highlighting its interactive nature. This commercial product also included RSCS (Remote Spooling Communication Subsystem) and IPCS (Interactive  Problem Control System), which ran under CMS.

VM itself was originally not very popular within IBM, but, through quite an interesting story, survived. VM/370 became VM/SE and then VM/SP. There was also a low-end variant called VM/IS. Then there was VM/SP HPO before we had VM/XA SF, VM/XA SP, then VM/ESA, and now z/VM.

But returning to CMS, it was popular because you could do so much with it. You could develop, debug, and run programs, manage data files, communicate with other systems or users, and much more. When you start (IPL CMS) CMS, it loads a profile exec, which sets up your virtual environment in exactly the way that you want it to be.

Two products that made CMS users very happy were PROFS and REXX. PROFS (PRofessional OFfice System) became available in 1981 and was originally developed by IBM in Dallas, in conjunction with Amoco. It provided e-mail, shared calendars, and shared document storage and management. It was so popular that it was renamed OfficeVision and ported to other platforms. OfficeVision/VM was dropped in 2003, with IBM recommending that users migrate to Lotus Notes and Domino, which it had acquired by taking over Lotus.

REXX (Restructured Extended Executor) is an interpreted programming language developed by Mike Cowlishaw and released by IBM in 1982. It is a structured, high-level programming language that's very easy to get the hang of. In fact, it was so popular, that it was ported to most other platforms. REXX programs are often called REXX EXECs because it replaced EXEC and EXEC2 as the command language of choice.

CMS users will remember XEDIT, which is an edit program written by Xavier de Lamberterie that was released in 1980. XEDIT supports automatic line numbers, and many of the commands operate on blocks of lines. The command line allows user to type editor commands. It replaced EDIT SP as the standard editor. Again, XEDIT was very popular and ported to other platforms.

CMS provided a way to maximize the number of people who could concurrently use mainframe facilities at a time when these facilities were fairly restricted. It was a hugely successful environment, spawning tools that themselves were ported to other platforms because they were so successful. I used CMS and VM a lot back in the day, and even wrote two books about VM. Like many users, I have very fond memories of using CMS and what could be achieved by using CMS.

Big SQL

Suppose you wanted to access ‘big data’ stored in HDFS or HBase. What would you do? Well, for many people, the first step is to find out what we’re talking about. So, let’s start with big data – it’s data that’s so large and complex that it’s difficult to process using standard and familiar database management tools or applications.

According to Wikipedia, there are issues around data capture, curation, storage, search, sharing, analysis, and visualization. You’re probably thinking: why not go back to using smaller and manageable data? It seems that people want access to larger and larger amounts of data because additional information can be gained from it – allowing people to “spot business trends, determine quality of research, prevent diseases, link legal citations, combat crime, and determine real-time roadway traffic conditions”.

Now that’s clear, what are HDFS and HBase? HDFS stands for Hadoop Distributed File System. It’s a distributed, scalable, and portable file system written in Java for the Hadoop framework. HDFS stores large files across multiple machines, and replicates the data across multiple hosts. HBase is an open source, non-relational, distributed database and is also written in Java. It was developed as part of Apache Software Foundation’s Apache Hadoop project and runs on top of HDFS (Hadoop Distributed File System), providing a fault-tolerant way of storing large quantities of data.

Each node in a Hadoop instance typically has a single namenode; a cluster of datanodes form the HDFS cluster. So what’s needed is some way to access that cluster. At the moment, the choices are basically Hive, Impala, and Big SQL.

Again, a search on Wikipedia informs me that “Hive supports analysis of large datasets stored in Hadoop-compatible file systems such as Amazon S3 filesystem. It provides an SQL-like language called HiveQL while maintaining full support for map/reduce. To accelerate queries, it provides indexes, including bitmap indexes. By default, Hive stores metadata in an embedded Apache Derby database, and other client/server databases like MySQL can optionally be used. Currently, there are three file formats supported in Hive, which are TEXTFILE, SEQUENCEFILE, and RCFILE”

The Cloudera Impala project allows users to query data, whether stored in HDFS or HBase – including SELECT, JOIN, and aggregate functions – in real time. Furthermore, it uses the same metadata, SQL syntax (Hive SQL), ODBC driver, and user interface (Hue Beeswax) as Apache Hive. To avoid latency, Impala circumvents MapReduce to directly access the data through a specialized distributed query engine.

When you look up information about these things, names like Apache, Cloudera, Amazon, Facebook, Google crop up, but not IBM. You might think that’s a bit strange. Wouldn’t IBM be the organization you’d expect to have experience of big data? I mean just think of those massive IMS databases. So, why haven’t I mentioned IBM? The answer is because I haven’t got to Big SQL yet.

IBM claims that Big SQL provides robust SQL support for the Hadoop ecosystem:

•  it has a scalable architecture;
• it supports SQL and data types available in SQL '92, plus it has some additional capabilities;
• it supports JDBC and ODBC client drivers;
• it has efficient handling of ‘point queries’;
• there are a wide variety of data sources and file formats for HDFS and HBase that it supports;
• And, although it isn’t open source, it does interoperate well with the open source ecosystem within Hadoop.

The really interesting thing about this is that all the information is available in one place – Big Data University (http://bigdatauniversity.com). I’m looking forward to taking the course. Big data isn’t going away any time soon.

Virtual IMS Connection user group - under threat

Virtual IMS Connection is a vendor-neutral, independently-operated, virtual user group for IMS (IBM's Information Management System) professionals. It can currently be found at www.virtualims.com. It was launched at the beginning of November 2007.

Virtual user group meetings take place every other month and a guest speaker gives a presentation relevant to the IMS user community. The user group meetings last a little over an hour in total. Participation in meetings is free to members. The Virtual IMS Connection user group is also free to join.

User group members also get a bi-monthly newsletter. You can see the flip-book version here and the PDF version here.

The Web site contains details of future meetings, information about IMS-related articles (published elsewhere on the Web) and events, a list of IMS tools (software - the listing is free to vendors) and services, a list of IMS consultants, and a job bank (with resumes - CVs - and job opportunities).

New IMS products and new versions of existing products are always included in the News page. There's a link to the IMS-L listserv, where IMS professionals can ask questions and share information.

Members can access PDF versions of presentations and PDF versions of newsletters.

The idea was to create a one-stop shop for IMS professionals, where individuals using IBM’s IMS hierarchical database and transaction processing systems could exchange information, learn new techniques, and advance their skills with the product.

Obviously, there are sponsorship opportunities available to vendors of IMS-related products to help fund this work. Vendors can place banner adverts, etc. However, since its inception, NEON Enterprise Software has provided Web hosting for the site, and funded the WebEx technology that we have used to make the regular webinars possible, as well as meeting other costs. This is now coming to an end, and the Virtual IMS Connection user group is looking for a new home.

If your organization might be able to help keep this important user group running, please contact me on trevor@itech-ed.com.

You can also find the user group on Twitter at twitter.com/VirtualIMS.
You can become a fan on Facebook at www.facebook.com/#/pages/Virtual-IMS-Connection-user-group/282385116069.

Let's hope that the Virtual IMS Connection user group can continue to share ideas, information, and skills into the future.

The recession is over, IBM is in profit!

Well, half right at least! IBM has reported its first quarter results for 2010 saying that profits are up 13 percent to $2.6 billion and sales are up 5 percent at $22.9 billion. As a consequence, IBM has raised its profit guidance for the year by 20 cents to $11.20 per share.

The UK managed an 8 percent increase in revenues. The Americas had 6 percent growth. Europe, as a whole, provided revenue growth of 5 percent. And Asia/Pacific provided 4 percent revenue growth.

In terms of hardware – provided by the Systems and Technology Group – sales were up by 5 percent. However, System z mainframe sales fell by 17 percent. This can be explained by the fact the new z11 mainframes are due later this year and people will be holding of purchasing until those new mainframes become available. Trouble-makers may suggest that hardware sales are down because people are running their mainframe applications on TurboHercules boxes!

Software grew by 10.6 percent to just over $5 billion. Sales of Tivoli management software increased by 23 percent. WebSphere middleware sales climbed by 13 percent. Income from Rational development tools was up by 7 percent. Disappointingly, but perhaps not surprisingly, Lotus software increased by only 1 percent.

Global Services was responsible for $13.7 billion of sales resulting in a 4 percent increase.

But life isn’t all good news for IBM with that antitrust claim from the European Commission brought by TurboHercules. It was only a couple of weeks ago that I was talking about the 10th birthday of Linux on a mainframe – and Linux is open source software. IBM seems unable to make up its mind whether it is the king of open source (Linux) or the arch-enemy of Intellectual Property rights in the open source community (TurboHercules). IBM claims that its battle with TurboHercules in Europe is a result of the hidden hand of Microsoft pulling the strings.

I’m pleased to see IBM making a profit. I earn my living by working with people who work on IBM mainframes. However, recently, I have had a strange nagging doubt. What if you could run mainframe software on a reasonably beefy x86 box with some kind of emulator like TurboHercules? What if the whole mainframe hardware strategy that IBM has been driving for all these years is little more than a house of cards? If that were the case, then no wonder IBM is doing its best to get rid of any company offering low-entry access to mainframe software. I wonder...