Anyone who’s been on holiday this week would probably have been holidaying on Mars to have missed Tuesday’s excited announcement from IBM of its new zBC12 mainframe computer! The zBC12 follows last year’s announcement of the zEC12 (Enterprise Class), 2011 saw the z114, and 2010 gave us the z196. So what’s special about the new baby?
Well, the zEnterprise BC12 (zBC12) features a 4.2GHz processor, and, for cloud computing, it can consolidate up to 40 virtual servers per core or up to 520 in a single footprint. It offers similar capabilities to the IBM zEC12, including specialty processors. There’s the Integrated Facility for Linux (IFL) engine for running Linux applications, and the System z Application Assist Processor (zAAP) and System z Integrated Information Process (zIIP) for off-core workload processing. The big difference is that the zBC12 has a maximum memory of 496GB compared to the zEC12’s maximum of 3TB, and it has more limited connectivity options.
If you need to ask the price, as they say, you can’t afford it, but a new zBC12 starts at $75,000, which is a very good price tag. IBM also claims that it can help customers save up to 55% over their x86 distributed environments.
One big difference with this year’s announcement is that IBM is also launching a Linux-only based version of the zBC12, which it’s calling the Enterprise Linux Server (ELS). This is aimed at first-time zEnterprise users. The deal is that the product includes hardware, hypervisor, and three-years of maintenance service, plus, it can be upgraded to analytics and cloud mainframe products. I think that’s called upselling!
IBM also released Version 6.3 of z/VM, which supports up to 1TB of real memory, enabling support for more virtual servers than any other platform in a single footprint. It’s also enabled for OpenStack for advanced enterprise-wide service management. In addition, there’s z/OS Version 2.1, which, among all its other features, is very good at securing private cloud workloads.
IBM claims that when integrated with DB2 Analytics Accelerator, the zBC12 can perform business analytics workloads with response times up to nine times faster, with 10 times better price performance, and 14 percent lower total cost of acquisition than the closest competitor. I wonder who they mean by that??
For cloud computing, IBM has enhanced its OMEGAMON for z/OS family to better detect performance problems in the cloud and minimize impact to the business and increase analytics visibility.
The new z/OS 2.1 operating system (mentioned earlier) enables the latest zEnterprise hardware features, including zEDC (Enterprise Data Compression) and SMC-R (Shared Memory Communications over RDMA – and that one stands for Remote Direct Memory Access). The so-called ‘Crypto as a Service’ enables Linux on System z applications to use z/OS services to encrypt data, thereby providing more secure encryption. Additionally, enhancements to z/OS Management Facility improve start up times and provides services for automating workflow, further reducing costs.
Of course, you may have missed IBM’s baby announcement because the news was swamped with Kate and William’s baby prince!
Sunday, 28 July 2013
Sunday, 21 July 2013
Disappointing figures all round?
It seems that now might not be a good time to invest in IT firms with both IBM and Intel reporting drops – Intel saw second quarter profits down 29% from last year at $2bn, and IBM’s earnings fell by 17% to $3.23bn.
When you look at revenues, you’ll see Intel’s fell by 5% and IBM’s by 3%. You’ll remember that last quarter IBM cut a large number of jobs (over 3,000) and started to make a big deal of its Big Data strategy and cloud computing. Intel is really being hit because of the drop in PC sales as people migrate to smart phones and tablets – and around 95% of those use ARM chips.
IBM’s sales were down 3.3 per cent to $24.92bn, and IBM reported adjusted second-quarter earnings of $32.91 a share. It also raised its guidance for the year and said it expects at least $16.90 a share in adjusted earnings. IBM saw zero revenue growth in its “growth markets”, the so-called emerging markets. Its earnings per share were reasonable and that was due to the earlier cost-cutting measures. IBM announced $1 billion write-off for downsizing. Services sales dropped by 5%, hardware sales dropped by 12%, but software revenue went up by 4%. Intel’s shares fell 3.7% to $23.23 after the company cut its forecast for the year.
It was also bad news for eBay, whose shares fell 6.4% to $53.70 after announcing their second-quarter results.
SAP AG, one of the biggest makers of business-management software, experienced its first software-sales decline in more than three years. Software licences fell around 3% to 982 million euros. Its software sales in Asia dropped by 9%.
Back in May, PC vendor, Dell, reported a 79% drop in net profit, which they attributed to a crash in PC sales as consumers migrate to smartphones and tablets. Dell’s net profit fell to $130m on revenue down 2% to $14bn. Michael Dell is still waiting for a vote on his proposal to buy out the company. He plans to transform the company from a PC vendor to one pushing higher-margin software and services to enterprises. He’s apparently got a $2bn loan from Microsoft.
But it’s not all doom and gloom, doing well is Sandisk, who reported a 43% jump in second-quarter revenue. Yahoo! posted an attributable net income of $331.15m for the second-quarter, which is an increase of 46%, which it attributes to its investment in Chinese e-commerce firm Alibaba and product overhauls. However, its revenues declined by 7% to $1.13bn and that was attributed to a decrease in advertising revenues and Web traffic.
Interestingly, in the USA, tech advertising has increased by 30% with Microsoft, now being the biggest spender. Its spending on advertising has gone up 200% in an attempt to drum up business for Windows 8 and its Surface tablet. It will be interesting to see what effect this has on the company’s bottom line.
So, the picture isn’t all bad for IT companies, and some companies are trying to do something about it, and perhaps dividends are OK, but, taken together, the figures don’t make the happiest of reading for the IT industry as a whole.
When you look at revenues, you’ll see Intel’s fell by 5% and IBM’s by 3%. You’ll remember that last quarter IBM cut a large number of jobs (over 3,000) and started to make a big deal of its Big Data strategy and cloud computing. Intel is really being hit because of the drop in PC sales as people migrate to smart phones and tablets – and around 95% of those use ARM chips.
IBM’s sales were down 3.3 per cent to $24.92bn, and IBM reported adjusted second-quarter earnings of $32.91 a share. It also raised its guidance for the year and said it expects at least $16.90 a share in adjusted earnings. IBM saw zero revenue growth in its “growth markets”, the so-called emerging markets. Its earnings per share were reasonable and that was due to the earlier cost-cutting measures. IBM announced $1 billion write-off for downsizing. Services sales dropped by 5%, hardware sales dropped by 12%, but software revenue went up by 4%. Intel’s shares fell 3.7% to $23.23 after the company cut its forecast for the year.
It was also bad news for eBay, whose shares fell 6.4% to $53.70 after announcing their second-quarter results.
SAP AG, one of the biggest makers of business-management software, experienced its first software-sales decline in more than three years. Software licences fell around 3% to 982 million euros. Its software sales in Asia dropped by 9%.
Back in May, PC vendor, Dell, reported a 79% drop in net profit, which they attributed to a crash in PC sales as consumers migrate to smartphones and tablets. Dell’s net profit fell to $130m on revenue down 2% to $14bn. Michael Dell is still waiting for a vote on his proposal to buy out the company. He plans to transform the company from a PC vendor to one pushing higher-margin software and services to enterprises. He’s apparently got a $2bn loan from Microsoft.
But it’s not all doom and gloom, doing well is Sandisk, who reported a 43% jump in second-quarter revenue. Yahoo! posted an attributable net income of $331.15m for the second-quarter, which is an increase of 46%, which it attributes to its investment in Chinese e-commerce firm Alibaba and product overhauls. However, its revenues declined by 7% to $1.13bn and that was attributed to a decrease in advertising revenues and Web traffic.
Interestingly, in the USA, tech advertising has increased by 30% with Microsoft, now being the biggest spender. Its spending on advertising has gone up 200% in an attempt to drum up business for Windows 8 and its Surface tablet. It will be interesting to see what effect this has on the company’s bottom line.
So, the picture isn’t all bad for IT companies, and some companies are trying to do something about it, and perhaps dividends are OK, but, taken together, the figures don’t make the happiest of reading for the IT industry as a whole.
Sunday, 14 July 2013
Mainframe apps
Who doesn’t have a smart phone these days? Or who goes to meetings without a tablet device? I thought it would be interesting to look at the mainframe apps available for Android devices – and there are quite a few.
The Mainframe IBM Interview QA app provides a wide range of questions that can be asked during an interview. The application answers the most commonly asked interview questions and they’ve been grouped into categories for ease.
Test Your Mainframe Skills!!! claims it will help you to assess your mainframe skills as well as develop them. It’s aimed at people who are looking for a new job and want to hone their technical skills. This app is designed so that you can go through different technical tests, assess your skills, and, simultaneously, hints are provided for each question to help you to understand and learn the information in better way!
Similarly, Mainframe interview questions provides over a 100 interview questions for assistance in getting your next mainframe-related job.
Mainframe Translator costs $1.99, and makes it easier to translate mainframe error codes and even machine instructions without having to always look them up in a manual. It also allows you to view the messages and codes on your device without having to minimize your source code or error screen. You can choose a specific code to translate or choose a translation type and scroll through all of the codes. The current translation options are: ASCII, EBCDIC, Machine Instruction, COBOL File Status, CICS EIBRESP, CICS EIBFN, DB2 SQL Errors, and FTP Codes. It also functions as a calculator performing decimal, hexadecimal, and binary maths. The dual pane feature in landscape mode allows for calculator and translation codes to be seen on the same screen. Users can customize colours and default settings.
Mainframe Guides explains how to code mainframe applications. It includes all the topics that are needed to learn about mainframes with a basic programing knowledge. Currently this app plays tutorials in high quality video.
Emulator Access 3270 cost $5.99, and allows users to easily access their employer’s Web-based emulator and mainframe with this utility. BNSF users get automatic login and their username and password are saved. It’s compatible with all Open Connect / 3270 emulator systems common in transportation, government, manufacturing, and financial industries.
TN3270, costing $16.72, is a client for connecting to mainframes with the telnet 3270 protocol. Features include: connecting to standard 3270 mainframes; text size configuration through buttons; and keyboard configuration.
Mocha TN3270 for Android provides TN3270 emulation for IBM mainframe terminal access. The paid version includes keys f1-f24.
Julian Date Conv Calendar is a simple calendar that displays a whole month of both standard (Gregorian) and Julian days together in the same display for easy conversion between the two. It’s useful for those who work with Julian dates, eg mainframe programmers and schedulers.
I’m sure there are others, but those are the ones that leaped out at me when I searched for mainframe apps. I thought it was interesting to see what was currently available.
The Mainframe IBM Interview QA app provides a wide range of questions that can be asked during an interview. The application answers the most commonly asked interview questions and they’ve been grouped into categories for ease.
Test Your Mainframe Skills!!! claims it will help you to assess your mainframe skills as well as develop them. It’s aimed at people who are looking for a new job and want to hone their technical skills. This app is designed so that you can go through different technical tests, assess your skills, and, simultaneously, hints are provided for each question to help you to understand and learn the information in better way!
Similarly, Mainframe interview questions provides over a 100 interview questions for assistance in getting your next mainframe-related job.
Mainframe Translator costs $1.99, and makes it easier to translate mainframe error codes and even machine instructions without having to always look them up in a manual. It also allows you to view the messages and codes on your device without having to minimize your source code or error screen. You can choose a specific code to translate or choose a translation type and scroll through all of the codes. The current translation options are: ASCII, EBCDIC, Machine Instruction, COBOL File Status, CICS EIBRESP, CICS EIBFN, DB2 SQL Errors, and FTP Codes. It also functions as a calculator performing decimal, hexadecimal, and binary maths. The dual pane feature in landscape mode allows for calculator and translation codes to be seen on the same screen. Users can customize colours and default settings.
Mainframe Guides explains how to code mainframe applications. It includes all the topics that are needed to learn about mainframes with a basic programing knowledge. Currently this app plays tutorials in high quality video.
Emulator Access 3270 cost $5.99, and allows users to easily access their employer’s Web-based emulator and mainframe with this utility. BNSF users get automatic login and their username and password are saved. It’s compatible with all Open Connect / 3270 emulator systems common in transportation, government, manufacturing, and financial industries.
TN3270, costing $16.72, is a client for connecting to mainframes with the telnet 3270 protocol. Features include: connecting to standard 3270 mainframes; text size configuration through buttons; and keyboard configuration.
Mocha TN3270 for Android provides TN3270 emulation for IBM mainframe terminal access. The paid version includes keys f1-f24.
Julian Date Conv Calendar is a simple calendar that displays a whole month of both standard (Gregorian) and Julian days together in the same display for easy conversion between the two. It’s useful for those who work with Julian dates, eg mainframe programmers and schedulers.
I’m sure there are others, but those are the ones that leaped out at me when I searched for mainframe apps. I thought it was interesting to see what was currently available.
Sunday, 7 July 2013
IBM’s approach to Big Data
IBM has taken lots of the open source Big Data technologies – like Hadoop, MapReduce, HBase – and added its own technology – like Big Sheets, DB2, DataStage – to create something hugely more powerful.
IBM’s InfoSphere BigInsights builds on open source Hadoop capabilities for enterprise class deployments. The enterprise-level capabilities can be grouped together as: visualization and exploration, development tools, advanced engines, connectors, workload optimization, and administration and security.
IBM claims the business benefits are: quicker time-to-value because of IBM’s technology and support, reduced operational risk, enhanced business knowledge with a flexible analytical platform, and it leverages and complements existing software.
In terms of administration and security, the Web console can start and stop services, run and monitor jobs (applications), explore and modify the file system, and built-in apps make it easy to do common tasks.
The connectors link to databases like DB2, Netezza, Oracle, Teradata. And there’s integration with: InfoSphere Data Stage (data collection and integration), InfoSphere Streams (real-time streams processing), InfoSphere Guardium (security and monitoring), Cognos Business Intelligence (Business Intelligence capabilities), and IBM Platform Computing (cluster/grid infrastructure and management), and more. Big SQL is coming with BigInsights V2.1. This will provide SQL access to data stored in BigInsights through JDBC/ODBC and use rich standard SQL to leverage Map/Reduce parallelism or achieve low-latency.
Advanced engines include an advanced text analytics engine that can automatically identify and understand key information in text. Text Analytics is really useful because most of the world’s data is in unstructured or semi-structured text; social media is full of discussions about products and services; internal information in organizations is locked in blobs, description fields, and sometimes even discarded. It’s been suggested that over 80% of stored information is unstructured – such as e-medical records, hospital reports, case files, police records, emergency calls, tech notes, call logs, online media, insurance claims, Twitter, Facebook, blogs, and forums.
In terms of development tools, there is an Eclipse-based development environment for building and deploying applications. There are developer tools and a set of analytic extractors for fast adoption that reduce coding and debugging time by up to 30% (IBM claims). There are also plug-ins for text analytics, MapReduce programming, Jaql development, Hive query, etc.
Visualization and exploration has Big Sheets, providing Web-based analysis and visualization for users with a familiar spreadsheet-like interface that can define and manage long-running data collection jobs.
Meanwhile, Microsoft has identified Hadoop users as a useful market to get into. Speaking recently at the Hadoop summit, Quentin Clark, corporate VP of data platforms said: “We believe Hadoop is the cornerstone of a sea change coming to all businesses”.
Microsoft is integrating Hadoop with its products and services. And, Clark says that Microsoft intends to stick to the principles of open source by contributing to the Hadoop project, rather than simply using it and adding its own stuff. Hortonworks recently announced management packs for Microsoft System Center Operations Manager and Microsoft System Center Virtual Machine Manager – both products for administering the Hortonworks Data Platform (HDP) distribution.
Apparently Microsoft is positioning itself as a big data player with a powerful set of Business Intelligence (BI) tools. Data Explorer for Excel 2013 is a self-service BI add-in allowing users to import data from a variety of sources, including Hadoop. SQL Server 2012 Parallel Data Warehouse (PDW) is a massively parallel processing data warehousing appliance designed for Hadoop integration. Microsoft is also trying to bring Hadoop into the cloud using Windows Azure.
Businesses can’t ignore Hadoop, and the fact that major software vendors are getting behind it means it’s not going to be some flash-in-the-pan idea. Certainly, I can imagine major organizations looking to get a huge business advantage by embracing the technology now – to be ahead of their competitors. Smaller organizations will probably take a few years before they see a business case for it. By then the IBM products (and Microsoft’s) will be very mature and eminently suitable.
IBM’s InfoSphere BigInsights builds on open source Hadoop capabilities for enterprise class deployments. The enterprise-level capabilities can be grouped together as: visualization and exploration, development tools, advanced engines, connectors, workload optimization, and administration and security.
IBM claims the business benefits are: quicker time-to-value because of IBM’s technology and support, reduced operational risk, enhanced business knowledge with a flexible analytical platform, and it leverages and complements existing software.
In terms of administration and security, the Web console can start and stop services, run and monitor jobs (applications), explore and modify the file system, and built-in apps make it easy to do common tasks.
The connectors link to databases like DB2, Netezza, Oracle, Teradata. And there’s integration with: InfoSphere Data Stage (data collection and integration), InfoSphere Streams (real-time streams processing), InfoSphere Guardium (security and monitoring), Cognos Business Intelligence (Business Intelligence capabilities), and IBM Platform Computing (cluster/grid infrastructure and management), and more. Big SQL is coming with BigInsights V2.1. This will provide SQL access to data stored in BigInsights through JDBC/ODBC and use rich standard SQL to leverage Map/Reduce parallelism or achieve low-latency.
Advanced engines include an advanced text analytics engine that can automatically identify and understand key information in text. Text Analytics is really useful because most of the world’s data is in unstructured or semi-structured text; social media is full of discussions about products and services; internal information in organizations is locked in blobs, description fields, and sometimes even discarded. It’s been suggested that over 80% of stored information is unstructured – such as e-medical records, hospital reports, case files, police records, emergency calls, tech notes, call logs, online media, insurance claims, Twitter, Facebook, blogs, and forums.
In terms of development tools, there is an Eclipse-based development environment for building and deploying applications. There are developer tools and a set of analytic extractors for fast adoption that reduce coding and debugging time by up to 30% (IBM claims). There are also plug-ins for text analytics, MapReduce programming, Jaql development, Hive query, etc.
Visualization and exploration has Big Sheets, providing Web-based analysis and visualization for users with a familiar spreadsheet-like interface that can define and manage long-running data collection jobs.
Meanwhile, Microsoft has identified Hadoop users as a useful market to get into. Speaking recently at the Hadoop summit, Quentin Clark, corporate VP of data platforms said: “We believe Hadoop is the cornerstone of a sea change coming to all businesses”.
Microsoft is integrating Hadoop with its products and services. And, Clark says that Microsoft intends to stick to the principles of open source by contributing to the Hadoop project, rather than simply using it and adding its own stuff. Hortonworks recently announced management packs for Microsoft System Center Operations Manager and Microsoft System Center Virtual Machine Manager – both products for administering the Hortonworks Data Platform (HDP) distribution.
Apparently Microsoft is positioning itself as a big data player with a powerful set of Business Intelligence (BI) tools. Data Explorer for Excel 2013 is a self-service BI add-in allowing users to import data from a variety of sources, including Hadoop. SQL Server 2012 Parallel Data Warehouse (PDW) is a massively parallel processing data warehousing appliance designed for Hadoop integration. Microsoft is also trying to bring Hadoop into the cloud using Windows Azure.
Businesses can’t ignore Hadoop, and the fact that major software vendors are getting behind it means it’s not going to be some flash-in-the-pan idea. Certainly, I can imagine major organizations looking to get a huge business advantage by embracing the technology now – to be ahead of their competitors. Smaller organizations will probably take a few years before they see a business case for it. By then the IBM products (and Microsoft’s) will be very mature and eminently suitable.
Sunday, 30 June 2013
Interesting week!
It’s been an interesting week – the auld enemies, Microsoft and Oracle, have made a joint announcement, SPDY is finding its way into IE11, and IBM has been showing off its mastery of big data at Wimbledon. So let’s take a look at what’s been going on.
It’s not quite a return to the 1960s with love breaking out all around, but large organizations are realizing that customers and, more importantly, potential customers, use hardware and software from other vendors – and the best way to do more business is to recognize this and be, at the very least, compatible. And so, with this in mind (I expect), Microsoft and Oracle have joined forces in a new cloud venture that means Microsoft’s Windows Azure cloud-computing service will now run Oracle’s database software, Java programming tools, and application-connecting middleware. Azure customers will be able to run Java, Oracle Database, Oracle WebLogic Server, and even Oracle Linux on Windows Server Hyper-V or Windows Azure, and Oracle will deliver full certification and support.
The benefits are that Microsoft gains additional customers for Azure, and Oracle gains customers for their technology who want to use it in the cloud. And it gives Microsoft something that VMware and Amazon Web services don’t have. Companies using Azure cloud service, which lets them build and run programs online, will be able to put information into Oracle’s database.
With many organizations thinking seriously about moving to the cloud, this alliance provides better choices for IaaS (Infrastructure as a Service), allowing them to rent computing power, storage, and database software over the Internet.
Also this week, Microsoft announced that Internet Explorer 11 will support SPDY (pronounced speedy), the Google-backed protocol for speeding up downloads from Web sites. The open source networking protocol achieves this by prioritizing and multiplexing the transfer of Web page subresources, so that only one connection per client is required. It uses Transport Layer Security (TLS) encryption and transmission headers are compressed. Faster Web pages has got to be a good thing – Firefox and Chrome already use it.
Meanwhile, Google has built QUIC (Quick UDP Internet Connections) into developer versions of Chrome. It’s an alternative to TCP (Transmission Control Protocol) and is designed to cut the round-trip time of the back-and-forth communications between computers on the Internet. User Datagram Protocol (UDP) is faster than TCP, but doesn’t have TCP’s error-checking reliability. QUIC is based on UDP and now provides its own error-correction technology.
So what new things could IBM bring to Wimbledon? It’s been providing up-to-date information for 24 years now, so what’s different in 2013? Well, the answer is more social media involvement.
Last year, it seems, there were around 100 tweets per second during the men’s final, won by Roger Federer against Andy Murray. So this year, IBM is providing social sentiment analysis, using its content analytics software. That means it can, for example, gauge how popular Andy Murray is in different parts of the UK!
IBM’s SPSS predictive analytics software is at the core of SlamTracker, which deploys a mixture of predictive analytics software, data visualization, and kinetic tracking to see what people consistently did when they won. IBM’s Second Sight technology measures player speed, distance, and stamina. This year it’s integrated with HawkEye, the ball-tracking and line-calling technology.
So an interesting week all round.
It’s not quite a return to the 1960s with love breaking out all around, but large organizations are realizing that customers and, more importantly, potential customers, use hardware and software from other vendors – and the best way to do more business is to recognize this and be, at the very least, compatible. And so, with this in mind (I expect), Microsoft and Oracle have joined forces in a new cloud venture that means Microsoft’s Windows Azure cloud-computing service will now run Oracle’s database software, Java programming tools, and application-connecting middleware. Azure customers will be able to run Java, Oracle Database, Oracle WebLogic Server, and even Oracle Linux on Windows Server Hyper-V or Windows Azure, and Oracle will deliver full certification and support.
The benefits are that Microsoft gains additional customers for Azure, and Oracle gains customers for their technology who want to use it in the cloud. And it gives Microsoft something that VMware and Amazon Web services don’t have. Companies using Azure cloud service, which lets them build and run programs online, will be able to put information into Oracle’s database.
With many organizations thinking seriously about moving to the cloud, this alliance provides better choices for IaaS (Infrastructure as a Service), allowing them to rent computing power, storage, and database software over the Internet.
Also this week, Microsoft announced that Internet Explorer 11 will support SPDY (pronounced speedy), the Google-backed protocol for speeding up downloads from Web sites. The open source networking protocol achieves this by prioritizing and multiplexing the transfer of Web page subresources, so that only one connection per client is required. It uses Transport Layer Security (TLS) encryption and transmission headers are compressed. Faster Web pages has got to be a good thing – Firefox and Chrome already use it.
Meanwhile, Google has built QUIC (Quick UDP Internet Connections) into developer versions of Chrome. It’s an alternative to TCP (Transmission Control Protocol) and is designed to cut the round-trip time of the back-and-forth communications between computers on the Internet. User Datagram Protocol (UDP) is faster than TCP, but doesn’t have TCP’s error-checking reliability. QUIC is based on UDP and now provides its own error-correction technology.
So what new things could IBM bring to Wimbledon? It’s been providing up-to-date information for 24 years now, so what’s different in 2013? Well, the answer is more social media involvement.
Last year, it seems, there were around 100 tweets per second during the men’s final, won by Roger Federer against Andy Murray. So this year, IBM is providing social sentiment analysis, using its content analytics software. That means it can, for example, gauge how popular Andy Murray is in different parts of the UK!
IBM’s SPSS predictive analytics software is at the core of SlamTracker, which deploys a mixture of predictive analytics software, data visualization, and kinetic tracking to see what people consistently did when they won. IBM’s Second Sight technology measures player speed, distance, and stamina. This year it’s integrated with HawkEye, the ball-tracking and line-calling technology.
So an interesting week all round.
Sunday, 23 June 2013
DB2 goes big and mobile
Let’s start at the big end. IBM announced DB2 Version 10.5
recently and included in it are a set of acceleration technologies code-named
BLU – apparently standing for Big data, Lightning fast, and Ultra easy! BLU is
a bundle of new techniques for columnar processing, data deduplication,
parallel vector processing, and data compression – everything you’d need if you
were working on Big Data in memory.
So, BLU enables databases to be “memory optimized” – which
means that they will run in memory, but not everything has to be put in memory.
BLU has also removed the need for hand-tuning SQL queries to optimize
performance.
IBM is saying that this new version of DB2 can speed up data
analysis by over 25 times. That means databases don’t need to be sized so they
everything fits in memory, and there’s no need to purchase separate in-memory
databases for fast data analysis and transaction processing jobs. IBM has been
showing an example of a 32-core system using BLU technology executing a query
against a 10TB data set in less than a second.
This kind of processing ability makes DB2 a better choice in
some cases than using Hadoop. The data is compressed in the order in which it
is stored, allowing predicate operations to be executed without decompressing
the data set. The software also keeps a metadata table that lists the high and
low key values for each data page or column of data. The advantage of this is
that when a query is executed, the database can check whether any of the
required values are on the data page.
IBM is using BLU in its DB2 SmartCloud IaaS (Infrastructure
as a Service) to add power for data analysis and data reporting jobs.
Meanwhile, DB2 and MongoDB are getting together to announce
a new standard to make it easier for organizations to implement data-intensive
apps for the Web and mobile devices. MongoDB, you say, what’s that? MongoDB is
owned by 10gen and utilizes NoSQL database technology. It’s used for lots of mobile
and Web apps.
Developers will be able to use Eclipse tools with IBM
Worklight Studio to integrate MongoDB APIs using the MongoDB query language.
That allows developers to more easily query JSON (JavaScript Object Notation)
documents in DB2. JSON documents are frequently used for storing Web-based
data. A NoSQL database allows data to be added without a predefined schema and
allows a wider range of choices when scaling up.
The plan is that later this year developers will be able to
combine the WebSphere eXtreme Scale data grid platform with MongoDB, and
they’ll be able to run MongoDB apps directly on DB2. Developers will be able to
write apps using MongoDB’s query language to interact with data stored in DB2
and WebSphere, making the vast amount of data in IBM data stores available to
modern application environments. IBM hopes to broaden the API and is already
working on open source code for security, extended transaction support, and
extended join support, among others.
So DB2 is growing at the big end of the database world and
the little (mobile) end. Interesting!
Sunday, 16 June 2013
Getting started with Big Data
So, you’ve decided that you’re taking your organization down the route of Big Data, what components do you need? What are the available components that make Big Data work? Well. Let’s take a brief overview.
In terms of hardware, you’ll need lots of servers grouped into a very large cluster, with each server having its own internal disk drives. Ideally, you’d have Linux, but you might have Windows. And, of course, you could use Linux on System z if you have a mainframe.
You’re going to need a file system and that’s HDFS (Hadoop Distributed File System). Data in a Hadoop cluster gets broken down into smaller pieces that are called blocks, and these are distributed throughout the cluster. Any work on the data can then be performed on manageable pieces rather than on the whole mass of data.
Next you want a data store – and that’s HBase. HBase is an open source, non-relational, distributed database modelled after Google’s BigTable and is written in Java. It’s a column-oriented database management system (DBMS) that runs on top of HDFS. HBase applications are written in Java.
As a runtime, there’s MapReduce – a programming model for processing large data sets with a parallel, distributed algorithm on a cluster.
What about workload management, what options do you have for that? Your open source choices are ZooKeeper, Oozie, Jaql, Lucerne, HCatalog, Pig, and Hive. According to Apache, ZooKeeper is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. Similarly, according to Apache, Oozie is a workflow scheduler system to manage Hadoop jobs. Oozie Workflow jobs are Directed Acyclical Graphs (DAGs) of actions. Oozie Coordinator jobs are recurrent Oozie Workflow jobs triggered by time (frequency) and data availabilty. Oozie is integrated with the rest of the Hadoop stack supporting several types of Hadoop jobs out of the box (such as MapReduce, Streaming MapReduce, Pig, Hive, Sqoop, and Distcp) as well as system specific jobs (such as Java programs and shell scripts). Jaql is primarily a query language for JavaScript Object Notation (JSON). It allows both structured and non-traditional data to be processed. Lucerne is an information retrieval software library from Apache that was originally created in Java. HCatalog is a table and storage management service for data created using Hadoop. Pig, also from Apache is a platform for analysing large data sets. It consists of a high-level language for expressing data analysis programs, coupled with infrastructure for evaluating these programs. The structure of Pig programs allows substantial parallelization, which enables them to handle very large data sets. Finally on the list is Hive, which is a data warehouse system for Hadoop that facilitates easy data summarization, ad hoc queries, and the analysis of large datasets stored in Hadoop compatible file systems.
So what are your integration options? Apache Flume is a distributed, reliable, and available system for efficiently collecting, aggregating and moving large amounts of log data from many different sources to a centralized data store. There’s also Sqoop, which is a tool designed for efficiently transferring bulk data between Hadoop and structured datastores such as relational databases.
And finally, is there an open source advanced analytic engine? There is and it’s called R. R is a programming language and a software suite used for data analysis, statistical computing, and data visualization. It is highly extensible and has object-oriented features and strong graphical capabilities. It is well-suited for modelling and running advanced analytics.
That will pretty much get you started and on your way. You may feel that you’d like more integration products, some form of administration, or some kind of visualization and discovery product. But this is where you need to go to specific vendors. I’m expecting to be at liberty to talk more about how IBM is looking at this in future blogs.
In terms of hardware, you’ll need lots of servers grouped into a very large cluster, with each server having its own internal disk drives. Ideally, you’d have Linux, but you might have Windows. And, of course, you could use Linux on System z if you have a mainframe.
You’re going to need a file system and that’s HDFS (Hadoop Distributed File System). Data in a Hadoop cluster gets broken down into smaller pieces that are called blocks, and these are distributed throughout the cluster. Any work on the data can then be performed on manageable pieces rather than on the whole mass of data.
Next you want a data store – and that’s HBase. HBase is an open source, non-relational, distributed database modelled after Google’s BigTable and is written in Java. It’s a column-oriented database management system (DBMS) that runs on top of HDFS. HBase applications are written in Java.
As a runtime, there’s MapReduce – a programming model for processing large data sets with a parallel, distributed algorithm on a cluster.
What about workload management, what options do you have for that? Your open source choices are ZooKeeper, Oozie, Jaql, Lucerne, HCatalog, Pig, and Hive. According to Apache, ZooKeeper is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. Similarly, according to Apache, Oozie is a workflow scheduler system to manage Hadoop jobs. Oozie Workflow jobs are Directed Acyclical Graphs (DAGs) of actions. Oozie Coordinator jobs are recurrent Oozie Workflow jobs triggered by time (frequency) and data availabilty. Oozie is integrated with the rest of the Hadoop stack supporting several types of Hadoop jobs out of the box (such as MapReduce, Streaming MapReduce, Pig, Hive, Sqoop, and Distcp) as well as system specific jobs (such as Java programs and shell scripts). Jaql is primarily a query language for JavaScript Object Notation (JSON). It allows both structured and non-traditional data to be processed. Lucerne is an information retrieval software library from Apache that was originally created in Java. HCatalog is a table and storage management service for data created using Hadoop. Pig, also from Apache is a platform for analysing large data sets. It consists of a high-level language for expressing data analysis programs, coupled with infrastructure for evaluating these programs. The structure of Pig programs allows substantial parallelization, which enables them to handle very large data sets. Finally on the list is Hive, which is a data warehouse system for Hadoop that facilitates easy data summarization, ad hoc queries, and the analysis of large datasets stored in Hadoop compatible file systems.
So what are your integration options? Apache Flume is a distributed, reliable, and available system for efficiently collecting, aggregating and moving large amounts of log data from many different sources to a centralized data store. There’s also Sqoop, which is a tool designed for efficiently transferring bulk data between Hadoop and structured datastores such as relational databases.
And finally, is there an open source advanced analytic engine? There is and it’s called R. R is a programming language and a software suite used for data analysis, statistical computing, and data visualization. It is highly extensible and has object-oriented features and strong graphical capabilities. It is well-suited for modelling and running advanced analytics.
That will pretty much get you started and on your way. You may feel that you’d like more integration products, some form of administration, or some kind of visualization and discovery product. But this is where you need to go to specific vendors. I’m expecting to be at liberty to talk more about how IBM is looking at this in future blogs.
Subscribe to:
Comments (Atom)