IOD news

In a week that saw Windows 8 and that funny iPad mini being announced, the really hot news was coming out of IBM’s Information On Demand (IOD) conference in Vegas.

Running under the ‘Think Big’ tag we saw IBM announce IBM PureData System, InfoSphere Guardium V9.0, InfoSphere Information Server V9.1, InfoSphere Master Data Management V10.1, IBM Cognos Insight Personal Edition, IBM Cognos 10.2, IBM Cognos TM1.1.1, IBM Cognos Disclosure Management, IBM Analytical Decision Management, IBM SPSS Statistics, IBM SPSS Modeler, Intelligent Investigation Manager, Patient Care and Insights, Datacap Taskmaster Capture, and Content Manager OnDemand for Multiplatforms.

The third member in IBM’s PureSystems family is the PureData System, which is designed for big data cloud appliances by providing data services to various applications. This new PureData System family comprises: PureData System for Transactions, which is aimed at improving data management costs; and PureData System for Analytics, which is designed to analyse large volumes of data.

InfoSphere Guardium V9.0: introduces Hadoop Activity Monitoring to protect sensitive data in Big Data environments; enhances data security for System z with improved performance, resiliency and scalability; further reduces TCO and provides simplified scalability with Guardium grid/load balancing; introduces Security Content Automation Protocol (SCAP) reporting in Vulnerability Assessment; and extends in-depth data security with new security solutions integrations, such as Security Intelligence with QRadar, and IDS (Intrusion Detection System) insight with F5.

InfoSphere Information Server V9.1 provides a scalable, secure, and robust data integration platform. V9.1 offers new features to help rationalize increasingly complex environment and address evolving data integration requirements, so that data is accessible, authoritative, consistent, timely, and in context for analysis and reporting.

InfoSphere Master Data Management V10.1 builds on the unification of InfoSphere Master Data Management Server, Initiate Master Data Service, and InfoSphere Master Data Management Collaboration Server. New features include: Master Data Policy Management; enhanced Business Process Management (BPM) integration capabilities and sample workflows; enhanced probabilistic matching and searching; enhancements to the InfoSphere MDM Application Toolkit; advanced Rules Management; and advanced Catalog Management.

Cognos Insight Personal Edition, IBM’s personal analytics solution enables business users and analysts to access business information and share it with others. IBM Cognos 10.2 lets users import personal or business data from spreadsheets or csv files, select visualizations of the data, drill-down for trends, and more. They can use drag-and-drop data visualization, what-if scenario modelling, and dashboard style delivery.

IBM Cognos TM1.1.1 delivers capabilities in personal desktop analysis for data exploration, prototyping, and results sharing for personal, workgroup, and enterprise levels. Users get: easier modelling for planning and analysis solutions; improved server performance with the new IBM Cognos TM1 Operations Console; and advances in distributed architecture for increased scale and interactivity.

The IBM Analytical Decision Management integrates predictive analytics, local rules, scoring, and optimization techniques into an organization’s systems, and then delivers real-time recommendations at the point of impact so organization can consistently drive better outcomes.
IBM SPSS Modeler is a high-performance predictive and text analytics workbench that enables organizations to gain insight from their data and deliver a positive return on investment by embedding predictive analytics into business processes.

Intelligent Investigation Manager optimizes fraud investigation and analysis for insurance, banking, healthcare, and other customers, and public safety programmes for government and law enforcement organizations. It dynamically coordinates and reports on cases, provides analysis and visualization, and enables more efficient and effective investigations.

Patient Care and Insights is an integrated and configurable set of solutions that brings together advanced analytics and care management capabilities to help healthcare organizations maximize the value of information for treating patients.

Content Manager OnDemand for Multiplatforms delivers instant access to bills, statements, and other print archive information. This helps them quickly respond to customer inquiries and manage access to electronic reports used to communicate activities and performance across an organization Content Manager OnDemand helps organizations reduce print, storage and distribution costs in support of a greener content delivery solution.

Obviously, much more detail can be found on IBM’s Web site.

Guide Share Europe annual conference

The Guide Share Europe (GSE) UK Annual Conference is taking place on 13-14 November at Whittlebury Hall, Whittlebury, Near Towcester, Northamptonshire NN12 8QH, UK.

Sponsors this year include IBM, Computacenter, PKWARE, CA Technologies, Attachmate Suse, Vanguard, Compuware, GT Software, RSM Partners, Blenheim Software, Sett, and Red Hat. And there will be over 30 vendors in the associated exhibition.

There’s the usual amazing range of streams – and, to be honest, there are a number of occasions when I would like to be in two or more places at once over the two days. The streams are: CICS, IMS, DB2, Enterprise Security, Large Systems Working Group, Network Management Working Group, zLinux, Storage Management, TWS (Tivoli Workload Schedular), Automation, New Technologies, Software Asset Management, MQ, Application Development, and Assembler Training.

There are also keynotes from Mark Anzani, VP and CTO for System z at IBM. And there’s the not-to-be-missed “Dealing with difficult people” session from Resli Costabell.

That means that at this year’s conference there will be well over 140 hours of education covering most aspects of mainframe technology – more than last year. This year, there will be 14 streams of ten sessions over the two days, plus Assembler Training, plus four keynotes.

There is still time to register, and the organizers are expecting the daily total of delegates to exceed 300 – as it did last year.

A number of the streams have 101 sessions at different times during the two days to give newcomers and those unfamiliar with parts of the mainframe infrastructure a basic understanding of the that mainframe technology and how it works.

You can find out more details about the conference at www.gse.org.uk/tyc/.

If you’re still debating whether to go, let me recommend it to you. The quality of presentations is always excellent. And the networking opportunities are brilliant. If you are going, I look forward to seeing you there.

IOD – so near yet so far away!

The IBM Information On Demand 2012 global conference is one of those conferences that you just don’t want to miss. And yet, like me, when you’re based in the UK, that’s just what’s going to happen. I won’t be rubbing shoulders with the people attending, listening to the speakers, or visiting the exhibitors stands. But I will be keeping up-to-date with what’s going on!

IOD 2012 runs from this coming Sunday, 21 October, to Thursday 25 at Mandalay Bay resort Las Vegas, Nevada, USA. This year, the old IBM slogan, “Think”, has been turned into “Think BIG”. And the whole conference is billed as the largest System z software event in the world. The Web site at www-01.ibm.com/software/os/systemz/conference/iod/ goes on to say that it delivers “the know-how you need to optimize business performance, drive more value from existing business analytics deployments, and get the latest innovations and success strategies from IBM experts, analysts, and customers.”

I chair the Virtual IMS user group – you can find us at www.fundi.com/virtualims – so I’m very interested in the IMS stream at IOD, particularly Betty J Patterson’s (a Distinguished Engineer with IBM) session, “Taking IMS to New Heights: What the Future Holds for IMS” and Dinesh Nirmal’s (IMS Director, IBM) “IMS Futures Roadmap” keynote seminar. But there is so much more of interest. You can see the session details at www-01.ibm.com/software/os/systemz/conference/iod/ims.html.

It’s not just IMS that’s featured, there are DB2 sessions, business analytics/data warehousing sessions, and tools and utilities sessions, totalling, they say, 180 executive, business leadership, and technical sessions in all.

And while it might seem unfair to highlight any of the many sessions going on at IOD, I guess that’s what everyone has to do in order to set a personal agenda for the four days. I’d like to attend some of the client-led sessions and see what real users are doing with their mainframes. What problems they’ve been experiencing and learn how they’ve overcome them. And I’d like to get my hands dirty in the hands-on labs and workshops. Perhaps dig a little deeper than I’ve ever had the chance to do with a production system.

Going to IOD lets you chat to other users. To find out that perhaps you’re not alone dealing with the pressures of fewer staff and tightening budgets, and listening to what tools and techniques other people have used to overcome these issues – and perhaps get an early warning about issues that haven’t come your way yet!

I can’t be there in person, but as an IBM Champion for the past four years, you may see my picture on display. I will definitely be using my laptop and tablet to keep up-to-date with the news and information already coming from the IOD 2012 Web site. I’ll be reading press releases and the blogs (www.ibmiodblog.com/), and I’ll be watching out for people tweeting and reading with interest what they have to say. And I’m hoping that my friends, who are there, will keep me in the loop with regular and frequent e-mails.

And to those of you who are lucky enough to attend – enjoy!

The Arcati Mainframe Yearbook 2013

The Arcati Mainframe Yearbook has been the de facto reference work for IT professionals working with z/OS (and its forerunner) systems since 2005. It includes an annual user survey, an up-to-date directory of vendors and consultants, a media guide, a strategy section with papers on mainframe trends and directions, a glossary of terminology, and a technical specification section. Each year, the Yearbook is downloaded by around 15,000 mainframe professionals. The current issue is still available at www.arcati.com/newyearbook12.

Very shortly, many mainframe professionals will receive an e-mail telling them that Mark Lillycrop and I have started work on the 2013 edition of the Arcati Mainframe Yearbook. If you don’t hear from us, then e-mail trevor@itech-ed.com and I will add you to our mailing list.

As in previous years, we’re inviting mainframe professionals to complete the annual user survey, which will shortly be up and running at www.arcati.com/usersurvey13. The more users who complete the survey, the more accurate and therefore useful the survey report will be. All respondents before Friday 7 December will receive a free PDF copy of the survey results on publication. The identity and company information of all respondents is treated in confidence and will never be divulged to third parties. And any comments made by respondents will be anonymized before publication. If you go to user group meetings, IOD, GCE Europe, etc, or just hang out with mainframers from other sites, please pass on the word about this survey. We’re hoping that this year’s user survey will be the most comprehensive survey ever. Current estimates suggest that there are somewhere between 6,000 and 8,000 companies using mainframes spread over 10,000 sites worldwide.

Anyone reading this who works for a vendor, consultant, or service provider, can ensure their company gets a free entry in the vendor directory section by completing the form, which will be at www.arcati.com/vendorentry. This form can also be used to amend last year’s entry.

Also, as in previous years, there is an opportunity for organizations to sponsor the Yearbook or take out a half-page advertisement. Half-page adverts (5.5in x 8.5in max landscape) cost $750 (UK£460). Sponsors get a full-page advert (11in x 8.5in) in the Yearbook; inclusion of a corporate paper in the Mainframe Strategy section of the Yearbook; a logo/link on the Yearbook download page on the Arcati Web site; and a brief text ad in the Yearbook publicity e-mails sent to users. All this for just $2200 (UK£1300).

To put that cost into perspective: for every dollar you spend on an advert, you reach around 20 mainframe professionals.

The Arcati Mainframe Yearbook 2013 will be freely available for download early in January next year.

IMS's HALDBA again

Last time we looked at the first part of Neil Price’s presentation to the Virtual IMS user group back in February. Neil is a Senior DBA and Systems Programmer with TNT Express ICS. Here’s some more highlights from his presentation.

Neil suggested that at his site key-range partitions can be so different from each other that they need to be treated like independent databases. For example if the volume of data in one partition is growing much faster than in the others, it might need a lot more free space so that you don’t have to resize or split it too soon, and you might want to set the free space warning threshold lower, so that you have time to react. Another example is where the average database record length in one partition is much lower than in the others. If you want to be alerted when the record lengths increase, the threshold for this partition would need to be correspondingly lower.

When it comes to tuning – what Neil called “Performance Tweaks” – they have various processes where values are obtained in physical sequential order from the root keys of one database and used to access one or more related databases with the same root key. If the databases involved are all HDAM and use the DFSHDC40 randomizing routine, this works well because the randomizer tends to keep keys in more or less the same sequence regardless of the total number of Root Anchor Points, as long as this is much greater than the number of roots. This means that all the databases are accessed in physical sequential order, which more or less guarantees that each database block is read only once and so minimizes I/O, as well as generating a sequential access pattern that maximizes read hits in disk cache and more recently has enabled them to take advantage of OSAM Sequential Buffering. But once you go to PHDAM, the keys are only kept in this sequence within each partition. Optimal performance is only restored when all the databases involved are HALDBs with identical partitioning arrangements.

Neil Price identified something that he said confuses a lot of people, including some in IBM support, not least because the manuals are unclear. Normally when a KSDS data CI fills up, it split roughly in half. That’s not good for ascending keys such as the timestamps used by most of their HIDAM databases, because the old or “lower” CI will stay half-empty forever. In the same way, a CA split will leave the old CA with half its CIs empty and the rest usually half-empty, in other words only about a quarter full. If the insert rates are high, the KSDS can end up with a very high proportion of unusable free space and might approach the 4GB limit. However if Sequential Mode is used for the inserts, CI and CA splits are done at the point of the insert – which for ascending keys means that it starts a new, empty CI or CA and leaves the old one as it is. This is good for performance – CA splits have been known to take seconds to complete – as well as greatly slowing down the dataset growth.

It’s possible to specify sequential-mode inserts in the IMS buffer pool specification member by coding INSERT=SEQ on the OPTIONS statement, but that would apply to every KSDS. Instead they include a DBD statement for each index they want treated this way and specify FREESPACE=YES, which is the dataset-level equivalent. Sequential-mode inserts also honour the free space specifications for the KSDS. This means that if free space is specified and all inserts are done this way, the free space will never get used and is a complete waste of space. All their inserts are done through the online system, so FREESPACE=YES implies that there should be no free space specified for the cluster!

FREESPACE=YES would work almost as well with constantly descending keys. It might also be beneficial for some indexes whose keys are generally, rather than strictly, ascending or descending, but until they’ve got IMS 12 and can change buffer pool parameters dynamically it’s going to be difficult to determine. Currently they specify FREESPACE=YES for 6 of their PSINDEXes, of which only one has a strictly ascending key and the rest start with a date. There used to be more, but after analysing the VSAM statistics, Neil reverted some to normal inserts and non-zero free space. This reduced the rate of CA splits, ie growth, and also the overall I/O. Neil reminded us that PSINDEX records can be relatively large, which makes the total index much larger than its non-HALDB version. It also means that fewer records are retrieved in one I/O, which is bad for sequential processing.

For Neil’s site, most of their indexes are accessed with a partial key, which in some cases will match hundreds of entries. Neil attempted to minimize the effects by making the VSAM buffer pools bigger, although he knew he wouldn’t be able to get the I/O rates back down to pre-HALDB levels. He reasoned that the buffers ought to hold the same number of VSAM data records as before, so he aimed to increase the number of buffers in line with the increase in the record length. When he calculated the new subpool sizes he discovered that some came out greater than the limit of 32,767 buffers, and in any case there wasn’t enough virtual storage available for such large increases, so he had to scale them all back anyway. But the I/O rates for most of the OSAM subpools are higher than for any of the VSAM ones, so this is maybe not the area with the most potential for I/O reduction.

To sum up, Neil informed the user group that HALDBs can change your life if you’re a DBA! Once you’ve converted your databases, that’s when the fun really begins

You might like to know that the Virtual IMS user group can be found at www.fundi.com/virtualims. The next meeting is on 9 October at 11:30am EDT, when the guest speaker will be Aurora Emanuela Dell’Anno, a Senior Engineering Services Architect with CA Technology, who’ll be talking about “IMS performance – taming the beast”. It will be well worth attending – and being a virtual meeting, there’s no need to leave your desk. See the Web site for more details.

IMS’s HALDBA

The Virtual IMS user group at www.fundi.com/virtualims has a regular virtual meetings every other month. In fact, the next one is on 9 October at 10:30 Central Daylight Time. Our guest speaker is Aurora Emanuela Dell'Anno, a Senior Engineering Services Architect with CA Technology, and she will be talking about “IMS performance – taming the beast”. It will be well worth attending – and being a virtual meeting, there’s no need to leave your desk. See the Web site for more details.

Back in February, Neil Price, a Senior DBA and Systems Programmer with TNT Express ICS gave an excellent presentation entitled, “Memoirs of a HALDBA”. Neil first encountered IBM mainframes as a student in 1971 and more recently he has been Chairman of the GSE UK IMS Working Group.

Neil started his presentation by giving the user group a clear idea of what hardware and software TNT Express use. This is relevant because it affects the decisions that Neil and his organization had to make. They have two machines – one hosting the main production LPAR; and the other running the network, testing, and one or two other things. This has a total disk capacity of about 18TB, of which just over 4TB are mirrored to their Disaster Recovery site, about 100 miles away in London.

Neil informed us that they always used OSAM because they believe it’s more efficient. More recently, of course, it gave them the ability to have datasets up to 8GB in size. Very few of their databases ever get disorganized enough to need a reorg unless they’re running out of space. The use of HDAM for all of the most volatile databases is a major factor in this. A few are reorganised weekly, 3 of them in order to remove logically-deleted data with a user exit, and a few quarterly.

In terms of HALDB, Neil told us that since they started investigating and testing HALDB in 2004, they’ve converted 18 HDAM databases, which were getting close to the 8GB OSAM dataset size limit, including the whole of the CCDB (Central Consignment Data Base, consisting of 15 HALDBs of 6 partitions each) and one with an index that was threatening to hit 4GB between weekly reorgs. Just 2 of their HALDBs are partitioned by key range. Most of the HALDBs got up to 16 partitions on model-3 logical volumes over time, but went down to 6 partitions when they moved them to model-9s. Apart from Consignment, each HALDB has at least one index. 5 have 28 indexes between them, and 13 of the indexes are also partitioned.

Neil was pleased to have avoided certain challenges at his site. For example, the DBRC question – they decided to have DBRC forced registration in every system from the very beginning, and, because there’s virtually no DL/I batch, there’s just one copy of each database and it’s permanently online. It didn’t take much thinking to decide that their 14 cloned databases would stay separate when they became HALDBs. Anything else would have involved major application code changes. The databases using a partitioning exit all use the Consignment key, so they’ve effectively had to write only one simple exit routine. They still maintain separate modules, though, just in case they need to vary the logic at any time.

To simplify everything they treat the HALDBs just the same as before except in very exceptional circumstances and haven’t changed their names. In particular, not taking individual partitions offline means that the applications don’t need to handle partitions being unavailable.

In terms of challenges they faced, Neil said that the first database they converted was one using the Consignment key, and it was fairly obvious that they wouldn’t get a predictable spread of data using key ranges. In the end they adapted the example in the IBM Redbook “The Complete IMS HALDB Guide” (SG24-6945) to select a partition based on a portion of the key, in their case the trailing characters of the 9-digit character numeric Consignment number. With equally-spaced “limits”, this has resulted in a very even spread of data across the partitions. They started with 2 digits, but once they went from 10 to 12 and then 16 partitions it seemed sensible to start using 3 digits for a slightly more even spread. Of course the overall physical sequence of records isn’t the same as in HDAM. For databases with key-range partitioning, they used the HALDB Migration Aid utility DFSMAID0 to choose the limits.

When it comes to monitoring challenges, Neil told the group that for non-HALDBs they get a list by e-mail of any thresholds that have been exceeded, for example the number of database records or the number of HDAM roots not stored in their “home” block. This is sent by a simple SAS program that processes a daily report of all metrics and threshold exceptions from their Pointer Checker statistics repository. The statistics are gathered as part of the daily online Image Copy backups. They find the e-mail preferable to having to navigate ISPF panels or a GUI to find out what exceptions have occurred, if any.

HALDB statistics, on the other hand, are written to a completely different repository, which can be viewed through a GUI but didn’t come with facilities to generate reports or e-mails. So they segregated the HALDBs into their own backup jobs, which direct the Pointer Checker report to a file. An additional SAS step then processes the file, writes the results to a SAS database and a summary report, and generates an e-mail if the percentage of usable free space in any partition falls below a hard-coded threshold.

Find out more about Neil Price’s experience next week.

CICS communication history

In the beginning, CICS programmers used conversational transactions – but not any more. Pseudo-conversational coding is less resource-intensive, user think time costs are eliminated, and sites can run more simultaneous transactions. On the down side, programming is more difficult, application design requires re-starting programs, it’s more difficult to debug programs, and programs require an area to store data between tasks.
 
And so we get the Communication Area (COMMAREA). It’s a terminal-based scratchpad that’s dynamic in size (0 to 32,500 bytes), and is private and chained to the terminal. The COMAREA can hold data between pseudo-conversations, has a maximum in size of 32K, can be passed from program to program, can be empty, partially full, or full, and is automatically cleaned up when not used.

However, COMMAREAs have obvious limitations, and so IBM gave us channels and containers. There are no size limitations on data in containers, the number of containers in a channel is unlimited, and there can be separate containers for input and output.
Another way round COMMAREA limitations is to use an XML (eXtensible Mark-up Language) front-end application. Associated with this are:

• Extensible Stylesheet Language Transformation (XSLT) is the recommended style sheet language of XML.
• XML namespaces – in XML, element names are defined by a developer.
• XML schema is an XML document that describes the structure and constrains the contents of other XML documents.
• XML parser is a program that is invoked by an application to process an XML document, ensure that it meets all the rules of XML as well as the syntax of the DTD or schema, making the data available to the calling application.

With the introduction of Services Oriented Architecture (SOA), programmers now use CICS Web services. A Web service is a collection of operations that are network accessible through standardized XML messaging. The Web services architecture is based on interactions between three components: a service provider, which is the platform that hosts access to the service; a service requester, which is the application that is looking for and invoking or initiating an interaction with a service; and service registry, which is a place where service providers publish their service descriptions, and where service requesters find them.

Universal Description, Discovery and Integration (UDDI) is a specification for distributed Web-based information registries of Web services. The service provider “owns” the service, creates the WSDL (Web Service Description Language – see below), publishes the WSDL, and processes the requests. The service requester “finds” the service, binds to the service, and invokes the service using the WSDL. The service registry hosts the service description and is optional for statically bound requesters.

WSDL is an XML application for describing Web services. WSDL comprises:

• Types – the data types in the form of XML schemas
• Message – an abstract definition of the data in the form of a message
• PortType – an abstract set of operations mapped to one or more end points
• Binding – the concrete protocol and data formats for the operations
• Service – a collection of related end points.

Simple Object Access Protocol (SOAP) is an XML-based protocol for the exchange of information in a distributed environment. A SOAP message is encoded as an XML document.
 
The CICS Web services assistant is a set of batch utilities that can help users transform existing CICS applications into Web services and to enable CICS applications to use Web services provided by external providers. The CICS Web services assistant comprises two utility programs:

• DFHLS2WS generates a Web service binding file from a language structure. This utility also generates a Web service description.
• DFHWS2LS generates a Web service binding file from a Web service description. This utility also generates a language structure that you can use in your application programs.

You can use Rational Developer for System z for Web services and XML development. A CICS pipeline is responsible for dealing with SOAP headers. It’s implemented as a series of programs, and can be configured by end users by using message handlers. A message handler is a program in which you can perform your own processing of Web service requests and responses.