Last time we looked at the first part of Neil Price’s presentation to the Virtual IMS user group back in February. Neil is a Senior DBA and Systems Programmer with TNT Express ICS. Here’s some more highlights from his presentation.
Neil suggested that at his site key-range partitions can be so different from each other that they need to be treated like independent databases. For example if the volume of data in one partition is growing much faster than in the others, it might need a lot more free space so that you don’t have to resize or split it too soon, and you might want to set the free space warning threshold lower, so that you have time to react. Another example is where the average database record length in one partition is much lower than in the others. If you want to be alerted when the record lengths increase, the threshold for this partition would need to be correspondingly lower.
When it comes to tuning – what Neil called “Performance Tweaks” – they have various processes where values are obtained in physical sequential order from the root keys of one database and used to access one or more related databases with the same root key. If the databases involved are all HDAM and use the DFSHDC40 randomizing routine, this works well because the randomizer tends to keep keys in more or less the same sequence regardless of the total number of Root Anchor Points, as long as this is much greater than the number of roots. This means that all the databases are accessed in physical sequential order, which more or less guarantees that each database block is read only once and so minimizes I/O, as well as generating a sequential access pattern that maximizes read hits in disk cache and more recently has enabled them to take advantage of OSAM Sequential Buffering. But once you go to PHDAM, the keys are only kept in this sequence within each partition. Optimal performance is only restored when all the databases involved are HALDBs with identical partitioning arrangements.
Neil Price identified something that he said confuses a lot of people, including some in IBM support, not least because the manuals are unclear. Normally when a KSDS data CI fills up, it split roughly in half. That’s not good for ascending keys such as the timestamps used by most of their HIDAM databases, because the old or “lower” CI will stay half-empty forever. In the same way, a CA split will leave the old CA with half its CIs empty and the rest usually half-empty, in other words only about a quarter full. If the insert rates are high, the KSDS can end up with a very high proportion of unusable free space and might approach the 4GB limit. However if Sequential Mode is used for the inserts, CI and CA splits are done at the point of the insert – which for ascending keys means that it starts a new, empty CI or CA and leaves the old one as it is. This is good for performance – CA splits have been known to take seconds to complete – as well as greatly slowing down the dataset growth.
It’s possible to specify sequential-mode inserts in the IMS buffer pool specification member by coding INSERT=SEQ on the OPTIONS statement, but that would apply to every KSDS. Instead they include a DBD statement for each index they want treated this way and specify FREESPACE=YES, which is the dataset-level equivalent. Sequential-mode inserts also honour the free space specifications for the KSDS. This means that if free space is specified and all inserts are done this way, the free space will never get used and is a complete waste of space. All their inserts are done through the online system, so FREESPACE=YES implies that there should be no free space specified for the cluster!
FREESPACE=YES would work almost as well with constantly descending keys. It might also be beneficial for some indexes whose keys are generally, rather than strictly, ascending or descending, but until they’ve got IMS 12 and can change buffer pool parameters dynamically it’s going to be difficult to determine. Currently they specify FREESPACE=YES for 6 of their PSINDEXes, of which only one has a strictly ascending key and the rest start with a date. There used to be more, but after analysing the VSAM statistics, Neil reverted some to normal inserts and non-zero free space. This reduced the rate of CA splits, ie growth, and also the overall I/O. Neil reminded us that PSINDEX records can be relatively large, which makes the total index much larger than its non-HALDB version. It also means that fewer records are retrieved in one I/O, which is bad for sequential processing.
For Neil’s site, most of their indexes are accessed with a partial key, which in some cases will match hundreds of entries. Neil attempted to minimize the effects by making the VSAM buffer pools bigger, although he knew he wouldn’t be able to get the I/O rates back down to pre-HALDB levels. He reasoned that the buffers ought to hold the same number of VSAM data records as before, so he aimed to increase the number of buffers in line with the increase in the record length. When he calculated the new subpool sizes he discovered that some came out greater than the limit of 32,767 buffers, and in any case there wasn’t enough virtual storage available for such large increases, so he had to scale them all back anyway. But the I/O rates for most of the OSAM subpools are higher than for any of the VSAM ones, so this is maybe not the area with the most potential for I/O reduction.
To sum up, Neil informed the user group that HALDBs can change your life if you’re a DBA! Once you’ve converted your databases, that’s when the fun really begins
You might like to know that the Virtual IMS user group can be found at www.fundi.com/virtualims. The next meeting is on 9 October at 11:30am EDT, when the guest speaker will be Aurora Emanuela Dell’Anno, a Senior Engineering Services Architect with CA Technology, who’ll be talking about “IMS performance – taming the beast”. It will be well worth attending – and being a virtual meeting, there’s no need to leave your desk. See the Web site for more details.
Showing posts with label Neil Price. Show all posts
Showing posts with label Neil Price. Show all posts
Sunday, 30 September 2012
Sunday, 23 September 2012
IMS’s HALDBA
The Virtual IMS user group at www.fundi.com/virtualims has a regular virtual meetings every other month. In fact, the next one is on 9 October at 10:30 Central Daylight Time. Our guest speaker is Aurora Emanuela Dell'Anno, a Senior Engineering Services Architect with CA Technology, and she will be talking about “IMS performance – taming the beast”. It will be well worth attending – and being a virtual meeting, there’s no need to leave your desk. See the Web site for more details.
Back in February, Neil Price, a Senior DBA and Systems Programmer with TNT Express ICS gave an excellent presentation entitled, “Memoirs of a HALDBA”. Neil first encountered IBM mainframes as a student in 1971 and more recently he has been Chairman of the GSE UK IMS Working Group.
Neil started his presentation by giving the user group a clear idea of what hardware and software TNT Express use. This is relevant because it affects the decisions that Neil and his organization had to make. They have two machines – one hosting the main production LPAR; and the other running the network, testing, and one or two other things. This has a total disk capacity of about 18TB, of which just over 4TB are mirrored to their Disaster Recovery site, about 100 miles away in London.
Neil informed us that they always used OSAM because they believe it’s more efficient. More recently, of course, it gave them the ability to have datasets up to 8GB in size. Very few of their databases ever get disorganized enough to need a reorg unless they’re running out of space. The use of HDAM for all of the most volatile databases is a major factor in this. A few are reorganised weekly, 3 of them in order to remove logically-deleted data with a user exit, and a few quarterly.
In terms of HALDB, Neil told us that since they started investigating and testing HALDB in 2004, they’ve converted 18 HDAM databases, which were getting close to the 8GB OSAM dataset size limit, including the whole of the CCDB (Central Consignment Data Base, consisting of 15 HALDBs of 6 partitions each) and one with an index that was threatening to hit 4GB between weekly reorgs. Just 2 of their HALDBs are partitioned by key range. Most of the HALDBs got up to 16 partitions on model-3 logical volumes over time, but went down to 6 partitions when they moved them to model-9s. Apart from Consignment, each HALDB has at least one index. 5 have 28 indexes between them, and 13 of the indexes are also partitioned.
Neil was pleased to have avoided certain challenges at his site. For example, the DBRC question – they decided to have DBRC forced registration in every system from the very beginning, and, because there’s virtually no DL/I batch, there’s just one copy of each database and it’s permanently online. It didn’t take much thinking to decide that their 14 cloned databases would stay separate when they became HALDBs. Anything else would have involved major application code changes. The databases using a partitioning exit all use the Consignment key, so they’ve effectively had to write only one simple exit routine. They still maintain separate modules, though, just in case they need to vary the logic at any time.
To simplify everything they treat the HALDBs just the same as before except in very exceptional circumstances and haven’t changed their names. In particular, not taking individual partitions offline means that the applications don’t need to handle partitions being unavailable.
In terms of challenges they faced, Neil said that the first database they converted was one using the Consignment key, and it was fairly obvious that they wouldn’t get a predictable spread of data using key ranges. In the end they adapted the example in the IBM Redbook “The Complete IMS HALDB Guide” (SG24-6945) to select a partition based on a portion of the key, in their case the trailing characters of the 9-digit character numeric Consignment number. With equally-spaced “limits”, this has resulted in a very even spread of data across the partitions. They started with 2 digits, but once they went from 10 to 12 and then 16 partitions it seemed sensible to start using 3 digits for a slightly more even spread. Of course the overall physical sequence of records isn’t the same as in HDAM. For databases with key-range partitioning, they used the HALDB Migration Aid utility DFSMAID0 to choose the limits.
When it comes to monitoring challenges, Neil told the group that for non-HALDBs they get a list by e-mail of any thresholds that have been exceeded, for example the number of database records or the number of HDAM roots not stored in their “home” block. This is sent by a simple SAS program that processes a daily report of all metrics and threshold exceptions from their Pointer Checker statistics repository. The statistics are gathered as part of the daily online Image Copy backups. They find the e-mail preferable to having to navigate ISPF panels or a GUI to find out what exceptions have occurred, if any.
HALDB statistics, on the other hand, are written to a completely different repository, which can be viewed through a GUI but didn’t come with facilities to generate reports or e-mails. So they segregated the HALDBs into their own backup jobs, which direct the Pointer Checker report to a file. An additional SAS step then processes the file, writes the results to a SAS database and a summary report, and generates an e-mail if the percentage of usable free space in any partition falls below a hard-coded threshold.
Find out more about Neil Price’s experience next week.
Back in February, Neil Price, a Senior DBA and Systems Programmer with TNT Express ICS gave an excellent presentation entitled, “Memoirs of a HALDBA”. Neil first encountered IBM mainframes as a student in 1971 and more recently he has been Chairman of the GSE UK IMS Working Group.
Neil started his presentation by giving the user group a clear idea of what hardware and software TNT Express use. This is relevant because it affects the decisions that Neil and his organization had to make. They have two machines – one hosting the main production LPAR; and the other running the network, testing, and one or two other things. This has a total disk capacity of about 18TB, of which just over 4TB are mirrored to their Disaster Recovery site, about 100 miles away in London.
Neil informed us that they always used OSAM because they believe it’s more efficient. More recently, of course, it gave them the ability to have datasets up to 8GB in size. Very few of their databases ever get disorganized enough to need a reorg unless they’re running out of space. The use of HDAM for all of the most volatile databases is a major factor in this. A few are reorganised weekly, 3 of them in order to remove logically-deleted data with a user exit, and a few quarterly.
In terms of HALDB, Neil told us that since they started investigating and testing HALDB in 2004, they’ve converted 18 HDAM databases, which were getting close to the 8GB OSAM dataset size limit, including the whole of the CCDB (Central Consignment Data Base, consisting of 15 HALDBs of 6 partitions each) and one with an index that was threatening to hit 4GB between weekly reorgs. Just 2 of their HALDBs are partitioned by key range. Most of the HALDBs got up to 16 partitions on model-3 logical volumes over time, but went down to 6 partitions when they moved them to model-9s. Apart from Consignment, each HALDB has at least one index. 5 have 28 indexes between them, and 13 of the indexes are also partitioned.
Neil was pleased to have avoided certain challenges at his site. For example, the DBRC question – they decided to have DBRC forced registration in every system from the very beginning, and, because there’s virtually no DL/I batch, there’s just one copy of each database and it’s permanently online. It didn’t take much thinking to decide that their 14 cloned databases would stay separate when they became HALDBs. Anything else would have involved major application code changes. The databases using a partitioning exit all use the Consignment key, so they’ve effectively had to write only one simple exit routine. They still maintain separate modules, though, just in case they need to vary the logic at any time.
To simplify everything they treat the HALDBs just the same as before except in very exceptional circumstances and haven’t changed their names. In particular, not taking individual partitions offline means that the applications don’t need to handle partitions being unavailable.
In terms of challenges they faced, Neil said that the first database they converted was one using the Consignment key, and it was fairly obvious that they wouldn’t get a predictable spread of data using key ranges. In the end they adapted the example in the IBM Redbook “The Complete IMS HALDB Guide” (SG24-6945) to select a partition based on a portion of the key, in their case the trailing characters of the 9-digit character numeric Consignment number. With equally-spaced “limits”, this has resulted in a very even spread of data across the partitions. They started with 2 digits, but once they went from 10 to 12 and then 16 partitions it seemed sensible to start using 3 digits for a slightly more even spread. Of course the overall physical sequence of records isn’t the same as in HDAM. For databases with key-range partitioning, they used the HALDB Migration Aid utility DFSMAID0 to choose the limits.
When it comes to monitoring challenges, Neil told the group that for non-HALDBs they get a list by e-mail of any thresholds that have been exceeded, for example the number of database records or the number of HDAM roots not stored in their “home” block. This is sent by a simple SAS program that processes a daily report of all metrics and threshold exceptions from their Pointer Checker statistics repository. The statistics are gathered as part of the daily online Image Copy backups. They find the e-mail preferable to having to navigate ISPF panels or a GUI to find out what exceptions have occurred, if any.
HALDB statistics, on the other hand, are written to a completely different repository, which can be viewed through a GUI but didn’t come with facilities to generate reports or e-mails. So they segregated the HALDBs into their own backup jobs, which direct the Pointer Checker report to a file. An additional SAS step then processes the file, writes the results to a SAS database and a summary report, and generates an e-mail if the percentage of usable free space in any partition falls below a hard-coded threshold.
Find out more about Neil Price’s experience next week.
Saturday, 5 November 2011
Guide Share Europe – an impression
I could only make Day 1 of this year’s Guide Share Europe conference on the 1st and 2nd of November – which was a huge disappointment. For those of you who weren’t there, I thought I’d give you a flavour of my experience.
Firstly, it was at Whittlebury Hall again – which is a magnificent location just over the border from Buckinghamshire into Northamptonshire. The location is stunning and the facilities are excellent. It is in the countryside, so if you’re travelling by train, there’s a long taxi ride to get there. If you travel by car, there’s a huge car park.
The exhibition hall is big, but not so big you get lost in it. By having lunch and coffee in the hall, there were plenty of opportunities to engage with vendors and chat to other attendees. I always find it’s a great opportunity to catch up with old colleagues and make new friends. The quality of the coffee and food was good – which translates as excellent when compared to some venues!
But the point of GSE is not the food, it’s the presentations. I chair the Virtual IMS user group and the Virtual CICS user group, so I was torn between the CICS and IMS streams. In the end, I split my time between them. I watched Circle’s Ezriel Gross present on Using CICS to Deploy Microsoft .Net Winforms with Smart Client Technology – which was really fascinating. I’m sure we’re going to see more sites integrating their Windows technology with the power of mainframe subsystems. Ezriel made quite a complicated integration seem straightforward and obvious.
Next I watched IBM’s Alison Coughtrie talk about IMS 12 Overview. Another knowledgeable speaker with a lot of information to get over in the time. I certainly think I have a clearer idea of what’s new, and perhaps a small insight into where IBM is taking the product.
After lunch it was Neil Price, who works for TNT Express and chairs the IMS group for GSE, with a presentation entitled Memoirs of a HALDBA. I was so impressed with Neil’s real-life descriptions that I’ve asked him to speak to the Virtual IMS user group. Neil could have gone on for much longer than the time allowed. And I could happily have gone on listening.
Next up in the IMS stream was IBM’s Dougie Lawson. Dougie is another fantastically knowledgeable IBMer, who you may have come across when you’ve had an IMS problem. He talked about The Why and How of CSL. A real bits and bytes expert, who could have talked much longer.
I felt it was time to sit in on the CICS stream and the session I chose was IBM’s Ian Burnett talking about CICS Scalability. Yet again, a fact-filled presentation that would be hard to criticize. I felt my knowledge about CICS (and I used to edit CICS Update) making more sense and falling more into place.
But all work and no play makes Jack a dull boy – as they say. And the evening presentation was How To Cope With Pressure & Panics Without Going Into Headless Chicken Mode from Resli Costabell. A mixture of psychology, NLP, and audience participation made this a memorable session. If you get a chance to see her anywhere – don’t miss it!
After that there were drinks in the exhibition hall sponsored by Attachmate/Suse and Computacenter, followed by dinner sponsored by EMC and Computacenter. Both were very enjoyable in their own way, and they were an opportunity to chat more informally with vendors and real mainframe users. Obviously, I was telling vendors about sponsorship opportunities with the Arcati Mainframe Yearbook, and asking users to complete the user survey.
In conversation, I asked a few of the vendors how business was going. No-one admitted that double-dip recession was taking them out of business, but most suggested that they were keeping their heads above water and business generally was flat – but there was some business being done.
An IBMer suggested that over 30 z196s had been sold in the UK and eight of the new z114s. So, that’s good news for them.
My overall impression of the conference was that it was excellent. I bumped into Mark Wilson (the GSE technical coordinator) during the day as he rushed around making sure everything was going smoothly. And that’s why the conference works so well, because people like Mark work so hard to ensure it does.
Well done everyone who organized it and spoke at it. And if you missed it, go next year.
Firstly, it was at Whittlebury Hall again – which is a magnificent location just over the border from Buckinghamshire into Northamptonshire. The location is stunning and the facilities are excellent. It is in the countryside, so if you’re travelling by train, there’s a long taxi ride to get there. If you travel by car, there’s a huge car park.
The exhibition hall is big, but not so big you get lost in it. By having lunch and coffee in the hall, there were plenty of opportunities to engage with vendors and chat to other attendees. I always find it’s a great opportunity to catch up with old colleagues and make new friends. The quality of the coffee and food was good – which translates as excellent when compared to some venues!
But the point of GSE is not the food, it’s the presentations. I chair the Virtual IMS user group and the Virtual CICS user group, so I was torn between the CICS and IMS streams. In the end, I split my time between them. I watched Circle’s Ezriel Gross present on Using CICS to Deploy Microsoft .Net Winforms with Smart Client Technology – which was really fascinating. I’m sure we’re going to see more sites integrating their Windows technology with the power of mainframe subsystems. Ezriel made quite a complicated integration seem straightforward and obvious.
Next I watched IBM’s Alison Coughtrie talk about IMS 12 Overview. Another knowledgeable speaker with a lot of information to get over in the time. I certainly think I have a clearer idea of what’s new, and perhaps a small insight into where IBM is taking the product.
After lunch it was Neil Price, who works for TNT Express and chairs the IMS group for GSE, with a presentation entitled Memoirs of a HALDBA. I was so impressed with Neil’s real-life descriptions that I’ve asked him to speak to the Virtual IMS user group. Neil could have gone on for much longer than the time allowed. And I could happily have gone on listening.
Next up in the IMS stream was IBM’s Dougie Lawson. Dougie is another fantastically knowledgeable IBMer, who you may have come across when you’ve had an IMS problem. He talked about The Why and How of CSL. A real bits and bytes expert, who could have talked much longer.
I felt it was time to sit in on the CICS stream and the session I chose was IBM’s Ian Burnett talking about CICS Scalability. Yet again, a fact-filled presentation that would be hard to criticize. I felt my knowledge about CICS (and I used to edit CICS Update) making more sense and falling more into place.
But all work and no play makes Jack a dull boy – as they say. And the evening presentation was How To Cope With Pressure & Panics Without Going Into Headless Chicken Mode from Resli Costabell. A mixture of psychology, NLP, and audience participation made this a memorable session. If you get a chance to see her anywhere – don’t miss it!
After that there were drinks in the exhibition hall sponsored by Attachmate/Suse and Computacenter, followed by dinner sponsored by EMC and Computacenter. Both were very enjoyable in their own way, and they were an opportunity to chat more informally with vendors and real mainframe users. Obviously, I was telling vendors about sponsorship opportunities with the Arcati Mainframe Yearbook, and asking users to complete the user survey.
In conversation, I asked a few of the vendors how business was going. No-one admitted that double-dip recession was taking them out of business, but most suggested that they were keeping their heads above water and business generally was flat – but there was some business being done.
An IBMer suggested that over 30 z196s had been sold in the UK and eight of the new z114s. So, that’s good news for them.
My overall impression of the conference was that it was excellent. I bumped into Mark Wilson (the GSE technical coordinator) during the day as he rushed around making sure everything was going smoothly. And that’s why the conference works so well, because people like Mark work so hard to ensure it does.
Well done everyone who organized it and spoke at it. And if you missed it, go next year.
Subscribe to:
Posts (Atom)