Showing posts with label SQData. Show all posts
Showing posts with label SQData. Show all posts

Sunday, 18 December 2011

2011 at iTech-Ed Ltd

Well, as another year comes partying to an end, and everyone stops checking their e-mails on their smartphones or tablets and finally starts to let their hair down and enjoy a glass of something alcoholic, I thought I’d review the year through the lens of my company – iTech-Ed Ltd (www.itech-ed.com).

January started the year, as most Januaries do, with the publication of the Arcati Mainframe Yearbook. The 2011 edition is still available for download from www.arcati.com/newyearbook11. The 2012 edition will be available in a couple of weeks. As always the Arcati Mainframe Yearbook  includes its annual user survey, an up-to-date directory of vendors and consultants, a media guide, a strategy section with papers on mainframe trends and directions, a glossary of terminology, and a technical specification section. And each year, it gets downloaded by around 15,000 mainframe professionals.

February saw the launch of the new series of Virtual IMS user group meetings. The user group is now sponsored by Fundi Software and hosted at www.fundi.com/virtualims. The first speaker was Jim Martin from Fundi Software, whose presentation was called, “Solving the problem when IMS isn't the cause”.

In March, everyone seemed to be talking about cloud computing.

April’s meeting of the Virtual IMS user group included a presentation from Ron Haupert, a Senior Technologist with Rocket Software. His talk was called, “Simplify and improve database administration by leveraging your storage system”.

In May, Mark Lillycrop, Director of Arcati Ltd and I took part in a ‘Scheduled Chat’ in the ‘House of Mainframe’ section of CA’s May Mainframe Madness month. May also witnessed the launch of the new Virtual CICS user group – again sponsored by Fundi – with its Web site at www.fundi.com/virtualcics. Our opening presentation was from Fundi’s Jim Martin talking about, “Solving the problem when CICS isn't the cause”.

In June, I was asked by ITToolbox to lead a discussion in the Data Center Infrastructure section of their Web site. At the Virtual IMS user group meeting, Gary Weinhold a Systems Engineer and Verna Bartlett Head of Marketing with Data Kinetics talked about, “MSU reduction due to in-memory table management with (any) IMS applications”.

In July, I was selected for the Destination z (www.destinationz.org/) member spotlight. The Virtual CICS user group saw a presentation from Jeff Geminder, Principal Consultant with CA, called, “Cross-enterprise application performance monitoring and CICS-specific drill-down: approaches to finding the performance problem needle in the heterogeneous haystack”. I was also a guest blogger on the Destination z Web site.

In August, my article CICS Top Performance and Tuning Issues was published in z/Journal. I had a guest blog published on Destination z. The Virtual IMS user group had a presentation from Scott Quillicy, CEO and Founder of SQData. His talk was called, “IMS replication for high-availability”.

For the September meeting, Charles Jones, from the Product Management group at Rocket Software, gave a talk to the Virtual CICS user group called, “CICS TS 4.2: Leveraging event processing and high-performance Java”. I wrote a guest blog for the Destination z Web site.

October saw a presentation from Rosemary Galvan, Principal Software Consultant – IMS, with BMC. Her talk to the Virtual IMS user group was called, “Database Performance – Could Have, Should Have, Would Have”. I had a guest blog on the Destination z Web site.

In November, my Mainframe Update blog at mainframeupdate.blogspot.com was a finalist in the Computer Weekly Social Media Awards 2011. Also in November the Arcati Mainframe Yearbook user survey was launched. And Eugene S Hudders, president of C\TREK Corp, gave a presentation to the Virtual CICS user group called, “CICS TS Performance – Tuning LSR Pools”. I also had a guest blog on the Destination z Web site.

And finally, in December, I had an article entitled, Ways to Save Money and Improve IT Services published in z/Journal. The final speaker for the year at the Virtual IMS user group was Suzie Wendler, a Consulting IT Specialist in the IBM IMS Advanced Technical Skills organization, who talked about, “IMS V12”. I chaired a webinar for SQData entitled, “How Important is Continous Availability of Critical Applications to Your Company?”And there was a guest blog on the Destination z Web site.

What else, well apart from a full year of writing and consultancy work,  I was made an IBM Champion for the third year running.

Looking forward to 2012, we have the launch of the Arcati Mainframe Yearbook in January, and a presentation from Andrew Smithson of IBM Hursley on CICS Transaction Gateway V8.1 for the Virtual CICS user group.

If you do celebrate it, Merry Christmas and a happy New Year. I’ll be back blogging in January.
Trevor Eddolls

Saturday, 19 November 2011

Continuous availability – no longer a dream?

Zero downtime is a goal that many companies are striving for. It sounds so straighforward, and yet it’s not that simple to achieve – especially when it involves the continuous availability of large, high-volume databases. One of the inherent problems is that data replication for high-availability is filled with many nuances that need to be addressed for a successful deployment, including maintaining sub-second latency, active/active considerations, scalability options, conflict detection/resolution, recovery, exception processing, and verifying that the source/target are synchronized properly.

One of the problems that organizations face is the need to address lots of different business issues using, what often involves, multiple software packages. Integrating these different pieces of software – perhaps even from different vendors – can add an extra level of complexity to the job in hand. What those organizations really need is a single piece of software that’s flexible enough to provide a comprehensive solution for changed data capture, replication, enhancing existing ETL (Extract, Transform, and Load) processes, and data migrations/conversions. Quite a big ask.

Wouldn’t you be interested in software that offers industrial-strength, near-real-time data integration solutions that include high-performance Changed Data Capture (CDC), data replication, data synchronization, enhanced ETL and business event publishing? And what if it was equally simple to experience the high-speed delivery of mainframe data (IMS, DB2, VSAM, etc) into data warehouses and downstream applications? Too good to be true?

If you’re like me, you carry around a list of capabilities in your head, and tick them off – or more often don’t tick them off – when you give software the once over. So here’s the kind of things I’d have on my list for an integration engine. In general I’d expect:
  • Concurrent operation across multiple operating system platforms
  • Multi-step processes within a single script (UNION)
  • Simultaneous multi-record type file handling
  • Multi-level array handling (repeating groups) of source data store records/rows
  • Data filtering and cleansing
  • Dynamic look-up table processing
  • Support for data transfer and communication using TCP/IP and MQSeries
  • Preservation of referential integrity (RI) rules on target updates
  • Joins/Merges of heterogeneous databases/files.
In terms of data transformation I’d like to see:
  • Case (If/Else) logic
  • Extensive date cleansing and formatting
  • Arithmetic functions (add, subtract, multiply, etc)
  • Aggregation functions (sum, min, max, avg, etc)
  • Data type conversions
  • String functions
  • Data filtering
  • XML data formatting
  • Delimited data formatting.
When it comes to datastore processing I’d want:
  • High performance bulk data transfer
  • Concurrent processing of multiple data store types
  • Creation of target data stores from source data store format
  • Insert/append to existing target data stores
  • Update/replace existing target data stores
  • Delete from existing target data stores
  • New column/field creation Data Movement.
And for Data Movement, my list includes MQSeries, TCP/IP, and FTP.

If there was also some kind of Integration Center that had an easy-to-use Graphical User Interface (GUI) enabling users to quickly develop data integration interfaces from a single control point – that would be good. Additionally, some way to develop, deploy and maintain data interfaces, create relational DDL (Data Definition Language), XML (Extensible Mark-up Language ) and C/C++ structures from COBOL Copybooks, monitor the status of integration engines, and contain an integrated metadata repository – that would be a real plus.

I’d definitely want to find out more about a single piece of software that provided high-performance Changed Data Capture (CDC) and Apply, data replication, event publishing, Extract, Transformation, and Load (ETL), and data conversions/migrations.

So, if you’re like me and want to know more, there’s a webinar from SQData’s Scott Quillicy on 1 December at 2pm GMT (8am CST). To join the webinar from your PC, you need to register before the event at https://www1.gotomeeting.com/register/844029904. I’ll see you there.