Sunk without trace

There was a time when using the trace facility was really the final strategy. You’d perhaps have tried everything else to find what was going wrong first. And when nothing seemed to have worked, you’d equip yourself with all the necessary manuals – and that could be quite a few – and run the trace and start the hard job of interpreting the results. And then try to fix the problem. Those days are long gone thanks to more modern software tools, but, to many people, the memories linger on!

I recently bumped into William Data Systems’ Tony Amies, who took the time to show me some of the things he was working on. And one of those things was making trace much, much, more user friendly.

Tony showed me WDS’s ZEN product, which, as you may know, allows lots of network monitoring information to be collated and viewed from anywhere using a browser. Information can appear as coloured boxes, which once you clicked on them display more-and-more information in a clever drill-down manner. Fairly quickly, you can identify the component that has exceeded some predetermined threshold.

WDS has a number of products in the ZEN family and you can use buttons on the browser to switch between them – giving you information about different aspects of performance. ZIM the ZEN IP MONITOR can detect error conditions, then ZEN TRACE and SOLVE (ZTS – which used to be called EXIGENCE) can be used to start, stop, and view traces. Now that has got to be so much easier than in the Old Days!

Tony showed how a TCP trace could be carried out in seconds, explaining that there were lots of commands embedded in it. Tony explained how network tracing can be so difficult. For example, using Enterprise Extender, which allows SNA applications to run over TCP networks, results in encapsulated messages. Tony demonstrated software that was able to look inside the message to see what was there – in terms of different types of header. He then explained how this works with FMH5, UDP, IP, APPN, HPR, and more. He explained that sites using the Cisco load balancing GRE tunnelling protocol can also be opened to see the true header for the message. All very clever stuff – and no manuals in sight.

In fact, on a number of occasions a right mouse click on some information in the display would produce a pop-up box explaining exactly what some term or other actually meant. So there was no need for any manuals. The display could show delays, highlight response time problems, and the TCP window size.

Tony also showed me a piece of software that drew a diagram of a Sysplex Distributor – which shows the IP addresses and links on a mainframe system. The software also highlighted where there were issues. And, like the rest of the software we looked at, you could drill down to find exactly where any problem were. In fact, Tony was sure that this would allow customers to identify potential issues before their users did. Behind the scenes, information from netstat and other commands were being used to drive the display.

We talked about customers being able to build business service views of what was going on their system and how useful that would be for each of their customers. That kind of bespoke requirement wasn’t something that Tony could necessarily build into the software, but all it requires is a knowledge of REXX to make it happen. And most z/OS sites have at least one person who code in REXX.

Lastly, we talked about problem resolution when you have two or more systems that don’t seem to be talking to each other. Currently, you need to log into each system and run traces to find out which of the systems has the problem. Tony plans to implement a ‘grouptrace’ feature that allows the user to tell the software to run a trace on these two (or more) systems. The results will come back from both systems and be visible from the browser. The results will be displayed in timestamp order and it will be possible to see on which of the systems the problem is. As easy that.

Too often we’d be sunk without a trace facility. Now we have an example of a way to be able to use trace across multiple systems and simply click to drill down to identify the problem.

Network management issues

I want to know what’s going on on my network – what do I do? Well this seemingly simple question has a complicated answer, which is historical and all to do with the separate worlds of the mainframes and everyone else coming together to give us our current complicated environment. Just to summarize, mainframes used SNA and VTAM to communicate and everyone else used TCP/IP. Sometime in the 1990s, all this came together. In addition, we now  have people on iPads and smartphones using our networks – in fact, we can have people on iPads and smartphones controlling our networks!

So, in terms of software, what are our choices for network management?

IBM has Tivoli OMEGAMON XE for mainframe networks, which allows you to monitor VTAM and TCP/IP networks and identify problems within those networks.

CA has its CA NetMaster Network Management for TCP IP and CA NetMaster Network Management for SNA products, which allow users to “proactively manage” their network. And there are a number of other more specialized products including CA NSM NetMaster Option, which lets you “manage your mainframe networks from CA Unicenter Network and Systems Management”.

SDS has Vital Signs for IP and Vital Signs for VTAM, which are both monitoring software.

William Data Systems has a whole range of products including ZEN OSA MONITOR for OSA performance monitoring and management, ZEN IP MONITOR (IMPLEX) for IP performance monitoring and management, ZEN EE MONITOR (FERRET) for APPN/HPR and EE monitoring and management, ZEN TRACE and SOLVE (EXIGENCE) for network problem determination, ZEN FTP CONTROL (FTPALERT) for audited FTP monitoring and management, ZEN EE SECURITY (APIAS) for secure EE deployment across any network, and ZEN AUTOMATION for automating network and operational tasks.

In fact, William Data Systems has just release Version 4.7 of ZEN TRACE and SOLVE, which includes more customer-suggested z/OS network tracing and solution capabilities to help busy z/OS network support people do more in less time – they claim.

If you are plagued by mainframe network issues, you may well be interested in the new features they’ve incorporated into ZEN TRACE and SOLVE V4.7, including:

• The new “Peek” function, which provides in-flight viewing of active IP traces. This means that it is now possible to view an active trace while it is still running, tracking its progress ‘in-flight', and then stopping it when the condition being sought in the trace is detected. This can save considerable CPU costs and user time.
• Traces originally captured in OSAENTA, libpcap, or Sniffer format can now be imported into ZTS. This can make network problem solving both faster and easier.
• As well as being able to accept commands from the operator console, ZEN TRACE and SOLVE now adds the ability to issue any commands (including ZTS commands) from JCL. When combined with the capabilities provided by the new import features, this enables different trace formats to be automatically imported into ZTS for analysis. This greatly simplifies the whole network problem solving process.
• Tracing of IPv6 is now faster and easier with the provision of improved Flow display text descriptions for recognizable packet contents, as well as the capability to search using IPv6 format addresses.
• Tracing packets in large Sysplex environments is inherently complex, but ZEN TRACE and SOLVE v4.7 now makes it easier by providing improved facilities for expanding encapsulated data used by Sysplex protocols such as GRE and VIPA Route. Being able to view and expand encapsulated payload data is a vital tool for rapid problem diagnosis in large Enterprise networks.

If you’re finding network management is an issue, it might well be worth a look.