Mainframes and AIOps

When I first started work as an operator on a mainframe, it was a different world to the current mainframe environment. The company I worked for bought a mainframe with flashing lights on it that impressed visitors to our air-conditioned mainframe room. We used to have removable DASD, and we would test our strength by trying to lift one in each hand level with our shoulders. We spent most of our days loading tapes, feeding in punched cards (and paper tape for one particular job), and loading multi-line paper in the printers. And we also tried to fix problems when they occurred. On the night shift, we would often play golf, using the holes in the floor as the targets for our shots. Jobs were mainly submitted by people working in the huge open-plan office, with a few working from another office. We did have people using CICS, but they were in a special part of the open-plan office.

It seems like a hundred years ago. There was no Internet, no-one working from a browser, no API economy linking our applications with apps running elsewhere. No mobile computing. No cloud. No containers. And, if we couldn’t fix a problem, there could be a long time-delay until someone did fix it. Like I say, a different world.

Nowadays, there is more technology, and everyone expects everything to be operational all the time. People expect optimal performance. And one of the ways to achieve that is to make use of Artificial Intelligence for IT Operations (AIOps). AIOps can automate and enhance IT operations. It can quickly identify problems and remediate them. Using Machine Learning (ML), it can learn to perform better. And using big data, it can quickly identify problems as the start to occur. Putting these two things together means that AIOps can work in situations that could be too complex or changing too quickly for a human operator to work well in.

When I was enjoying myself as an operator, everything I needed to check was a short walk from the console. Nowadays, devices can be many miles away, and information for applications can be coming from anywhere on the planet. In addition, I thought I was a bit of an expert on mainframes, nowadays, I would need to have expertise on cloud, distributed, mobile, Internet of Things (IoT), and who knows what else.

But AIOps isn’t replacing human operators. What it is doing is allowing operations teams to work faster and smarter, dealing with issues as they start to arise and before customers notice something is wrong. And those human operators can deal with issues that haven’t come up before – which makes the job much more interesting than it was in my day.

As well as collecting and combining large volumes of data coming from applications and monitoring tools, AIOps can sift through the data and filter out anything that isn’t important. It can then take what’s left and either automatically deal with it or report it for the operations team to deal with. That often involves the AIOps software reporting the root cause of the issue and suggesting possible remediation strategies. As mentioned earlier, ML will use previous results to update algorithms or even create new ones in order to work even more effectively.

AIOps can also collect data from across the IT environment, and can then make suggestions about what might not be performing well and what remedial action might need to take place to the appropriate people. That might be the applications team, or the network team, or the storage team, etc. That way, the old silos that so often held up the work to solve problems are being broken down. In addition, it’s worth getting everyone to understand that using an AIOps tool in each of these different areas will not allow the software to see the bigger picture and will not have access to relevant data from outside the silo to correlate information. So, it is imperative that the AIOps tool runs across all the different IT areas of the business.

In addition to AIOps reducing the time taken to fix problems, AIOps also moves the approach to problems from being reactive to being proactive, which means that problems are solved before they become noticeable as problems.

For mainframe sites that are moving to a cloud environment – whether that’s public cloud, private cloud, or a combination – AIOps can be used to provide centralized visibility across the different environments, which helps the operations team to identify and remediate problems much more quickly than without the software.

There is an IBM Cloud Pak for Watson AIOps that can correlate data across the various tools that are used together in complex tasks (toolchain) and uncover information and problems quickly.

A survey by Micro Focus of sites using AIOps found a 76% reduction in the number of incidents and a 400% reduction in the mean time to repair. The survey also found that the risk of not evolving to AIOps is $1.2M on incident escalations that could often be avoided.

AIOps definitely seems to be the way of the future. As IT environments become more distributed, more dynamic, more hybrid, and even more componentized; and as everything has to happen so quickly and down time just can’t be countenanced, then the agility that comes with AIOps definitely seems to be the way to go for most mainframe sites.

Another advantage is that the operations team can work from anywhere, whether that’s a network operations centre or from home, and still be receiving updates from the AIOps software and talk to the teams that are best suited to fix whatever issue has been highlighted. All this without having to sift through reams and reams of data to identify where the fault is located.

The days of machine room golf are long gone!

Working from home – is VPN safe anymore?

The pandemic struck and most of those people who usually worked from an office started working from home. Even mainframers were working on a laptop from home. The IT team needed a speedy way to get these people securely working from home and VPN became a three-letter acronym known to millions of people. Users were happy, they could work from home. Management were happy because their employees could work remotely. And hackers were happy because they could now pretty much gain access to every organization that they wanted!

For some people, the idea that virtual private networks (VPNs) aren’t secure must come as a shock, but anyone looking at the news this year will realize that VPN security has been a problem.

Before we look at what’s been in the news, let’s quickly remind ourselves what VPN is meant to do and how it works.

While everyone was safely inside their corporate offices working, they could access data and applications safely. However, once they were working remotely, they really needed a dedicated cable running from their laptop to the company network. This would be a private network just for them, and this would keep safe all the information passing backwards and forwards. However, this was never going to happen. So, users connected from home to office over the Internet. And in order to keep the packets of information safe, they used a virtual private network. In effect, there’s a virtual tunnel through the Internet from one end to the other. It keeps the information secure, and the activity anonymous. Sounds like the problem’s solved, doesn’t it?

Capcom, the video games developer in Japan, was hacked in November last year. It appears that Capcom’s US subsidiary retained an older VPN service as a backup, and this was used by the hackers to get into North American and Japanese networks, where they knocked out email and file servers. Apart from a ransomware demand, 390,000 individuals may have had their data compromised.

Zyxel has recently warned customers that its devices are being attacked, including security appliances having remote management or SSL VPN enabled, namely in the USG/ZyWALL, USG FLEX, ATP, and VPN series running on-premise ZLD firmware. Once hackers can access the device, they can connect to previously unknown accounts hardwired into the devices.

Worryingly, we hear that what's thought to be a North Korean hacking group has got into South Korea’s atomic research agency. Hackers breached the Korea Atomic Energy Research Institute (KAERI) network on 14 May using a VPN system vulnerability.

LimeVPN, VPN provider, has been hacked and 69,000 users have had their personal information stolen. A backup server was hacked that included a database of the details of all of LimeVPN's customers. The hackers claim to now possess the private key of every user, which means their data could be decrypted.

In the first quarter of this year, there was a 1,916% increase in attacks against Fortinet’s SSL-VPN. This was probably due to three different problems with the Fortinet FortiOS, Firstly, it seems that unauthenticated hackers could use specific HTTP resource requests on the SSL VPN web portal, which allowed them to download system files. Secondly, unauthenticated attackers on the same subnet could impersonate an LDAP server and intercept information. Thirdly, simply by changing the case of their username, hackers could successfully log in without being prompted for a second authentication factor. Clearly, anyone using Fortinet devices could be hacked, giving bad actors access to their network. The company has released patches to fix the vulnerabilities.

Also, in the first quarter of the year it’s been revealed that there has been a 1,527% increase in attacks on Pulse Connect Secure VPNs. In fact, in April the Metropolitan Transportation Authority (MTA) in New York revealed it was breached by hackers linked to the Chinese government. The Metropolitan Water District of Southern California and communications company, Verizon, were also targeted. It seems there was a zero-day vulnerability in Pulse Connect Secure VPN, which has now been patched.

Another attack using VPN was on Colonial Pipeline, which operates a pipeline from Texas to New Jersey and provides around 45 percent of the USA’s East Coast’s oil supplies. In this attack, hackers used an old VPN account that still provided access to the network. All they needed was a username and a password. The recommendation is that everyone uses multi-factor authentication when users login. And, all old accounts are removed, so they can’t be used to gain access to the network.

The problem is so serious that Help Net Security ran a headline in June saying, “VPN attacks up nearly 2000% as companies embrace a hybrid workplace”. Adding to everyone’s concern is the fact that US and British authorities have issued a joint advisory notice saying that Unit 26165, part of Russia's military spy agency, had been using VPNs and Tor to conduct "widespread, distributed, and anonymised brute force access attempts against hundreds of government and private sector targets". These include government offices, political parties, energy companies, law firms, and media organizations.

It makes sense now for every organization using VPN for their working-from-anywhere employees – and that includes companies that also have a mainframe – review the security of their VPN setup, and replace products where necessary or patch everything to the latest version. It might be a good idea to look at zero trust networks to keep checking that users are authorized and only doing what they’re authorized to do. And keep an ear to the ground for any news of VPN hacks and quickly respond. With nation states now using hacking teams in addition to criminal gangs, no-one is completely safe.

Security and the pandemic

Having spent a number of hours each week talking and writing about security, I kind of assume that it’s a topic that everyone is interested in and that everyone is pretty much clued up about these days. Every company of any size uses external penetration testing (pentesting) experts to check that their systems are secure (even mainframes), and most companies nowadays seem to run dummy phishing attacks just to see which of their employees are still clicking on dodgy links and downloading questionable attachments. So, it was interesting to see the results of the IBM Consumer Survey: Security Side Effects of the Pandemic. The survey was carried out in March by Morning Consult, which asked 22,000 people around the world about their online security habits.

The key findings of the survey were:

• Global respondents shifted further into digital interactions during COVID-19 and are likely to continue digital-first interactions in life after the COVID-19 pandemic.
• Across all categories, global consumers created about 15 new online accounts during the pandemic. Younger respondents created more new accounts during the pandemic across categories, and created more accounts across each category than any other age group or generation.
• Over four in five (82%) global respondents are re-using the same credentials that they have used for other accounts at least some of the time. Younger respondents are more likely to say they always or mostly re-use the same credentials that they have used for other accounts.
• Many would still rather place an order digitally – even if there were security/privacy concerns. Over four in ten (44%) global respondents, and 51% of millennials would rather place and pay for an order digitally than go to a physical location or call to place an order even if they had concerns about the website/app’s safety or privacy.
• A majority of global respondents (63%) accessed COVID-related services via digital channels – including mobile apps, websites, email, and text messages.
• Nearly half (44%) of respondents do not plan to delete or deactivate any of the new accounts they created during the pandemic after society returns to pre-pandemic norms.

The report says that “consumers not only increased their reliance on digital channels during the pandemic, but also that this ‘digital dependence’ is expected to linger even after society returns to pre-pandemic norms. Consumers reported they will continue to rely on digital services at higher rates than before the pandemic, and many say they will not delete any of the new accounts they created during that time.”

The survey found that respondents under 50 are most likely to predict they will interact through digital formats in life after the COVID-19 pandemic. Although the average number of new accounts was 15, millennials created over 18 new online accounts during the pandemic, more than any other generation. The only slightly good news was that the survey found that 56% respondents would remove permission for an application to track behaviour if the app requested permission to do so.

The survey concludes that “consumers’ increased reliance on digital channels during the pandemic may have caused more lax attitudes towards security – with the convenience of digital ordering often outweighing security and privacy concerns. Many consumers (particularly younger generations) say they would rather place an order digitally, even if there were security or privacy concerns with the application. Additionally, consumers rarely decline to use a new digital platform due to security or privacy concerns. This surge in new accounts may also be creating password fatigue, with consumers reporting high levels of password reuse across their accounts. This means many of the new accounts created during the pandemic likely relied on reused credentials, which may have been compromised in previous data breaches.

The survey also found that 35% of respondents have accepted terms they were uncomfortable with so they could use a service. 41% would avoid using an online platform to shop or place an order over concerns over app/website security, and 38% would avoid using online platforms if they had concerns around privacy.

For vendors, a bad online user experience can lead to people giving up on an online purchase, application, or transaction based on negative experiences. 42% said they’d done that when logging in, 41% when signing up, and 41%, again, when completing payment. Younger respondents are more likely to give up than older people. The survey also found that, on average, respondents across all age groups would attempt about 3-4 logins before they decided to reset their login credentials.

While 59% of respondents expect to spend between 1-5 minutes setting up a new digital account, 57% would reconsider setting up a non-essential digital account after spending 1-5 minutes. 44% of respondents keep online account information in their memory, and 32% have it written on paper. And while passwords are the preferred method to log in, respondents under 35 are more likely than older generations to prefer single sign-on or biometrics. It’s good to see that around two thirds of respondents have used two-factor or multi-factor authentication to access an online account. 65% of respondents are very or somewhat familiar with the concept of digital credentials, and 76% of respondents would be very or somewhat likely to use digital credentials.

63% of respondents have accessed COVID-related services via digital channels, and younger respondents were more likely to have accessed COVID related services digitally.

Overall, it’s an interesting, but worrying survey. IBM Security did offer companies the following guidance:

“Zero Trust Approach: given increasing risks, companies should consider evolving to a ‘zero trust’ security approach, which operates under the assumption that an authenticated identity, or the network itself, may already be compromised – therefore, it continuously validates the conditions for connection between users, data, and resources to determine authorization and need. This approach requires companies to unify their security data and approach, with the goal of wrapping security context around every user, every device, and every interaction.

“Modernizing Consumer IAM: investing in a modernized Consumer Identity and Access Management (CIAM) strategy can help companies increase digital engagement – providing a frictionless user experience across digital platforms and using behavioural analytics to decrease the risk of fraudulent account use.

“Data Protection & Privacy: having more digital users means that companies will also have more sensitive consumer data to protect. Organizations must ensure that strong data security controls are in place to prevent unauthorized access – from monitoring data to detect suspicious activity, to encrypting sensitive data wherever it travels. Companies should also implement the right privacy policies on premise, and in the cloud, in order to maintain consumer trust.

“Put Security to the Test: with usage and reliance on digital platforms changing rapidly, companies should consider dedicated testing to ensure the security strategies and technologies they’ve relied on previously still hold up in this new landscape. Re-evaluating the effectiveness of incident response plans, and testing applications for security vulnerabilities, are both important components of this process.”

Good advice.