SharePoint 2007 site collection auditing

We start the year with another in our series of SharePoint hints and tips from our expert Darren Pritchard. This time he’s looking at how to sucessfully audit SharePoint site collections.

To enable/disable SharePoint auditing:

1. Open the site that you wish to audit
2. Click ‘Site Actions’
3. Select ‘Site Settings’
4. Click ‘Modify All Site Settings’
5. Under ‘Site Collection Administration’ you have ‘Site collection audit settings’
6. You would see Figure 1.
7. Here you can select what you wish to audit
8. Be very careful, SharePoint will not automatically purge these audit logs. They will continue to grow as long as they are enabled. I have seen content databases with 80GB of audit files!

Figure 1: Configure Audit settings page

To trim audit files:

1. Create the batch file shown in Figure 2. Note: CONTENTDATABASE is the name of the database within SQL.
2. Save it as ‘Purge Audit Logs.bat’
3. This will need to be run on your SharePoint Web frontend server as an Administrator. Do not be alarmed if it takes a little while to run. The time it takes depends on the amount of audit logs you are trying to trim.

SET STSADM="c:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\12\bin\STSADM.EXE"

%stsadm% -o trimauditlog -date -databasename

Pause

Figure 2: Purge Audit Logs.bat

I would suggest running this on a system without users because it may have an impact on performance.

If you have large amounts of audit log files, I would recommend running the trim command on a month’s worth of audit logs at a time. The SQL transaction log file will grow very large during the trim process.

Once you have trimmed the audit logs you will need to run a shrink database task within SQL to reclaim the space used during the trim process.

I'd like to thank Darren for his continuing contributions, and a happy New Year to everyone.

2011 at iTech-Ed Ltd

Well, as another year comes partying to an end, and everyone stops checking their e-mails on their smartphones or tablets and finally starts to let their hair down and enjoy a glass of something alcoholic, I thought I’d review the year through the lens of my company – iTech-Ed Ltd (www.itech-ed.com).

January started the year, as most Januaries do, with the publication of the Arcati Mainframe Yearbook. The 2011 edition is still available for download from www.arcati.com/newyearbook11. The 2012 edition will be available in a couple of weeks. As always the Arcati Mainframe Yearbook  includes its annual user survey, an up-to-date directory of vendors and consultants, a media guide, a strategy section with papers on mainframe trends and directions, a glossary of terminology, and a technical specification section. And each year, it gets downloaded by around 15,000 mainframe professionals.

February saw the launch of the new series of Virtual IMS user group meetings. The user group is now sponsored by Fundi Software and hosted at www.fundi.com/virtualims. The first speaker was Jim Martin from Fundi Software, whose presentation was called, “Solving the problem when IMS isn't the cause”.

In March, everyone seemed to be talking about cloud computing.

April’s meeting of the Virtual IMS user group included a presentation from Ron Haupert, a Senior Technologist with Rocket Software. His talk was called, “Simplify and improve database administration by leveraging your storage system”.

In May, Mark Lillycrop, Director of Arcati Ltd and I took part in a ‘Scheduled Chat’ in the ‘House of Mainframe’ section of CA’s May Mainframe Madness month. May also witnessed the launch of the new Virtual CICS user group – again sponsored by Fundi – with its Web site at www.fundi.com/virtualcics. Our opening presentation was from Fundi’s Jim Martin talking about, “Solving the problem when CICS isn't the cause”.

In June, I was asked by ITToolbox to lead a discussion in the Data Center Infrastructure section of their Web site. At the Virtual IMS user group meeting, Gary Weinhold a Systems Engineer and Verna Bartlett Head of Marketing with Data Kinetics talked about, “MSU reduction due to in-memory table management with (any) IMS applications”.

In July, I was selected for the Destination z (www.destinationz.org/) member spotlight. The Virtual CICS user group saw a presentation from Jeff Geminder, Principal Consultant with CA, called, “Cross-enterprise application performance monitoring and CICS-specific drill-down: approaches to finding the performance problem needle in the heterogeneous haystack”. I was also a guest blogger on the Destination z Web site.

In August, my article CICS Top Performance and Tuning Issues was published in z/Journal. I had a guest blog published on Destination z. The Virtual IMS user group had a presentation from Scott Quillicy, CEO and Founder of SQData. His talk was called, “IMS replication for high-availability”.

For the September meeting, Charles Jones, from the Product Management group at Rocket Software, gave a talk to the Virtual CICS user group called, “CICS TS 4.2: Leveraging event processing and high-performance Java”. I wrote a guest blog for the Destination z Web site.

October saw a presentation from Rosemary Galvan, Principal Software Consultant – IMS, with BMC. Her talk to the Virtual IMS user group was called, “Database Performance – Could Have, Should Have, Would Have”. I had a guest blog on the Destination z Web site.

In November, my Mainframe Update blog at mainframeupdate.blogspot.com was a finalist in the Computer Weekly Social Media Awards 2011. Also in November the Arcati Mainframe Yearbook user survey was launched. And Eugene S Hudders, president of C\TREK Corp, gave a presentation to the Virtual CICS user group called, “CICS TS Performance – Tuning LSR Pools”. I also had a guest blog on the Destination z Web site.

And finally, in December, I had an article entitled, Ways to Save Money and Improve IT Services published in z/Journal. The final speaker for the year at the Virtual IMS user group was Suzie Wendler, a Consulting IT Specialist in the IBM IMS Advanced Technical Skills organization, who talked about, “IMS V12”. I chaired a webinar for SQData entitled, “How Important is Continous Availability of Critical Applications to Your Company?”And there was a guest blog on the Destination z Web site.

What else, well apart from a full year of writing and consultancy work,  I was made an IBM Champion for the third year running.

Looking forward to 2012, we have the launch of the Arcati Mainframe Yearbook in January, and a presentation from Andrew Smithson of IBM Hursley on CICS Transaction Gateway V8.1 for the Virtual CICS user group.

If you do celebrate it, Merry Christmas and a happy New Year. I’ll be back blogging in January.
Trevor Eddolls

Sunk without trace

There was a time when using the trace facility was really the final strategy. You’d perhaps have tried everything else to find what was going wrong first. And when nothing seemed to have worked, you’d equip yourself with all the necessary manuals – and that could be quite a few – and run the trace and start the hard job of interpreting the results. And then try to fix the problem. Those days are long gone thanks to more modern software tools, but, to many people, the memories linger on!

I recently bumped into William Data Systems’ Tony Amies, who took the time to show me some of the things he was working on. And one of those things was making trace much, much, more user friendly.

Tony showed me WDS’s ZEN product, which, as you may know, allows lots of network monitoring information to be collated and viewed from anywhere using a browser. Information can appear as coloured boxes, which once you clicked on them display more-and-more information in a clever drill-down manner. Fairly quickly, you can identify the component that has exceeded some predetermined threshold.

WDS has a number of products in the ZEN family and you can use buttons on the browser to switch between them – giving you information about different aspects of performance. ZIM the ZEN IP MONITOR can detect error conditions, then ZEN TRACE and SOLVE (ZTS – which used to be called EXIGENCE) can be used to start, stop, and view traces. Now that has got to be so much easier than in the Old Days!

Tony showed how a TCP trace could be carried out in seconds, explaining that there were lots of commands embedded in it. Tony explained how network tracing can be so difficult. For example, using Enterprise Extender, which allows SNA applications to run over TCP networks, results in encapsulated messages. Tony demonstrated software that was able to look inside the message to see what was there – in terms of different types of header. He then explained how this works with FMH5, UDP, IP, APPN, HPR, and more. He explained that sites using the Cisco load balancing GRE tunnelling protocol can also be opened to see the true header for the message. All very clever stuff – and no manuals in sight.

In fact, on a number of occasions a right mouse click on some information in the display would produce a pop-up box explaining exactly what some term or other actually meant. So there was no need for any manuals. The display could show delays, highlight response time problems, and the TCP window size.

Tony also showed me a piece of software that drew a diagram of a Sysplex Distributor – which shows the IP addresses and links on a mainframe system. The software also highlighted where there were issues. And, like the rest of the software we looked at, you could drill down to find exactly where any problem were. In fact, Tony was sure that this would allow customers to identify potential issues before their users did. Behind the scenes, information from netstat and other commands were being used to drive the display.

We talked about customers being able to build business service views of what was going on their system and how useful that would be for each of their customers. That kind of bespoke requirement wasn’t something that Tony could necessarily build into the software, but all it requires is a knowledge of REXX to make it happen. And most z/OS sites have at least one person who code in REXX.

Lastly, we talked about problem resolution when you have two or more systems that don’t seem to be talking to each other. Currently, you need to log into each system and run traces to find out which of the systems has the problem. Tony plans to implement a ‘grouptrace’ feature that allows the user to tell the software to run a trace on these two (or more) systems. The results will come back from both systems and be visible from the browser. The results will be displayed in timestamp order and it will be possible to see on which of the systems the problem is. As easy that.

Too often we’d be sunk without a trace facility. Now we have an example of a way to be able to use trace across multiple systems and simply click to drill down to identify the problem.

The future - gamification and augmented reality

I remember many years ago saying to my children that one day, when they walked around London or any capital city, they’d be able to hold up their phone in front of a statue or building and information would appear on screen explaining what the statue commemorated, etc.
 
But how about if you could hold up your phone in front of the mainframe or some x86 server, and on screen would appear statistics about usage and performance? You could then take appropriate action to resolve hot spots and capacity issues. All just a dream? Apparently not.
 
Beverley Head’s blog at IT Wire (www.itwire.com/cloud-computing/51364-bmc-sniffs-out-next-generation-tools) from last week suggests that BMC is exploring how it can harness gamification and augmented reality techniques in the next generation of its systems management tools. Beverley reports Suhas Kelkar, a chief technology officer for BMC, describing the server example I gave above. Suhas adds: “If someone comes across an intelligent solution they should add it to the knowledge base. But hardly anyone does it. But what if you gamify the system and reward people for doing that?”
 
So there we have it... Augmented reality is the appearance on your phone of information about server capacity. And it could be about anything else. Wouldn’t it be great to hold your phone over a cable and read off the upstream and downstream broadband speeds?
 
Gamification – a new word, so try to drop into conversations, if you want to sound up-to-date – then is the fun part of using software. The part that is all too often missing!
 
Interestingly, I found an article about gamification from back in May this year at www.dnitza.com/2011/05/21/gamifcation-making-fun-of-the-web/. Daniel Nitsikopoulos talks about “Gamification: Making fun of the web”. He asserts that: “Gamification is one of the newest and I believe one of the biggest movements in the creative world today. It is the concept that you can apply game mechanics (elements that make games fun, engaging, and in some cases competitive) to things that aren’t typically considered a game, or even fun! From work, to health, to socialising, to cooking, to just about anything!”
 
So if BMC is looking at gamification and augmented reality, you can bet CA Technologies is as well. And that other big software supplier, IBM! But I would bet that the really exciting stuff is going to come from smaller companies. And I would also predict that these smaller companies will one-by-one be swallowed up by the existing software giants.
 
It definitely gets my vote as a direction I’d like technology to move in. Some equivalent to Google Goggles that not only identifies what you’re looking at (the Web server, or the z/Linux LPAR, or whatever) and provides current performance information. And then makes it fun to resolve any problems that might have been identified. Maybe when you look at the x86 server, it appears in red if there are issues. Then the length of time you take to resolve the problem is entered onto a leader board. And at the end of the week you can see who is the fastest techie in your team! Or perhaps the only green screen you’ll see will mean ‘game over’!
 

Managing expectations

Have you ever been out for a few drinks with friends. Maybe you’ve had more to drink than usual. What happens next? Well the answer seems to depend on which country you and the people you’re drinking with come from.

It seems that in some countries, people take the view that alcohol is so strong and people are so weak that anything is permissible. You can stand up in court and explain your actions – whatever they may be – by saying that you’d drunk too much. In other countries – like Italy – alcohol is grouped with food in the minds of people. You drink when you eat. You eat and drink with your friends and family. Using the defence of excessive alcohol would seem as absurd as using the defence of having eaten too many burgers to explain antisocial behaviour.

And it’s exactly the same with users. If they expect nanosecond response times to a CICS transaction they will be miffed when a response takes a second or two. Whereas, if they are used to a response taking a few seconds, they will be pleased when it takes less than two seconds for their screen to refresh.

Managing expectations can be the difference between happy users and unhappy users. In the same way it can be the difference between alcoholic destruction of everything on the way home and a great night out.

Banks seem to use the opposite technique. They pretend that they offer great service, but as every customer knows, they don’t. The news is always full of demands that the banks should lone more – particularly to small businesses. Speaking as the owner of a small business, I think this is not the real problem. I think the problem for most small businesses is the fact that banks charge too much for their services.

Now I don’t mind banks charging for the work they do – that’s the same model I use to stay in business! What I object to is the amount they charge. And I think this is part of the problem most small businesses face. For example, here in the UK, I get a lot of dollar cheques from the USA. I get an exchange rate that’s clearly in the bank’s favour and then I get charged for paying the money into my account. I get charged for paying in UK cheques. And I get charged even more for paying in cash!

So I guess my expectations are that banks are going to rip me off. They do nothing to manage that and make things better. And they really are the reason that a lot of small businesses are having a hard time during this recession – or whatever we’re calling it.

Just revisiting the psychology again. There are experiments where two groups of students were given free drinks all evening. Both groups got equally drunk. Then the experimenters explained that one group had drunk alcohol and the other group hadn’t. Once this second group were told they hadn’t had any alcohol, they immediately sobered up. Their expectations changed completely and they now behaved in a different way.

So, while IT strives to offer the best service to its users. It’s important that conversations take place between the two groups so that users can describe their expectations of the service they want to receive, and IT can explain how the service is being delivered and give a realistic idea of what an end user shoould expect. Most sites have SLAs (Service Level Agreements), but these tend to be gathering dust somewhere rather than being constantly referred to. The importance of the conversation is to manage expectations and make sure both groups can continue to work, happy in the knowledge that they are getting or delivering the level of service that everyone expects.

Don’t forget that on Thursday 1 December there’s a webinar entitled: “How Important is the Continuous Availability of Your Critical Applications?” at 2pm GMT. You can register for the event at https://www1.gotomeeting.com/register/844029904.

And this is the last week that you can complete the Arcati Mainframe Yearbook user survey at http://www.arcati.com/usersurvey12. We need all the completed surveys by Friday evening.

Continuous availability – no longer a dream?

Zero downtime is a goal that many companies are striving for. It sounds so straighforward, and yet it’s not that simple to achieve – especially when it involves the continuous availability of large, high-volume databases. One of the inherent problems is that data replication for high-availability is filled with many nuances that need to be addressed for a successful deployment, including maintaining sub-second latency, active/active considerations, scalability options, conflict detection/resolution, recovery, exception processing, and verifying that the source/target are synchronized properly.

One of the problems that organizations face is the need to address lots of different business issues using, what often involves, multiple software packages. Integrating these different pieces of software – perhaps even from different vendors – can add an extra level of complexity to the job in hand. What those organizations really need is a single piece of software that’s flexible enough to provide a comprehensive solution for changed data capture, replication, enhancing existing ETL (Extract, Transform, and Load) processes, and data migrations/conversions. Quite a big ask.

Wouldn’t you be interested in software that offers industrial-strength, near-real-time data integration solutions that include high-performance Changed Data Capture (CDC), data replication, data synchronization, enhanced ETL and business event publishing? And what if it was equally simple to experience the high-speed delivery of mainframe data (IMS, DB2, VSAM, etc) into data warehouses and downstream applications? Too good to be true?

If you’re like me, you carry around a list of capabilities in your head, and tick them off – or more often don’t tick them off – when you give software the once over. So here’s the kind of things I’d have on my list for an integration engine. In general I’d expect:

• Concurrent operation across multiple operating system platforms
• Multi-step processes within a single script (UNION)
• Simultaneous multi-record type file handling
• Multi-level array handling (repeating groups) of source data store records/rows
• Data filtering and cleansing
• Dynamic look-up table processing
• Support for data transfer and communication using TCP/IP and MQSeries
• Preservation of referential integrity (RI) rules on target updates
• Joins/Merges of heterogeneous databases/files.

In terms of data transformation I’d like to see:

• Case (If/Else) logic
• Extensive date cleansing and formatting
• Arithmetic functions (add, subtract, multiply, etc)
• Aggregation functions (sum, min, max, avg, etc)
• Data type conversions
• String functions
• Data filtering
• XML data formatting
• Delimited data formatting.

When it comes to datastore processing I’d want:

• High performance bulk data transfer
• Concurrent processing of multiple data store types
• Creation of target data stores from source data store format
• Insert/append to existing target data stores
• Update/replace existing target data stores
• Delete from existing target data stores
• New column/field creation Data Movement.

And for Data Movement, my list includes MQSeries, TCP/IP, and FTP.

If there was also some kind of Integration Center that had an easy-to-use Graphical User Interface (GUI) enabling users to quickly develop data integration interfaces from a single control point – that would be good. Additionally, some way to develop, deploy and maintain data interfaces, create relational DDL (Data Definition Language), XML (Extensible Mark-up Language ) and C/C++ structures from COBOL Copybooks, monitor the status of integration engines, and contain an integrated metadata repository – that would be a real plus.

I’d definitely want to find out more about a single piece of software that provided high-performance Changed Data Capture (CDC) and Apply, data replication, event publishing, Extract, Transformation, and Load (ETL), and data conversions/migrations.

So, if you’re like me and want to know more, there’s a webinar from SQData’s Scott Quillicy on 1 December at 2pm GMT (8am CST). To join the webinar from your PC, you need to register before the event at https://www1.gotomeeting.com/register/844029904. I’ll see you there.

Guest blog – Mainframe security: who needs it?

This week, for a change, I’m publishing a blog entry from Peter Goldberg, a senior solution architect at Liaison Technologies, a global provider of cloud-based integration and data management services and solutions based in Atlanta. He works directly with customers to identify their unique data security and integration challenges and helps to design solutions to suit their organizations’ requirements. A frequent speaker at industry conferences on eBusiness security issues and solutions, he can be reached at pgoldberg@liaison.com.

I’ve been helping companies on both sides of the pond solve their data security problems for many years now. If I’ve learned one thing, it’s this: when I go into an organization that runs Windows, there’s little question of the need for data security. The organization knows it and so do I. When I visit a company whose IT infrastructure revolves around a mainframe, however, the mindset is often quite the opposite. In fact, the biggest data security misconception I encounter is the belief that the mainframe environment is inherently secure. Most IT staff view the mainframe as just another network node. Why? Because it’s universally perceived as a closed environment and, therefore, invulnerable to hackers.

In some cases, it’s the mainframe IT pros who hold this conviction. In other instances, it’s the executive management team. Lack of management attention allows “bad practices” to continue. I can tell you this without reserve: data stored in mainframes needs protection just as much as sensitive information stored on a Windows server or anywhere else. And, as systems continue to support more data, users, applications, and services, effective security management in the mainframe environment becomes significantly more difficult.

News flash: mainframes can be hacked!

For that simple reason, mainframe security should not be taken for granted.

Even though the mainframe is a mature platform, there is a real shortage of mainframe-specific security skills in the market. And, the few mainframe security practitioners who are out there spend a lot of time implementing configuration and controls within their environments as well as putting into place security systems like RACF, which provide access control and auditing functionality. As for other security measures, in my experience, the mainframe people know about encryption, but they’re not terribly aware of newer data security techniques like tokenization as it relates to protecting data within the mainframe environment and beyond.

Tokenization is a data security model that substitutes surrogate values for sensitive information in business systems. A rapidly rising method for reducing corporate risk and supporting compliance with data security standards and data privacy laws, it can be used to protect cardholder information as well as Personally Identifiable Information (PII) and Protected Health Information (PHI).

In fact, for companies that need to comply with the Payment Card Industry’s Data Security Standard (PCI DSS), tokenization has been lauded for its ability to reduce the cost of compliance by taking entire systems out of scope for PCI assessments. And, even in companies that do not deal with PCI DSS or other mandates, tokenization has proven effective for managing the duplication of data across LPARs and for facilitating the usage of potentially sensitive data for development purposes.

Too often, compliance audits skim over mainframe control weaknesses and there are also fewer mainframe-specific security guidelines. But this does not mean that significant risk is not there. You can apply a risk-based, defence-in-depth approach within the mainframe environment by using stronger mainframe host security controls and by using tokenization to protect the data itself.

To beef up data security on a mainframe, here’s my advice:

1. Bring in mainframe security experts to identify and remediate risks, and to develop and enforce security policies and procedures.
2. Develop in-house capabilities and skilled professionals across the mainframe platform to support security initiatives.
3. Evaluate available security configuration and administration tools – there are some really good ones out there.
4. Apply an in-depth security strategy that includes secure access and authentication controls, and use them appropriately.
5. Adopt encryption and tokenization to protect sensitive information. Through their proper implementation, it’s really not that hard to achieve a true high level of protection within the mainframe environment.

Protecting sensitive and/or business-critical data is essential to a company’s reputation, profitability, and business objectives. In today’s global market, where business and personal information know no boundaries, traditional point solutions that protect certain devices or applications against specific risks are insufficient to provide cross-enterprise data security. Combining encryption and tokenization, along with centralized key management, as part of a corporate data protection programme works well – including in mainframe-centric environments – for protecting information while reducing corporate risk and the cost of compliance with data security mandates and data privacy laws.

Don’t be fooled: your mainframe isn’t inherently secure. Doing nothing is no longer an option!

Thanks Peter for your guest blog.
And remember, there's still time to complete the mainframe user survey or place a vendor entry in the Arcati Mainframe Yearbook 2012.