I started setting up contingency plans and disaster recovery (DR) sites for financial services firms ten years ago. Initially, the goal was to reduce the expected recovery effort from an insane week to a few difficult days, but in relatively short time the new standard became less than 24 hours. Given the SEC’s increased scrutiny of business continuity (BC) plans most investment management firms should now have a detailed plan that includes access to offsite servers, and a level of routine data updates to an offsite facility.
To ensure that your BC plan is successful follow these simple rules:
1. Plan to fail
Most plans are too optimistic. When things go awry, they typically go from bad to worse — rapidly. Planning to fail means envisioning and detailing potential failure scenarios and documenting your contingency plan for each situation.
For example, we recommend three forms of backup (with the assumption that one or two of the backups may not be sufficient). While many of the firms we work with could probably get away with a single backup method, each method that we implement lowers the overall risk of losing any data.
2. Don’t set it and forget it
Remember, there is no silver bullet. The day you don’t check your systems is the day you should expect them not to work. Systems are ultimately managed by people and even the most competent people sometimes make mistakes. Beware of vendors that tell you, “It’s automatic. You don’t have to do anything.”
If you have a metered Internet backup service and you are getting billed monthly, the invoice amount should never be the same. If it is, it may indicate that the data being backed up isn’t changing.
Absolute vigilance is required to be successful at planning for a contingency.
3. Establish strict and coherent responsibilities
Who is the steward of your firm’s plan? How do they validate the plan? How does the plan work? Our real-world experience indicates that multiple parties need to understand and check the plan for problems on an ongoing basis. When new systems are implemented a disciplined approach to adding and updating the contingency plan needs to be executed.
4. Institute operational checks and balances
We recommend a multi-faceted approach designed to ensure that multiple parties independently share ultimate responsibility for backing up your company’s data and validating that the contingency plan works. Your firm cannot afford to make assumptions about whether those responsibilities are being met.
If you think you’re ready, test it. Ask your IT folks to throw the switch with little or no warning to see how well your plan really works. You may want to think about this carefully since some plans are like having a gun that can only shoot one bullet. In order to test the system again you may need to rebuild the system.
5. Continuously improve and refine
Contingency plans fall into five basic categories: non-existent, poor, okay, good and excellent. As a decision maker and responsible party at your firm, do you know how your firm’s plan would rate? Moreover, IT systems are in a nearly constant state of change. If your plan was “good” last year, is it as “good” today?
A company that has an excellent contingency plan for a catastrophic event may not have a good plan for the more likely event of losing Internet access at their office tomorrow.
There is always room to improve your plan.
About the Author:
Kevin Shea is President of InfoSystems Integrated, Inc. (ISI); ISI provides a wide variety of outsourced IT solutions to investment advisors nationwide. For details, please visit isitc.com or contact Kevin Shea via phone at 617-720-3400 x202 or e-mail at kshea@isitc.com.
