Critical Communication Recovery

EDBOK Guide

What is Critical Communications Recovery?

Critical communications and your company’s print and mail operations are vital to your organization’s survival. Invoices, statements, policies, payroll – if you cannot print and mail these documents, your company can suffer fines, loss of cash flow, or compliance issues. Having a recovery plan in place for these documents is essential. With the variations and intricacies in today’s print environment, recovery efforts must meet the specific requirements of your company to be successful. Having the right plans in place, which often means working with a recovery partner, are the milestones to a successful recovery.

This high level overview reviews the steps to take to develop, test, and maintain a critical document recovery plan.

Business Impact Analysis: Defining Requirements

A Business Impact Analysis (BIA) is a tool used by continuity and recovery professionals to assess a company’s disaster preparedness and resulting financial risks relative to a major business disruption. If your company has completed a BIA, the risks of losing your critical document production facilities should have been detailed and you already have an understanding of the recovery scope needed to ensure that your company’s production needs can be met. If you are starting from the ground up and need to provide upper management with evidence of organizational impact should your critical document processing facilities suffer a business disruption, the following are some high level questions and scenarios to review.

Organizational Impact

Consider what types of business disruptions may affect your ability to produce your critical printed documents. Natural disasters, power outages, pandemic, and work stoppages are overall concerns. In a print and insert production environment, technical issues must also be noted as potential disaster scenarios. Interruptions to printer or inserter hardware or software due to technical difficulties can affect your company’s ability to meet your Service Level Agreements (SLA’s), may last longer than expected, or cause such stress on your capabilities that your recovery partner may need to assist.

In a total loss scenario, replacement of production equipment must include realistic timelines. The days of replacing a printer in one to two weeks or an inserter in four to six weeks are long gone. Specialized equipment requires additional manufacturing time and set-up once delivered. Planning for a lengthy outage to your production facilities will impact your recovery scope. With a long-term disaster both high-level critical documents and communications that are lower on the priority list must be included in the planning.

Critical Documents

What documents are crucial to the continued success of your company? How long could your company manage its day-to-day operations without the flow of your critical communications? How will your company:

• maintain cash flow,
• keep customers informed,
• maintain regulatory compliance, and
• avoid service level penalties and fines.

All customer-facing communications must be reviewed for inclusion in your recovery plan based on these considerations.

Business Units Print/Mail Expectations

Determining what documents should be skipped or included in your recovery plan is a monumental task. All company departments that utilize or are affected by these documents should be included in the recovery scope discussion.

Once the critical documents are identified, priority and service level requirements must be detailed. Remember to include start-up time at the recovery partner site. Once a disaster is declared, data, forms, envelopes, and other materials must be transferredto the print and mail recovery facilities. These delays, although part of the normal recovery process, will affect your standard SLA’s and must be taken into account. In addition, consider set-up time at the recovery facility. Although their printers and inserters may equal the production speeds of your own, they cannot go from zero to one hundred in one day. Documenting these start-up obstacles will provide realistic expectations for all parties.

Internal Recovery Considerations

A successful critical communications recovery plan requires inclusion of all business units working with, or affected by, print and mail operations in recovery planning discussions. They must all have knowledge of the plan details and objectives. Units that process or generate printed documents may include:

• IT – Main Processing Site
• Mid-Range, Server, Satellite Sites
• Business Units
• Support Groups
• Remote Offices
• Organization Partners
• Processing Centers

Multiple Print and Mail Operations

If your company operates multiple print and mail production sites and utilizes these facilities as their recovery plan, consider these questions to determine if you can meet the criteria for actual recoverability.

• Do they print and/or insert similar jobs?
• Are the facilities mirror images of each other (data formats utilized; print hardware and software; insert hardware, software, barcode methodology, and finishing requirement)?
• Do they routinely transmit jobs between sites for processing?
• Are the facilities running at or near capacity?
• Are they staffed for influx of additional production volume?
• Can all jobs be produced at the alternate sites?

Answers to these questions will highlight if your multiple locations can truly act as recovery sites for each other. As with all recovery plans, testing of data transfer, print, and insert capabilities should be done on an annual or semi-annual basis to assess the continuity plan and reveal any hidden gaps.

Outsourcing

If your company outsources any or all of your critical document production processes, it is important to understand that compliance and any service level agreements are ultimately your responsibility. The recovery of your critical documents should be managed and maintained by internal controls and processes rather than those of your outsource vendor.

Outsource vendors with multiple facilities may provide a recovery solution to you that includes the use of their secondary facilities. If so, be sure to use the checklist above regarding multiple locations to confirm that, in the event of a disaster, your production requirements can be met by the outsource vendor’s secondary location(s).

If your outsource vendor operates only one facility and does not have a formal back-up or business resumption plan, it is imperative that you work together to find a recovery solution that will meet your print and mail requirements. In this instance, bringing in a third-party print and mail recovery partner may be your best solution.

Print and Mail Recovery Requirements

After completing your Business Impact Analysis, you should have an understanding of the critical documents that will need to be replicated in the event of a business disruption. Discuss the following details with your recovery partner so that they have an understanding of your print and insert needs. Some specifications are more crucial for implementation rather than at start-up, but the more information you can provide to your vendor at the beginning will allow for a more thorough understanding of your environment.

Print Equipment and Volumes

Provide a detailed list of all printer types (and quantity), models, and specifications that reside in your production environment.

Provide a spreadsheet of print volumes required for recovery. This may not represent all print produced in your facility, but should include critical applications identified during the impact analysis phase. Volumes should be provided as total images (8 ½” x 11” (or A4) images or impressions) and should be grouped into the following sub- categories:

• Cut Sheet (indicate if simplex, duplex, or both)
• Continuous (indicate if simplex, duplex, or both)
• Highlight Color Cut Sheet (indicate if simplex, duplex, or both)
• Highlight Color Continuous (indicate if simplex, duplex, or both)
• Full Color Cut Sheet (indicate if simplex, duplex, or both)
• Full Color Continuous (indicate if simplex, duplex, or both)
• Any MICR requirements

If there are print jobs that use legal size or non-standard size pages, note them.

Print Data Transfer

There are several options for transferring critical print data to your recovery partner. Your options will depend on your circuit availability (FTP, VPN, T1, etc.) and your print and/or host systems. The following is a list of options that your print and mail recovery environment/vendor should be able to support. If your data and print environment requires something non-standard, discuss options with your recovery planning team.

Provide your recovery partner with the types of host systems that drive your print process. There may be more than one type. Some common options:

• IBM Mainframe
  • JES2/JES3
  • PSF /Infoprint Manager / Process Director
  • Prisma
  • Custom Statement Formatter (CSF)
  • XPAF
  • VPS
  • JQP
• Operating System Level – ZOS V_.R_
• User Modifications / System Exits
  • JES2/JES3 exits
  • PSF exits
  • VPS/JQP exits
  • Other exits
• Unix/Linux
  • SUN
  • HP
  • Hitachi
  • Generic OS Level
  • IBM I-series OS Level
• Windows/Other OS Level

Print Data

Provide the types of data streams used in your print environment:

• Line Data
• LCDS
• Metacode
• AFP / IPDS
• PDF
• Postscript
• PCL
• Unformatted Other

Print Software and Processing Reconstruction

The recovery plan requires details of any print management tools involved in your print process. The following identifies some of the common tools, but consult with your print operations team to ensure what tools are in use, the release levels of the tools in use, and what additional scripts might me in place. All of these must be documented and available in the recovery environment.

Crawford Technology Production Manager	LRS VPS / VPX	Pitney Bowes JESConnect	Solimar SOLitrack
Canon Océ Prisma	Ricoh InfoPrint Manager	Pitney Bowes FlexServer	Solimar SPDE
GMAC Software	Ricoh TotalFlow Producer	Pitney Bowes VIP Server	Rochester Software QDirect
BARR Host Comunication Suite	Ricoh InfoPrint ProcessDirector	Xerox XPAF	MPITech
BARR Enterprise Print Server	Mirus Remote Sync	Xerox FreeFlow

Are any of the following applications used to generate the print data stream:

GMC Software PrintNet	HP Exstream Dialogue	EMC Document Sciences Compuset / Autograph	Xerox VIPP
GMC Software Inspire	ISIS Papyrus DocExec	EMC Document Sciences xPression	Xerox XICS
Thunderhead	SCRIPT-based command languages (DCF / BookMaster / Waterloo Script)	COBOL or other programmed formatting	Xerox XMPie
Custom Statement Formatter (CSF)	DocuMerge	Napersoft CCM Author	Bitstream Pageflex
Pitney Bowes Doc1	Objectif Lune / Printsoft PRES / New Leaf	Objectif Lune PlanetPress	EZ-Letter
Inventive Designers Scriptura Engage	Transformations Uluro	Doxee

Are post-composition re-engineering products in use?

Crawford Technologies Product Suite	Solimar Product Suite	Sefas Product Suite	Actuate Product Suite
TargetStream	Rochester Software Product Suite	Pitney Bowes Product Suite	Canon Océ Prisma
Ricoh Infoprint Product Suite	Compart Product Suite

Remember to describe any special processing tools for managing data to the printer that may not be detailed above. This includes network to mainframe protocols, legacy resource files like electronic overlays, and other special requirements.

Insert Equipment and Volumes

Provide a detailed list of all inserter types (and quantity), models, and specifications that reside in your production environment.

Provide a spreadsheet of insert volumes required for recovery.  This may not represent all insert/mail packages produced in your facility, but should include critical applications identified during the impact analysis phase. Volumes should be provided by envelope size and should be grouped into the following sub-categories:

• #10 (indicate average page count per envelope)
• 6” x 9” (indicate average page count per envelope)
• 9” x 12”or other flat size (indicate average page count per envelope)
• Pressure Seal Requirements
• Hand Inserting or Special Packaging Requirements

Any special processing completed on your inserters should also be identified, including two- or three-way matching, selective inserts, stapling, booklets, and post-insertion imprinting.

Insert Technical Details

Along with providing the types of insert equipment used in your facility, your recovery partner will need to know some further specifics in order to replicate your production environment. These details can include the following:

Feeding	Face-up / Face-down
Edge Leading	Top / Bottom
Collation	A to Z, Z to A
Address Location	Top / Bottom
Print Orientation	North/South East/West
Average Page Accumulation	Page Count
Maximum Page Accumulation	Page Count
Fold Type (Letter)	Z, Half, Roll fold, Other
Barcode Configuration	2D, 3of9, 2of5, OMR(4th 6th 8th), Other Logic
Envelope Size	Identify size
Envelope Type	Window, Closed Self
Imprinting Requirements	Identify

A copy of your barcode methodology is needed by your recovery partner to assist in understanding how your applications are assembled on the inserter (number of pages per envelope, selective inserts, match requirements, etc.) and any document tracking requirements.

Postal Requirements  

Postage is a significant segment of your print and mail costs. Ideally any postal savings applied during normal operations should be duplicated in the event of a business disruption. Your recovery partner (internal or external) should have the ability to sort mail online, or have access to a pre-sort vendor that can apply the optimal postal discounts. Discuss your postal strategy with your recovery partner and confirm that they can match your efforts.

Develop a plan for moving postage money from your local accounts to the recovery partner. If metering is required, an allocation of funds via check or electronic funds transfer (EFT) to a meter account will be needed. If you are using indicia (permit) for your mail, your recovery partner must check with their local post office to ensure your permit will be allowed to mail out of the recovery location at time of disaster.

Quality, Reprints, Waste

Your recovery partner should have their own standard quality control procedures, but in the event of a disaster declaration your operational procedures should be maintained as closely as possible. It is important to share your company’s quality requirements with your partner, even internal partners, so they may follow your procedures and ensure the integrity of your printed and inserted documents.

A process for handling reprints should also be discussed. Since most recovery sites will work with print image files (instead of raw data), they may be unable to reproduce the document at the recovery facility. Verify during drills that all file types can be processed as needed.

Any waste as a result of print or insert technical issues must be reported for inclusion in the next day’s print volume. Exact procedures for reporting waste details will vary by organization and may include reporting a sequence number, name and address, account number, or some other identifying information on the damaged document.

As with any document containing personal information, waste material must be handled as securely as possible. Your recovery partner must demonstrate the ability to contain all waste in secured receptacles and offer document shredding services, either on-site or offsite, with reporting documentation. The security of the personal information on the printed documents is ultimately your responsibility and therefore every measure taken in your own production environment should be followed at the recovery site.

Security and Compliance Requirements

Security and compliance is a major part of your company’s day-to-day operations and must also be part of business continuity plans. Your recovery partner must maintain the same security and compliance requirements as your own print and mail production facilities.

Compliance requirements will vary based on your core business and geographical location. For example, HIPAA regulations are required for the healthcare industry and PCI certification is required for the financial industry. Your company’s compliance manager will have the security details you must follow and which will be required of your recovery partner. Most companies will require some type of security and compliance questionnaire to be completed by an outside vendor upon selection, with yearly updates needed. The questionnaire may also be required of internal recovery partners.

Vendor Criteria

If you are choosing a vendor recovery partner, choosing the right vendor is critical to assuring the continuity of your print and mail operations. A recovery partner should have facilities dedicated to handling print and mail production requirements, or at the very least dedicated equipment to handle a potential disaster. Look for the following as you evaluate your potential partner:

General:

• Recovery as their core business, industry knowledge, and history,
• Dedicated recovery facility and/or dedicated equipment capabilities,
• Multiple locations or secondary back-up facility,
• Business continuity plan in the event of a disruption to the recovery site,
• Disaster declaration experience (how many, how long),
• Standard test procedures and processes along with the ability to follow your specific test requirements, and
• Security qualifications similar or higher than your own: ISO, PCI, GLB, HIPAA, etc.

Facilities:

• Multiple communication transfer options for delivery of data from your facility to print and mail recovery facilities,
• Printer hardware, software, and volume capabilities to duplicate your production requirements,
• Insert hardware (with any special functionality, i.e. barcode methodology, matching, booklet making, stapling, etc.) and volume capabilities to duplicate your production requirements,
• Document tracking procedures equal or similar to your environment,
• Ability to meet your postal requirements (pre-sort, permit mail, manifest, etc.) with postal facilities on-site or near facility capable of handling additional mail volume,
• Document shredding for secure handling of all production waste,
• Daily tracking reports and procedures for notifying you of project status,
• Standard quality control procedures, and the ability to implement any specific requirements you may need,
• Dedicated staff: Customer service, IT, Network, Print, Insert, Postal, Quality, and
• 24/7 operations.

Recovery Plan Implementation

The implementation of your print and mail recovery plan can begin as soon as your recovery partner has an understanding of your print, insert, and mail specifications.

Your implementation team should include members from the following departments: DR/Business Continuity, IT, Network, Print Operations, Insert Operations, and Postal. If you use outside vendors for data recovery they should be part of initial implementation to ensure successful data transfer.

Your recovery partner may request an onsite review of your production facilities.  This visit can be an important step in understanding your specific print and insert production requirements, giving your vendor the tools to better duplicate the recovery effort in their own facility.

The following is a general list of action items you should expect to complete for a successful implementation of your recovery plan. Some items may not apply to your environment, and there may be other requirements not detailed below.

	Critical Communications Recovery Implementation Checklist
	Action Item	Responsibility	Completion Date
	Proof of Concept Test
☐	Print and Insert Grids to Print/Mail Recovery Vendor	Customer
☐	Pre-Test Discussion to determine test scope, expectations	Print/Mail Recovery Vendor
☐	Review data, print, and mail specifications with customer	Print/Mail Recovery Vendor
☐	Barcode logic and sample MRDF files to Print/Mail Recovery Vendor for test set up	Customer
☐	Pre-Test Samples to Print/Mail Recovery Vendor	Customer
☐	Proof of Concept Test at Print/Mail Recovery Vendor	Print/Mail Recovery Vendor
☐	Proof of Concept Test Report to Customer	Print/Mail Recovery Vendor
	Account Implementation - General
☐	Review non-tested application specifics	Customer and Print / Mail Recovery Vendor
☐	Gather samples and detailed specifications of all non-tested applications	Customer and Print / Mail Recovery Vendor
☐	Gather additional barcode logic or MRDF files (if needed)	Customer and Print / Mail Recovery Vendor
☐	Quality Control Procedures to Print/Mail Recovery Vendo	Customer
☐	Additional security procedures required (if any) to Print/Mail Recovery Vendor	Customer
☐	Declaration Authorization Information to Print / Mail Recovery Vendor	Customer
☐	Review postal procedures for disaster declaration	Customer and Print / Mail Recovery Vendor
	Data Description of Data Transmission method requested
☐	Configure and test method of transmission	Customer and Print / Mail Recovery Vendor
	Finalize automation of file receipt decryption and spooling notification, if needed	Print/Mail Recovery Vendor
☐	Confirm resource requirements, and set up monthly transfers, if requested	Customer and Print / Mail Recovery Vendor
	Print Description of Print Requirements
☐	Determine initial control document for print priority (at time of disaster)	Customer and Print / Mail Recovery Vendor	Prior to next test date
	To understand what files are being sent to the recovery facility.	Print/Mail Recovery Vendor
☐	Maintain print specifications on the print grid, samples, etc.	Customer and Print / Mail Recovery Vendor
	Insert Description of Insert Requirements
	Maintain insert specification on insert grid, samples, barcode methodology, postal requirements, etc.	Customer and Print / Mail Recovery Vendor
	Account Maintenance
☐	Change Management Updates	Print/Mail Recovery Vendor to Customer
	Print/Mail Recovery Vendor will send Change Management memo for updates on changes to volume, equipment, applications, etc.		More frequently upon client request
☐	Inventory
	Annual Testing
☐	Determine annual testing schedule (quarterly, semi-annual, etc)	Customer	Upon receipt of contract
☐	Send test date requests to Print/Mail Recovery Vendor	Customer	Upon receipt of contract

Testing

Testing your print and mail recovery plans will assure that your recovery partner can support your requirements if and when you need them. The more you test, the more comprehensive your tests are – the smoother your recovery will be.

A typical print and mail recovery exercise should include the following production processes: transfer of data to the print and mail recovery facility, printing, and inserting of identified critical applications. Standard test procedures should include a variety of print and insert applications to insure you can match all of your requirements.

A Proof of Concept test should be performed prior to recovery plan acceptance. Yearly, semi-annual, or quarterly testing should be completed to maintain recoverability and test any changes to your environment that take place throughout the year. It may be wise to schedule ad hoc tests when new applications are added to production.

It is important to combine your data recovery test with your critical document testing to insure seamless transition of data from your normal operations to the recovery location. Your data recovery tests should include backup and recovery of all print systems and applications so that you can confirm all recovery time objectives. If your print and mail recovery plan calls for transmission of data to the print recovery site within eight hours, but the data recovery plan priorities or timeframes do not match this requirement, the success of your print and mail operational recovery is at risk. Work with your IT and Data departments to detail your critical applications and timeframes so that recovery scope is understood by everyone involved with the process. Remember that there may be regulatory requirements that guide recovery times.

Disaster Declaration

Your print and mail recovery partner should maintain formal disaster declaration procedures, with contacts and instructions for movement of materials, postage, and other requirements. Your internal recovery plans should include who at your organization is authorized to formally declare a disaster and who will manage the critical document recovery efforts. Appropriate staff members at the company and the recovery partner should be identified and all contact information should be exchanged.

Material Management

Movement of required materials for print and insert production should be detailed in your recovery plans.  It is suggested that you manage stock levels at your print sites so that in the event of a disaster you can request immediate shipment of materials to the print and mail recovery site. Most recovery sites have warehouse availability, but due to paper sensitivity it is not suggested to keep materials onsite for long periods of time. However, keeping a small amount of material is suggested for set-up and testing purposes. Discuss your material requirements (paper, forms, inserts, envelopes) with your recovery partner to determine your best options for quick delivery.

Plan Maintenance and Change Management

As technology evolves, changes to your print, insert, and mail processes are inevitable. Modifications and upgrades to data or communications environment, printer hardware and software, insert technology, and postal requirements will impact your business continuity plans. It is vital to report any changes in your environment to your recovery partner as soon as possible as they may require alterations to your service. Revisions should be communicated to your recovery partner as they happen, but it may help to set up quarterly or semi-annual review meetings with internal departments and your recovery partner to discuss any changes to your environment.

Summary

Print and mail production environments have evolved into technical wonderlands that can no longer be easily replicated at the time of a disaster. A recovery plan is essential. Understanding the requirements needed to produce your critical documents in the event of a business disruption, partnering for print and mail recovery that can match your specific needs, and testing these recovery efforts can be a cumbersome task, but a necessary one. As compliance and service level requirements become more intense, recovery efforts must match the ever-increasing demands of your internal environments. The right plan + the right vendor = recovery success.

If you would like to learn more about critical document recovery, the following resources may help you find a consultant or vendor that can assist: http://www.drj.com and http://www.continuityinsights.com.