Electrical Loads: How Should Data Center Systems be Protected?

We are routinely asked how various data center systems should be protected. Usually the customer wonders what systems should be protected by an uninterruptible power supply (UPS) and what loads should be placed on a backup generator.

In the average data center we find several critical systems: the enterprise itself, precision cooling, fire suppression, life safety, physical security, etc. All of these systems are critical to normal operations, but in the event of a power outage, how are these systems prioritized and how are they protected? The answer is relatively straight forward; the UPS should be dedicated to supporting the enterprise. Regardless of the facility’s operation strategy for power outages, (be it to achieve an orderly shut down of the enterprise or to operate the enterprise until the facility’s backup generator comes online) the UPS is present to provide short term, interim protection for enterprise hardware.

The reason for this prioritization is straight forward, the loads associated with the cooling plant, such as compressors, pumps and fans are so large as to make it impractical to support them on UPS power. In addition, most data centers can operate during a brief outage of the cooling plant. For these reasons cooling plant components can be protected by backup generator power without a requirement for UPS power.

As for fire protection systems, data center fire detection and suppression systems should be placed on the backup generator supplied emergency power circuits, but must also incorporate their own battery backup system. Typically a maintenance free battery and charging system is built into the facility fire panel. It is imperative that the battery be maintained and replaced per the manufacturer’s recommendations. Failure to do so could result in the system not activating when needed, or in some failure modes, causing an inadvertent discharge of the suppression system.

Likewise, key life safety systems, such as emergency lighting should be engineered to operate independently of the UPS. As with the fire protection systems, emergency lighting batteries must be maintained per manufacturer’s recommendations and be connected to generator supplied emergency power system.

Normal facility systems, such as lighting and comfort cooling outside of the data center, are somewhat of a mixed bag. Often normal data center lighting will be placed on backup generation to allow continued operations in the facility. Generally speaking comfort cooling is not a priority as it is not necessary for most data center operations.

Management may also wish to consider defining telephone equipment as mission critical and placing it on UPS power as well. This will ensure uninterrupted voice capability in the event of a power outage. While a power outage may curtail normal office operations, communications in and out of the building would be an essential service for data center personnel and any skeleton staff asked to occupy the building during an extended outage.

While on the subject of UPS and generator systems, deployment of a UPS and/or generator is only the first step in protecting a data center. A commitment to periodic testing and maintenance of these systems must be made by data center personnel and management. While UPS and generator systems are very robust, they are like any mechanical system and require ongoing maintenance to ensure they work properly when needed. UPS systems should be maintained by factory trained personnel at least once a year.

Back up generators should be tested at least once a month and allowed to run long enough to attain normal operating temperatures. Like UPS systems, generators should be inspected and serviced at least once a year by factory authorized technician,s to ensure the system is fully functional and available when it is needed. Generators also require a programmatic approach to ensure availability. Data center or facilities managers should ensure their generator has sufficient onsite fuel storage to run continuously for extended periods of time. Generally 24 hours of operation at full load is a good target to aim for. Also, management should have pre-existing contracts in place for re-fueling and be assured that their fuel vendor will be able to deliver fuel even in adverse conditions.

A few other items that should be kept in mind in regard to generators include maintaining basic spare parts onsite. Extra engine oil, serpentine belts, filters, hoses and basic tools will give you a fighting chance to make simple repairs if needed. One item that is often overlooked is the generator starting battery. Generator batteries should be float charged and tested periodically to ensure they are in excellent condition and able to start the generator when needed.

UPS and backup power systems are essential for reliable data center operation. Design and deployment of these systems is not difficult, but do require solid planning and a commitment to good maintenance and operational practices. A few dollars spent on system consulting from a qualified vendor will not only ease the system design and deployment process, but will also reduce the chance that your essential back up systems will fail when needed most.

Human Error in the Data Center Environment

One of the things we stress with our customers is the fact that no matter how robust data center facility systems are designed, the "human factor" can always find a way to wreak havoc.  Computer World published a great article on this topic in their August 12, 2010 online edition titled: "Stupid Data Center Tricks".

The take away of the article, and of our discussions with our clients, is that training of data center staff is an essential part of data center operations.  If staff isn't intimately familiar with systems operatation, or how to perform in time of an emergency, the investment in redundant systems can be negated in a heartbeat.

We've Moved!

We're pleased to announce that we have relocated our offices to larger, more comfortable quarters in Tampa's Cypress Point Office Park.  One of the main "perks" of our new offices are expanded conference facilities and we'll be offering a number of in-office and webinar "Lunch & Learn" sessions in which manufacturer and industry leaders will present on topics of interest to the data center community.  Watch our blog and Twitter posts for updates.

Our new address is:
Pergravis, LLC
10012-B North Dale Mabry Highway
Suite 109
Tampa, FL 33618

Now Offering Global Colocation Services!

We are very pleased to announce that Pergravis now offers global colocation services to our customers.  If you need high availability data center space to support disaster recovery operations, reduce latency for critical applications or support remote office operations, we can offer service at any one of 35 global facilities including locations in London, Paris, Beijing, Cape Town, Seoul, and Hanoi.  We also offer domestic colocation services at our network of 13 data centers.

While our colocation servcies are new, our hardware migration services are not.  We can provide total support in the relocation of existing hardware or installation of new hardware in the colocation facility of your choice.  All moves are fully insured, executed by trained IT personnel and hardware is packaged to some of the most stringent requirements in the industry.  Feel free to contact us at your convenience for your colocation and physical migration needs.

Small UPS Maintenance

If your office is like ours, you probably have several small UPS units protecting individual desktop computers, perhaps a switch in a remote communications closet or perhaps a SOHO class telephone system. Many times these systems are installed and then promptly forgetten about as they collect dust under a desk or languish in a seldom visited rack. Well, belive it or not, small UPS units require as much, if not more, attention as an enterprise class UPS.

An enterprise class UPS usually features advanced battery technology that enable batteries to remain in service for many years. Unfortunately, the same is not true of smaller UPS units. The typical small UPS uses basic sealed lead acid batteries which have a much shorter life span than those found in large UPS systems.

We recommend to our customers that they replace the batteries in small UPS units every 24 months. If the UPS protects an extremely critical load, consider reducing this period to 18 months. When replacing batteries in these units, place a label on the UPS with the date for the next battery replacement and set a reminder in your departmental calendar as well.

One final note, make sure to dispose of used batteries properly. Most manufacturers offer some type of recycling program and if that option is not practical, many electronics and battery retailers offer free battery collection programs. Your municipality may also offer free battery recycling. Contact your solid waste authority for options in your community.

Keeping Your Data Center Resolutions?

Well, here we are, in the middle of the first quarter of 2010 and I’m willing to bet that some of those New Year’s resolutions that were made on December 31st still haven’t been put into practice. In an attempt to help all of you that made resolutions to improve your data center operations this year, here are a few of our favorites that will get you moving down the right path before we enter 2011. In general they are simple, inexpensive and will definitely help to improve operations in your facility.

Conduct a Data Center Walk Through
What could be easier? A walk through is simply a row by row, wall to wall walk through your data center. During the walk through take note of all the things you would like to change. Consider it a “Blue Sky” exercise and note everything that comes to mind as you view the facility. Are floor tiles worn? Write it down. Is cable management a mess? Write it down. Have a lot of empty boxes or dead hardware stacked up in the corner? Write it down. Notice warm areas during your stroll? You get the idea.

Once you’ve finished your walk through and assembled your list, organize the list into two groups:

- Stuff that can be done for free or on the cheap such as removing empty boxes and decommissioned hardware from the data center, replacing filters, installing blanking plates, relocating perforated tiles to improve air flow, cleaning up cable bundles;

- Things that need dollars to fix such as a Computer Room Air Conditioner repair, UPS battery replacement, super-cleaning the data center, etc.

Once the two lists are assembled, prioritize each using risk to the enterprise as the guide for prioritization. For the “fix for free” items, work on each issue in order of priority; the items that require money, budget on the basis of priority.

Establish an Annual Maintenance Schedule for Data Center Hardware
Address simple items such as air filter replacement and fan belt inspection and replacement on precision air conditioning systems. Schedule more complex services such as load testing battery banks on UPS systems, conducting battery maintenance and replacement and annual data center cleanings.

Conduct an Annual Load v. System Capacity Analysis
Data center systems, such as the precision cooling system and UPS have finite capacities. Understanding the load v. capacity these systems is essential to uninterrupted operations.

A load v. capacity analysis ensures any hardware that has been added or removed from the enterprise is properly accounted for and any risk to overloading critical systems is found before it becomes a problem.

Check Data Center Fire Suppression Systems
Have your fire suppression vendor check the condition and charge of the facility central clean agent fire suppression system and all handheld fire extinguishers.

Inspect the Condition of all CRAC Unit Belts
CRAC units are equipped with fan belts. The belt connects an electric motor to the CRAC unit fans. As belts age they can stretch and crack leading to slippage and reduced efficiency. Inspect all belts and tighten or replace as necessary.

Inspect the Condition of Humidification/De-humidification Systems
Periodically inspect the humidification/de-humidification systems in CRAC units with particular attention to trays and condensate lines. Look for signs of leakage or tray overflow. Also look for signs of mineral deposits. Take corrective actions as necessary.

Inspect and Replace as Necessary all Filter Media in CRAC Units
Just like a furnace or HVAC unit in you home, the CRAC units in the data center need clean filters in order to efficiently move the vast quantities of air required by the enterprise. Clogged filters reduce overall system cooling efficiency and may cause data center temperatures to rise. Also, clogged filters can cause CRAC unit fan motors to work harder. This, in turn, increases wear on the motors and increases electrical consumption.

Clean the Data Center Floor
If the data center is equipped with a raised access floor system, inspect and clean as necessary the sub-floor plenum beneath raised access floor. This should include any unused cabling that may have accumulated in addition to the usual dust and debris. If the data center is on a slab/deck, clean the floor as necessary to remove dust and debris. Be aware that static electricity is always a danger if vacuums are used and it is best to either refer the cleaning to a qualified vendor or take all necessary static dissipation measure to prevent damage to enterprise hardware.

Ensure a Good Seal along the Raised Floor Perimeter
Inspect the wall/floor seal between the data center walls and raised access floor. Seal any gaps that are allowing conditioned air to leak through.

Install Blanking Plates to Improve Cooling Performance
Install blanking plates on all empty “U”. This is an easy, inexpensive and highly effective method of improving cooling performance. Need blanking plates? Pergravis manufactures its own blanking plates and we can offer very aggressive pricing.

Inspect the Ceiling
If the data center is equipped with a suspended ceiling, inspect the space above the ceiling for cleanliness, condition and any signs of water intrusion in the deck above the ceiling. Also check the ceiling tiles for signs of damage or erosion. If the tiles are of the compressed cellulose “acoustical” type, consider replacing them for a clean room rated tile.

Dress Data & Power Cables
Properly dress any cabling that has been installed in the last year and not properly incorporated into the structured cable plant. Always use at least 1” wide “hook & loop” fasteners and not cable ties. “Hook & loop” fasteners reduce stress on cables and makes adding/removing cables from cable bundles a breeze. Pergravis also manufacturers “Hook & Loop” cable organizers including custom wraps with company logos. Give us a call for pricing.

Review Emergency & Contingency Plans with all Staff
Review the data center emergency and contingency plans with everyone who is party to the plan. This is an excellent way to ensure smooth execution of all necessary procedures in time of emergency. In addition to review of the plans, review items such as call trees, availability of procedural manuals, availability of hardware manuals, contact lists for essential vendors, etc. as well. Commit to reviewing the plan twice a year.

There you have it, a “Baker’s Dozen” of simple, relatively inexpensive tips for improving data center operations. If you have your own favorite, inexpensive data center tips, feel free to send them along and we’ll include them in future blog entries.