Datacenter and Colocation Discussions

Another local datacenter outage to report. Voicenet facility located in Northeast Philadelphia had a major network outage last night. Reports from a collegue of mine that manages equipment there indicates the outage lasted for about 45-60 minutes. The outage even took out Voicenet’s phone system so customers who have equipment in the Voicenet Datacenter couldn’t even call in and complain.

I cant say it enough… Dont colocate equipment in datacenters that dont have true diverse network connectivity. Voicenet claims redundant fiber, but its just a single fiber ring. Yes, a ring has redundant fiber and two paths to prevent a fiber breakage, but its still a single ring operated by a single entity, with an equipment SPOF (single point of failure) at the other end. The only thing a ring protects against is back-how digging up the street or a tree falling down. Datacenters needs to have true diverse fiber. That means separate fiber paths coming in via separate entrances, and the fiber itself must be owned and operated by separate entities with completely separate routing platforms.

The scary thing is there are serveral datacenters in the Delaware Valley that operate off fiber ring topologies. Stay away from these datacenters, its just an outage waiting to happen.

We all hear how its good to buy our produce locally, but what about buying web hosting services locally. Interestingly, your company will be better off if you host with a local provider, and it has nothing to do with better support and everything to do with search engines and regionality.

Search Engines such as Google or Bing like LOCAL results

When you do an internet search, the search engine knows where you are and in turn will display search results that are close to your region. For example, if you do a search for lawyers, Google will undoubtedly return results for law offices in your area. It does this using IP geography. From your IP address, Google has a rough idea of why you are in the world.

The same technology that determines where you are located can be used to determine where a website/company is located. Now lets be honest, search engines know that most websites are hosted outside their served region. However, if your website is hosted from an IP with a geographic footprint of say Ohio, and your business is in Ohio and mentions this in the indexable content on your site (mailing address, area code, ectc.,.), the combination of these two things is very positive. It tells Google you are physically local to that area and cyber local to that area. All of this is good stuff for your ranking.

So go ahead and host your website with a local provider!

The top two new techniques for email harvesting is Craigslist Ad’s and Facebook Friend Requests.

If you use Facebook, I’m sure you’ve received a few random friend requests from people you don’t know. These requests are actually email harvesting attempts. If you reply or accept the friend request, they will most likely grab your real email – which was the original intended purpose.

The Craigslist Ad technique is similar. If you post an Ad on criagslist there is a very good chance someone will reply. Not all the replies are real people however, in fact, many of the replies are from robots. The response will be generic, for example, “I’m interested, contact me”. If you reply to that email, boom, the robot just grabbed your real email address.

Being involved in multiple data center build projects, I wanted to share some opinions on what I think are the most important core features to focus on.

The first hurdle of any building project is electrical power. Any location with nearby poles will have access to either 7,200 or 13,200 volt 3-phase high voltage. But some areas dont have enough capacity on the pole to support a typical data center load of 1-2 Megawatts. And if the local grid can support it, the power might not be diverse, or might be mostly aerial. Now some design firms stress having underground diverse power. Underground power is easy if your in an urban Metro, diverse not so much. I have come to a surprising conclusion however, it doesn’t really matter!

Today’s Generators are nothing like those of the past…

If your facility has a single aerial power feed, thats not too big of a deal these days especially if you intend to install an N+1 redundant generator infrastructure with a few 1000 gallons of fuel. I mean, thats enough capacity to run for a few days, and with multiple generators you can survive a significant localized mechanical failure on top of your grid failure. And lets face it, when was the last time an area lost power for more than 5 days? Modern generators have an amazing performance record and can run for days on end. If you have three 1.5MW generators for a datacenter with a 1.5MW running load, lets face it, your never going dark.

Location is the most important aspect to a datacenter…

This location criteria is two fold. First, you want your location to be in an area of minimized natural disasters. So the probably of flooding and natural conditions that could cause power lose are limited. Second, you want to be very close to diverse fiber optic routes. The number one cause of datacenter outage time is not power or cooling, its IP access. If your facility sits at the cross roads of multiple major networks and has access to fully diverse fiber routes, you greatly eliminate the possibility of an outage. Mechanicals can be solved by buying the right gear and applying good design, but your network quality will also hinge upon your access to fiber.

I’ve been seeing this more and more lately and its time to clear the air. In the past, dry chemical fire suppression was the standard. Either Halon or FM200 dry chemical gas would suppress the fire by removing all oxygen from the space.

Nowadays, many datacenters are cutting back on dry chemical systems. Instead they advertise that they have “Dry Pipe Preaction Sprinklers”. Sounds good doesn’t it! Well its a fancy way of saying we use building code required overhead sprinklers. Preaction simply means that the sprinkler pipes dont have water pressure in them. An action has to trigger the building pumps which pressurize the pipes, i.e. smoke alarms and such. Dry pipe means that after the system is triggered (either a real alarm event or planned maintenance) it is drained.

The play on words tricks people into thinking that “Dry Pipe” makes a connection to dry chemical – of which there is none!

Sprinklers are code in all buildings… period. The sprinkler heads only open after a temperature fuse breaks – normally around 175 degrees. Well, if a datacenter gets up to 175 degrees, its all over. Thats why you use dry chemical like FM200 or Halon to kill the fire immediately before sprinklers heads open.

Point of the story is… Never point your equipment in a datacenter that does not have true dry chemical fire suppression.

For those readers who expect colocation topics, I am sorry. But when you have a blog you have the ability to let people know of your experiences. I was so shocked by how British Airways scammed me that I feel compeled to tell the story here.

I recently purchased an international flight via British Airways. Total cost was $846. There were two other carriers (USair and Delta) who were $10-$20 cheaper, plus a few carriers who were more expensive. I did alot of shopping around on sites like Kayak.com and Orbitz.com to find the best price. Ultimately I decided to go with British Airways.

I went to the British Airways site and purchased my ticket online. When the whole process was done, I had my confirmaion number, final total was $846. Then I login to select my seats. To my surprise, it will cost a total of $90 to select my departure and return seats (1-stop flights). So in essence, this ticket really cost me $936. What pisses me off is I could have went with Delta for $834 and had ZERO seat fees.

British Airways (BA) says you can wait until 24 hours before departure and select seats for free, but we all know when that time comes, there probably wont be any seats left, especially since they overbook flights.

Nowhere in the order process does BA disclose the seat fee. I even clicked and read all their terms and conditions, doesn’t mention seat selection fees anywhere. When I called to complain, they told me to do a google search of their site for “seating” which returned a page stating the fees. Yeah, that helps, what am I supposed to be a mind reader. If you dont disclose it in the order process how am I supposed know about it.

I am disputing the charge with AMEX to get the ticket refunded so I can fly Delta. As a business owner, it really annoys me how some businesses abuse customers and trick them into higher pricing. I will never purchase a BA ticket for the rest of my life because of this.

We recently received our first CAREL brand centrifugal atomizing humidifier, the HumiDisk65. In addition to providing 7 liters per hour of humidification, the unit only uses about 200Watts. The resulting adiabatic cooling comes to about a 1-ton reduction in hourly BTU’s. Additional units may be installed over the next few months depending on the output of this first unit.

CAREL is one of the few manufactures that make industrial grade centrifugal humidifiers for the telecom/datacenter space.

Everyone should know by now that datacenter humidity is very important. Dry air has lower heat transfer capability, damages electronics, and causes excessive static build up. Humid air (45% RH) has better heat transfer, is less caustic on materials, and captures airborne dust particles more efficiently.

The problem is humidification in the datacenter is expensive and time consuming. Older options have been steam electrode canisters and infrared heat lamp with water pan. Both options have significant power consumption requirements. Infrared heat or IR uses about 50% of the rated cooling power load. So if your CRAC unit uses 8KW, the IR pan system uses about 4KW. Steam electrode is about 25% of the rated cooling load, so same 2KW on a similar 8KW rated cooling unit. Annually, thats alot of money.

Maintenance is a hassle too. Steam canisters need to be replaced every 4-6 months, infrared systems need constant maintenance with cleaning mineral deposits out of the pan tray, replacing bulbs, and so on.

They are a few low power options out there. The most popular is ultrasonic humidification. Ultrasonic waves literally atomize the water droplets into vapor. It uses very little power, but unfortunately it requires significant reverse osmosis water treatment, and the water treatment canisters needs to be replaced every few months too.

The last option is mother nature. Humidification by accelerated air evaporation. This is what you may see in your house. A porous filter (sponge or paper) absorbs water for evaporation that is triggered by a fan. Why not use this in a datacenter? Well, at Quonix were designing a system around it – patent pending.

A datacenter is a closed loop system. Why get all condensate water be re-evaporated back into the room? John Von Essen, founder of Quonix, is trying to design such a system. The system uses no additional electricity, and very little water.

Cooling units already have blower fans, so why not engineer a water filter or sponge that can absorb condensate before it goes down the drain and re-evaporate it using the blower that is already blowing? Stay tuned as we continue to design and implement our first prototype.