Sacramento Colocation Blog

Update:  ADC has just been covered in this Sacramento Business Journal article.

I’ve seen the Advanced Data Centers web site when I was scanning through Loopnet, but I was oblivious to the obvious connection to the Sacramento area (a Loopnet listing for Sacramento, company web site associated with the listing, ummm, duh).  In any case, I’d like to stand corrected.

I actually made note of the Loopnet listing in my last post when working through the various happenings in the local data center market, but Advanced Data Centers is the actual company behind the listing.  Here’s a brief backgrounder on the company from Datacenterknowledge so I won’t bother regurgitating it.   Apparently the site was a former USAF radar repair facility at McClellan.  Some old inside building shots and history are here.  This page has some newer shots and data center specifications.

I mainly thought at 45-50MW their capacity was interesting.  Their SMUD substation size is identical to Ragingwire’s and Herakles‘ though their power density seems to be similar to Ragingwire, who has 46MW capacity.   Ragingwire is working at around 200-250W/sq ft and ADC appears to be running in essentially an identical watts/sq ft range, which works out to approximately 15-17kW/rack (I believe - someone correct my math).

Drawing upon ADC’s VP Sales Blog and their own web site, their market focus seems to be users who want their own private, large DC suites (about 10K sq. ft.).  If you’ve been in Ragingwire lately, this doesn’t seem like an entirely bad idea as there are several users who are taking large swaths of their newly built out space - 10K sq. foot on up believe it or not is not uncommon in there.

The advertised power density makes it feasible, for example, to have 3-4 HP c-class blade chassis (or up to 64 servers per cabinet!).  The image at the top of this blog is from this Flickr photo, which shows 4xc7000 blade chassis in one cabinet so this is definitely a viable configuration.  Not surprisingly though HP does not recommend more than 2x loaded blade chassis per cabinet, but if you have the power and thermal handling capacity (a single c7000 blade chassis uses about 15K BTUs/hr so you’d be pushing out major heat out the back if you had 3-4 chassis/cabinet) - why not, right?  The only other higher density scenario I can imagine is something like 45-46 of these type of dual-motherboard 1U servers, which is not likely to be remotely possible as you’re looking at at least 20-25kW/cabinet, unless you have a pretty custom scenario like chimnied cabs that suck the hot air directly into the plenum or cabinets that have self-contained cooling, which are extremely costly and seems pointless in a data center (the data center really should be handling all your cooling needs, right?).  I’ve heard three-phase power into a c7000 is more power efficient as well.

In any case, another interesting player in the local Sacramento data center market.  They seem to be competing directly  with Ragingwire as their facility specs is essentially identical to RW’s.

Also, I don’t want to fail to mention that the data center I was speculating about on my last post is not only very much for real, but I already have a presence in there.  The company is Rely.net (valuable domain name by itself, eh?) and is indeed in business and ready to take customers.  Their cabinet, power, and bandwidth pricing is among the best in the region so please contact them directly if you have a need for cabinets.   They have about 60 new 45U Wright-Line cabinets setup and ready to go.  They also have Level(3) and XO coming in over fiber directly and are thus one of the few places in town that have Level(3) in the building other than, of course, Level(3)’s DC in West Sacramento on Triangle Court.

Filed under: Colocation Market Notes on June 5th, 2008 | No Comments »

Not really sure, but these guys could be setting up a data center for public consumption in Rancho Cordova. Hints are here. Former JPSers could be involved, guys with a history with O1 and Ragingwire. Beenup2, a local mobile-oriented web site, may be hosted there presently. May be a private DC for the company though. Based on the Beenup2 threads, connectivity to the site appears to be XO with Level3 fiber coming in soon. Hints from the principals involved seem to indicate colo may be a service offering coming out from that location. The data center looks like it may have been a former Verizon and/or Brooks data center, probably quite some time ago, located at 2999 Gold Canal, Rancho Cordova. The location is small - a bit above 4K raised floor space, but probably not a bad start for what seems to amount to a small colo provider. This link also hints at the site’s original purpose and that it was up for lease relatively recently. You may notice that the live.com maps don’t reconcile exactly to the building elevation in the lease PDFs so I think my pincushion is off by one or two buildings - though I can’t figure which one yet.

This new provider is centered in an area that is fairly dense with fiber and some local data centers. For example, if you study this listing of hazmat sites in Sacramento you’ll see that there are many data centers and telecom POPs within spitting distance (admittedly you’d have to be a pretty skilled spitter) of each other in the White Rock-Gold Canal-Gold Camp-Kilgore Road area: eBay, Sprint, Qwest, Barclays, and of course the Stephen P. Teale Data Center. In general Rancho Cordova is rather rich with glass, data centers, and telecom POPs with Franklin Templeton, Vision Service Plan, EDS, Progressive, Integra Telecom.

Note the relative density of data centers and telecom POPs in this Rancho Cordova triangle (here’s the actual link to the live.com collection):

The cluster of small buildings in the Ramos Circle-Rockingham-Placer Lane-Old Placerville Road about two miles away from this cluster also has a small group of data center/telecom providers like the former Brooks/ELI site @ 10316 Placer Lane and Lanset/DataNOC at 10321 Placer Lane.

All this could make for an interesting development in the local data center/colocation market, especially if these new providers can price around O1’s tier - and O1 just bumped up their pricing considerably. With Ragingwire planning to add 200K+ square feet, the Sacramento data center market is becoming more interesting. With this and this basically built out data centers available for occupation there are potentially ripe opportunities out there for DCs. I just have to wonder how these little guys will fare.

Filed under: Colocation Market Notes on March 25th, 2008 | No Comments »

Well, it was a long night and early morning, but Serverminds entire colocation operation successfully moved from Herakles Data Center to O1 Communications Data Center at 1515 K st. in Downtown Sacramento.

When we finally got the load of equipment down to O1 it was about 9:30pm. The most immediately noticeable difference between the sprawl of concrete tilt-up warehouses in Natomas and Downtown is of course the presence of a nightlife proximal to the data center. The O1 Lofts, directly opposite the O1 offices on 16th st., house O1’s main offices on the second floor (in addition to the offices and cubes in the 1515 K st. building) have drawn in a tony crowd of youthful denizens at Mizuni and PF Chang’s on the ground floor of the building and there’s another loft building next to it that has additional eateries. The crowds were big, the limos long, and I think I spotted a Ferrari in front. It almost felt like The City’s Financial District at night. Sacramento isn’t quite as much of a brute cow town as it used to be I suppose.

Capitol Garage, one of the older local coffee shops, is directly across the street from O1 on K st. I’m not a big coffee person, but the brews I’ve had there are quite good.

O1 is in a location that doesn’t make it real easy to move hardware in–basically you have to fight for some alley or street parking to really move in, but, once you’ve got your equipment inside, the differences between it and Herakles aren’t considerable. I can say I much prefer their 46U, 36″ deep Wright-Line cabinets over Herakles smallish Rittal cabs. There is far more room for cabling and for front and rear access to servers.

What went wrong with the move?

1.  I spent too much time trying to bring a server up that would port forward to the new IP space at O1.  It turns out there was some sort of issue with the Netgear switch I had in there and I delayed our move out by probably about 45 minutes dinking around with it and trying to get the connectivity up.  One of my customers wanted his site to have a maintenance page up during the transition and I had spent much of the day preparing this new server for this and a variety of other tasks especially pushing DNS to the new IP space.  The server was ready but the switch connectivity never came up properly so I had to ditch the idea.

2.   I hadn’t considered the IP routing setup closely enough.  I received a class C IP space allocation but didn’t think much about the way they had assigned it to me until the day of the move and by then it was too late to do a network change in time for the move.  This means I’ve had to reconfigure the network setup again which, while not a huge change, is still not fun.

3.  I probably needed one more body to help me.  A customer of mine helped me greatly with the move but he had his own gear to focus on getting live and it was up to me to bring up the rest of the hardware up.  It would have been helpful to have someone racking gear, logging on to verify connectivity, etc. while I solved other problems in the move and we shifted back and forth performing grunt work.

What went right with the move?

1.  Bringing up most hardware happened easily.

2.  The hardware that didn’t come up right away took no more than 30 minutes total to bring up.

3.  I didn’t lose any cords, nuts, bolts, rails, switches and, most importantly, servers in the move.  Always a bonus not to lose your customers’ gear.

Of course the move was a success - I got moved, after all - but it wasn’t as smooth as I would have liked.  And everything was back up about 2-3 hours beyond where I had really wanted everything to be back up.

My commit to O1 is long-term so I won’t need to start even remotely considering another move any time soon but as the business grows into 2, 3, 4, etc. cabinets a move will definitely get more complex and will probably need to occur in stages rather than in a single shot.

Filed under: Colocation Notes on February 26th, 2008 | 1 Comment »

Well, it’s happened. I’ve made the decision and signed the contracts: Serverminds is moving its entire colocation operation to O1 Communications in Downtown Sacramento.

When I originally was looking at starting this business, it was a close race between O1 and Herakles. I decided on Herakles because they had insurance requirements that could be waived, which made the monthly payments for a half-cabinet (what I started with) about the same between the two locations.

In retrospect, this turned out to be a fundamental mistake for a couple reasons:

• I didn’t get competitive price quotes for the liability and workmen’s compensation policies I need. It turned it the basic policy requirements I have to be in O1 aren’t as expensive as I thought they’d be back then. This colored how I weighed out my startup costs.
• The bigger mistake was not looking at how much it would cost to grow; e.g., how much was it going to cost me to grow from a half- to a full-cabinet? How much were additional power circuits, bandwidth, IPs, etc. going to cost?

Had I weighed this out more carefully I would have seen that O1 allowed me to grow far more cost effectively over the long term, even if the initial costs were a bit higher. Had I seen that I would have been able to keep a lot more money to grow my business rather than sinking it into large setup costs and bandwidth overage charges that I’ve gotten hit with over the last year.

And, let me be careful to say this is not a deep criticism of Herakles or their pricing. Based on what I’ve seen Herakles’ customer base is typically the small customer that has some kind of revenue stream that facilitates growth and helps pay for the costs associated with having a cabinet, cage, etc. For example it could be a customer–a small bank, for example–who needs an offsite disaster recovery location that is secure, air conditioned, reliable, etc. and doesn’t see it as a big deal to have a few servers in a cabinet there. It’s just part of the IT budget.

Or it’s a big customer - Qwest, Symantec, eBay are all customers there - that just needs that conditioned space and doesn’t want to build it out themselves. eBay swallows up DC space all over Sacramento - in Herakles, Ragingwire, and I believe they are in what was once Sprint’s DC in Rancho Cordova. Though I’m sure they negotiate hard for the pricing the costs are relatively inconsequential for them because they have a huge revenue stream to draw upon.

My business couldn’t be more different. Small server colocation is a microscopically incremental growth in revenue for every new customer - say $50 to $300/mo. growth per customer. This means when I filled up the half-cabinet I was filling it up with small, incremental amounts of revenue, not a big customer that was paying me thousands of dollars a month for services. This meant every need - a new power circuit, more bandwidth, and more space - was far ahead of the revenue growth trajectory. The two just simply did not track each other very well at all.

In spite of this things always seemed to work out - e.g., I’d get a consulting gig that would cover those expenses, usually - but in retrospect it stings to realize those extra dollars could not really be reinvested into the business in a way that might significantly add to my revenue. In other words, spending, say, $1000 to perform various upgrades wasn’t matched with an additional $1000 in revenue coming in. While there is, of course, always a cost to doing business, it felt like shoving money down a hole because I had positioned the business in a place that didn’t address this fundamental incremental growth need. I was short-sighted or, at the time I started this business up, mostly penny wise and pound foolish.

Herakles and Ragingwire and probably most DCs simply do not have pricing designed for this kind of growth trajectory. It does not mean they are “bad” it just means they know their markets and they know how best to make money. After all, running a data center is not cheap at all. If you’ve ever been inside Ragingwire, for example–just imagine what their electrical costs must be every month! Wow! Imagine roughly a football field of dense power requirements: power-sucking servers, switches, routers, SANs. They have to charge a lot of money to make a profit and keep their business growing. I am realistic that I have chosen an inherently expensive business to be in.

The learning experience for me is I now understand better how to grow the kind of business I am in. Sure, this business may change and grow in other directions, but in the meantime I have had to discover–sometimes painfully–how to lay a foundation for this growth without losing my shirt. O1 is better aligned to this kind of incremental revenue growth. They have a smaller data center, have only built out with cabinets and have deliberately eschewed customers that want to swallow up half their data center with cage space. They’ve stuck with small guys like me that need just a little bit of space and don’t want to pay a bunch of money for it and a bunch of money to grow it, when need be.

In short, I’m pretty excited about the move and about what it means in terms of helping grow this little business of mine.

Filed under: Uncategorized on February 8th, 2008 | No Comments »

After many months of development I’ve finally decided to release a beta of Fastdeploy, my take on automated operating system install solutions.

There are at least a few Web based front-ends that work to modify distribution installer scripts (e.g., Kickstart, Debian preseed) but Fastdeploy may be one of the first that is OS-agnostic and has no great dependency on the underlying distribution or operating system it’s running on.   I’ve tried to write it to be as modular as possible, with modules or plug-ins for each supported operating system and an underlying design that maps the operating system to the database to the automated script file.

I see Fastdeploy as very compelling for technology environments that have to deploy or redeploy many servers or workstations each month.   Its value proposition is that it largely eliminates much of the manual process of install operating systems and concomitant server software (mail server, web server, database server, etc.), a burden that is often put upon smaller IT departments that can’t afford a complex and expensive automation or imaging solution and don’t have the time or budget to develop their own.

I support Fastdeploy commercially but the code (written in PHP) is completely open source.  Feel free to change it as you please, offer suggestions, report bugs, etc.  Go to http://www.fastdeploy.com to learn more, see some demos, and download it.

Filed under: Uncategorized on April 11th, 2007 | No Comments »

Man, am I hyperanalyzing this or just trolling for new articles?

Well, in all seriousness, I see these kinds of posts on WHT, as though it truly was a mystery what it’s like or what’s involved in colocating. It’s all very simple really.

Say it’s the day of your install. You come prepared with your hardware, power cables, an OS install CD (just in case–generally you have your OS installed and configured with an assigned IP address beforehand), a power screwdriver, your rack rails or shelf, a 5′-7′ CAT5 patch cable, a buddy to help you lift your box in place if it’s that heavy (a 2U box with 6 drives and redundant power can get pretty heavy), and probably a laptop. Leave the donuts and soda outside as few data centers (none?) permit eating and drinking inside the DC itself. Make sure your security and access to the data center is determined beforehand as you may have to step through that process the first time you arrive.

If you have your hardware delivered to the DC there is probably a space where you can bring it and stage the hardware, connect it to the network, etc. Most DCs are good about providing space to customers to get hardware ready before loading it into a rack/cabinet.

What about tools? Some people are pretty fanatical about having a big toolkit for testing the network, making cables, etc. I personally have a nice RJ-45 crimp tool, wire strippers, and usually RJ-45 ends. I have screwdrivers and try to always remember to bring my DeWalt 9V power driver. When you’re holding up a firewall, switch, router, or smaller server with one hand trying to get it screwed in and mounted having a power driver could save your bacon and your hardware.

What about rails, screws and mounting hardware? Most DCs I’ve been in have a pool of screws and square mounting lugs. You generally don’t have to provide that. You do need rails suitable for the environment. For example Dell hardware has two kinds of rails, Rapid Rails and Versa Rails. Versa rails are designed to fit into any cabinet whereas Rapid Rails are designed specifically for Dell Racks but they also generally work with EIA-310 compliant cabinets (e.g., APC Netshelter, Wright-Line, Rittal, Chatsworth, etc.). Rapid Rails are nicer because you just click the rails in place in the square mounts and do not have to screw them in. You should not buy static rails (i.e., fixed-length rails) as they may not fit in a shared cabinet where there’s existing equipment and it’s impossible or near-impossible to readjust the front or back mounting plates. If it’s your own cabinet or rack you can do what you want but in an existing shared environment you may be stuck if you have static rails.

What if you don’t have or can’t find rails for your 1U+ server or you have a tower? Expect to bring a shelf or try the generic Gruber rails which you can find cheaply on eBay. You can go to a local Fry’s Electronics (my favorite place this side of heaven) to find rack shelves or order online from any number of sources. In short, don’t expect your DC or provider to have the shelves or rails unless you specifically ask and pay for it. Rack solutions is also an excellent resource if you want or need to center-mount a 1 or 2U server in a 2-post rack or just can’t find the right kind of rails for a 4-post setup. They are not cheap but I’ve used their stuff before and it’s quality hardware. Also, avoid getting some monster shelf to house a 1U server because you could end up wasting 1 or more Us with just the shelf and might be charged accordingly. Some server vendors sell center-mount rail kits but they are pretty rare. In general even the smallest shelves (except the Gruber shelves) will use 2Us so be aware of that when planning a space and buying shelves. Take into consideration the weight requirements of what you’re putting on the shelf as many cheap aluminum shelves are only rated to about 50#. I would mention the 100#+ loaded Catalyst 6509 I once put on a tiny aluminum shelf, but let’s not go there shall we?

I prefer mounting hardware with as much internal hardware as possible removed; i.e., I’ll pull all the hard drives and power supplies. HP/Compaq DL series Proliant servers for example have (for some reason) very heavy drive trays so I pull them all out and stack them in the order I took them out–and of course I have to remember that order otherwise I’m dead meat. Dell PowerEdge has sturdy but lighter drive trays so it may not be worth the trouble to pull the drives. Power supplies can be quite heavy and by the time you’ve pulled as much hot-swap hardware as possible the system might be manageable for one person or more manageable for two people. Why give yourself a hernia or hurt your back if you don’t have to?

After you’ve got all your gear mounted — and you’d be surprised how much time you can spend just in the racking process — then the real work begins: making sure it all works. If you come in with a couple servers and a firewall you may spend some hours getting it all working. On the other hand preconfigured hardware may just require racking and plugging in a few Ethernet connections and you’re good to go. Most sensible admins though will do a fair amount of testing to verify they can access their hardware remotely so they’re not stuck once they leave the DC.

That leads to another topic: Remote access. Of course the operating system itself will generally either have shell/SSH access (Linux, xBSD, Solaris, and other UNIX or UNIX-like systems), RDP (Windows), and/or Web-based management tools like cPanel, Plesk, DirectAdmin and so on that have became extremely popular with dedicated server and hosting/VPS providers. But what about so-called “out of band” management when your box is physically unable to boot properly or you’re stuck on a prompt and can’t get into the actual operating system? In that case there are a few ways to handle remote access:

1. Serial console. Dell, HP, Sun, and other vendors support serial console redirection from the BIOS. This means you can use a serial console device (see Cyclades (now Avocent), Digi) to get into the BIOS and see what’s happening during a boot failure. A serial console has the advantage of being cheaper than an IP-KVM but the disadvantage of not supporting VGA or higher video. Basically if it’s not pure 80×25 ASCII text you won’t see it on a serial console. Many Linux/UNIX/Solaris shops will use serial consoles because it’s much easier to manage those operating systems in a serial console vs. Windows. Sun SPARC hardware for example–and perhaps their x86/x64 hardware–has very good serial console support to the extent that you can easily install, manage, and troubleshoot the entire OS from a serial line. Generic x86 hardware though generally does not support serial console from the BIOS but most OSes will support serial lines from the boot loader (e.g., LILO, GRUB) into the kernel itself so a lot of times you can see a kernel boot failure / problem even if the BIOS doesn’t support serial console redirection.

2. Internal management cards and devices. HP has iLO and rLO, Dell has DRAC, Sun has LOM, all internal management devices that permit remote out-of-band management over a network line, in most cases even when the server is having serious problems short of a total power supply problem or failure. IBM xSeries no doubt has a solution as well but I don’t know what that is.

3. IPMI. IPMI is designed to be a full generic vendor neutral remote OOB (out-of-band - in other words your server is toast and you need to get a console) protocol. It supports management like a serial console over a network line so you don’t necessarily need both a serial console and a “lights-out” remote management board at the same time. Most server vendors now support IPMI off one of more internal network cards in their servers. I’ve never actually used IPMI though I have servers that support it but I’ve heard it has problems especially with the early implementations (but isn’t that always the case?). More generic barebone server vendors like Supermicro, ASUS, Tyan, and MSI are likely to support IPMI instead of developing their own remote management board solutions.

4. KVM-over-IP. Last by not least is KVMoIP. Simple enough - get your KVM console over a network connection, not respective of any remote management boards, remote management protocols, etc. In my experience KVMoIP is nice but has its issues such as lousy console interfaces–nearly all are written in Java which is usually slow and clunky–and awful mouse synchronization for Windows-type environments. Actual keyboard and mouse support can also be tricky but it’s always been that way with KVMs. I once had a Belkin Omnipro that to be physically rebooted every time I added a device. This was a 16-device KVM so that meant I had to unplug every single connection (because the KVM also derived its power from any keyboard or video that might be connected) then reconnect it every time I plugged in a new server or device. Long story, but what a nightmare and I’ll never use Belkin KVMs because of that experience. There are fully integrated KVM-over-IP devices (see Avocent again or look at other vnedors like Minicom, Dell’s rebranded and usually cheaper Avocent solutions, and rebranded HP, IBM, etc. KVMoIP products) or KVM-over-IP head devices that plug into an existing KVM (see Aten CN-6000 , the Avocent DSR1024, or the Startech SV1110IPEXT)  which are relatively cheap ways to get IP KVM access using existing KVMs without breaking the bank on a big Avocent solution.   Ridiculously expensive rebadged vendors like Black Box are probably options here but I don’t like to mention them.

Or… you could just use a rolling monitor and keyboard cart that most DC providers place throughout their facilities (usually).   You could be in a world of hurt at 3AM though if you have to crawl out of bed and fix the hardware.  Remote management is of course essential in 24/7/365 environments where hardware could be halfway across the world.  Even it’s 30 minutes away it’s usually easier, better, and faster to have a remote management solution that you can operate from a PC at home.   The bit of extra money you spend up front for the better hardware that supports remote management will almost definitely save you time and therefore money down the line.

Filed under: Uncategorized on February 16th, 2007 | No Comments »

Let’s start off by first asking and trying to answer a simple question: What is colocation anyway?

Colocation generally means putting your server(s), storage equipment, and possibly network hardware in a large-ish air conditioned raised floor data center facility that has lots of power backup (batteries for power conditioning, UPSes for short-term power blip and for the automatic transfer switch to kick on the genset, and generator sets to provide long-term power in the event of power failure), in-building fiber optic services so you can scale to massive bandwidth if you need it (e.g., you become a Youtube over night), in-building tier1-3 Internet providers (e.g., Level(3), Qwest, Verizon, AT&T, Global Crossing, etc.), and general physical security so it’s not terribly easy for creepnoids to smash through a door or window and make off with lots of spendy hardware.

Once you choose your provider, you sign a contract and/or service agreement, toss your pile of servers in the back of your car (ok, maybe not toss, maybe set gently), drive over to the data center, get your security access setup and taken care of (this means getting a badge frequently with an associated PIN and biometric certification so a hand print or retina scan will certify you are who you say you are), then take your hardware to the cabinet(s) inside the data center itself and get your equipment up and running on a direct or filtered/firewalled Internet connection.

Where can you colocate?

There are hundreds of colocation facilities across the United States alone and many in Europe, Asia, Australia, and so on. If I had to pick the “biggies” Equinix, Switch & Data, NTT/Verio, and Level(3) probably qualify as some of the largest though the Bells like AT&T and Verizon probably have many people in colocation facilities as well. There are also REITs like Digital Realty Trust and the big shot telco hotel buildings like Market Post Tower in San Jose, 365 Main in San Francisco, and One Wilshire in Los Angeles. These buildings tend to aggregate many, many colocation service providers, web hosts, telecom companies, ISPs, fiber optics providers, and so on and a lot of public and private peering arrangements occur in these major concentrations of data and bandwidth. I’m not sure where they sit in terms of total market share, but Equinix is probably doing the most colocation business globally. Basically any time they open up a new facility it’s usually at least 50% occupied before the doors open and their existing facilities’ space is so scarce that generally the only space that opens up is the rare 1-3 cabinet vacancy by an existing customer who leaves. In short, they have been extremely successful in the colocation business.

If you put together the sum of all colocation providers in the major metro areas in the US, most if not all are doing extremely well. Some perhaps in far-flung smaller metros may be hurting for business but certainly just about anything in LA, San Francisco Bay Area, Dallas, Miami, Atlanta, New York, Boston, and Chicago are doing very well.

Why we’re were we are (or can I come up with a more confusing sentence?)

You see what happened when the dot-coms flamed out in originally in 2000-2001 is you had just tons and tons and tons of this very expensive built-out data center infrastructure all over the country. There were numerous “dark” buildings all over the map even in the Bay Area and Silicon Valley, LA, and New York. AboveNet, MFN, Colo.com, C&W, Digital Island, Global Crossing, etc. all went bankrupt and left not just billions in debts but these beautiful, sometimes grossly overbuilt data center buildings that had megs of gensets, massive fiber, rows and rows of racks and cabinets and ladders, thousands of batteries to condition power and so on. I mean it was beautiful stuff and landlords were saddled with these very expensive but then-unsaleable/unleaseable data centers.

What happened though is that in spite of the capsizing economy the web hosting business just kept on growing and growing and growing then the dedicated server business came along probably seriously in force in 2001-2003 and the hosting industry has never been the same. Some very smart investors and REITs saw what was happening, knew about this massive infrastructure that was sitting idle all over the country and got it often for pennies on the dollar out of bankruptcy courts. And then the vanguard like Equinix came along and started swallowing up these cheap facilities as fast as they could, building up a portfolio of elite facilities all over the country. So–even though 9/11 happened and the economy flew off a cliff for many of us the hosting and Internet access business still grew like crazy.

Where we are now

The end result is that customers now have a lot of options for colocation and hosting and Internet access in most major metro areas–and even in smaller cities.  But demand is making it harder to make the business work in larger metro areas where space availability is pretty limited.

The nice thing about colocation for most customers is that they can put a few servers in a rack and their hardware can have a fast connection to Internet quickly and easily (no more 45-90 days turnaround times waiting for a lowly T1 to come in when you can have 100Mbps+ access in the matter of a few days!). In many cases if they don’t have the technical expertise their provider has it or someone in the building they’re in has it so they don’t have to go far to get the support they need. It’s a good value proposition for the customer whether they need 1U of rack space or they need their own cage space for several racks or cabinets.

Filed under: Colocation Notes on February 13th, 2007 | No Comments »

There are blogs about 50 billion boring and insipid topics so why not explore yet another micro-niche?

My motivation to do so rests with the following:

1. I am starting up a colocation service in Sacramento, in Herakles Data Center. It’s called Serverminds.
2. I have more or less kept my ear to the ground regarding the Sacramento area “tech scene” (as if that were a cool thing) for the last several years.
3. For some nutty reason I think there might be this microscopic demographic of people out there that are interested in some of the same esoteric topics I’m interested in; for example, there may be local “techies” who are curious about what kind of options there might be to have a local data center for their startup operation, hot or warm (or even cold) site disaster recovery / failover site, or just to colo a box they want to host their own personal projects on.

Ergo a blog concerned with a topic as specific as Sacramento colocation and as broad as colocation as a subject in general.

So… let’s get down to business.

Here is a link to my local.live.com “Bird’s Eye” links to known local data centers, colo facilities, and some telecom sites of general interest. Most of Sactown is covered by the Bird’s Eye views so you should get some visuals of exciting concrete tilt-up warehouses, gensets, and close proximity to lots of power. I realize some of these may be wrong and it’s far (far!) from comprehensive. There are probably numerous good private corporate and State of California data centers I am not aware of (Teale Data Center is one I can think of off the top of my head that I haven’t included). Also my building locations could be completely wrong as live.com’s mapping is abysmal in the Bird’s Eye mode.

For now, let’s hit the high notes of what I think you might want to know about Sacramento Data Center and colocation providers:

1. RagingWire Enterprise Services. Probably the “king” of the local DC market, RagingWire is a very nice facility in Natomas just up the street from probably its main competitor, Herakles Data. RES focuses primarily on servicing the enterprise Fortune 1000 customer and has filled up most of their large warehouse with cage and a limited amount of cabinet space. They have built their business on providing technical support and management services to their customers and have done very well building up the technical staff to support the customers in the building. Their “known” customers including Alibris, Applied Materials, Photronics, Blackrock, Flextronics, and, based on their AS report, they may have Raleys and KLA-Tencor as customers. Their management is comprised of a lot of former semiconductor industry veterans including two pinched directly from Applied Materials’ executive management. RagingWire will typically quote high on pricing but you can probably get them to compete with Herakles’ pricing (well, you used to anyway…). I worked for a company that was in Ragingwire and there is a lot of big iron in the building, so to speak. If you want to see some impressive build-outs just go for a tour sometime.

2. Herakles Data Center. Herakles has been in business about as long as RagingWire and it’s fair to say they come from a more storied history. The DC itself was built on the cusp of the dot-com bust in 2000-2001. A former president and CFO of a long defunct company that was going to barnstorm across the country and build similar data centers, Wavve Telecommunications, is now the president and CEO of Herakles and has been since its inception. I guess basically he picked up the bankrupt remains of Wavve’s facility and decided to make a go of it. Herakles is probably best described as the middle tier between the low-end DC providers and the high-end like Ragingwire. They’ve only relatively recently focused on managed services, but, given they’ve survived and seemingly prospered for the last 5+ years, it’s a good company to work with and their pricing is a bit more reasonable than Ragingwire. I don’t know what their current customer list is like but way back when I first toured the facility Qwest had some kind of search engine in there–there were dozens and dozens of cabinets of Dell servers in there and they occupied a good third of more of the DC space that was open–but I’ve heard Qwest has long since pulled out. I think Herakles’ fortunes have improved considerably since then however. Back then the other 25K square foot of the original build out was actually purposefully dark to save on costs. I think now they are pretty close to running out of room.

3. O1 Communications. O1 Communications operates a former Inflow facility at 1515 K st. It’s a small data center–maybe 10-11K square feet total. O1 will kill on price: They are the cheapest in town that I’ve come across. You can get third, half, or full cabinets very reasonably priced. Anyone with any history of the local Sacramento ISP scene will know who started O1–Brad Jenkins of JPS.net fame (or infamy, depending on how you look at it). JPS.net, for those who don’t recall, introduced the wildly successful $99 a year Internet service. Though every local ISP (including the one I worked for at the time) slammed JPS.net because it just had an awful reputation, most of us secretly envied its success. Anyway, Brad took a little bit of the money he got from selling JPS.net to Onemain (and later Earthlink in the twilight of the dialup ISP biz) and started O1. O1 became successful selling managed modem pools back, ironically, to the same dialup ISPes that JPS.net once competed against. O1 is basically a telecom now, a facilities-based CLEC and sells local phone service, Internet service, VoIP, and colocation.

4. Lanset/DataNOC. I don’t know much about these guys other than they never responded to sales requests. That and I didn’t really want to go all the way out to Rancho Cordova to do colo. I believe Lanset was once–maybe still is? Are these things still around any more?–a computer store. They got into the ISP and webhosting / colo business. DataNOC is their colo business. Their pricing seems good, but I know nothing about their facility, bandwidth, power, backup generators, etc.

5. All the rest:

Surewest. Surewest, formerly Roseville Telephone, picked up the bankrupt remains of high-flying Fiber-to-the-Home provider Winfirst a few years back for pennies on the dollar. Winfirst’s killer data center in McClellan business park came along with that acquisition. They do sell colocation though I’m not sure how aggressively as their sales staff also never responded to repeated attempts to contact them. Must be nice to have business be so good.

Level 3. Of course L3 is one of the leading national fiber / tier-1 ISPs. They have DCs and facilities and long-haul fiber optic networks all over the country. In general their bandwidth has an excellent reputation. In general they are not cheap. I believe they have a facility in a dumpy cinder block building along the railroad tracks on Triangle Court in West Sacramento and they may also now run/own the former Wiltel building in the equally and perhaps more dumpy neighborhood across the river from downtown around 2nd and B streets in Sacramento.

MCI/Worldcom (now Verizon Business). Though Bernie Ebbers is now rotting in a jail for the rest of his life one of the few good things MCI/Worldcom did during its high-flying days was build a pretty tight tier-1 ISP starting with probably the first true commercial ISP, UUnet. That network lives on and still has a pretty good rep. They have a largeish facility on KOVR Drive in West Sacramento, unsurprisingly next to KOVR the TV station. I have no idea if they offer colo there or not. It may just be a CO and data center for the exclusive use of Verizon. At one time I believe many people worked out of that building so others may be able to speak to what its real purpose is.

XO Communications has a colo facility in Roseville. Because I live West of West Sacramento I have zero interest in driving an hour to Roseville to colo hardware, but more centrally located folk might be interested in their facility. XO has had a very up-and-down reputation over the years but apparently it’s not hard to get them to compete on pricing.

Electric Lightwave may do colo in their Rancho facility. In a land long ago and very far far away ELI was the bomb for local dialup ISPs. They were an excellent source for cheap T1s, PRIs, and cheap and relatively decent fast T1+ Internet access. They were a good CLEC to work with. In fact, when I worked for one of these local ISPs we visited ELI when they very first put their central office in at 660 J st., a small smoked glass building that adjoins Downtown Plaza. That was probably around 1996 or ‘97. When they bought Brooks Fiber I believe they consolidated their facilities in Rancho. I don’t even know if they maintain a physical POP on J st. any more. Way back then I also visited Brooks’ colo facility in Rancho–it was very small–but that’s another story.

Who else? Of course there is AT&T but you probably need CLEC certification to even consider putting hardware in one of their COs. Verizon owns a mostly barren former GTE facility off Freeport Blvd. I have no idea if there is a decent data center there but by all appearances there is indeed something there (I used to drive past it frequently on I-5), but it may be private or limited to a pool of Verizon’s big enterprise type customers. If you go on loopnet.com you can see that the adjoining office space has been up for lease for quite a while now.

What about the big tier1s and tier2s like Cogent and Internap? I don’t know if they sell colo space somewhere in Sacramento, but they may. Buildings like 815 L st were once considered “telecom hotels” but I’m not sure if there is still any significant telecom presence in this building though I’m sure like most large buildings there is at the very least a big ma bell fiber fridge in the basement.

Filed under: Colocation Notes on February 8th, 2007 | No Comments »