When Jon needed to open a Backblaze Storage Pod for maintenance in our Phoenix data center, it wasn’t as straightforward as one might think. With a steel case, 60 hard drives, backplanes, a pair of power supplies and other components, each pod can weigh up to 150 pounds.
However, there was even a bigger challenge than the pod’s weight. A Storage Pod is divided into two main sections, the drive section and the processing section, each with separate access panels. To replace a drive, you need to open the access panel at the front, which requires sliding the Storage Pod out of the front of the cabinet. To replace a power supply or perhaps reseat a SATA card or cable, you’d prefer to slide the pod out the back of the cabinet because that gives you better access to the panel at the rear top of the pod.
Backblaze’s 6.0 Storage Pod with 60 drives (front)
The problem was that doing that was difficult, if not impossible, with all the power cables that connected the pods to the power distribution unit (PDU) at the rear of the cabinet. That left Jon with only one choice: slide the pod out of the front of the cabinet even when he wanted to access the rear access panel, which took more time and often required two people.
Identifying the Problem — the PDU
As Backblaze’s Phoenix data center site manager, Jon realized that the job would be much easier if he could change one component, the PDU. The Phoenix data center used vertically-mounted power distribution units (PDUs) at the back of the cabinets that ran all the way from the top to the bottom of the cabinet. All the cables from the ten pods to the PDU blocked access to the back of the pods in the cabinet.
Vertically-mounted PDU blocking rear access to Storage Pods
What’s a PDU?
A power distribution unit (PDU) is a device fitted with multiple outputs designed to distribute electric power to racks of computers and networking equipment located within a data center. Some PDUs have additional capabilities, including power filtering, intelligent load balancing, and remote monitoring and control by LAN or SNMP.
Data center IT managers remotely monitor PDU performance to ensure continual service, improve efficiency, and plan for growth.
Jon knew that the vertical PDU forced his team to spend more time than needed getting into the pods for service. If they needed to replace a component at the rear of the pod, that meant taking the entire Storage Pod completely out of the cabinet using the server lift. If Jon could find a better option for the PDU, everyone on the team would have more time to focus on other data center matters, like setting up more cabinets to fill with Storage Pods and customers’ data.
Guido, A Valued Member of Backblaze’s Operations Team
Sometimes we do have to completely remove heavy pods from a cabinet, but a special member of the team helps with that challenge. Our server lift Guido has no trouble lifting and moving 150 pound Storage Pods and IT gear when needed.
Our server lift, Guido (on the right), helping Joe with the heavy lifting in our Phoenix data center
The Backblaze Storage Pods and Cabinets
Backblaze’s Storage Pod racks are standard full size data center cabinets that are 42 rack units (U or RU) high — a rack unit is 44.50 millimeters (1.75 inches). Equipment that fits into these racks is typically 1U, 2U, 3U, or 4U high. Backblaze’s Storage Pods are 4U high, so ten of them can fit into a single rack. With a small switch at the top occupying one of those rack units, that leaves just 1U of space.
If Jon could use that 1U of space in the cabinet for a horizontally-mounted PDU, he could get rid of the vertically-mounted PDU that was causing the access problem. The PDU had more power outlets than needed, anyway, as well as extra monitoring circuitry that wasn’t required for Zabbix, the software monitoring suite we use to track the health of all the components in our data centers. That made it more complex and expensive than was necessary — two factors that go against Backblaze’s philosophy of keeping things as simple and inexpensive as possible to keep costs low for our customers. (For a bit of history on this, see this post on how Backblaze got started.)
A Better PDU
Jon made a list of the requirements he wanted in a PDU that would fit Backblaze’s needs. It didn’t seem to him that it would be that hard to find one ready to drop into the cabinet.
Jon’s PDU Requirements
- 1 rack unit high
- 3-phase power
- Horizontally mounted
- Metering to remotely monitor circuit loads
- 12 C13 power outlets
- 10 outlets for Storage Pods
- 1 outlet for small switch
- 1 outlet for crash cart to service the pods
Finding a PDU that fit the list turned out to be harder than he expected. Jon searched to see if anyone made a 3-phase 1U horizontal mount PDU, and the only one he could find didn’t have the right type of power outlets (C13) or monitoring circuitry.
The only remaining option was to design a custom PDU. Jon remembered that he and Larry, Backblaze’s data center manager, had run into a PDU manufacturer, Geist, at an IT trade show in San Jose. Jon contacted our vendor, Mirapath, whom Jon had successfully worked with on other projects for Backblaze. Mirapath got the project rolling with Geist, worked out all the kinks, and were instrumental in bringing the project to completion.
The Custom PDU
The result is a custom PDU that fits Jon’s requirements. The PDU fits horizontally in the center-back of the cabinets and doesn’t block access from the back of the cabinet. It takes up only 1U of cabinet space, which allows Jon to put ten Storage Pods in each cabinet — five above the PDU in the center of the cabinet and five below. It has the correct type (C13) and number (12) of power outlets, which support the ten pods, one switch, and the crash cart. It also contains the power monitoring circuitry needed to collect data for Zabbix.
 |
 |
 |
||
|
Custom PDU |
Custom PDU (back) |
Custom PDU ports and interface |
The custom PDU enables Jon and his team to access the Storage Pods from the back of the cabinet. Jon estimates that the new PDU enables him to complete a boot drive replacement in a Storage Pod in half the time it used to take with the previous PDU, and he doesn’t need the help of our server lift Guido for the job. That saved time adds up, especially when you need to replace boot drives in forty Storage Pods, as Jon did recently.
Custom PDU in a cabinet between two Storage Pods
Storage Pod open at rear of cabinet
Storage Pod open at rear top
We Value Our Culture of Doing Things Differently
If you’re a regular reader of this blog, you’re already familiar with Backblaze’s history. Backblaze’s founders started the company because they thought people should back up their computers and it could be done at $5 per month. The problem was that no storage system available at the time would enable a sustainable business at that price. They did what they had to do: designed and built their own solution. The Backblaze Storage Pods, vault architecture, and Reed-Solomon encoding enabled a globally scalable storage system. After eleven years, three data centers, and seven hundred petabytes of customer data, we’re still able to sustainably offer the most affordable storage available anywhere.
Continuing the Backblaze Tradition
Hardworking innovators like Jon and our operations team find new ways every day to make our operations more efficient. This allows us to continuously reduce our costs while driving our growing, global footprint.
Thanks Jon. Well done!
Jon with two Backblaze cabinets, each with 10 Storage Pods, one switch, and one custom PDU
Editor’s Note: Anyone interested in obtaining information about availability and pricing for the PDU described above can contact Mirapath.
