“Hybrid cloud” has jumped into the IT vernacular over the last few years. Hybrid cloud solutions intelligently divide processing and data storage between on-premises and off-premises resources to maximize efficiency. Businesses seamlessly extend their data processing and data storage capabilities to the cloud enabling them to manage unusual or fluctuating demands for services. More recently, businesses are utilizing cloud computing and storage resources for their day to day operations instead of building out their own infrastructure.
Companies in the media and entertainment industry are candidates for considering the hybrid cloud as on any given day these organizations ingest and process large amounts of data in the form of video and audio files. Effectively processing and storing this data is paramount in managing the cost of a project and keeping it on schedule. Below we’ll examine the data storage aspects of the hybrid cloud when considering such a solution for a media and entertainment organization.
The Classic Storage Environment
In the media and entertainment industry, much of the video and audio collected is either never used or reviewed once and then archived. A rough estimate is that 10% of all the audio and video collected is used in the various drafts produced. That means that 90% of the data is archived: stored on the local storage systems or perhaps saved off to tape. This archived data can not be deleted until the project owner agrees, an event that can take months and sometimes years.
Using local storage to keep this archived data means you have to “overbuy” your on-premises storage to accommodate the maximum amount of data you might ever need to hold. While this allows the data to be easily accessed and restored, you have to purchase or lease substantially more storage than you really need.
As a consequence, many organizations decided to use tape storage for their archived data to reduce the need for on-premises data storage. They soon discovered the hidden costs of tape systems: ongoing tape maintenance, supply costs, and continuing personnel expenses. In addition, to recover an archived video or audio file from tape was often slow, cumbersome, and fraught with error.
Hybrid Cloud Storage
Cantemo’s Media Asset Management Portal can identify and automatically route video and audio data to a storage destination—on-premises, cloud, tape, etc.—as needed. Let’s consider a model where 20% of the data ingested is needed for the duration of a given project. The remaining 80% is evaluated and then determined that it can be archived, although we might need to access a video or audio clip at a later time. What is the best destination for the Cantemo Portal to route video and audio that optimizes both cost and access? Let’s review each of our choices: on-premises disk, tape, and cloud storage.
Data Destinations
To compare the three solutions, we’ve considered the cost of each system over a five year period for: initial purchase cost, ongoing costs and supplies, maintenance costs, personnel cost for operations, and subscription costs.
- On-Premises Disk Storage: On-premises storage can range from a 1PB network attached storage (NAS) system to a multi-petabyte storage area network (SAN). The cost ranges from $12/terabyte/month to $20/terabyte/month (or more). These figures assume new equipment at “street” prices where available. These systems are used for instant access to the data over a high speed network connection. The data, or a proxy, can be altered multiple times and versioning is required.
- Tape Storage: Typically these are linear tape-open (LTO) systems with a minimum of two local tape systems, operational costs, etc. The data is stored, typically in batch mode, and accessed infrequently. The tapes can be stored on-site or off-site. Off-site storage costs more. The cost for LTO tape ranges from $7/terabyte/month to $10/terabyte/month, with much of that being the ongoing operational costs. The design includes one incremental tape per day, two week retention, first week on-site, second week off-site, with weekly pickup/drop-off. Also included are weekly, monthly, and yearly full backups, rotated on/off-site as needed for tape testing, data recovery, etc.
- Cloud Storage: The cost of cloud storage has come down over the last few years and currently ranges from $5/terabyte/month to $25/terabyte/month for storage depending on the vendor. Video and audio stored in cloud storage is typically easy to locate and readily available for recovery if needed. In most cases, there are minimal operational costs as, for example, the Cantemo Portal software is designed to locate and recover files that are required, but not present on the on-premises storage system.
Of course, a given organization will have their own costs, but in general they should fall within the ranges noted above.
Comparing Storage Costs
In comparing costs of the different methods noted above, we’ll present three scenarios. For each scenario we’ll use data storage amounts of 100TB, 1PB, and 2PB. Each table is the same format, all we’ve done is change how the data is distributed: on-premises, tape, or cloud. The math can be adapted for any set of numbers you wish to use.
SCENARIO 1: 100% of data is in on-premises storage.
| Scenario 1 | Data Stored | Data Stored | Data Stored | ||
|---|---|---|---|---|---|
| Data stored on-premises: 100% | 100TB | 1,000TB | 2,000TB | ||
| On-premises cost range | Monthly Cost | Monthly Cost | Monthly Cost | ||
| Low: $12/TB/month | $1,200 | $12,000 | $24,000 | ||
| High: $20/TB/month | $2,000 | $20,000 | $40,000 | ||
SCENARIO 2: 20% of data is in on-premises storage and 80% of data is on LTO Tape.
| Scenario 2 | Data Stored | Data Stored | Data Stored | ||
|---|---|---|---|---|---|
| Data stored on-premises: 20% | 20TB | 200TB | 400TB | ||
| Data stored on tape: 80% | 80TB | 800TB | 1,600TB | ||
| On-premises cost range | Monthly Cost | Monthly Cost | Monthly Cost | ||
| Low: $12/TB/month | $240 | $2,400 | $4,800 | ||
| High: $20/TB/month | $400 | $4,000 | $8,000 | ||
| LTO tape cost range | Monthly Cost | Monthly Cost | Monthly Cost | ||
| Low: $7/TB/month | $560 | $5,600 | $11,200 | ||
| High: $10/TB/month | $800 | $8,000 | $16,000 | ||
| TOTAL Cost of Scenario 2 | Monthly Cost | Monthly Cost | Monthly Cost | ||
| Low | $800 | $8,000 | $16,000 | ||
| High | $1,200 | $12,000 | $24,000 | ||
Using tape to store 80% of the data can reduce the cost 33% over just using on-premises data storage.
SCENARIO 3: 20% of data is in on-premises storage and 80% of data is in cloud storage.
| Scenario 3 | Data Stored | Data Stored | Data Stored | ||
|---|---|---|---|---|---|
| Data stored on-premises: 20% | 20TB | 200TB | 400TB | ||
| Data stored in the cloud: 80% | 80TB | 800TB | 1,600TB | ||
| On-premises cost range | Monthly Cost | Monthly Cost | Monthly Cost | ||
| Low: $12/TB/month | $240 | $2,400 | $4,800 | ||
| High: $20/TB/month | $400 | $4,000 | $8,000 | ||
| LTO tape cost range | Monthly Cost | Monthly Cost | Monthly Cost | ||
| Low: $5/TB/month | $400 | $4,000 | $8,000 | ||
| High: $25/TB/month | $2,000 | $20,000 | $40,000 | ||
| TOTAL Cost of Scenario 3 | Monthly Cost | Monthly Cost | Monthly Cost | ||
| Low | $640 | $6,400 | $12,800 | ||
| High | $2,400 | $24,000 | $48,000 | ||
Storing 80% of the data in the cloud can lead a 46% savings on the low end, but could actually be more expensive depending on the vendor selected.
Separate the Costs
Often, cloud storage costs are combined with cloud computing costs in the hybrid cloud model, thus hiding the true cost of the cloud storage, perhaps, until it’s too late. The savings gained by using cloud computing services a few times a day may be completely offset by the high cost of cloud storage, which you would be using the entire time. Here are some recommendations.
- Ask to have your hybrid cloud costs broken out into computing and storage costs, it should be clear what you are paying for each service.
- Consider moving the cloud data storage cost to a low cost provider such as Backblaze B2 Cloud Storage, which charges only $5/terabyte/month for cloud storage. This is particularly useful for archived data that still needs to be accessible as Backblaze B2 Cloud Storage is readily available.
- If compute, data distribution, and data archiving services are required, the Cantemo Portal allows you to designate different cloud storage vendors depending on the usage. For example, data requiring computing services can be stored with Amazon S3 and data designated for archiving can be stored in Backblaze. This allows you optimize access, while minimizing costs.
Considering Hybrid Data Storage?
Today, most companies in the media and entertainment industry have large amounts of data. The hybrid cloud has the potential to change how the industry does business by moving to cloud-based platforms that allow for global collaboration around the clock. In these scenarios, the amount of data created and stored will be staggering, even by today’s standards. As a consequence, it will be paramount for you to know the most cost efficient way to store and access your data.
The latest version of Cantemo Portal includes native integration to Backblaze B2 Cloud Storage, making it easy to create custom rules for archiving to the cloud and access archived files when needed.
(Author’s note: I used on-premises throughout this post as it is the common vernacular used in the tech industry. Apologies to those grammatically offended.)
