Getting Your Data Into the Cloud Is Just the Beginning

Organizations should consider not just the cost of getting their data into the cloud, but also long-term costs for storage and retrieval when deciding which cloud storage solution meets their needs.

As cloud storage has become ubiquitous, organizations large and small are joining in. For larger organizations, the lure of reducing capital expenses and their associated operational costs is enticing. For smaller organizations, cloud storage often replaces an unmanageable closet full of external hard drives, thumb drives, SD cards, and other devices. With terabytes or even petabytes of data, the common challenge facing organizations, large and small, is how to get their data up to the cloud.

Transferring Data to the Cloud

The obvious solution for getting your data to the cloud is to upload your data from your internal network through the internet to the cloud storage vendor you’ve selected. Cloud storage vendors don’t charge you for uploading your data to their cloud, but you, of course, have to pay your network provider and that’s where things start to get interesting. Here are a few things to consider.

• The initial upload: Unless you are just starting out, you will have a large amount of data you want to upload to the cloud. This could be data you wish to archive or have had archived previously, for example data stored on LTO tapes or kept stored on external hard drives.
• Pipe size: This is the amount of upload bandwidth of your network connection. This is measured in megabits per second. Remember, your data is stored in megabytes, so an upload connection of 80 Mbps will transfer no more than 10MB of data per second and most likely a lot less.
• Cost and caps: In some places, organizations pay a flat monthly rate for a specified level of service (speed) for internet access. In other locations, internet access is metered, or pay as you go. In either case, there can be internet service caps that limit or completely stop data transfer once you reach your contracted threshold.

One or more of these challenges has the potential to make the initial upload of your data expensive and potentially impossible. You could wait until cloud storage companies start buying up internet providers and make data upload cheap (Or free with Amazon Prime!), but there is another option.

Data Transfer Devices

Given the potential challenges of using your network for the initial upload of your data to the cloud, a handful of cloud storage companies have introduced data transfer or data ingest services. Backblaze has the Fireball, Amazon has Snowball (and other similar devices), and Google recently introduced their Transfer Appliance.

These services work as follows:

• The provider sends you a portable (or somewhat portable) storage device.
• You connect the device to your network and load some amount of data on the device over your internal network connection.
• You return the device, loaded with your data, to the provider, who uploads your data to your cloud storage account from inside their own data center.

Data Transfer Devices Save Time

Assuming your internet connection is a flat rate service that has no caps or limits and your organizational operations can withstand the traffic, you still may want to opt to use a data transfer service to move your data to the cloud. Why? Time. For example, if your initial data upload is 100TB, here’s how long it would take using different network upload connection speeds:

This assumes you are using most of your upload connection to upload your data, which is probably not realistic if you want to stay in business. You could potentially rent a better connection or upgrade your connection permanently, both of which add to the cost of running your business.

Speaking of cost, there is of course a charge for the data transfer service that can be summarized as follows:

• Backblaze Fireball: Up to 96TB of data per trip for $550 for 30 days in use at your site.
• Amazon Snowball: Up to 50TB of data per trip for $200 for 10 days use at your site, plus $15/day each day in use at your site thereafter.
• Google Transfer Appliance: Up to 100TB of data per trip for $300 for 10 days use at your site, plus $10/day each day in use at your site thereafter.

These prices do not include shipping, which can range from $100 to $900 depending on shipping method, location, etc.

Both Amazon and Google have transfer devices that are larger and cost more. For comparison purposes below we’ll use the three device versions listed above.

The Real Cost of Uploading Your Data

If we stopped our review at the previous paragraph and we were prepared to load up our transfer device in 10 days or less, the clear winner would be Google. But, this leaves out two very important components of any cloud storage project; the cost of storing your data and the cost of downloading your data.

Let’s look at two examples:

Example 1: Archive 100TB of data:

• Use the data transfer service to move 100TB of data to the cloud storage service.
• Accomplish the transfer within 10 days.
• Store that 100TB of data for one year.

Results:

• Using the Backblaze Fireball to store data in Backblaze B2 saves you $16,650 over a one-year period versus the Google solution.
• The payback period for using a Backblaze Fireball versus a Google Transfer Appliance is less than one month.

Example 2: Store and use 100TB of data:

• Use the data transfer service to move 100TB of data to the cloud storage service.
• Accomplish the transfer within 10 days.
• Store that 100TB of data for one year.
• Add 5TB a month (on average) to the total stored.
• Delete 2TB a month (on average) from the total stored.
• Download 10TB a month (on average) from the total stored.

Results:

• Using the Backblaze Fireball to store data in Backblaze B2 saves you $28,764 over a one-year period versus the Google solution.
• The payback period for using a Backblaze Fireball versus a Google Transfer Appliance is less than one month.

Editor’s Note: Fireball capacity is now 96TB, which would require fewer trips and provide even greater cost savings. See Backblaze Fireball.

Notes:

• All prices listed are based on list prices from the vendor websites as of the date of this blog post.
• We are accomplishing the transfer of your data to the device within the 10 day “free” period specified by Amazon and Google.
• We are comparing cloud storage services that have similar performance. For example, once the data is uploaded, it is readily available for download. The data is also available for access via a web GUI, CLI, API, and/or various applications integrated with the cloud storage service. Multiple versions of files can be kept as desired. Files can be deleted any time.

To be fair, it requires Backblaze three trips to move 100TB while it only takes one trip for the Google Transfer Appliance. This adds some cost to prepare, monitor, and ship three Backblaze Fireballs versus one Transfer Appliance. Even with that added cost, the Backblaze B2 solution will still be significantly less expensive over the one year period and beyond.

Have a Data Transfer Device Owner

Before you run out and order a transfer device, make sure the transfer process is someone’s job once the device arrives at your organization. Filling a transfer device should only take a few days, but if it is forgotten, you’ll find you’ve had the device for two or three weeks. While that’s not much of a problem with a Fireball, it could start to get expensive otherwise.

Just the Beginning

As with most “new” technologies and services, you can expect other companies to jump in and provide various data ingest services. The cost will get cheaper or even free as cloud storage companies race to capture and lock up the data you have kept locally all these years. When you are evaluating cloud storage solutions, it’s best to look past the data ingest loss-leader price, and spend a few minutes to calculate the long-term cost of storing and using your data.