{"id":109872,"date":"2023-09-28T09:04:26","date_gmt":"2023-09-28T16:04:26","guid":{"rendered":"https:\/\/www.backblaze.com\/blog\/?p=109872"},"modified":"2025-12-11T12:24:45","modified_gmt":"2025-12-11T20:24:45","slug":"ai-101-do-the-dollars-make-sense","status":"publish","type":"post","link":"https:\/\/www.backblaze.com\/blog\/ai-101-do-the-dollars-make-sense\/","title":{"rendered":"AI 101: Do the Dollars Make Sense?"},"content":{"rendered":"\r\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"583\" class=\"wp-image-109898\" src=\"https:\/\/www.backblaze.com\/blog\/wp-content\/uploads\/2023\/09\/bb-bh-Cost-of-AI-Cloud-Storage-1-1024x583.png\" alt=\"A decorative image showing a cloud reaching out with digital tentacles to stacks of dollar signs.\" srcset=\"https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2023\/09\/bb-bh-Cost-of-AI-Cloud-Storage-1-1024x583.png 1024w, https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2023\/09\/bb-bh-Cost-of-AI-Cloud-Storage-1-300x171.png 300w, https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2023\/09\/bb-bh-Cost-of-AI-Cloud-Storage-1-768x437.png 768w, https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2023\/09\/bb-bh-Cost-of-AI-Cloud-Storage-1-560x319.png 560w, https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2023\/09\/bb-bh-Cost-of-AI-Cloud-Storage-1.png 1440w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\r\n\r\n\r\n\r\n<div class=\"wp-block-spacer\" style=\"height: 15px;\" aria-hidden=\"true\">\u00a0<\/div>\r\n\r\n\r\n\r\n<p class=\"has-drop-cap\">Welcome back to AI 101, a series dedicated to breaking down the realities of artificial intelligence (AI). Previously we\u2019ve <a href=\"\/blog\/ai-101-how-cognitive-science-and-computer-processors-create-artificial-intelligence\/\" target=\"_blank\" rel=\"noreferrer noopener\">defined artificial intelligence, deep learning (DL), and machine learning (ML)<\/a> and dove into the <a href=\"\/blog\/ai-101-gpu-vs-tpu-vs-npu\/\" target=\"_blank\" rel=\"noreferrer noopener\">types of processors that make AI possible<\/a>. Today we\u2019ll talk about one of the biggest limitations of AI adoption\u2014how much it costs. Experts have already flagged that the <a href=\"https:\/\/www.ftc.gov\/policy\/advocacy-research\/tech-at-ftc\/2023\/06\/generative-ai-raises-competition-concerns\" target=\"_blank\" rel=\"noreferrer noopener\">significant investment necessary for AI<\/a> can cause antitrust concerns and that AI is driving up <a href=\"https:\/\/www.wsj.com\/articles\/rising-data-center-costs-linked-to-ai-demands-fc6adc0e\" target=\"_blank\" rel=\"noreferrer noopener\">costs in data centers<\/a>.\u00a0<\/p>\r\n\r\n\r\n\r\n<p>To that end, we\u2019ll talk about:\u00a0<\/p>\r\n\r\n\r\n\r\n<ul class=\"wp-block-list\">\r\n<li>Factors that impact the cost of AI.<\/li>\r\n\r\n\r\n\r\n<li>Some real numbers about the cost of AI components.\u00a0<\/li>\r\n\r\n\r\n\r\n<li>The AI tech stack and some of the industry solutions that have been built to serve it.<\/li>\r\n\r\n\r\n\r\n<li>And, uncertainty.<\/li>\r\n<\/ul>\r\n\r\n\r\n\r\n<h2 class=\"wp-block-heading\">Defining AI: Complexity and Cost Implications<\/h2>\r\n\r\n\r\n\r\n<p>While ChatGPT, DALL-E, and the like may be the most buzz-worthy of recent advancements, AI has already been a part of our daily lives for several years now. In addition to generative AI models, examples include virtual assistants like Siri and Google Home, fraud detection algorithms in banks, facial recognition software, URL <a href=\"https:\/\/www.backblaze.com\/cloud-storage\/case-studies\/urlscan-io\" target=\"_blank\" rel=\"noreferrer noopener\">threat analysis services<\/a>, and so on.\u00a0<\/p>\r\n\r\n\r\n\r\n<p>That brings us to the first challenge when it comes to understanding the cost of AI: The type of AI you\u2019re training\u2014and how complex a problem you want it to solve\u2014has a huge impact on the computing resources needed and the cost, both in the training and in the implementation phases. AI tasks are hungry in all ways: they need a lot of processing power, storage capacity, and specialized hardware. As you scale up or down in the complexity of the task you\u2019re doing, there\u2019s a huge range in the types of tools you need and their costs.\u00a0\u00a0\u00a0<\/p>\r\n\r\n\r\n\r\n<p>To understand the cost of AI, several other factors come into play as well, including:\u00a0<\/p>\r\n\r\n\r\n\r\n<ul class=\"wp-block-list\">\r\n<li>Latency requirements: How fast does the AI need to make decisions? (e.g. that split second before a self-driving car slams on the brakes.)<\/li>\r\n\r\n\r\n\r\n<li>Scope: Is the AI solving broad-based or limited questions? (e.g. the best way to organize this library vs. how many times is the word \u201ccat\u201d in this article.)<\/li>\r\n\r\n\r\n\r\n<li>Actual human labor: How much oversight does it need? (e.g. does a human identify the cat in cat photos, or does the AI algorithm identify them?)<\/li>\r\n\r\n\r\n\r\n<li>Adding data: When, how, and what quantity new data will need to be ingested to update information over time?\u00a0<\/li>\r\n<\/ul>\r\n\r\n\r\n\r\n<p>This is by no means an exhaustive list, but it gives you an idea of the considerations that can affect the kind of AI you\u2019re building and, thus, what it might cost.<\/p>\r\n\r\n\r\n\r\n<h2 class=\"wp-block-heading\">The Big Three AI Cost Drivers: Hardware, Storage, and Processing Power<\/h2>\r\n\r\n\r\n\r\n<p>In simple terms, you can break down the cost of running an AI to a few main components: hardware, storage, and processing power. That\u2019s a little bit simplistic, and you\u2019ll see some of these lines blur and expand as we get into the details of each category. But, for our purposes today, this is a good place to start to understand how much it costs to ask a bot to <a href=\"https:\/\/neural.love\/ai-art-generator\/1ee21e1c-c5dd-636e-a103-c3977fa7b99c\/squirrel-playing-a-red-electric-guitar-photograph-by-marina-dieul-and-craola-remy\" target=\"_blank\" rel=\"noreferrer noopener\">create a squirrel holding a cool guitar.<\/a><\/p>\r\n\r\n\r\n<div class=\"wp-block-image\">\r\n<figure class=\"alignleft size-full\"><a href=\"\/blog\/wp-content\/uploads\/2023\/09\/AI-101-Cost_1_Rockin-Squirrel-1.jpg\" data-rel=\"lightbox-gallery-VGrq8Oue\" data-rl_title=\"\" data-rl_caption=\"\" title=\"\"><img loading=\"lazy\" decoding=\"async\" width=\"708\" height=\"900\" class=\"wp-image-109883\" src=\"https:\/\/www.backblaze.com\/blog\/wp-content\/uploads\/2023\/09\/AI-101-Cost_1_Rockin-Squirrel-1.jpg\" alt=\"An AI generative image of a squirrel holding a guitar. Both the squirrel and the guitar and warped in strange, but not immediately noticeable ways.\" srcset=\"https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2023\/09\/AI-101-Cost_1_Rockin-Squirrel-1.jpg 708w, https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2023\/09\/AI-101-Cost_1_Rockin-Squirrel-1-236x300.jpg 236w, https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2023\/09\/AI-101-Cost_1_Rockin-Squirrel-1-560x712.jpg 560w\" sizes=\"auto, (max-width: 708px) 100vw, 708px\" \/><\/a>\r\n<figcaption class=\"wp-element-caption\">Still not quite there on the guitar. Or the squirrel. How much could this really cost?<\/figcaption>\r\n<\/figure>\r\n<\/div>\r\n\r\n\r\n<div class=\"wp-block-spacer\" style=\"height: 10px;\" aria-hidden=\"true\">\u00a0<\/div>\r\n\r\n\r\n\r\n<h2 class=\"wp-block-heading\">First Things First: Hardware Costs<\/h2>\r\n\r\n\r\n\r\n<p>Running an AI takes specialized processors that can handle complex processing queries. We\u2019re early in the game when it comes to <a href=\"https:\/\/www.cbinsights.com\/research\/nvidia-generative-ai-investments\/\" target=\"_blank\" rel=\"noreferrer noopener\">picking a \u201cwinner\u201d<\/a> for specialized processors, but these days, the most common processor is a graphical processing unit (GPU), with Nvidia\u2019s hardware and platform as an industry favorite and front-runner.\u00a0<\/p>\r\n\r\n\r\n\r\n<p>The most common \u201cworkhorse chip\u201d of AI processing tasks, the <a href=\"https:\/\/www.nvidia.com\/en-us\/data-center\/a100\/\" target=\"_blank\" rel=\"noreferrer noopener\">Nvidia A100<\/a>, <a href=\"https:\/\/www.cnbc.com\/2023\/03\/13\/chatgpt-and-generative-ai-are-booming-but-at-a-very-expensive-price.html\" target=\"_blank\" rel=\"noreferrer noopener\">starts at about $10,000 per chip<\/a>, and a set of eight of the most advanced processing chips can cost about $300,000. When Elon Musk wanted to invest in his generative AI project, <a href=\"https:\/\/arstechnica.com\/information-technology\/2023\/04\/elon-musk-reportedly-purchases-thousands-of-gpus-for-generative-ai-project-at-twitter\/\" target=\"_blank\" rel=\"noreferrer noopener\">he reportedly bought 10,000 GPUs<\/a>, which equates to an estimated value in the tens of millions of dollars. He\u2019s gone on record as saying that <a href=\"https:\/\/economictimes.indiatimes.com\/tech\/technology\/chip-giant-nvidia-rides-ai-wave-as-profits-soar\/articleshow\/103017274.cms?from=mdr\" target=\"_blank\" rel=\"noreferrer noopener\">AI chips can be harder to get than drugs<\/a>.\u00a0<\/p>\r\n\r\n\r\n\r\n<p>Google offers folks the ability to <a href=\"https:\/\/cloud.google.com\/tpu\/pricing\" target=\"_blank\" rel=\"noreferrer noopener\">rent their TPUs through the cloud<\/a> starting at $1.20 per chip hour for on-demand service (less if you commit to a contract). Meanwhile, Intel released a <a href=\"https:\/\/www.intel.com\/content\/www\/us\/en\/developer\/articles\/tool\/neural-compute-stick.html\" target=\"_blank\" rel=\"noreferrer noopener\">sub-$100 USB stick<\/a> with a full NPU that can plug into your personal laptop, and folks have created their own models at home with the help of open sourced developer toolkits. <a href=\"https:\/\/quantumobile.com\/rd-blog\/a-test-drive-of-the-intel-neural-compute-stick-2\/\" target=\"_blank\" rel=\"noreferrer noopener\">Here\u2019s a guide<\/a> to using them if you want to get in the game yourself.\u00a0<\/p>\r\n\r\n\r\n\r\n<p>Clearly, the spectrum for chips is vast\u2014from under $100 to millions\u2014and the <a href=\"https:\/\/research.aimultiple.com\/ai-chip-makers\/\" target=\"_blank\" rel=\"noreferrer noopener\">landscape for chip producers<\/a> is changing often, as is the strategy for monetizing those chips\u2014which leads us to our next section.\u00a0<\/p>\r\n\r\n\r\n\r\n<h2 class=\"wp-block-heading\">Using Third Parties: Specialized Problems = Specialized Service Providers<\/h2>\r\n\r\n\r\n\r\n<p>Building AI is a challenge with so many moving parts that, in a business use case, you eventually confront the question of whether it\u2019s more efficient to outsource it. It\u2019s true of storage, and it\u2019s definitely true of AI processing. You can already see one way Google answered that question above: create a network populated by their TPUs, then sell access.\u00a0\u00a0\u00a0<\/p>\r\n\r\n\r\n\r\n<p>Other companies specialize in broader or narrower parts of the AI creation and processing chain. Just to name a few, diverse companies: there\u2019s <a href=\"https:\/\/huggingface.co\/\" target=\"_blank\" rel=\"noreferrer noopener\">Hugging Face<\/a>, <a href=\"https:\/\/inflection.ai\/\" target=\"_blank\" rel=\"noreferrer noopener\">Inflection AI<\/a>, and <a href=\"https:\/\/www.vultr.com\/\" target=\"_blank\" rel=\"noreferrer noopener\">Vultr<\/a>. Those companies have a wide array of product offerings and resources from open source communities like Hugging Face that provide a menu of models, datasets, no-code tools, and (frankly) rad developer experiments to bare metal servers like Vultr that enhance your compute resources. How resources are offered also exist on a spectrum, including proprietary company resources (i.e. Nvidia\u2019s platform), open source communities (looking at you, Hugging Face), or a mix of the two.\u00a0<\/p>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image size-full\"><a href=\"\/blog\/wp-content\/uploads\/2023\/09\/AI-101-Cost_2_Data-Superhero-1.png\" data-rel=\"lightbox-gallery-VGrq8Oue\" data-rl_title=\"\" data-rl_caption=\"\" title=\"\"><img loading=\"lazy\" decoding=\"async\" width=\"936\" height=\"558\" class=\"wp-image-109885\" src=\"https:\/\/www.backblaze.com\/blog\/wp-content\/uploads\/2023\/09\/AI-101-Cost_2_Data-Superhero-1.png\" alt=\"An AI generated comic showing various iterations of data storage superheroes.\" srcset=\"https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2023\/09\/AI-101-Cost_2_Data-Superhero-1.png 936w, https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2023\/09\/AI-101-Cost_2_Data-Superhero-1-300x179.png 300w, https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2023\/09\/AI-101-Cost_2_Data-Superhero-1-768x458.png 768w, https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2023\/09\/AI-101-Cost_2_Data-Superhero-1-560x334.png 560w\" sizes=\"auto, (max-width: 936px) 100vw, 936px\" \/><\/a>\r\n<figcaption class=\"wp-element-caption\">A comic generated on Hugging Face\u2019s <a href=\"https:\/\/huggingface.co\/spaces\/jbilcke-hf\/ai-comic-factory\" target=\"_blank\" rel=\"noreferrer noopener\">AI Comic Factory<\/a>.<\/figcaption>\r\n<\/figure>\r\n\r\n\r\n\r\n<div class=\"wp-block-spacer\" style=\"height: 10px;\" aria-hidden=\"true\">\u00a0<\/div>\r\n\r\n\r\n\r\n<p>This means that, whichever piece of the AI tech stack you\u2019re considering, you have a high degree of flexibility when you\u2019re deciding where and how much you want to customize and where and how to implement an out-of-the box solution.\u00a0<\/p>\r\n\r\n\r\n\r\n<p>Ballparking an estimate of what any of that costs would be so dependent on the particular model you want to build and the third-party solutions you choose that it doesn\u2019t make sense to do so here. But, it suffices to say that there\u2019s a pretty narrow field of folks who have the infrastructure capacity, the datasets, and the business need to create their own network. Usually it comes back to any combination of the following: whether you have existing infrastructure to leverage or are building from scratch, if you\u2019re going to sell the solution to others, what control over research or dataset you have or want, how important privacy is and how you\u2019re incorporating it into your products, how fast you need the model to make decisions, and so on.\u00a0<\/p>\r\n\r\n\r\n\r\n<h2 class=\"wp-block-heading\">Welcome to the Spotlight, Storage<\/h2>\r\n\r\n\r\n\r\n<p>And, hey, with all that, let\u2019s not forget storage. At the most basic level of consideration, AI uses <em>a ton <\/em>of data. How much? Going knowledge says at least an order of magnitude more examples than the problem presented to train an AI model. That means you want <a href=\"https:\/\/postindustria.com\/how-much-data-is-required-for-machine-learning\/#:~:text=The%20most%20common%20way%20to,parameters%20in%20your%20data%20set.\" target=\"_blank\" rel=\"noreferrer noopener\">10 times more examples<\/a> than parameters.\u00a0<\/p>\r\n\r\n\r\n\r\n<div class=\"abstract\" style=\"line-height: 1.8; margin: 24px 12px; padding: 24px 12px 10px 12px;\">\r\n<h4>Parameters and Hyperparameters<\/h4>\r\n<p>The easiest way to think of <a href=\"https:\/\/towardsdatascience.com\/parameters-and-hyperparameters-aa609601a9ac\" target=\"_blank\" rel=\"noopener\">parameters<\/a> is to think of them as factors that control how an AI makes a decision. More parameters = more accuracy. And, just like our other AI terms, the term can be <a href=\"https:\/\/machinelearningmastery.com\/difference-between-a-parameter-and-a-hyperparameter\/\" target=\"_blank\" rel=\"noopener\">somewhat inconsistently applied<\/a>. Here\u2019s what ChatGPT has to say for itself:<\/p>\r\n<\/div>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image size-full\"><a href=\"\/blog\/wp-content\/uploads\/2023\/09\/AI-101-Cost_3_GPT-Parameters-1.png\" data-rel=\"lightbox-gallery-VGrq8Oue\" data-rl_title=\"\" data-rl_caption=\"\" title=\"\"><img loading=\"lazy\" decoding=\"async\" width=\"936\" height=\"336\" class=\"wp-image-109886\" src=\"https:\/\/www.backblaze.com\/blog\/wp-content\/uploads\/2023\/09\/AI-101-Cost_3_GPT-Parameters-1.png\" alt=\"A screenshot of a conversation with ChatGPT where it tells us it has 175 billion parameters.\" srcset=\"https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2023\/09\/AI-101-Cost_3_GPT-Parameters-1.png 936w, https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2023\/09\/AI-101-Cost_3_GPT-Parameters-1-300x108.png 300w, https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2023\/09\/AI-101-Cost_3_GPT-Parameters-1-768x276.png 768w, https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2023\/09\/AI-101-Cost_3_GPT-Parameters-1-560x201.png 560w\" sizes=\"auto, (max-width: 936px) 100vw, 936px\" \/><\/a><\/figure>\r\n\r\n\r\n\r\n<div class=\"wp-block-spacer\" style=\"height: 10px;\" aria-hidden=\"true\">\u00a0<\/div>\r\n\r\n\r\n\r\n<p>That 10x number is just the amount of data you store for the initial training model\u2014clearly the thing learns and grows, because we\u2019re talking about AI.\u00a0<\/p>\r\n\r\n\r\n\r\n<p>Preserving both your initial training algorithm and your datasets can be incredibly useful, too. As we talked about before, the more complex an AI, the higher the likelihood that <a href=\"https:\/\/www.quantamagazine.org\/the-unpredictable-abilities-emerging-from-large-ai-models-20230316\/\" target=\"_blank\" rel=\"noreferrer noopener\">your model will surprise you<\/a>. And, as many folks have pointed out, deciding whether to leverage an already-trained model or to build your own doesn\u2019t have to be an either\/or\u2014oftentimes the best option is to <a href=\"https:\/\/venturebeat.com\/ai\/neural-network-complexity-is-it-getting-better\/\" target=\"_blank\" rel=\"noreferrer noopener\">fine-tune an existing model<\/a> to your narrower purpose. In both cases, having your original training model stored can help you roll back and identify the changes over time.\u00a0<\/p>\r\n\r\n\r\n\r\n<p>The size of the dataset absolutely <a href=\"https:\/\/www.washingtonpost.com\/technology\/2023\/06\/05\/chatgpt-hidden-cost-gpu-compute\/\" target=\"_blank\" rel=\"noreferrer noopener\">affects costs <em>and <\/em>processing times.<\/a> The best example is that ChatGPT, everyone\u2019s favorite model, has been rocking GPT-3 (or 3.5) instead of GPT-4 on the general public release because GPT-4, which works from a much larger, updated dataset than GPT-3, is too expensive to release to the wider public. It also returns results much more slowly than GPT-3.5, which means that our current love of instantaneous search results and <a href=\"\/blog\/stable-diffusion-and-backblaze-create-a-masterpiece-from-a-bucket-of-your-own-images\/\" target=\"_blank\" rel=\"noreferrer noopener\">image generation<\/a> would need an adjustment.\u00a0<\/p>\r\n\r\n\r\n\r\n<p>And all of that is true because GPT-4 was updated with more information (by volume), more up-to-date information, <em>and<\/em> the model was given more parameters to take into account for responses. So, it has to both access more data per query and use more complex reasoning to make decisions. That said, it also reportedly has <a href=\"https:\/\/www.washingtonpost.com\/technology\/2023\/03\/14\/gpt-4-has-arrived-it-will-blow-chatgpt-out-water\/\" target=\"_blank\" rel=\"noreferrer noopener\">much better results<\/a>.<\/p>\r\n\r\n\r\n\r\n<h3 class=\"wp-block-heading\">Storage and Cost<\/h3>\r\n\r\n\r\n\r\n<p>What are the real numbers to store, say, a primary copy of an AI dataset? Well, it\u2019s hard to estimate, but we can ballpark that, if you\u2019re training a large AI model, you\u2019re going to have at a minimum tens of gigabytes of data and, at a maximum, petabytes. OpenAI considers the size of its training database proprietary information, and we\u2019ve found sources that cite that number as\u00a0 anywhere from <a href=\"https:\/\/updf.com\/knowledge\/what-is-gpt-4\/\" target=\"_blank\" rel=\"noreferrer noopener\">17GB<\/a> to <a href=\"https:\/\/www.sciencefocus.com\/future-technology\/gpt-3\" target=\"_blank\" rel=\"noreferrer noopener\">570GB<\/a> to <a href=\"https:\/\/transmitter.ieee.org\/how-big-will-ai-models-get\/\" target=\"_blank\" rel=\"noreferrer noopener\">45TB of text data<\/a>.\u00a0<\/p>\r\n\r\n\r\n\r\n<p>That\u2019s not actually a ton of data, and, even taking the highest number, it would only cost $225 per month to store that data in Backblaze B2 (45TB * $5\/TB\/mo), for argument\u2019s sake. But let\u2019s say you\u2019re training an AI on video to, say, make a robot vacuum that can navigate your room or <a href=\"https:\/\/black.ai\/\" target=\"_blank\" rel=\"noreferrer noopener\">recognize and identify human movement<\/a>. Your training dataset could easily reach into petabyte scale (for reference, one petabyte would cost $5,000 per month in Backblaze B2). Some research shows that <a href=\"https:\/\/www.lesswrong.com\/posts\/asqDCb9XzXnLjSfgL\/trends-in-training-dataset-sizes#Dataset_size_trends\" target=\"_blank\" rel=\"noreferrer noopener\">dataset size is trending up<\/a> over time, though other folks point out that <a href=\"https:\/\/www.scientificamerican.com\/article\/small-data-are-also-crucial-for-machine-learning\/\" target=\"_blank\" rel=\"noreferrer noopener\">bigger is not always better<\/a>.<\/p>\r\n\r\n\r\n\r\n<p>On the other hand, if you\u2019re the guy with the Intel Neural Compute stick we mentioned above and a <a href=\"https:\/\/www.raspberrypi.com\/\" target=\"_blank\" rel=\"noreferrer noopener\">Raspberry Pi<\/a>, you\u2019re talking the cost of the ~$100 AI processor, ~$50 for the Raspberry Pi, and any incidentals. You can choose to add external hard drives, network attached storage (NAS) devices, or even servers as you scale up.<\/p>\r\n\r\n\r\n\r\n<h3 class=\"wp-block-heading\">Storage and Speed<\/h3>\r\n\r\n\r\n\r\n<p>Keep in mind that, in the above example, we\u2019re only considering the cost of storing the primary dataset, and that\u2019s not very accurate when thinking about how you\u2019d be using your dataset. You\u2019d also have to consider temporary storage for when you\u2019re actually training the AI as your primary dataset is transformed by your AI algorithm, and nearly always you\u2019re splitting your primary dataset into discrete parts and feeding those to your AI algorithm in stages\u2014so each of those subsets would also be stored separately. And, in addition to needing a lot of storage, where you physically locate that storage makes a huge difference to how quickly tasks can be accomplished. In many cases, the difference is a matter of seconds, but there are some tasks that just can\u2019t handle that delay\u2014think of tasks like self-driving cars.\u00a0<\/p>\r\n\r\n\r\n\r\n<p>For huge data ingest periods such as training, you\u2019re often talking about a compute process that\u2019s assisted by powerful, and often specialized, supercomputers, with repeated passes over the same dataset. Having your data physically close to those supercomputers saves you huge amounts of time, which is pretty incredible when you consider that it breaks down to as little as milliseconds per task.<\/p>\r\n\r\n\r\n\r\n<p>One way this problem is being solved is via caching, or creating temporary storage on the same chips (or motherboards) as the processor completing the task. Another solution is to keep the whole processing and storage cluster on-premises (at least while training), as you can see in the Microsoft-OpenAI setup or as you\u2019ll often see in universities. And, unsurprisingly, you\u2019ll also see <a href=\"https:\/\/blogs.nvidia.com\/blog\/2022\/02\/17\/what-is-edge-ai\/\" target=\"_blank\" rel=\"noreferrer noopener\">edge computing solutions<\/a> which endeavor to locate data physically close to the end user.\u00a0<\/p>\r\n\r\n\r\n\r\n<p>While there can be benefits to on-premises or co-located storage, having a way to quickly add more storage (and release it if no longer needed), means cloud storage is a powerful tool for a holistic AI storage architecture\u2014and can help control costs.\u00a0<\/p>\r\n\r\n\r\n\r\n<p>And, as always, effective backup strategies require at least one off-site storage copy, and the easiest way to achieve that is via cloud storage. So, any way you slice it, you\u2019re likely going to have cloud storage touch some part of your AI tech stack.\u00a0<\/p>\r\n\r\n\r\n\r\n<h2 class=\"wp-block-heading\">What Hardware, Processing, and Storage Have in Common: You Have to Power Them<\/h2>\r\n\r\n\r\n\r\n<p>Here\u2019s the short version: any time you add complex compute + large amounts of data, you\u2019re talking about a ton of money and a ton of power to keep everything running.\u00a0<\/p>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image size-full\"><a href=\"\/blog\/wp-content\/uploads\/2023\/09\/AI-101-Cost_4_Power-1.png\" data-rel=\"lightbox-gallery-VGrq8Oue\" data-rl_title=\"\" data-rl_caption=\"\" title=\"\"><img loading=\"lazy\" decoding=\"async\" width=\"936\" height=\"524\" class=\"wp-image-109884\" src=\"https:\/\/www.backblaze.com\/blog\/wp-content\/uploads\/2023\/09\/AI-101-Cost_4_Power-1.png\" alt=\"A disorganized set of power cords and switches plugged into what is decidedly too small of an outlet space.\" srcset=\"https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2023\/09\/AI-101-Cost_4_Power-1.png 936w, https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2023\/09\/AI-101-Cost_4_Power-1-300x168.png 300w, https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2023\/09\/AI-101-Cost_4_Power-1-768x430.png 768w, https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2023\/09\/AI-101-Cost_4_Power-1-560x314.png 560w\" sizes=\"auto, (max-width: 936px) 100vw, 936px\" \/><\/a>\r\n<figcaption class=\"wp-element-caption\">Just flip the switch, and you have AI. <a href=\"https:\/\/unsplash.com\/photos\/qAShc5SV83M\">Source.<\/a><\/figcaption>\r\n<\/figure>\r\n\r\n\r\n\r\n<div class=\"wp-block-spacer\" style=\"height: 10px;\" aria-hidden=\"true\">\u00a0<\/div>\r\n\r\n\r\n\r\n<p>Fortunately for us, other folks have done the work of figuring out how much this all costs. This <a href=\"https:\/\/www.semianalysis.com\/p\/the-inference-cost-of-search-disruption\" target=\"_blank\" rel=\"noreferrer noopener\">excellent article from SemiAnalysis<\/a> goes deep on the total cost of powering searches and running generative AI models. The <a href=\"https:\/\/www.washingtonpost.com\/technology\/2023\/06\/05\/chatgpt-hidden-cost-gpu-compute\/?utm_campaign=wp_main&amp;utm_medium=social&amp;utm_source=facebook&amp;fbclid=IwAR0LN3Jri1C-JQRqgTC1n6n2IL5H6MEfl9zArV4oEi-zR1jZV_daOlPGv0k&amp;mibextid=Zxz2cZ\" target=\"_blank\" rel=\"noreferrer noopener\">Washington Post<\/a> cites Dylan Patel (also of SemiAnalysis) as estimating that a single chat with ChatGPT could cost up to 1,000 times as much as a simple Google search. Those costs include everything we\u2019ve talked about above\u2014the capital expenditures, data storage, and processing.\u00a0<\/p>\r\n\r\n\r\n\r\n<p>Consider this: Google spent several years putting off publicizing a frank accounting of their power usage. When <a href=\"https:\/\/www.nytimes.com\/2011\/09\/09\/technology\/google-details-and-defends-its-use-of-electricity.html#:~:text=Google%20also%20released%20an%20estimate,be%20difficult%20to%20understand%20intuitively.\" target=\"_blank\" rel=\"noreferrer noopener\">they released numbers in 2011<\/a>, they said that they use enough electricity to power 200,000 homes. And that was in <em>2011<\/em>. There are widely varying claims for how much a single search costs, but even the most conservative say .03 Wh of energy. There are <a href=\"https:\/\/www.demandsage.com\/google-search-statistics\/#:~:text=Google%20Search%20Statistics(Top%20Picks,billion%20unique%20queries%20every%20day.\" target=\"_blank\" rel=\"noreferrer noopener\">approximately 8.5 billion Google searches per day<\/a>. (That\u2019s just an incremental cost by the way\u2014as in, how much does a single search cost in <em>extra <\/em>resources on top of how much the system that powers it costs.)\u00a0<\/p>\r\n\r\n\r\n\r\n<p>Power is a <a href=\"https:\/\/cc-techgroup.com\/data-center-energy-consumption\/#:~:text=Data%20centers%20usually%20use%20a,and%20cool%20down%20their%20components.\" target=\"_blank\" rel=\"noreferrer noopener\">huge cost in operating data centers<\/a>, even when you\u2019re only talking about pure storage. One of the biggest single expenses that <a href=\"https:\/\/www.serverroomenvironments.co.uk\/blog\/calculating-heat-loads-and-server-room-cooling-requirements\" target=\"_blank\" rel=\"noreferrer noopener\">affects power usage is cooling systems<\/a>. With high-compute workloads, and particularly with GPUs, the amount of work the processor is doing generates a ton more heat\u2014which means more money in cooling costs, and more power consumed.\u00a0<\/p>\r\n\r\n\r\n\r\n<h2 class=\"wp-block-heading\">So, to Sum Up<\/h2>\r\n\r\n\r\n\r\n<p>When we\u2019re talking about how much an AI costs, it\u2019s not just about any single line item cost. If you decide to build and run your own models on-premises, you\u2019re talking about huge capital expenditure and ongoing costs in data centers with high compute loads. If you want to build and train a model on your own USB stick and personal computer, that\u2019s a different set of cost concerns.\u00a0<\/p>\r\n\r\n\r\n\r\n<p>And, if you\u2019re talking about querying a generative AI from the comfort of your own computer, you\u2019re still using a comparatively high amount of power somewhere down the line. We may spread that power cost across our national and international infrastructures, but it\u2019s important to remember that it\u2019s coming from somewhere\u2014and that the bill comes due, somewhere along the way.\u00a0<\/p>\r\n","protected":false},"excerpt":{"rendered":"<p>Next up in AI 101: let&#8217;s talk about what it costs to build and use an AI. <\/p>\n","protected":false},"author":182,"featured_media":109898,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"content-type":"","footnotes":""},"categories":[7,434],"tags":[489,468],"class_list":["post-109872","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-cloud-storage","category-featured-1","tag-ai-ml","tag-b2cloud","entry"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.4 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>AI 101: Understanding the Economics of Cloud Storage for Top AI Companies<\/title>\n<meta name=\"description\" content=\"Want to build an AI? Let&#039;s talk about some of the things you&#039;ll need to consider: hardware, processing power, and storage.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.backblaze.com\/blog\/ai-101-do-the-dollars-make-sense\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"AI 101: Understanding the Economics of Cloud Storage for Top AI Companies\" \/>\n<meta property=\"og:description\" content=\"Want to build an AI? Let&#039;s talk about some of the things you&#039;ll need to consider: hardware, processing power, and storage.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.backblaze.com\/blog\/ai-101-do-the-dollars-make-sense\/\" \/>\n<meta property=\"og:site_name\" content=\"Backblaze Blog | Cloud Storage &amp; Cloud Backup\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/backblaze\" \/>\n<meta property=\"article:published_time\" content=\"2023-09-28T16:04:26+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-12-11T20:24:45+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2023\/09\/bb-bh-Cost-of-AI-Cloud-Storage-1.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1440\" \/>\n\t<meta property=\"og:image:height\" content=\"820\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Stephanie Doyle\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@backblaze\" \/>\n<meta name=\"twitter:site\" content=\"@backblaze\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Stephanie Doyle\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"13 minutes\" \/>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"AI 101: Understanding the Economics of Cloud Storage for Top AI Companies","description":"Want to build an AI? Let's talk about some of the things you'll need to consider: hardware, processing power, and storage.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.backblaze.com\/blog\/ai-101-do-the-dollars-make-sense\/","og_locale":"en_US","og_type":"article","og_title":"AI 101: Understanding the Economics of Cloud Storage for Top AI Companies","og_description":"Want to build an AI? Let's talk about some of the things you'll need to consider: hardware, processing power, and storage.","og_url":"https:\/\/www.backblaze.com\/blog\/ai-101-do-the-dollars-make-sense\/","og_site_name":"Backblaze Blog | Cloud Storage &amp; Cloud Backup","article_publisher":"https:\/\/www.facebook.com\/backblaze","article_published_time":"2023-09-28T16:04:26+00:00","article_modified_time":"2025-12-11T20:24:45+00:00","og_image":[{"width":1440,"height":820,"url":"https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2023\/09\/bb-bh-Cost-of-AI-Cloud-Storage-1.png","type":"image\/png"}],"author":"Stephanie Doyle","twitter_card":"summary_large_image","twitter_creator":"@backblaze","twitter_site":"@backblaze","twitter_misc":{"Written by":"Stephanie Doyle","Est. reading time":"13 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.backblaze.com\/blog\/ai-101-do-the-dollars-make-sense\/#article","isPartOf":{"@id":"https:\/\/www.backblaze.com\/blog\/ai-101-do-the-dollars-make-sense\/"},"author":{"name":"Stephanie Doyle","@id":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/#\/schema\/person\/688f3962fd24d8155ef726bc94d75058"},"headline":"AI 101: Do the Dollars Make Sense?","datePublished":"2023-09-28T16:04:26+00:00","dateModified":"2025-12-11T20:24:45+00:00","mainEntityOfPage":{"@id":"https:\/\/www.backblaze.com\/blog\/ai-101-do-the-dollars-make-sense\/"},"wordCount":2595,"commentCount":0,"publisher":{"@id":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/#organization"},"image":{"@id":"https:\/\/www.backblaze.com\/blog\/ai-101-do-the-dollars-make-sense\/#primaryimage"},"thumbnailUrl":"https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2023\/09\/bb-bh-Cost-of-AI-Cloud-Storage-1.png","keywords":["AI\/ML","B2Cloud"],"articleSection":["Cloud Storage","Featured"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.backblaze.com\/blog\/ai-101-do-the-dollars-make-sense\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.backblaze.com\/blog\/ai-101-do-the-dollars-make-sense\/","url":"https:\/\/www.backblaze.com\/blog\/ai-101-do-the-dollars-make-sense\/","name":"AI 101: Understanding the Economics of Cloud Storage for Top AI Companies","isPartOf":{"@id":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.backblaze.com\/blog\/ai-101-do-the-dollars-make-sense\/#primaryimage"},"image":{"@id":"https:\/\/www.backblaze.com\/blog\/ai-101-do-the-dollars-make-sense\/#primaryimage"},"thumbnailUrl":"https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2023\/09\/bb-bh-Cost-of-AI-Cloud-Storage-1.png","datePublished":"2023-09-28T16:04:26+00:00","dateModified":"2025-12-11T20:24:45+00:00","description":"Want to build an AI? Let's talk about some of the things you'll need to consider: hardware, processing power, and storage.","breadcrumb":{"@id":"https:\/\/www.backblaze.com\/blog\/ai-101-do-the-dollars-make-sense\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.backblaze.com\/blog\/ai-101-do-the-dollars-make-sense\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.backblaze.com\/blog\/ai-101-do-the-dollars-make-sense\/#primaryimage","url":"https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2023\/09\/bb-bh-Cost-of-AI-Cloud-Storage-1.png","contentUrl":"https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2023\/09\/bb-bh-Cost-of-AI-Cloud-Storage-1.png","width":1440,"height":820,"caption":"A decorative image showing a cloud reaching out with digital tentacles to stacks of dollar signs."},{"@type":"BreadcrumbList","@id":"https:\/\/www.backblaze.com\/blog\/ai-101-do-the-dollars-make-sense\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/"},{"@type":"ListItem","position":2,"name":"AI 101: Do the Dollars Make Sense?"}]},{"@type":"WebSite","@id":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/#website","url":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/","name":"Backblaze Cloud Solutions Blog","description":"Cloud Storage &amp; Cloud Backup","publisher":{"@id":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/#organization","name":"Backblaze","url":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/i0.wp.com\/www.backblaze.com\/blog\/wp-content\/uploads\/2017\/12\/backblaze_icon_transparent.png?fit=512%2C512&ssl=1","contentUrl":"https:\/\/i0.wp.com\/www.backblaze.com\/blog\/wp-content\/uploads\/2017\/12\/backblaze_icon_transparent.png?fit=512%2C512&ssl=1","width":512,"height":512,"caption":"Backblaze"},"image":{"@id":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/backblaze","https:\/\/x.com\/backblaze","https:\/\/www.youtube.com\/user\/Backblaze","https:\/\/en.wikipedia.org\/wiki\/Backblaze"]},{"@type":"Person","@id":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/#\/schema\/person\/688f3962fd24d8155ef726bc94d75058","name":"Stephanie Doyle","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2022\/12\/headshot-4-1-e1670452405672-150x150.jpg","url":"https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2022\/12\/headshot-4-1-e1670452405672-150x150.jpg","contentUrl":"https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2022\/12\/headshot-4-1-e1670452405672-150x150.jpg","caption":"Stephanie Doyle"},"description":"Stephanie is the Technical Narrative Content Manager at Backblaze. She specializes in taking complex topics and writing relatable, engaging, and user-friendly content. You can most often find her reading in public places, and can connect with her on LinkedIn.","url":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/author\/stephanie\/"}]}},"jetpack_featured_media_url":"https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2023\/09\/bb-bh-Cost-of-AI-Cloud-Storage-1.png","_links":{"self":[{"href":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/wp-json\/wp\/v2\/posts\/109872","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/wp-json\/wp\/v2\/users\/182"}],"replies":[{"embeddable":true,"href":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/wp-json\/wp\/v2\/comments?post=109872"}],"version-history":[{"count":0,"href":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/wp-json\/wp\/v2\/posts\/109872\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/wp-json\/wp\/v2\/media\/109898"}],"wp:attachment":[{"href":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/wp-json\/wp\/v2\/media?parent=109872"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/wp-json\/wp\/v2\/categories?post=109872"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/wp-json\/wp\/v2\/tags?post=109872"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}