this post was submitted on 27 Aug 2023
110 points (91.7% liked)

No Stupid Questions

36169 readers
499 users here now

No such thing. Ask away!

!nostupidquestions is a community dedicated to being helpful and answering each others' questions on various topics.

The rules for posting and commenting, besides the rules defined here for lemmy.world, are as follows:

Rules (interactive)


Rule 1- All posts must be legitimate questions. All post titles must include a question.

All posts must be legitimate questions, and all post titles must include a question. Questions that are joke or trolling questions, memes, song lyrics as title, etc. are not allowed here. See Rule 6 for all exceptions.



Rule 2- Your question subject cannot be illegal or NSFW material.

Your question subject cannot be illegal or NSFW material. You will be warned first, banned second.



Rule 3- Do not seek mental, medical and professional help here.

Do not seek mental, medical and professional help here. Breaking this rule will not get you or your post removed, but it will put you at risk, and possibly in danger.



Rule 4- No self promotion or upvote-farming of any kind.

That's it.



Rule 5- No baiting or sealioning or promoting an agenda.

Questions which, instead of being of an innocuous nature, are specifically intended (based on reports and in the opinion of our crack moderation team) to bait users into ideological wars on charged political topics will be removed and the authors warned - or banned - depending on severity.



Rule 6- Regarding META posts and joke questions.

Provided it is about the community itself, you may post non-question posts using the [META] tag on your post title.

On fridays, you are allowed to post meme and troll questions, on the condition that it's in text format only, and conforms with our other rules. These posts MUST include the [NSQ Friday] tag in their title.

If you post a serious question on friday and are looking only for legitimate answers, then please include the [Serious] tag on your post. Irrelevant replies will then be removed by moderators.



Rule 7- You can't intentionally annoy, mock, or harass other members.

If you intentionally annoy, mock, harass, or discriminate against any individual member, you will be removed.

Likewise, if you are a member, sympathiser or a resemblant of a movement that is known to largely hate, mock, discriminate against, and/or want to take lives of a group of people, and you were provably vocal about your hate, then you will be banned on sight.



Rule 8- All comments should try to stay relevant to their parent content.



Rule 9- Reposts from other platforms are not allowed.

Let everyone have their own content.



Rule 10- Majority of bots aren't allowed to participate here.



Credits

Our breathtaking icon was bestowed upon us by @Cevilia!

The greatest banner of all time: by @TheOneWithTheHair!

founded 2 years ago
MODERATORS
 

Edit: To clarify:

Is it even possible, financially speaking, to keep adding storage? I mean, advertisements don't even make a lot of money, is the indefinite growth of server storage even sustainable?

Or will they do what Twitch does with old content and just delete them?

top 50 comments
sorted by: hot top controversial new old
[–] LetMeEatCake@lemm.ee 96 points 1 year ago* (last edited 1 year ago) (10 children)

Storage is cheap, especially at the corporate scale.

Make two simplifying assumptions: pretend that Google is paying consumer prices for storage, and pretend that Google doesn't need to worry about data redundancy. In truth Google will pay a lot less than consumer prices, but they'll also need more than 1 byte of storage for each byte of data they have, so for the sake of envelope math we can just pretend they cancel out.

Western Digital sells a 22TB HDD for $400. Seagate has a 20TB HDD for $310. I don't like Seagate but I do like round numbers, so for simplicity we'll call it $300 for 20TB. This works out to $15/TB. According to wikipedia, Youtube had just under $29b of revenue in 2021. If youtube spend just $100m of that — 0.34% — they'd be able to buy 6,666,666 of those hard drives. In a single year. That's 6,666,666x20TB = 133,333,333 TB of storage, also known as 133^note^ ^1^ exabytes.

That's a lot of storage. A quick search tells me that youtube's compression for 4k/25fps is 45Mbps, which is about 5.5 megabytes/s. That's 768,722 years of 4k video content. All paid for with 0.34% of youtube's annual revenue.

Note 1: Note that I am using SI units here. If you want to use 1024^n^ for data names, then the SI prefixes aren't correct. It'd be 115 exbibytes instead.

EDIT: I initially did the price wrong, fixed now.

[–] AceBonobo@lemmy.world 28 points 1 year ago (3 children)

I wouldn't assume Googe pays less for storage. They need to pay for land use in many countries, power usage, redundancy and the staff that manages all of it.

They also need powerful servers with fast caching storage and a lot of RAM. They also need to pay for the bandwidth.

As far as I know, they save multiple copies of each video in all resolutions they serve. So an 8K video will also have 4K + 1440p + 1980p + 720p + 480p + 240p + 144p Possibly also 60Hz and 30Hz for some of them and also HDR versions.

You have to add all that to the cost per TB. Finally, there is the question of how much additional storage they need per year, 100 PByr? Presumably also increasing yearly?

[–] LetMeEatCake@lemm.ee 30 points 1 year ago

I wasn't calculating server costs, just raw storage. Google is not buying hard drives at retail prices. I wouldn't be surprised if they're paying as little as 50% of the retail price to buy at volume.

All of what you say is true but the purpose was to get a back of the envelope estimation to show that the cost of storage is not a truly limiting factor for a company like youtube. My point was to answer the question.

With the level of compression youtube uses, the storage costs of everything below 4k is substantially lower than 4k by itself: for back of envelope purposes we can just ignore those resolutions.

[–] sarchar@programming.dev 6 points 1 year ago

Do you absolutely know they're storing those qualities individually? It's perfectly plausible that they do on the fly transcoding.

load more comments (1 replies)
load more comments (9 replies)
[–] redcalcium@lemmy.institute 43 points 1 year ago (2 children)

YouTube is known to reduce the quality of old videos. The resolution is often the same (e.g.1080p), but the image quality is way worse compared to when those videos were new. They're probably doing it to reduce their storage cost.

[–] riskable@programming.dev 22 points 1 year ago

They still keep the originals. They're degrading the quality of older videos for bandwidth reasons, not storage.

Also, it's not because of the price of bandwidth: It's because they can use up so much bandwidth in a given region that they can cause slowdowns (e.g. hogging too much of what's available).

[–] Anticorp@lemmy.ml 39 points 1 year ago (1 children)

I mean, advertisements don't even make a lot of money

Advertising has made Alphabet one of the richest companies in the world. I assure you that it does make a lot of money.

[–] Eranziel@lemmy.world 10 points 1 year ago

I think OP is conflating the amount that a YT channel sees per ad vs the amount that YT would keep. These are not the same thing.

Plus, YT gets their share of every single ad seen every day. The economy of scale obviously is paying off.

[–] httpjames@sh.itjust.works 26 points 1 year ago (1 children)

Google datacenters are global. They store petabytes of data in each and are constantly evolving to Google Cloud customer needs and their own, meaning they are always expanding their storage network with new servers and drives.

In addition to the hardware expansion, YouTube engineers are experimenting with ways to encode videos in much more efficient formats, such as AV1. Basically, encoding is how a video is stored. The engineers are trying new standards to retain original video quality in much smaller file sizes, leading to more video storage capacity without the need to upgrade servers as quickly.

load more comments (1 replies)
[–] scarabic@lemmy.world 25 points 1 year ago* (last edited 1 year ago)

Storage for YT is not like storage for your computer. The question to ask is not if Google has enough hard drive space to keep old videos, but what it costs Google to keep all YT videos available enough to meet demand.

First keep in mind that there isn’t just one giant server at YouTube. Everything is replicated onto many parallel servers. And enormous datasets that are too large for any one server are “sharded” across many. Perhaps it takes 1000 server clusters to store one copy of everything.

Now you have to parallelize copies of those 1000 so there are redundant servers that can scale up to meet viewer capacity. This is a server “grid.”

But only some videos are being watched millions of times today. Only those server nodes need 100x redundancy for scale. The long tail of less watched videos might barely need a single node to be kept available.

So there is a massive “head” of videos that need tremendous server capacity to be available enough, and a very long and thin “tail” of videos that don’t require much resources at all.

The “head” grows as YT’s overall audience gets bigger. It’s very resource hungry and is probably their main challenge.

The “tail” gets longer as the total library of videos grows. But the tail is thin and making it longer isn’t that expensive.

I’m sure there is also a threshold below which they will drop videos. Made over 1 year ago. More than 30 seconds long. Has never been viewed once. Auhthor account hasn’t been visited in a year either. Drop it. No one will ever know.

[–] Nemo@midwest.social 23 points 1 year ago (1 children)

No, they just add more servers.

[–] FuglyDuck@lemmy.world 5 points 1 year ago (1 children)

and maybe delete videos that have been, you know... abandoned....

load more comments (1 replies)
[–] yoz@aussie.zone 20 points 1 year ago (1 children)

Google is a trillion dollar company. TRILLION DOLLARS!

[–] xusontha@ls.buckodr.ink 4 points 1 year ago

If your product scales with your size, your pure revenue doesn't matter as much. Video is expensive

[–] nutsack@lemmy.world 17 points 1 year ago* (last edited 1 year ago) (1 children)

i imagine they will delete content. they have already started deleting google accounts with 2 years of inactivity.

[–] hikaru755@feddit.de 7 points 1 year ago

That specifically excludes accounts with YouTube channels though, if I remember correctly

[–] PM_ME_VINTAGE_30S@lemmy.sdf.org 16 points 1 year ago* (last edited 1 year ago) (20 children)

If you're asking if YouTube has a finite amount of storage, the answer is yes. Assuming no safeguards were in place, you could theoretically fill up all their storage.

If you're asking whether they will run out of storage... probably not while it is considered important. YouTube can buy additional storage space (the good ending), or they can delete content they deem unimportant (the bad ending). Or, they could decide that YouTube is "finished" and elect not to increase its storage. It's their storage, so they call the shots.

Really, everything hosted "in the cloud" is hosted locally on someone else's storage. If that storage dies, the data dies, unless you or someone else has a backup.

Edit: fixed for clarity

load more comments (20 replies)
[–] independantiste@sh.itjust.works 11 points 1 year ago* (last edited 1 year ago) (1 children)

There's a physical limit to everything (I'm talking more land space here, they can't just keep adding new drives for infinity), especially since a site like youtube will only get more expensive to run as time goes. I expect them to start deleting content that doesn't make them money like videos with under 5 views that are over a year old relatively soon especially with the current economy. This would make sense for them and would free up a lot of their storage. Google doesn't disclose the profit margins of youtube but I am pretty sure they are not very large especially now with 8K HDR videos being available

[–] Jamie@jamie.moe 18 points 1 year ago (1 children)

They're already wiping inactive google accounts and all related content. It's going to be problematic for old videos where the owners haven't used the account in some time.

I have a friend that passed away past the limit, I'm going to need to make sure to archive all of his stuff or else it'll all fall into the youtube void.

[–] glibg10b@lemmy.ml 5 points 1 year ago (1 children)

They’re already wiping inactive google accounts and all related content

GDPR requires that inactive accounts older than four years be wiped

[–] ChaoticNeutralCzech@feddit.de 3 points 1 year ago

I got the memo but accounts with YouTube videos seem to be exempt from this. This is good because I uploaded some content a few years back and forgot the passwords.

[–] amanaftermidnight@lemmy.world 9 points 1 year ago (1 children)

But first, will YouTube run out of video IDs? Tom Scott answers: https://youtube.com/watch?v=gocwRvLhDf8

[–] thessnake03@lemmy.world 9 points 1 year ago* (last edited 1 year ago) (2 children)

Just based on the url you shared. 11 characters needed. 26 lowercase letters + 26 uppercase letters + 10 numbers = 62 possible characters choices. 62^11=5.2x10^19 possible ids

Edit: oooh I was close

[–] amanaftermidnight@lemmy.world 6 points 1 year ago

Its base 64. 26 uppercase + 26 lowercase + 10 digits + - and _. And there's 11 places, so in total it's 64^11^.

[–] glibg10b@lemmy.ml 3 points 1 year ago* (last edited 1 year ago)

Your superscripts are messed up. Lemmy uses the syntax a^b^

[–] Yepthatsme@kbin.social 8 points 1 year ago

Go back into an old email account and try old links. I did that and found links that work from the early 00’s but there are a lot removed.

My first reddit link was from 2007. Memories…

[–] JohnDClay@sh.itjust.works 7 points 1 year ago

Google has said they'll start deleting long inactive accounts. I'm guessing they'll go for unlisted next. Eventually they'll try and prune their catalog to not have tons of unprofitable videos.

[–] nafri@lemmy.world 7 points 1 year ago (4 children)

can't wait until Google has a planet dedicated as its server ☺️

[–] zakobjoa@lemmy.world 14 points 1 year ago (5 children)

If we build a planet sized computer I got a question for it.

[–] pancakes@sh.itjust.works 10 points 1 year ago (1 children)

They could call it something clever, like Google Earth.

[–] guyrocket@kbin.social 5 points 1 year ago

Or the death star.

[–] atocci@kbin.social 5 points 1 year ago

Can't wait to live on the Jupiter Brain with the homies

[–] WtfEvenIsExistence@sopuli.xyz 4 points 1 year ago

Oh they might already have it: Planet Earth

[–] kratoz29@lemm.ee 7 points 1 year ago

I have wondered the same from time to time, but I really don't think so, every time tech becomes better and smaller (for the most part), it wouldn't be crazy to think we can double or triple the current amount of normal storage we have in a few years, without compromising size at all, and Google has the funds to adapt these new tendencies first than most folks.

[–] UncleBadTouch@lemmy.ca 4 points 1 year ago (1 children)

more than 6x10^80 bits might be a bit to much

[–] Kolanaki@yiffit.net 10 points 1 year ago

Once YouTube contains all the data in the universe, it will open its protective crust so as to scan itself, completing its thousand year task. Once their task is complete, they will ensure that no new information arises in the only way possible: By destroying the universe.

load more comments
view more: next ›