I remember reading some years ago that that's how facebook worked. Every
picture/video was converted and stored in all possible resolutions and
quality upon upload. Although I'm guessing that a few days after the
upload, both Youtube and Facebook have data that is rarely accessed, so
maybe they lose the extra copies.
I think there was an article on Facebook testing moving older
pictures/videos to a Blu-Ray based nearline storage that could store
10's of PB/rack, but would have several minutes of latency for reading.
On 3/1/2017 11:12 AM, Mike Hammett wrote:
They don't transcode live, IIRC, so they have each resolution and
quality version stored as well. I wouldn't be surprised if there was a
rack per day of non-redundant storage just for YouTube.
-----
Mike Hammett
Intelligent Computing Solutions <http://www.ics-il.com/>
<https://www.facebook.com/ICSIL><https://plus.google.com/+IntelligentComputingSolutionsDeKalb><https://www.linkedin.com/company/intelligent-computing-solutions><https://twitter.com/ICSIL>
Midwest Internet Exchange <http://www.midwest-ix.com/>
<https://www.facebook.com/mdwestix><https://www.linkedin.com/company/midwest-internet-exchange><https://twitter.com/mdwestix>
The Brothers WISP <http://www.thebrotherswisp.com/>
<https://www.facebook.com/thebrotherswisp>
<https://www.youtube.com/channel/UCXSdfxQv7SpoRQYNyLwntZg>
------------------------------------------------------------------------
*From: *"Nate Burke" <n...@blastcomm.com>
*To: *"Animal Farm" <af@afmug.com>
*Sent: *Wednesday, March 1, 2017 11:08:15 AM
*Subject: *[AFMUG] OT: Youtube Data Storage
I was reading an article the other day that referenced that youtube
users upload 400 hours of video every minute, or 65 years of video every
day. Is my math right in showing that at an average of 2.5mb/s for the
video size, that's 600 TB of data per day being uploaded? Given
redundant copies being made, Youtube is bringing online over a Petabyte
per day of storage.
Since I've only been involved with Small businesses, wrapping my head
around this amount of storage and $$ is hard.
Realizing that Google is custom, I'll use Backblaze for some math.
Backblaze will do 45 drives in 4U@600w power draw 47U rack height = 11
4U boxes = 495HD /rack If they're 8tb drives, that's 4PB/rack. If
Google is paying 1/2 retail cost for drives, that's $175/drive, or $90k
per rack (+ Hardware) 6.6kw power draw (+ Cooling).
So every 4 Days, they spend $100k in Hardware, and increases their
electric bill by 8kw?
I guess when you're dealing with Billions of Dollars, a $100k every
couple days' isn't such a big deal.
