Re: [AFMUG] OT: Youtube Data Storage
https://ir.netflix.com/events.cfm <https://ir.netflix.com/events.cfm> financials are there. Justin Wilson j...@mtin.net --- http://www.mtin.net Owner/CEO xISP Solutions- Consulting – Data Centers - Bandwidth http://www.midwest-ix.com COO/Chairman Internet Exchange - Peering - Distributed Fabric > On Mar 2, 2017, at 10:43 AM, Adam Moffett wrote: > > $12B per year? > $12B per month is staggering. > > -- Original Message -- > From: "Josh Reynolds" mailto:j...@kyneticwifi.com>> > To: af@afmug.com <mailto:af@afmug.com> > Sent: 3/1/2017 12:44:52 PM > Subject: Re: [AFMUG] OT: Youtube Data Storage > >> Haven't fact check it, but I heard today Netflix is pulling in something >> like $12 Billion a month now. >> >> On Mar 1, 2017 11:39 AM, "Robert Andrews" > <mailto:i...@avantwireless.com>> wrote: >> At the peak downloads of Netscape browser back in '97 we were _netting_ a >> million per day. That was back in 1997. yeah, I could pretty much spend >> anything I wanted to keep the downloads going. We foresaw a bump in >> downloads coming for the 2.0 browser and I went out and picked up $200K in >> SGI processors off their loading dock overnight with a credit card purchase >> (Amex Black for the win)...We were very small fry compared to what's >> going on now. Probably multiply those numbers by 10K or even 100K for >> today. >> >> On 03/01/2017 09:08 AM, Nate Burke wrote: >> I was reading an article the other day that referenced that youtube >> users upload 400 hours of video every minute, or 65 years of video every >> day. Is my math right in showing that at an average of 2.5mb/s for the >> video size, that's 600 TB of data per day being uploaded? Given >> redundant copies being made, Youtube is bringing online over a Petabyte >> per day of storage. >> >> Since I've only been involved with Small businesses, wrapping my head >> around this amount of storage and $$$ is hard. >> >> Realizing that Google is custom, I'll use Backblaze for some math. >> Backblaze will do 45 drives in 4U@600w power draw 47U rack height = 11 >> 4U boxes = 495HD /rack If they're 8tb drives, that's 4PB/rack. If >> Google is paying 1/2 retail cost for drives, that's $175/drive, or $90k >> per rack (+ Hardware) 6.6kw power draw (+ Cooling). >> >> So every 4 Days, they spend $100k in Hardware, and increases their >> electric bill by 8kw? >> >> I guess when you're dealing with Billions of Dollars, a $100k every >> couple days' isn't such a big deal.
Re: [AFMUG] OT: Youtube Data Storage
$12B per year? $12B per month is staggering. -- Original Message -- From: "Josh Reynolds" To: af@afmug.com Sent: 3/1/2017 12:44:52 PM Subject: Re: [AFMUG] OT: Youtube Data Storage Haven't fact check it, but I heard today Netflix is pulling in something like $12 Billion a month now. On Mar 1, 2017 11:39 AM, "Robert Andrews" wrote: At the peak downloads of Netscape browser back in '97 we were _netting_ a million per day. That was back in 1997. yeah, I could pretty much spend anything I wanted to keep the downloads going. We foresaw a bump in downloads coming for the 2.0 browser and I went out and picked up $200K in SGI processors off their loading dock overnight with a credit card purchase (Amex Black for the win)...We were very small fry compared to what's going on now. Probably multiply those numbers by 10K or even 100K for today. On 03/01/2017 09:08 AM, Nate Burke wrote: I was reading an article the other day that referenced that youtube users upload 400 hours of video every minute, or 65 years of video every day. Is my math right in showing that at an average of 2.5mb/s for the video size, that's 600 TB of data per day being uploaded? Given redundant copies being made, Youtube is bringing online over a Petabyte per day of storage. Since I've only been involved with Small businesses, wrapping my head around this amount of storage and $$$ is hard. Realizing that Google is custom, I'll use Backblaze for some math. Backblaze will do 45 drives in 4U@600w power draw 47U rack height = 11 4U boxes = 495HD /rack If they're 8tb drives, that's 4PB/rack. If Google is paying 1/2 retail cost for drives, that's $175/drive, or $90k per rack (+ Hardware) 6.6kw power draw (+ Cooling). So every 4 Days, they spend $100k in Hardware, and increases their electric bill by 8kw? I guess when you're dealing with Billions of Dollars, a $100k every couple days' isn't such a big deal.
Re: [AFMUG] OT: Youtube Data Storage
That sounds about right ….. highly distributed well connected storage is their key … It truly is staggering though when you think about it… > On Mar 1, 2017, at 12:08 PM, Nate Burke wrote: > > I was reading an article the other day that referenced that youtube users > upload 400 hours of video every minute, or 65 years of video every day. Is > my math right in showing that at an average of 2.5mb/s for the video size, > that's 600 TB of data per day being uploaded? Given redundant copies being > made, Youtube is bringing online over a Petabyte per day of storage. > > Since I've only been involved with Small businesses, wrapping my head around > this amount of storage and $$$ is hard. > > Realizing that Google is custom, I'll use Backblaze for some math. > Backblaze will do 45 drives in 4U@600w power draw 47U rack height = 11 4U > boxes = 495HD /rack If they're 8tb drives, that's 4PB/rack. If Google is > paying 1/2 retail cost for drives, that's $175/drive, or $90k per rack (+ > Hardware) 6.6kw power draw (+ Cooling). > > So every 4 Days, they spend $100k in Hardware, and increases their electric > bill by 8kw? > > I guess when you're dealing with Billions of Dollars, a $100k every couple > days' isn't such a big deal.
Re: [AFMUG] OT: Youtube Data Storage
Haven't fact check it, but I heard today Netflix is pulling in something like $12 Billion a month now. On Mar 1, 2017 11:39 AM, "Robert Andrews" wrote: > At the peak downloads of Netscape browser back in '97 we were _netting_ a > million per day. That was back in 1997. yeah, I could pretty much spend > anything I wanted to keep the downloads going. We foresaw a bump in > downloads coming for the 2.0 browser and I went out and picked up $200K in > SGI processors off their loading dock overnight with a credit card purchase > (Amex Black for the win)...We were very small fry compared to what's > going on now. Probably multiply those numbers by 10K or even 100K for > today. > > On 03/01/2017 09:08 AM, Nate Burke wrote: > >> I was reading an article the other day that referenced that youtube >> users upload 400 hours of video every minute, or 65 years of video every >> day. Is my math right in showing that at an average of 2.5mb/s for the >> video size, that's 600 TB of data per day being uploaded? Given >> redundant copies being made, Youtube is bringing online over a Petabyte >> per day of storage. >> >> Since I've only been involved with Small businesses, wrapping my head >> around this amount of storage and $$$ is hard. >> >> Realizing that Google is custom, I'll use Backblaze for some math. >> Backblaze will do 45 drives in 4U@600w power draw 47U rack height = 11 >> 4U boxes = 495HD /rack If they're 8tb drives, that's 4PB/rack. If >> Google is paying 1/2 retail cost for drives, that's $175/drive, or $90k >> per rack (+ Hardware) 6.6kw power draw (+ Cooling). >> >> So every 4 Days, they spend $100k in Hardware, and increases their >> electric bill by 8kw? >> >> I guess when you're dealing with Billions of Dollars, a $100k every >> couple days' isn't such a big deal. >> >>
Re: [AFMUG] OT: Youtube Data Storage
At the peak downloads of Netscape browser back in '97 we were _netting_ a million per day. That was back in 1997. yeah, I could pretty much spend anything I wanted to keep the downloads going. We foresaw a bump in downloads coming for the 2.0 browser and I went out and picked up $200K in SGI processors off their loading dock overnight with a credit card purchase (Amex Black for the win)...We were very small fry compared to what's going on now. Probably multiply those numbers by 10K or even 100K for today. On 03/01/2017 09:08 AM, Nate Burke wrote: I was reading an article the other day that referenced that youtube users upload 400 hours of video every minute, or 65 years of video every day. Is my math right in showing that at an average of 2.5mb/s for the video size, that's 600 TB of data per day being uploaded? Given redundant copies being made, Youtube is bringing online over a Petabyte per day of storage. Since I've only been involved with Small businesses, wrapping my head around this amount of storage and $$$ is hard. Realizing that Google is custom, I'll use Backblaze for some math. Backblaze will do 45 drives in 4U@600w power draw 47U rack height = 11 4U boxes = 495HD /rack If they're 8tb drives, that's 4PB/rack. If Google is paying 1/2 retail cost for drives, that's $175/drive, or $90k per rack (+ Hardware) 6.6kw power draw (+ Cooling). So every 4 Days, they spend $100k in Hardware, and increases their electric bill by 8kw? I guess when you're dealing with Billions of Dollars, a $100k every couple days' isn't such a big deal.
Re: [AFMUG] OT: Youtube Data Storage
I remember reading some years ago that that's how facebook worked. Every picture/video was converted and stored in all possible resolutions and quality upon upload. Although I'm guessing that a few days after the upload, both Youtube and Facebook have data that is rarely accessed, so maybe they lose the extra copies. I think there was an article on Facebook testing moving older pictures/videos to a Blu-Ray based nearline storage that could store 10's of PB/rack, but would have several minutes of latency for reading. On 3/1/2017 11:12 AM, Mike Hammett wrote: They don't transcode live, IIRC, so they have each resolution and quality version stored as well. I wouldn't be surprised if there was a rack per day of non-redundant storage just for YouTube. - Mike Hammett Intelligent Computing Solutions <http://www.ics-il.com/> <https://www.facebook.com/ICSIL><https://plus.google.com/+IntelligentComputingSolutionsDeKalb><https://www.linkedin.com/company/intelligent-computing-solutions><https://twitter.com/ICSIL> Midwest Internet Exchange <http://www.midwest-ix.com/> <https://www.facebook.com/mdwestix><https://www.linkedin.com/company/midwest-internet-exchange><https://twitter.com/mdwestix> The Brothers WISP <http://www.thebrotherswisp.com/> <https://www.facebook.com/thebrotherswisp> <https://www.youtube.com/channel/UCXSdfxQv7SpoRQYNyLwntZg> *From: *"Nate Burke" *To: *"Animal Farm" *Sent: *Wednesday, March 1, 2017 11:08:15 AM *Subject: *[AFMUG] OT: Youtube Data Storage I was reading an article the other day that referenced that youtube users upload 400 hours of video every minute, or 65 years of video every day. Is my math right in showing that at an average of 2.5mb/s for the video size, that's 600 TB of data per day being uploaded? Given redundant copies being made, Youtube is bringing online over a Petabyte per day of storage. Since I've only been involved with Small businesses, wrapping my head around this amount of storage and $$ is hard. Realizing that Google is custom, I'll use Backblaze for some math. Backblaze will do 45 drives in 4U@600w power draw 47U rack height = 11 4U boxes = 495HD /rack If they're 8tb drives, that's 4PB/rack. If Google is paying 1/2 retail cost for drives, that's $175/drive, or $90k per rack (+ Hardware) 6.6kw power draw (+ Cooling). So every 4 Days, they spend $100k in Hardware, and increases their electric bill by 8kw? I guess when you're dealing with Billions of Dollars, a $100k every couple days' isn't such a big deal.
Re: [AFMUG] OT: Youtube Data Storage
I know the "Big Ten" web companies (Google, Facebook, etc) are currently pushing for 800Gbps optics. Makes sense. On Mar 1, 2017 11:12 AM, "Mike Hammett" wrote: > They don't transcode live, IIRC, so they have each resolution and quality > version stored as well. I wouldn't be surprised if there was a rack per day > of non-redundant storage just for YouTube. > > > > - > Mike Hammett > Intelligent Computing Solutions <http://www.ics-il.com/> > <https://www.facebook.com/ICSIL> > <https://plus.google.com/+IntelligentComputingSolutionsDeKalb> > <https://www.linkedin.com/company/intelligent-computing-solutions> > <https://twitter.com/ICSIL> > Midwest Internet Exchange <http://www.midwest-ix.com/> > <https://www.facebook.com/mdwestix> > <https://www.linkedin.com/company/midwest-internet-exchange> > <https://twitter.com/mdwestix> > The Brothers WISP <http://www.thebrotherswisp.com/> > <https://www.facebook.com/thebrotherswisp> > > > <https://www.youtube.com/channel/UCXSdfxQv7SpoRQYNyLwntZg> > -- > *From: *"Nate Burke" > *To: *"Animal Farm" > *Sent: *Wednesday, March 1, 2017 11:08:15 AM > *Subject: *[AFMUG] OT: Youtube Data Storage > > I was reading an article the other day that referenced that youtube > users upload 400 hours of video every minute, or 65 years of video every > day. Is my math right in showing that at an average of 2.5mb/s for the > video size, that's 600 TB of data per day being uploaded? Given > redundant copies being made, Youtube is bringing online over a Petabyte > per day of storage. > > Since I've only been involved with Small businesses, wrapping my head > around this amount of storage and $$ is hard. > > Realizing that Google is custom, I'll use Backblaze for some math. > Backblaze will do 45 drives in 4U@600w power draw 47U rack height = 11 > 4U boxes = 495HD /rack If they're 8tb drives, that's 4PB/rack. If > Google is paying 1/2 retail cost for drives, that's $175/drive, or $90k > per rack (+ Hardware) 6.6kw power draw (+ Cooling). > > So every 4 Days, they spend $100k in Hardware, and increases their > electric bill by 8kw? > > I guess when you're dealing with Billions of Dollars, a $100k every > couple days' isn't such a big deal. > >
Re: [AFMUG] OT: Youtube Data Storage
They don't transcode live, IIRC, so they have each resolution and quality version stored as well. I wouldn't be surprised if there was a rack per day of non-redundant storage just for YouTube. - Mike Hammett Intelligent Computing Solutions Midwest Internet Exchange The Brothers WISP - Original Message - From: "Nate Burke" To: "Animal Farm" Sent: Wednesday, March 1, 2017 11:08:15 AM Subject: [AFMUG] OT: Youtube Data Storage I was reading an article the other day that referenced that youtube users upload 400 hours of video every minute, or 65 years of video every day. Is my math right in showing that at an average of 2.5mb/s for the video size, that's 600 TB of data per day being uploaded? Given redundant copies being made, Youtube is bringing online over a Petabyte per day of storage. Since I've only been involved with Small businesses, wrapping my head around this amount of storage and $$$ is hard. Realizing that Google is custom, I'll use Backblaze for some math. Backblaze will do 45 drives in 4U@600w power draw 47U rack height = 11 4U boxes = 495HD /rack If they're 8tb drives, that's 4PB/rack. If Google is paying 1/2 retail cost for drives, that's $175/drive, or $90k per rack (+ Hardware) 6.6kw power draw (+ Cooling). So every 4 Days, they spend $100k in Hardware, and increases their electric bill by 8kw? I guess when you're dealing with Billions of Dollars, a $100k every couple days' isn't such a big deal.
[AFMUG] OT: Youtube Data Storage
I was reading an article the other day that referenced that youtube users upload 400 hours of video every minute, or 65 years of video every day. Is my math right in showing that at an average of 2.5mb/s for the video size, that's 600 TB of data per day being uploaded? Given redundant copies being made, Youtube is bringing online over a Petabyte per day of storage. Since I've only been involved with Small businesses, wrapping my head around this amount of storage and $$$ is hard. Realizing that Google is custom, I'll use Backblaze for some math. Backblaze will do 45 drives in 4U@600w power draw 47U rack height = 11 4U boxes = 495HD /rack If they're 8tb drives, that's 4PB/rack. If Google is paying 1/2 retail cost for drives, that's $175/drive, or $90k per rack (+ Hardware) 6.6kw power draw (+ Cooling). So every 4 Days, they spend $100k in Hardware, and increases their electric bill by 8kw? I guess when you're dealing with Billions of Dollars, a $100k every couple days' isn't such a big deal.
