Re: [mythtv-users] commercial flagging idea - commercialfingerprinting
I just want to reiterate (since this thread is getting long), that it'd be best to fingerprint the show before the commercial break, and after the commercial break (or the entire show itself)... not fingerprint or ID the commercials themselves. Shows will always be constant. Commercials will not. Also, again... not a replacement for mythcommflag, just something to strengthen and verify it's results to make it smarter :) ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
Re: [mythtv-users] commercial flagging idea - commercialfingerprinting
On Wed, 2005-04-13 at 22:44 +, Risto Treksler wrote: Mike Benoit [EMAIL PROTECTED] said: I'm not a C programmer, so implementing MythRecommend directly in to MythTV isn't something I have the time to do at this point. Coding the client/server scripts though is what I do all day long, so its pretty quick. Though, I'm sure a Myth developer could implement MythRecommend in to MythTV in no time flat, the scripts store the data in the local mythconverg database, so its just a matter of running a simple query and making the interface to it. --PART II-- mythtv needs to draw people down the long tail from popular shows -to- obscure but related shows i haven't used the mythrecommend script recently so i forget whether the perl script already does that by genre - ie. recommend other *less known comedies* for Simpsons viewers - so, don't recommend CSI or Futurama, even if Simpsons watchers watch them Basically, how do you picture the myth interface for this? 1) have a mythrecommend module accessible by: manage recordings - schedule recordings - recommended shows - this kind of misses the point though, since you have to go out of the way to do it I think that for it to be effective the interface must be done on a per episode basis --so option number 1) is out-- 2) put it somewhere in the set recording options screen - where you currently have list upcoming episodes and program details - make option for list recommended shows? 3) or put it under the context menu of the episode -recording options - recommended shows - or even put it right uner recording options and job options either option 2 or 3 would bring up a kind of - a program finder that only lists recommended shows Ideally I would like to see all 3 options eventually. Option 1 is nice if you just want to get an entire overview of recommended shows. Option 2 is what I originally had in mind, but Option 3 seems to make a lot of sense too. Either option 2 or 3 would be a great start. The mythrecommend script at this point does not take in to account genre, since the user base is still pretty small, it makes doing any really fancy recommendations pretty useless at this point. However the script simply grabs the recommendations from the server and inserts them in to the local database, so all it takes is a couple joins and you can get the genre from the local program table and narrow it down that way if you want. Personally I wouldn't want it to recommend shows JUST in the same genre, but rather any shows that are popular and/or very highly rated among its viewers. (which is currently does) Other options would be to show a genre column in the recommend shows program finder or have an option to limit recommendations by genre only. Are you volunteering to do the implementation? -- Mike Benoit [EMAIL PROTECTED] signature.asc Description: This is a digitally signed message part ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
Re: [mythtv-users] commercial flagging idea - commercialfingerprinting
Hi there, I've been interested in this topic for quite some time now. I've even posted to the list regarding something very similar a few months back, however it was met with mixed results. I'm the author of MythRecommend (http://ipso.snappymail.ca:8080/mythrecommend/) which has been around for a fair amount of time now, and out of the 120 unique people who use it, they record 2275 unique programs in total. The top 345 shows are watched by over half of the people. So I think a system where commercial cut lists could be exchanged seamlessly would be a huge help. I'm willing to volunteer my time to work on a centralized repository similar to what MythRecommend has that can handle commercial cut lists. Thats the easy part though, the hard part is getting Myth to do its part. Here is a snippet of the most popular shows, and how many unique clients record them. title | count ---+--- The Simpsons |67 The Daily Show With Jon Stewart |48 Futurama |41 MythBusters |39 Desperate Housewives |36 CSI: Crime Scene Investigation |34 Scrubs |34 South Park |31 Stargate SG-1 |31 Star Trek: Enterprise |30 Family Guy |30 Lost |29 CSI: NY |26 Stargate Atlantis |26 The Apprentice |25 CSI: Miami |24 Aqua Teen Hungerforce |23 Malcolm in the Middle |22 Arrested Development |21 ER |21 Law Order: Special Victims Unit |20 24 |19 Law Order |19 Extreme Makeover: Home Edition |18 Good Eats |17 Joey |17 The West Wing |17 Nova |16 Harvey Birdman: Attorney at Law |16 Sealab 2021 |16 Everybody Loves Raymond |15 Frontline |15 King of the Hill |15 Battlestar Galactica |15 Will Grace |15 That '70s Show |15 Sex and the City |15 Star Trek: Voyager |14 American Chopper |14 Friends |14 60 Minutes |14 Star Trek: Deep Space Nine |14 Survivor: Vanuatu%2c Islands of Fire |14 Chappelle's Show |13 Law Order: Criminal Intent
Re: [mythtv-users] commercial flagging idea - commercialfingerprinting
Mike Benoit wrote: Hi there, I've been interested in this topic for quite some time now. I've even posted to the list regarding something very similar a few months back, however it was met with mixed results. I'm the author of MythRecommend (http://ipso.snappymail.ca:8080/mythrecommend/) which has been around for a fair amount of time now, and out of the 120 unique people who use it, they record 2275 unique programs in total. The top 345 shows are watched by over half of the people. So I think a system where commercial cut lists could be exchanged seamlessly would be a huge help. I'm willing to volunteer my time to work on a centralized repository similar to what MythRecommend has that can handle commercial cut lists. Thats the easy part though, the hard part is getting Myth to do its part. Here is a snippet of the most popular shows, and how many unique clients record them. title | count ---+--- The Simpsons | 67 The Daily Show With Jon Stewart | 48 Futurama | 41 MythBusters | 39 Desperate Housewives | 36 CSI: Crime Scene Investigation| 34 Scrubs| 34 South Park| 31 Stargate SG-1 | 31 Star Trek: Enterprise | 30 Family Guy| 30 Lost | 29 CSI: NY | 26 Stargate Atlantis | 26 The Apprentice| 25 CSI: Miami| 24 Aqua Teen Hungerforce | 23 Malcolm in the Middle | 22 Arrested Development | 21 ER| 21 Law Order: Special Victims Unit | 20 24| 19 Law Order | 19 Extreme Makeover: Home Edition| 18 Good Eats | 17 Joey | 17 The West Wing | 17 Nova | 16 Harvey Birdman: Attorney at Law | 16 Sealab 2021 | 16 Everybody Loves Raymond | 15 Frontline | 15 King of the Hill | 15 Battlestar Galactica | 15 Will Grace | 15 That '70s Show| 15 Sex and the City | 15 Star Trek: Voyager| 14 American Chopper | 14 Friends | 14 60 Minutes| 14 Star Trek: Deep Space Nine| 14 Survivor: Vanuatu%2c Islands of Fire | 14 Chappelle's Show | 13 Law Order: Criminal Intent | 13 Father of the
Re: [mythtv-users] commercial flagging idea - commercialfingerprinting
I'm not a C programmer, so implementing MythRecommend directly in to MythTV isn't something I have the time to do at this point. Coding the client/server scripts though is what I do all day long, so its pretty quick. Though, I'm sure a Myth developer could implement MythRecommend in to MythTV in no time flat, the scripts store the data in the local mythconverg database, so its just a matter of running a simple query and making the interface to it. Regardless though, MythRecommend and a distributed commercial flagging system are completely different and should be treated as such. In my opinion a start would be to figure out some sort of finger printing system, or some method of sharing cutlists that will work on the same program across different stations or that takes in to account that not every mythtv box will start recording at the exact second. Once we have that, creating a centralized system to share said cutlists is the easy part. On Wed, 2005-04-13 at 13:13 -0400, Mario Limonciello wrote: Mike Benoit wrote: Hi there, I've been interested in this topic for quite some time now. I've even posted to the list regarding something very similar a few months back, however it was met with mixed results. I'm the author of MythRecommend (http://ipso.snappymail.ca:8080/mythrecommend/) which has been around for a fair amount of time now, and out of the 120 unique people who use it, they record 2275 unique programs in total. The top 345 shows are watched by over half of the people. So I think a system where commercial cut lists could be exchanged seamlessly would be a huge help. I'm willing to volunteer my time to work on a centralized repository similar to what MythRecommend has that can handle commercial cut lists. Thats the easy part though, the hard part is getting Myth to do its part. Here is a snippet of the most popular shows, and how many unique clients record them. title | count ---+--- The Simpsons |67 The Daily Show With Jon Stewart |48 Futurama |41 MythBusters |39 Desperate Housewives |36 CSI: Crime Scene Investigation |34 Scrubs |34 South Park |31 Stargate SG-1 |31 Star Trek: Enterprise |30 Family Guy |30 Lost |29 CSI: NY |26 Stargate Atlantis |26 The Apprentice |25 CSI: Miami |24 Aqua Teen Hungerforce |23 Malcolm in the Middle |22 Arrested Development |21 ER |21 Law Order: Special Victims Unit |20 24 |19 Law Order |19 Extreme Makeover: Home Edition |18 Good Eats |17 Joey |17 The West Wing |17 Nova |16 Harvey Birdman: Attorney at Law |16 Sealab 2021 |16 Everybody Loves Raymond |15 Frontline
Re: [mythtv-users] commercial flagging idea - commercialfingerprinting
In my opinion a start would be to figure out some sort of finger printing system, or some method of sharing cutlists that will work on the same program across different stations or that takes in to account that not every mythtv box will start recording at the exact second. Once we have that, creating a centralized system to share said cutlists is the easy part. Another consideration is going to be the different resolutions people record at. If everyone used the same resolution you could choos e a few place son teh screen (like a real fingerprint sort of) and get a difference from a particular pixel or set of pixels (this should eleminate the contrast and sharpness settings). Then just create a sort of MD5 type algorthm for frames or something so you aren't passing the actual screen shots and thus copyrighted material. The you could look for a series of frames that each match the algorithm's results in the database for the specific cut point. All that said with different resolutions you get even more hairy than what I have described, which is admittedly more complex than it might have to be. How do you compare different resolutions for the same picture? I'm sure it can be done. John -- I have plenty of Gmail invites. Ask me if you'd like one. G Mail rocks for email lists. ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
Re: [mythtv-users] commercial flagging idea - commercialfingerprinting
Another consideration is going to be the different resolutions people record at. If everyone used the same resolution you could choos e a few place son teh screen (like a real fingerprint sort of) and get a difference from a particular pixel or set of pixels (this should eleminate the contrast and sharpness settings). Then just create a sort of MD5 type algorthm for frames or something so you aren't passing the actual screen shots and thus copyrighted material. The you could look for a series of frames that each match the algorithm's results in the database for the specific cut point. Someone might be interested in looking at the gqview image-similarity code. I don't know how much it will help, but it *is* capable of comparing and showing similarity between flat different-res images at different percentages (and it's quite accurate at its high setting). On the other hand, it's rather slow to generate the similarity vectors, so it would be virtually useless for use in matching an existing fingerprint to a show to import a cutlist. -Chris ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
Re: [mythtv-users] commercial flagging idea - commercialfingerprinting
Mike Benoit [EMAIL PROTECTED] said: I'm the author of MythRecommend (http://ipso.snappymail.ca:8080/mythrecommend/) which has been around for a fair amount of time now, and out of the 120 unique people who use it, they record 2275 unique programs in total. The top 345 shows are watched by over half of the people. --PART I-- i found the article that explains why it is crucial to have something like mythrecommend built right in I know it's long, but honestly it's a good read. http://www.wired.com/wired/archive/12.10/tail_pr.html http://wired-vig.wired.com/wired/images.html?issue=12.10topic=tailimg=2 for myth, in the above graph, x axis would be shows ranked by popularity y-axis would be the percentage of people who watch sub in 345 for 39,000, (at this point, y=50) the argument is that the area under the curve to the right of the line is actually greater than the area under the curve to the left of the line in your case there were 1930 shows in the long tail i am willing to bet that: if more people used your script, then the long tail would get longer, but there wouldn't be many more than 345 shows that more than half the people watch this is good news and bad news for any commercial fingerprinting scheme - good - we know which shows to fingerprint (if it can be done ;) the 345 hit shows that lie to the left - bad - even though more than half of the people watch these shows, they might only make up 10% of television viewing ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
Re: [mythtv-users] commercial flagging idea - commercialfingerprinting
Mike Benoit [EMAIL PROTECTED] said: I'm not a C programmer, so implementing MythRecommend directly in to MythTV isn't something I have the time to do at this point. Coding the client/server scripts though is what I do all day long, so its pretty quick. Though, I'm sure a Myth developer could implement MythRecommend in to MythTV in no time flat, the scripts store the data in the local mythconverg database, so its just a matter of running a simple query and making the interface to it. --PART II-- mythtv needs to draw people down the long tail from popular shows -to- obscure but related shows i haven't used the mythrecommend script recently so i forget whether the perl script already does that by genre - ie. recommend other *less known comedies* for Simpsons viewers - so, don't recommend CSI or Futurama, even if Simpsons watchers watch them Basically, how do you picture the myth interface for this? 1) have a mythrecommend module accessible by: manage recordings - schedule recordings - recommended shows - this kind of misses the point though, since you have to go out of the way to do it I think that for it to be effective the interface must be done on a per episode basis --so option number 1) is out-- 2) put it somewhere in the set recording options screen - where you currently have list upcoming episodes and program details - make option for list recommended shows? 3) or put it under the context menu of the episode -recording options - recommended shows - or even put it right uner recording options and job options either option 2 or 3 would bring up a kind of - a program finder that only lists recommended shows ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
Re: [mythtv-users] commercial flagging idea - commercialfingerprinting
John Williams wrote: Another consideration is going to be the different resolutions people record at. If everyone used the same resolution you could choos e a few place son teh screen (like a real fingerprint sort of) and get a difference from a particular pixel or set of pixels (this should eleminate the contrast and sharpness settings). Then just create a sort of MD5 type algorthm for frames or something so you aren't passing the actual screen shots and thus copyrighted material. The you could look for a series of frames that each match the algorithm's results in the database for the specific cut point. Not sure you'd want to compare pictures to pictures. Perhaps it would be better to record cutlists and fixes, and compare your cutlist to these. In other words, if user A had an autogenerated cutlist I, fixed the errors to make II, and both lists were stored, then user B would compare their cutlist to I, and if it matched, apply cutlist II instead. One to one, that wouldn't work too well, but if 500 people also had I, and 200 had went ahead and fixed it themselves to II, then you'd have a pretty good chance that II would work for you. Also, you'd need to compare only the show portion, because some channels may have 4 minute breaks and others 6 for the first break, and the opposite on the second, or spend more time on commercials between shows and less during them, but most shows have set commercial break points, so the parts that are the actual show would be the same length (maybe give or take a few seconds). In other words, don't record cutlists as commercial cut at 5:42 until 9:46, commercial cut at 15:20 until 21:20 and so on, but instead record Program data segment 1 length 5:42, program segment 2 length 5:36 and so on. When comparing, you'd compare the segment lengths you have to the segment lengths others had, then look up the modified segment lists those users made and sanity check them against what you have (with presumably less stringent checking, as obviously it was missed the first time). Also, for shows that are broken at different places by different stations (I see that a lot with older re-runs like original series Star Trek), people watching version 1 would match lists by other viewers watching version 1, and people watching version 2 would match lists by others watching version 2, and so on. Of course, some shows may be split any which way, in which case you can't get a match on them. In those cases, though, you are no worse off than you are now. Jeff. ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
RE: [mythtv-users] commercial flagging idea - commercialfingerprinting
-Original Message- From: [EMAIL PROTECTED] [mailto:mythtv-users- [EMAIL PROTECTED] On Behalf Of Matt Sent: Tuesday, April 12, 2005 12:48 PM To: Discussion about mythtv Subject: Re: [mythtv-users] commercial flagging idea - commercialfingerprinting On Apr 11, 2005 7:36 PM, Brad Templeton [EMAIL PROTECTED] wrote: Yup, you'll find threads (often including me) about man of these sorts of features. The long term version -- aggregating which parts of programs people watch at normal speed and which they don't -- provides the only undefeatable commercial detection algorithm, unless you are among the first to watch a show, but it's a long way off. Fingerprints based on various video and audio fingerprinting -- or even easier, on closed caption text when present -- is a lot easier.Some of these can actually be applied without collaboration, in that once you have identified a budweiser commercial by its closed caption text you can spot it again everywhere. (Of course not all commercials have captions.) A _really_ fancy fingerprinting system could actually fingerprint shows and notice when the same snippet of video appears in two different shows -- almost always a commercial, but not always always -- and then cleverly delete the redundant fragment and point to it. (Something myth can't yet do of course but may be planned for the future.) One reason that's interesting is that aside from spotting repeated commercials and repeated shows, it can also let you browse the commercials and start learning what they are. Ie. some people like to capture the movie trailers. Though web download is often a better idea there. Hi Brad! How about this: Instead of fingerprinting the commercials, fingerprint the show 30 seconds before and after commercials. For example. Users manually verify (and fix) detected commercial. Those users that do that, then automatically run a fingerprint before the start of the commercial cut and after the end of the commercial cut (when the show is back from break). This fingerprint could then identify the breaks in the show, not the commercials themselves. Also, the fingerprint alogrithm could include time information. Obviously, shows (in America at least), have a set length between commercial breaks. Each episode might be different, but this is not determined by the individual broadcast stations, so someone watching an episode of Friends in Texas will have the same ammount of time between commercials as the person watching in New York. One show constantly see issues on is Law Order where it comes back from commercial, and will show a the black screen with the text giving the location/date/time of where the scene is taking place... it almost always gets cut off. Just another example :) ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users [DC] I thought you were just working out the timing of the adds themselves (I assume all tv channels break at the same spot in a show) Not actually trying to identify the show themselves. Cheers, Dean ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
Re: [mythtv-users] commercial flagging idea - commercialfingerprinting
On Apr 12, 2005 11:56 AM, dean collins [EMAIL PROTECTED] wrote: [DC] I thought you were just working out the timing of the adds themselves (I assume all tv channels break at the same spot in a show) Not actually trying to identify the show themselves. If you read elsewhere in this thread (although not in the many lines you quoted in your short reply), commercial breaks can easily be longer or shorter on different affiliates of the same network. Here's a relevant quote from Meatwad: In a perfect world, all of the commercial breaks would all happen at the same time. Everywhere. In reality, they do not. Every master cable headend follows a local origination commercial insertion schedule sent daily by their traffic and billing department. However, if we were able to fingerprint the start and end of each show segment, then we don't care how long the comemrcial breaks are, or what the actual commercials were. . We could just look for those fingerprints and mark everything between the end of the previous segment and beginning of the next as a commercial. _Great_ idea... if it's feasable. ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
RE: [mythtv-users] commercial flagging idea -commercialfingerprinting
-Original Message- From: [EMAIL PROTECTED] [mailto:mythtv-users- [EMAIL PROTECTED] On Behalf Of Ramon Redondo Sent: Tuesday, April 12, 2005 1:06 PM To: Discussion about mythtv Subject: Re: [mythtv-users] commercial flagging idea - commercialfingerprinting On Apr 12, 2005 11:56 AM, dean collins [EMAIL PROTECTED] wrote: [DC] I thought you were just working out the timing of the adds themselves (I assume all tv channels break at the same spot in a show) Not actually trying to identify the show themselves. If you read elsewhere in this thread (although not in the many lines you quoted in your short reply), commercial breaks can easily be longer or shorter on different affiliates of the same network. Here's a relevant quote from Meatwad: In a perfect world, all of the commercial breaks would all happen at the same time. Everywhere. In reality, they do not. Every master cable headend follows a local origination commercial insertion schedule sent daily by their traffic and billing department. However, if we were able to fingerprint the start and end of each show segment, then we don't care how long the comemrcial breaks are, or what the actual commercials were. . We could just look for those fingerprints and mark everything between the end of the previous segment and beginning of the next as a commercial. _Great_ idea... if it's feasable. ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users [DC] Ok Ramon, but globally it wouldn't make sense, your idea of fingerprinting the thousand and thousands of advertisements just in the USA alone would add up to be a huge amount. Are you sure that add breaks aren't 'cut' the same on a global basis? it would then make sense in that globally the advertisement 'pre-cursor' and 'post-cursor' would most likely be the same Cheers, Dean ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
Re: [mythtv-users] commercial flagging idea - commercialfingerprinting
Ramon Redondo [EMAIL PROTECTED] said: However, if we were able to fingerprint the start and end of each show segment, then we don't care how long the comemrcial breaks are, or what the actual commercials were. . We could just look for those fingerprints and mark everything between the end of the previous segment and beginning of the next as a commercial. *tongue firmly in cheek* luckily, we already know exactly what preceeds and follows every commercial - a blank frame the obvious problems with fingerprinting is that - it can't work for newly aired shows and - it relies on someone not poisoning the database (much like p2p networks have been poisoned with fakes) - it couldn't hope to cover nearly enough shows to be useful (or am i being naive) ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
Re: [mythtv-users] commercial flagging idea - commercialfingerprinting
On Apr 12, 2005 12:31 PM, Risto Treksler [EMAIL PROTECTED] wrote: snip luckily, we already know exactly what preceeds and follows every commercial - a blank frame Except when the network gets creative and blends the end of a commerical with the beginning of the show. Happens frequently with the 15sec station identification blurbs that usually occur at the end of a commerical block. Finding the blank frame get hard or impossible at that point. -- Josh ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
Re: [mythtv-users] commercial flagging idea - commercialfingerprinting
Josh Burks [EMAIL PROTECTED] said: On Apr 12, 2005 12:31 PM, Risto Treksler [EMAIL PROTECTED] wrote: snip luckily, we already know exactly what preceeds and follows every commercial - a blank frame Except when the network gets creative and blends the end of a commerical with the beginning of the show. Happens frequently with the 15sec station identification blurbs that usually occur at the end of a commerical block. Finding the blank frame get hard or impossible at that point. but there is a blank frame before the station id block, right? personally, i don't mind watching up to a minute of commercials or station id blurbs as long as most ads are skipped and there aren't any false positives. ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
Re: [mythtv-users] commercial flagging idea - commercialfingerprinting
personally, i don't mind watching up to a minute of commercials or station id blurbs as long as most ads are skipped and there aren't any false positives. My problem with this is knowing I'm actually only watching 1 min of ads. If I'm not sure I end up forwarding and then sometimes get in the OOPS, went too far loop -- I have plenty of Gmail invites. Ask me if you'd like one. G Mail rocks for email lists. ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
Re: [mythtv-users] commercial flagging idea - commercialfingerprinting
Preston Crow [EMAIL PROTECTED] said: - it couldn't hope to cover nearly enough shows to be useful (or am i being naive) False. You're being naive. If it just gets popular prime time network shows, that's enough to cover probably 70% or more of the shows people really care about. Kind of like Amazon and iTunes make it possible to access obscure content, that is not available to people in other ways, MythTV lets people watch obscure shows in prime time. I just tend to think that one of the main features of a PVR is that you can watch political commentaries or real estate or cooking shows or whatever you like, so that you don't have to watch mainstream fare as much. I was looking at the program finder and the variety of available programming is astonishing. Now multiply by the number of different cable/satellite carriers. I guess I tend to agree that the traditional 80/20 rule does not apply to electronic mediums such as PVRs. In other words, I don't think that you can satisfy 80% of people by fingerprinting just 20% of the most popular shows, as conventional wisdom would have you think. Instead of a dozen shows, each with a million viewers think of a million shows, each with a dozen viewers Disclamer: the ideas presented above are heavily borrowed from a wired magazine article i read awhile back ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
Re: [mythtv-users] commercial flagging idea - commercialfingerprinting
John Williams [EMAIL PROTECTED] said: My problem with this is knowing I'm actually only watching 1 min of ads. If I'm not sure I end up forwarding and then sometimes get in the OOPS, went too far loop currently you can set it so that - a forward skip, immediately after a backward skip, is as long as the backward skip was (ie 5 seconds) i have been meaning to try to see if it would be better if it did the same the other way around ie. - a backward skip, immediately after a forward skip, is as long as the forward skip was (ie 30 seconds) so when you skip forward one too many times you don't have to skip back 6 times to undo the damage ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
Re: [mythtv-users] commercial flagging idea - commercialfingerprinting
On Apr 12, 2005 12:31 PM, Risto Treksler [EMAIL PROTECTED] wrote: luckily, we already know exactly what preceeds and follows every commercial - a blank frame Not always, I know networks that sometimes edits these out. But for the most part true. the obvious problems with fingerprinting is that - it can't work for newly aired shows and True, but you don't have to run mythcommflag right after a recording. You can schedule it for a later date. But this is a very valid point. - it relies on someone not poisoning the database (much like p2p networks have been poisoned with fakes) Good Point... it could happen, buy how/why? Besides, this is not a replacement for mythcommflag, just another thing it can add to it's already useful arsenal to make a better educated decision. Lets say it received poisened or bad data... it still should be able to look at that data and say does that make since based on what I already have analyzed or how many people have validated a show. Redundant fingerprints should eliminate that. - it couldn't hope to cover nearly enough shows to be useful Another valid point, but my argument for that is that mythcommflag can meet the needs of the millions of programs out there and people can happily do what they've already done. But how many times a day do different networks rerun Seinfeld or Law Order? Or how many people watch/record 24, Lost, etc? Tons. It's all about popularity :) (or am i being naive) maybe I am... this could all be a pipe-dream. =D ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
Re: [mythtv-users] commercial flagging idea - commercialfingerprinting
On Apr 12, 2005 1:09 PM, Josh Burks [EMAIL PROTECTED] wrote: On Apr 12, 2005 12:31 PM, Risto Treksler [EMAIL PROTECTED] wrote: snip luckily, we already know exactly what preceeds and follows every commercial - a blank frame Except when the network gets creative and blends the end of a commerical with the beginning of the show. Happens frequently with the 15sec station identification blurbs that usually occur at the end of a commerical block. Finding the blank frame get hard or impossible at that point. -- Josh VH1 and Food network (and tons of others) do this frequently. ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
Re: [mythtv-users] commercial flagging idea - commercialfingerprinting
On Apr 12, 2005 1:43 PM, Risto Treksler [EMAIL PROTECTED] wrote: I guess I tend to agree that the traditional 80/20 rule does not apply to electronic mediums such as PVRs. In other words, I don't think that you can satisfy 80% of people by fingerprinting just 20% of the most popular shows, as conventional wisdom would have you think. Instead of a dozen shows, each with a million viewers think of a million shows, each with a dozen viewers Could be? But I don't think the networks think that way. That's why they have prime-time hours. That's also why they have sweeps. ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
Re: [mythtv-users] commercial flagging idea - commercialfingerprinting
On Apr 12, 2005 1:06 PM, Ramon Redondo [EMAIL PROTECTED] wrote: If you read elsewhere in this thread (although not in the many lines you quoted in your short reply), commercial breaks can easily be longer or shorter on different affiliates of the same network. Here's a relevant quote from Meatwad: In a perfect world, all of the commercial breaks would all happen at the same time. Everywhere. In reality, they do not. Every master cable headend follows a local origination commercial insertion schedule sent daily by their traffic and billing department. I think this would probably still give you enough to create a cut-list that you could share. You'd have the show, the time it aired and the cable feed ( as defined by xmltv ). Even if the cutlist was only used as another data point for commflag. This would reduce the effect of cutlist poisoning. ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
Re: [mythtv-users] commercial flagging idea - commercialfingerprinting
On Tue, 2005-04-12 at 16:12, Colin Smillie wrote: I think this would probably still give you enough to create a cut-list that you could share. You'd have the show, the time it aired and the cable feed ( as defined by xmltv ). Uhm, would we have a problem with the zap2it licensing of the xml data? I haven't read the license recently, but before someone implements this, that should be reviewed. --PC ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
Re: [mythtv-users] commercial flagging idea - commercialfingerprinting
On Tue, Apr 12, 2005 at 02:24:09PM -0500, Matt wrote: On Apr 12, 2005 1:43 PM, Risto Treksler [EMAIL PROTECTED] wrote: I guess I tend to agree that the traditional 80/20 rule does not apply to electronic mediums such as PVRs. In other words, I don't think that you can satisfy 80% of people by fingerprinting just 20% of the most popular shows, as conventional wisdom would have you think. Instead of a dozen shows, each with a million viewers think of a million shows, each with a dozen viewers Could be? But I don't think the networks think that way. That's why they have prime-time hours. That's also why they have sweeps. There might be a million shows, and a million viewers, but how many _commercials_ are there? Far fewer. And you don't have to be able to identify them all, because if you get something that looks like: longer than 30 seconds gap 30 seconds unidentified gap known commercial gap 30 sec unidentified gap known commercial gap 3 minutes Well, you don't need to know what the unidentifieds are to know they are commercials (and after that you can remember them.) So I don't watch with captions on -- what percentage of commercials have captions now? Also, do these new inaudible sounds that Arbitron is putting into all programming for their meter appear different in commercials than in regular shows? ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
Re: [mythtv-users] commercial flagging idea - commercialfingerprinting
On Tue, Apr 12, 2005 at 04:12:31PM -0400, Colin Smillie wrote: On Apr 12, 2005 1:06 PM, Ramon Redondo [EMAIL PROTECTED] wrote: If you read elsewhere in this thread (although not in the many lines you quoted in your short reply), commercial breaks can easily be longer or shorter on different affiliates of the same network. Here's a relevant quote from Meatwad: In a perfect world, all of the commercial breaks would all happen at the same time. Everywhere. In reality, they do not. Every master cable headend follows a local origination commercial insertion schedule sent daily by their traffic and billing department. I think this would probably still give you enough to create a cut-list that you could share. You'd have the show, the time it aired and the cable feed ( as defined by xmltv ). Even if the cutlist was only used as another data point for commflag. This would reduce the effect of cutlist poisoning. Another cool trick that could be done, of strong benefit to us west-coast people, is that people with east-coast feeds could upload the results of their mythcommflags to a server, and other people could download those. Now even if the feed is not exactly the same, a smarter version of mythcommflag could use it as a starting map. For example, it could go to the cutpoints and hunt near them for the transitions it is looking for (logo on/off, blank frame, etc.) If it doesn't find them anywhere nearby, it gives up and re-flags the old way. But if it does find them it has a new mapping and can let you skip commercials while watching just a few minutes behind live -- with no CPU costs. And even if you watch much later it's a lot less CPU cost. ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
RE: [mythtv-users] commercial flagging idea - commercialfingerprinting
Centralized flagging makes a great idea and whilst a complex project should be possible. The musicbrainz reference is a great starting point. Does MythTV have a centralized bounty system? If so I'll kick it off with an offer of $100. Cheers, Dean -Original Message- From: [EMAIL PROTECTED] [mailto:mythtv-users- [EMAIL PROTECTED] On Behalf Of Matt Sent: Monday, April 11, 2005 5:25 PM To: Discussion about mythtv Subject: [mythtv-users] commercial flagging idea - commercialfingerprinting Hello All, =D Sorry this is long: For the short version, just read the first two paragraphs. I had an idea a few months ago, and just thought I'd throw it out there. I think that mythcommflag is getting better and better. However, sometimes there are times it'll goof up, and some shows are notroriously harder than others to flag correctly by their unique filmographic nature(Lost, Law Order, CSI, etc...). How about a method where users can validate mythcommflag's results and make that info available to the myth community to strengthen the accuracy of commercial flagging by using verified data. For example. You record an epsiode of Lost. You let mythcommflag run, and it detects commercials 90% correct. Then you go in and edit the recording and import the commercial flags by pressing Z... so far, everything like normal. Once the commercial markings are in there, you edit them so they are correct (cut off the begining commercials before the start of the show, make sure commercials are detected correctly, and make sure that there aren't any improperly detected cuts in the middle of the show, etc). Then, once you have explicitly defined the commercials, some process could identify these as Human Verified to be Correct. Then, the next time a user runs mythcommflag on that episode, the service can verify it's results based on those verified results and adjust accordingly. This is where it gets difficult and several different solutions might be available: 1) Digital fingerprints - Not sure if you anyone is familiar with MusicBrainz (http://www.musicbrainz.org/) or not. What it does is take an mp3 file, analysis it and make a digital finger print of it. Then, it sends that fingerprint to a database to compare with other fingerprints that people have submitted. It can then make a comparison and correctly identify the file and populate the ID3 tags of an otherwise un-identified file. The more people that ID a file, the smarter it becomes and the better it can do at identifying files. Once the program has been correctly marked, and some kind of fingerprint is made from the show, then it could be uploaded to a central database where other's can look for that show and episode to compare mythcommflag results so that the mythcommflag can do a better job based on the verified results sent by users who manually made sure the commercial detection was correct. 2) Time based - Possibly more difficult, however you have to assume that the time of the show between commercial breaks is going to be (nearly) identical. So, in almost the same way, if mythcommflag has a false positive for a commercial, it can review what others have verified as correct and say hey, there shouldn't be a commercial here because 40 other users say there is a time block of 15 minutes, not 7.5 + commericial + 7.5 more, etc. 3) Identify individual commercials (not the entire commercial break) - Along the same lines as #1, identify specific commercials. So, anytime an acme company commercial is recorded, mythcommflag could/would be able to identify it as a commercial soley because it's learned that it is. This could possibly have more overhead and possibly labor intensive. With the constant turnover of advertising, as well as locally broadcast commercials, it might not be worth it and it'd be a ton of information. However, it could see where this might be useful for short teaser clips you see at the begining or the end of certain shows or that certain networks frequently use. 4) combination of the above - I think if you went with the fingerprint, a time based solution would almost be included in that. 5) Other - I'm sure others would have other suggestions that might work for this. I know that this is a centralized approach, but seeings how it would be what I call a bonus feature, people that don't want to participate wouldn't have to sort of thing. Perhaps their's a decentralized approach as well. Mythcommflag could still work WITHOUT this feature, it's just something that can reinforce the accuracy of commercial flagging in myth. I'm also not sure about the legalities, PC, etc that a methodology like this would have or Issac's stance on the issue as I don't know if it's ever been thought of up to the point. Anyway, I'm not a developer, nor am I pleading or begging anyone to develop this... I just think it sounds cool
Re: [mythtv-users] commercial flagging idea - commercialfingerprinting
dean collins wrote: Centralized flagging makes a great idea and whilst a complex project should be possible. The musicbrainz reference is a great starting point. Does MythTV have a centralized bounty system? If so I'll kick it off with an offer of $100. Cheers, Dean The only issue I can really see here comes that this is required that people have to upload their verified commercials. I can forsee that there are many people who would rather be lazy and download this data than upload their own. There would have to be some sort of system to enforce that everyone puts in their own share of data. Perhaps if you use this feature, you are required to hand verify your own results for some pre defined percentage of the shows, and report those. This also brings up the point that of some shows won't have very much data - say for example a show local only to your area. This predefined percentage should take a look at what shows have the most data being reported, and then take a look at what shows you record. If your shows being recorded don't have very much data, you would be required to submit data for your shows. What do people think of this? ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
