Re: Reading video files from HDFS using OpenCV

[Madhav Sharan]

Thanks a lot Ron. It helps

--
Madhav Sharan


On Sun, Jul 24, 2016 at 2:19 PM, Ron Gonzalez  wrote:

> In a manner of speaking. I would imagine that you would like to take
> advantage of resource management that comes with yarn. If you're planning
> to make this a product that your customers can submit jobs to do some image
> processing, then you're better off creating a yarn application master that
> would handle taking files from Hdfs, localize then process as a local
> process.
>
> If not and more of a one-time effort, then it's as simple as:
>
> hadoop fs -copyToLocal hdfspath localpath
>
> You can then do whatever you want to do with this local file.
>
> Thanks,
> Ron
>
> On Jul 24, 2016 12:28 PM, Madhav Sharan  wrote:
>
> Hi Ron, Thanks for replying.
>
> Unfortunately I could not find a VideoCapture method accepting stream
> input.
>
> I will look into second option. Will it be similar to copying file from
> hdfs to a tmp directory and then using tmp file?
>
> --
> Madhav Sharan
>
>
> On Sun, Jul 24, 2016 at 12:08 PM, Ron's Yahoo! 
> wrote:
>
> You can create a stream for an HDFS file and have OpenCV read from the
> stream.
> Other option is to create a yarn job and you can resource localize the
> HDFS file so you can run OpenCV just like any other application.
>
> Thanks,
> Ron
>
> On Jul 24, 2016, at 12:03 PM, Madhav Sharan  wrote:
>
> Hi hadoop users,
>
> I need some help in reading video files from HDFS. I am processing videos
> using OpenCV and opencv is not able to read video file from hdfs. I tried
> prefixing path with "hdfs://" but it still does not works. Is there a way
> we can directly read from HDFS using opencv?
>
>
> [0] Video capture code -
> https://github.com/smadha/pooled_time_series/blob/master/src/main/java/gov/nasa/jpl/memex/pooledtimeseries/PoT.java#L441
> 
>
> [1] TMP file solution -
> https://github.com/smadha/pooled_time_series/blob/master/src/main/java/gov/nasa/jpl/memex/pooledtimeseries/OpticalTimeSeries.java#L66
> 
>
> --
> Madhav Sharan
>
>
>
>
>

Re: Reading video files from HDFS using OpenCV

[Ron Gonzalez]

In a manner of speaking. I would imagine that you would like to take advantage of resource management that comes with yarn. If you're planning to make this a product that your customers can submit jobs to do some image processing, then you're better off creating a yarn application master that would handle taking files from Hdfs, localize then process as a local process.
 If not and more of a one-time effort, then it's as simple as:
hadoop fs -copyToLocal hdfspath localpath
You can then do whatever you want to do with this local file.
Thanks,
Ron
On Jul 24, 2016 12:28 PM, Madhav Sharan  wrote:Hi Ron, Thanks for replying.Unfortunately I could not find a VideoCapture method accepting stream input.I will look into second option. Will it be similar to copying file from hdfs to a tmp directory and then using tmp file?--Madhav Sharan
On Sun, Jul 24, 2016 at 12:08 PM, Ron's Yahoo!  wrote:You can create a stream for an HDFS file and have OpenCV read from the stream.Other option is to create a yarn job and you can resource localize the HDFS file so you can run OpenCV just like any other application.Thanks,RonOn Jul 24, 2016, at 12:03 PM, Madhav Sharan  wrote:Hi hadoop users,I need some help in reading video files from HDFS. I am processing videos using OpenCV and opencv is not able to read video file from hdfs. I tried prefixing path with "hdfs://" but it still does not works. Is there a way we can directly read from HDFS using opencv? [0] Video capture code -  https://github.com/smadha/pooled_time_series/blob/master/src/main/java/gov/nasa/jpl/memex/pooledtimeseries/PoT.java#L441[1] TMP file solution - https://github.com/smadha/pooled_time_series/blob/master/src/main/java/gov/nasa/jpl/memex/pooledtimeseries/OpticalTimeSeries.java#L66--Madhav Sharan

Re: Reading video files from HDFS using OpenCV

[Madhav Sharan]

Hi Ron, Thanks for replying.

Unfortunately I could not find a VideoCapture method accepting stream input.

I will look into second option. Will it be similar to copying file from
hdfs to a tmp directory and then using tmp file?

--
Madhav Sharan


On Sun, Jul 24, 2016 at 12:08 PM, Ron's Yahoo!  wrote:

> You can create a stream for an HDFS file and have OpenCV read from the
> stream.
> Other option is to create a yarn job and you can resource localize the
> HDFS file so you can run OpenCV just like any other application.
>
> Thanks,
> Ron
>
> On Jul 24, 2016, at 12:03 PM, Madhav Sharan  wrote:
>
> Hi hadoop users,
>
> I need some help in reading video files from HDFS. I am processing videos
> using OpenCV and opencv is not able to read video file from hdfs. I tried
> prefixing path with "hdfs://" but it still does not works. Is there a way
> we can directly read from HDFS using opencv?
>
>
> [0] Video capture code -
> https://github.com/smadha/pooled_time_series/blob/master/src/main/java/gov/nasa/jpl/memex/pooledtimeseries/PoT.java#L441
> 
>
> [1] TMP file solution -
> https://github.com/smadha/pooled_time_series/blob/master/src/main/java/gov/nasa/jpl/memex/pooledtimeseries/OpticalTimeSeries.java#L66
> 
>
> --
> Madhav Sharan
>
>
>

Re: Reading video files from HDFS using OpenCV

[Ron's Yahoo!]

You can create a stream for an HDFS file and have OpenCV read from the stream.
Other option is to create a yarn job and you can resource localize the HDFS 
file so you can run OpenCV just like any other application.

Thanks,
Ron

> On Jul 24, 2016, at 12:03 PM, Madhav Sharan  wrote:
> 
> Hi hadoop users,
> 
> I need some help in reading video files from HDFS. I am processing videos 
> using OpenCV and opencv is not able to read video file from hdfs. I tried 
> prefixing path with "hdfs://" but it still does not works. Is there a way we 
> can directly read from HDFS using opencv? 
> 
> 
> [0] Video capture code -  
> https://github.com/smadha/pooled_time_series/blob/master/src/main/java/gov/nasa/jpl/memex/pooledtimeseries/PoT.java#L441
>  
> 
> 
> [1] TMP file solution - 
> https://github.com/smadha/pooled_time_series/blob/master/src/main/java/gov/nasa/jpl/memex/pooledtimeseries/OpticalTimeSeries.java#L66
>  
> 
> 
> --
> Madhav Sharan
> 

Reading video files from HDFS using OpenCV

[Madhav Sharan]

Hi hadoop users,

I need some help in reading video files from HDFS. I am processing videos
using OpenCV and opencv is not able to read video file from hdfs. I tried
prefixing path with "hdfs://" but it still does not works. Is there a way
we can directly read from HDFS using opencv?


[0] Video capture code -
https://github.com/smadha/pooled_time_series/blob/master/src/main/java/gov/nasa/jpl/memex/pooledtimeseries/PoT.java#L441

[1] TMP file solution -
https://github.com/smadha/pooled_time_series/blob/master/src/main/java/gov/nasa/jpl/memex/pooledtimeseries/OpticalTimeSeries.java#L66

--
Madhav Sharan