Re: [Libav-user] get RGB values from ffmpeg frame
Got the solution to my problem quite a while back. Posting it now as I
have time. Hope it helps someone else. Like Carl said, it'd be best for
a person to build the ffmpeg libraries by themselves.
I was using the 32bit dev builds from Zeranoe, and linking to the .lib
files caused the compiler to ask for the corresponding DLL files too,
but DLL's aren't part of the dev builds, so I had to take the DLL's of
the shared build.
Using it this way, my ffmpeg code worked, but used to strangely crash at
sws_scale.
But using purely the shared build's DLL's everything worked fine. Some
example code for the RGB values:
//This example is from the updated tutorial 01:
https://github.com/mpenkov/ffmpeg-tutorial
#include Windows.h
#include libavformat/avformat.h
#include libavcodec/avcodec.h//just as an example
int main()
{
const char * avfDLLpointer =
F:\\Nav\\VisualStudio\\ffmpegTestProgram\\ffmpegTestProgram\\avformat-54.dll;//you
can place this DLL in the same directory as your exe
const char * avcDLLpointer =
F:\\Nav\\VisualStudio\\ffmpegTestProgram\\ffmpegTestProgram\\avcodec-54.dll;.//just
as an example
HINSTANCE hinstLib_avformat = LoadLibrary(avfDLLpointer);
HINSTANCE hinstLib_avcodec = LoadLibrary(avcDLLpointer);//just as
an example
__av_dump_format av_dump_format_proc =
(__av_dump_format)GetProcAddress(hinstLib_avformat,
av_dump_format);
...blah blah...
sws_ctx = sws_getContext_proc(pCodecCtx-width, pCodecCtx-height,
pCodecCtx-pix_fmt, pCodecCtx-width, pCodecCtx-height, PIX_FMT_RGB24,
SWS_BILINEAR, NULL, NULL, NULL );
avpicture_fill_proc((AVPicture *)pFrameRGB, buffer, PIX_FMT_RGB24,
pCodecCtx-width, pCodecCtx-height);
...blah blah...
sws_scale_proc(sws_ctx, (uint8_t const * const *)pFrame-data,
pFrame-linesize, 0, pCodecCtx-height, pFrameRGB-data,
pFrameRGB-linesize );
for(int yy = 0; yy pCodecCtx-height; ++yy)
{
for(int xx = 0; xx pCodecCtx-width; ++xx)
{
int p = xx * 3 + yy * pFrameRGB-linesize[0];
int r = pFrameRGB-data[0][p];
int g = pFrameRGB-data[0][p+1];
int b = pFrameRGB-data[0][p+2];
}
}
...blah blah...
}
Navin
On 11/22/2012 4:32 AM, Carl Eugen Hoyos wrote:
Malik Cissémc@... writes:
Carl Eugen ... writes:
...compile FFmpeg with msvc.
This sounds interesting. Where can I find msvc sample project?
Please see the MSVC section on the platform page
of the fine documentation to find information on
how to compile FFmpeg with MSVC.
Carl Eugen
___
Libav-user mailing list
Libav-user@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/libav-user
___
Libav-user mailing list
Libav-user@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/libav-user
Re: [Libav-user] get RGB values from ffmpeg frame
Sorry, the correct code (with the necessary typedefs) is as such:
#include Windows.h
#include libavformat/avformat.h
#include libavcodec/avcodec.h
#include libswscale/swscale.h
typedef SwsContext* (__cdecl *__sws_getContext)(int srcW, int srcH, enum
AVPixelFormat srcFormat, int dstW, int dstH, enum AVPixelFormat
dstFormat, int flags, SwsFilter *srcFilter, SwsFilter *dstFilter, const
double *param);
typedef int (__cdecl *__sws_scale)(struct SwsContext *c, const uint8_t
*const srcSlice[], const int srcStride[], int srcSliceY, int srcSliceH,
uint8_t *const dst[], const int dstStride[]);
typedef int (__cdecl *__avpicture_fill)(AVPicture *picture, uint8_t
*ptr, enum AVPixelFormat pix_fmt, int width, int height);
int main()
{
const char * swsDLLpointer =
F:\\Nav\\VisualStudio\\ffmpegTestProgram\\ffmpegTestProgram\\swscale-2.dll;
const char * avcDLLpointer =
F:\\Nav\\VisualStudio\\ffmpegTestProgram\\ffmpegTestProgram\\avcodec-54.dll;
HINSTANCE hinstLib_swscale = LoadLibrary(swsDLLpointer);
HINSTANCE hinstLib_avcodec = LoadLibrary(avcDLLpointer);
__avpicture_fill avpicture_fill_proc =
(__avpicture_fill)GetProcAddress(hinstLib_avcodec,
avpicture_fill);
__sws_getContext sws_getContext_proc =
(__sws_getContext)GetProcAddress(hinstLib_swscale,
sws_getContext);
__sws_scalesws_scale_proc =
(__sws_scale) GetProcAddress(hinstLib_swscale, sws_scale);
...blah blah...
sws_ctx = sws_getContext_proc(pCodecCtx-width, pCodecCtx-height,
pCodecCtx-pix_fmt, pCodecCtx-width, pCodecCtx-height, PIX_FMT_RGB24,
SWS_BILINEAR, NULL, NULL, NULL );
avpicture_fill_proc((AVPicture *)pFrameRGB, buffer, PIX_FMT_RGB24,
pCodecCtx-width, pCodecCtx-height);
...blah blah...
sws_scale_proc(sws_ctx, (uint8_t const * const *)pFrame-data,
pFrame-linesize, 0, pCodecCtx-height, pFrameRGB-data,
pFrameRGB-linesize );
for(int yy = 0; yy pCodecCtx-height; ++yy)
{
for(int xx = 0; xx pCodecCtx-width; ++xx)
{
int p = xx * 3 + yy * pFrameRGB-linesize[0];
int r = pFrameRGB-data[0][p];
int g = pFrameRGB-data[0][p+1];
int b = pFrameRGB-data[0][p+2];
}
}
...blah blah...
}
It's basically that simple to get RGB values.
Navin
On 11/22/2012 4:32 AM, Carl Eugen Hoyos wrote:
Malik Cissémc@... writes:
Carl Eugen ... writes:
...compile FFmpeg with msvc.
This sounds interesting. Where can I find msvc sample project?
Please see the MSVC section on the platform page
of the fine documentation to find information on
how to compile FFmpeg with MSVC.
Carl Eugen
___
Libav-user mailing list
Libav-user@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/libav-user
___
Libav-user mailing list
Libav-user@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/libav-user
Re: [Libav-user] get RGB values from ffmpeg frame
Navin nkipe@... writes: Am using the latest ffmpeg-win32-dev and I've tried with a wmv, many mp4 (from youtube, h264 (high) (avc1/0x31637661), yuv420p 1280x720, 1139kb/s, 24fps) files and avi too, but they're all crashing with Unhandled exception, Access violation writing location 0x002a3000 at the line sws_scale. Did you look at doc/examples/scaling_video.c? Because I don't see the call to av_image_alloc() in your code... Partly related: If you are really interested in working with FFmpeg libraries, instead of downloading pre-compiled libraries, you should either install msys and mingw (which provide a debugger that would have helped you) or compile FFmpeg with msvc. Carl Eugen ___ Libav-user mailing list Libav-user@ffmpeg.org http://ffmpeg.org/mailman/listinfo/libav-user
Re: [Libav-user] get RGB values from ffmpeg frame
Carl Eugen ... writes: ...compile FFmpeg with msvc. This sounds interesting. Where can I find msvc sample project? Malik Cisse ___ Libav-user mailing list Libav-user@ffmpeg.org http://ffmpeg.org/mailman/listinfo/libav-user
Re: [Libav-user] get RGB values from ffmpeg frame
use migw in linux 1- in a virtual machine run linux 2- install mingw 3- install yasm 4- compile yasm , x264 and ffmpeg with migw On Wed, Nov 21, 2012 at 4:03 PM, Malik Cissé m...@enciris.com wrote: Carl Eugen ... writes: ...compile FFmpeg with msvc. This sounds interesting. Where can I find msvc sample project? Malik Cisse ___ Libav-user mailing list Libav-user@ffmpeg.org http://ffmpeg.org/mailman/listinfo/libav-user ___ Libav-user mailing list Libav-user@ffmpeg.org http://ffmpeg.org/mailman/listinfo/libav-user
Re: [Libav-user] get RGB values from ffmpeg frame
Been trying these techniques, but I keep getting crashes. Help please?:
avpicture_fill((AVPicture*) frame2, frame2_buffer,
PIX_FMT_RGB24, width2, height2);
//printf(data = %d, %d, %d, %d
\n,frame2-data[0],frame2-data[1],frame2-data[2],frame2-data[3]);
//printf(linesize = %d %d %d\n,frame2-linesize[0],
frame2-linesize[1], frame2-linesize[2]);
//printf(width = %d\n, pCodecCtx-width);
//printf(height = %d\n, pCodecCtx-height);
//std::cin.get();
int linesize = frame2-linesize[0];
for(int xx = 0; xx (linesize * width1)-1; xx += 3)
{
int r = frame2-data[0][xx];//int r = frame2-data[0][xx];
int g = frame2-data[0][xx+1];
int b = frame2-data[0][xx+2];
printf(xx=%d r=%d, g=%d, b=%d \n,xx, r,
g, b);
}
printf(frame%d done,i++);
//for(int xx = 0; xx width1; xx = xx + 3)
//{
//for(int yy = 0; yy height1; ++yy)
//{
////int p = xx*3 + yy*frame2-linesize[0];
////int p = xx * 3 + yy * linesize;
//printf(yy=%d xx=%d,yy,xx);
//int p = yy * linesize + xx;
//printf(p=%d\n,p);
//int r = frame2-data[0][p];
//int g = frame2-data[0][p+1];
//int b = frame2-data[0][p+2];
//printf([r=%d, g=%d, b=%d ]\n, r, g, b);
//}//for
//}//for
Nav
On 11/20/2012 8:52 AM, Nav wrote:
Hi! Glad to be part of this mailing list.
What I wanted to create, was a program which would receive a streaming
video, and when it decodes a frame of the video into either a bitmap
format or just pure RGB (perhaps stored in a char array), it would
notify another program that it has received a frame, and the other
program would take the RGB values and display it.
I've already asked this question here:
http://ffmpeg.zeranoe.com/forum/viewtopic.php?f=15t=805
and rogerdpack told me to post my question on the libav mailing list.
I have been through many websites, but they either use img_convert
(which doesn't work) or sws_scale, which crashes when I try to use it
with RGB.
Could anyone help with a complete piece of code which can give me the
RGB values of a frame?
This is a part of the YUV conversion that I tried initially.
i=0;
while(av_read_frame(pFormatCtx, packet) = 0)
{
// Is this a packet from the video stream?
if(packet.stream_index==videoStream)
{
// Decode video frame
avcodec_decode_video2(pCodecCtx, pFrame, frameFinished, packet);
// Did we get a video frame?
if(frameFinished)
{
// Convert the image into YUV format that SDL uses
sws_scale( sws_ctx, (uint8_t const * const *)pFrame-data,
pFrame-linesize, 0, pCodecCtx-height, pict.data, pict.linesize );
___
Libav-user mailing list
Libav-user@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/libav-user
Re: [Libav-user] get RGB values from ffmpeg frame
Have you looked at the tutorial concerning ffmpeg wit sdl, and/or the ffplay
sourcecode? Other things to check out would be the motion jpeg encoder - afaik
jpg images are in rgb space.
R
Navin nk...@tatapowersed.com wrote:
Been trying these techniques, but I keep getting crashes. Help please?:
avpicture_fill((AVPicture*) frame2, frame2_buffer,
PIX_FMT_RGB24, width2, height2);
//printf(data = %d, %d, %d, %d
\n,frame2-data[0],frame2-data[1],frame2-data[2],frame2-data[3]);
//printf(linesize = %d %d %d\n,frame2-linesize[0],
frame2-linesize[1], frame2-linesize[2]);
//printf(width = %d\n, pCodecCtx-width);
//printf(height = %d\n, pCodecCtx-height);
//std::cin.get();
int linesize = frame2-linesize[0];
for(int xx = 0; xx (linesize * width1)-1; xx += 3)
{
int r = frame2-data[0][xx];//int r = frame2-data[0][xx];
int g = frame2-data[0][xx+1];
int b = frame2-data[0][xx+2];
printf(xx=%d r=%d, g=%d, b=%d \n,xx, r,
g, b);
}
printf(frame%d done,i++);
//for(int xx = 0; xx width1; xx = xx + 3)
//{
//for(int yy = 0; yy height1; ++yy)
//{
////int p = xx*3 + yy*frame2-linesize[0];
////int p = xx * 3 + yy * linesize;
//printf(yy=%d xx=%d,yy,xx);
//int p = yy * linesize + xx;
//printf(p=%d\n,p);
//int r = frame2-data[0][p];
//int g = frame2-data[0][p+1];
//int b = frame2-data[0][p+2];
//printf([r=%d, g=%d, b=%d ]\n, r, g, b);
//}//for
//}//for
Nav
On 11/20/2012 8:52 AM, Nav wrote:
Hi! Glad to be part of this mailing list.
What I wanted to create, was a program which would receive a
streaming
video, and when it decodes a frame of the video into either a bitmap
format or just pure RGB (perhaps stored in a char array), it would
notify another program that it has received a frame, and the other
program would take the RGB values and display it.
I've already asked this question here:
http://ffmpeg.zeranoe.com/forum/viewtopic.php?f=15t=805
and rogerdpack told me to post my question on the libav mailing list.
I have been through many websites, but they either use img_convert
(which doesn't work) or sws_scale, which crashes when I try to use it
with RGB.
Could anyone help with a complete piece of code which can give me the
RGB values of a frame?
This is a part of the YUV conversion that I tried initially.
i=0;
while(av_read_frame(pFormatCtx, packet) = 0)
{
// Is this a packet from the video stream?
if(packet.stream_index==videoStream)
{
// Decode video frame
avcodec_decode_video2(pCodecCtx, pFrame, frameFinished,
packet);
// Did we get a video frame?
if(frameFinished)
{
// Convert the image into YUV format that SDL uses
sws_scale( sws_ctx, (uint8_t const * const *)pFrame-data,
pFrame-linesize, 0, pCodecCtx-height, pict.data, pict.linesize );
___
Libav-user mailing list
Libav-user@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/libav-user
___
Libav-user mailing list
Libav-user@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/libav-user
Re: [Libav-user] get RGB values from ffmpeg frame
Hi Navin,
You don't need RGB for bitmap. AFAIK BMP pictures can also embed YUV4:2:0
Here is a h264 decoding code that works fine. It may help you extracting the
right parts.
//#include stdafx.h
#include stdio.h
#include stdlib.h
#include avcodec/avcodec.h
static int FindStartCode (unsigned char *Buf, int zeros_in_startcode)
{
int info;
int i;
info = 1;
for (i = 0; i zeros_in_startcode; i++)
if(Buf[i] != 0)
info = 0;
if(Buf[i] != 1)
info = 0;
return info;
}
int getNextNal(FILE* inpf, unsigned char* Buf)
{
int pos = 0;
while(!feof(inpf) (Buf[pos++]=fgetc(inpf))==0);
int StartCodeFound = 0;
int info2 = 0;
int info3 = 0;
while (!StartCodeFound)
{
if (feof (inpf))
{
return -1;
}
Buf[pos++] = fgetc (inpf);
info3 = FindStartCode(Buf[pos-4], 3);
if(info3 != 1)
info2 = FindStartCode(Buf[pos-3], 2);
StartCodeFound = (info2 == 1 || info3 == 1);
}
fseek (inpf, -4, SEEK_CUR);
return pos - 4;
}
int main()
{
int i;
FILE * inpf;
FILE * outf;
inpf = fopen(test.264, rb);
outf = fopen(out1_720x576.yuv, wb);
int nalLen = 0;
unsigned char* Buf = (unsigned char*)calloc ( 100, sizeof(char));
unsigned char* Buf1 = (unsigned char*)calloc ( 100, sizeof(char));
int width = 720;
int height = 576;
//unsigned char yuvData[352 * 240 * 3];
AVCodec *codec; // Codec
AVCodecContext *c;// Codec Context
AVFrame *picture; // Frame
avcodec_init();
avcodec_register_all();
codec = avcodec_find_decoder(CODEC_ID_H264);
if (!codec) {
return 0;
}
//allocate codec context
c = avcodec_alloc_context();
if(!c){
return 0;
}
//open codec
if (avcodec_open(c, codec) 0) {
return 0;
}
//allocate frame buffer
picture = avcodec_alloc_frame();
if(!picture){
return 0;
}
while(!feof(inpf))
{
//printf(cntr: %d\n, cntr++);
nalLen = getNextNal(inpf, Buf);
//try to decode this frame
int got_picture, result;
result= avcodec_decode_video(c, picture, got_picture, Buf,
nalLen);
if(result 0)
{
//fwrite(picture-data[0],1,width*height,outf);
//fwrite(picture-data[1],1,width*height/4,outf);
//fwrite(picture-data[2],1,width*height/4,outf);
#if 0 //this is also ok!
for(i=0; ic-height; i++)
fwrite(picture-data[0] + i *
picture-linesize[0], 1, c-width, outf );
for(i=0; ic-height/2; i++)
fwrite(picture-data[1] + i *
picture-linesize[1], 1, c-width/2, outf );
for(i=0; ic-height/2; i++)
fwrite(picture-data[2] + i *
picture-linesize[2], 1, c-width/2, outf );
#endif
//http://ffmpeg.org/doxygen/trunk/group__lavc__picture.html#g99bc554a84681f7ee6422a384007a4ca
//Copy pixel data from an AVPicture into a buffer,
always assume a linesize alignment of 1.
avpicture_layout((AVPicture *)picture, c-pix_fmt,
c-width, c-height, Buf1, c-width*c-height*1.5);
fwrite(Buf1, c-width*c-height*1.5, 1, outf);
}
}
if(c) {
avcodec_close(c);
av_free(c);
c = NULL;
}
if(picture) {
av_free(picture);
picture = NULL;
}
return 1;
}
Malik,
-Original Message-
From: libav-user-boun...@ffmpeg.org [mailto:libav-user-boun...@ffmpeg.org] On
Behalf Of Navin
Sent: 20 November 2012 09:49
To: libav-user@ffmpeg.org
Subject: Re: [Libav-user] get RGB values from ffmpeg frame
Been trying these techniques, but I keep getting crashes. Help please?:
avpicture_fill((AVPicture*) frame2, frame2_buffer, PIX_FMT_RGB24,
width2, height2);
//printf(data = %d, %d, %d, %d
\n,frame2-data[0],frame2-data[1],frame2-data[2],frame2-data[3]);
//printf(linesize = %d %d %d\n,frame2-linesize[0],
frame2-linesize[1], frame2-linesize[2]);
//printf(width = %d\n, pCodecCtx-width);
//printf(height = %d\n, pCodecCtx-height);
//std::cin.get();
int linesize = frame2-linesize[0];
for(int xx = 0; xx (linesize * width1)-1; xx += 3)
{
int r
Re: [Libav-user] get RGB values from ffmpeg frame
Malik Cissé mc@... writes: You don't need RGB for bitmap. AFAIK BMP pictures can also embed YUV4:2:0 I didn't know bmp supports yuv. Do you have a sample or a source that confirms this? Carl Eugen ___ Libav-user mailing list Libav-user@ffmpeg.org http://ffmpeg.org/mailman/listinfo/libav-user
Re: [Libav-user] get RGB values from ffmpeg frame
Navin nkipe@... writes: I have been through many websites, but they either use img_convert (which doesn't work) or sws_scale, which crashes when I try to use it with RGB. I may miss something, but I guess this is the important problem you should work on (see doc/examples). Possibly unrelated: In your source code, I did not find information which decoder you are using. Some decoders (everything mpeg-related) output yuv, and if you want the data in rgb, you have to convert it (using the sofware scaler), others (some lossless formats) output rgb, that may already be the data you want. Carl Eugen ___ Libav-user mailing list Libav-user@ffmpeg.org http://ffmpeg.org/mailman/listinfo/libav-user
Re: [Libav-user] get RGB values from ffmpeg frame
René J.V. Bertin rjvbertin@... writes: afaik jpg images are in rgb space. No. (Lossless jpg supports rgb and FFmpeg supports reading and writing lossless jpg but this is a very unusual feature and nearly all non-FFmpeg software will not support the images.) Carl Eugen ___ Libav-user mailing list Libav-user@ffmpeg.org http://ffmpeg.org/mailman/listinfo/libav-user
Re: [Libav-user] get RGB values from ffmpeg frame
Hey thanks everyone! Still trying/struggling with the code sent by Malik
because avcodec_decode_video is deprecated.
Thought I'd post the entire code here for anyone who could help or for
anyone who needs such code in future. This is the corrected version of
ffmpeg's tutorial02, with SDL added to it too.
Problems are:
1. Since I was facing trouble with RGB, I thought I'd use SDL to output
the frames as bitmaps, but although I could store the bitmap in an area
in memory, it threw an exception when I tried extracting it out of memory.
2. Besides, when at least trying to output the bitmap to hard disk, I'm
unable to output it unless SDL_DisplayYUVOverlay(bmp, rect); is called
(and I don't want to call it because I don't want the video to be
displayed. I just want the bitmap to be output).
3. Main problem is still with extracting RGB, so would be grateful for
any help. If not RGB, then at least with question 1 (how to store and
extract the BMP from SDL memory).
The code:
extern C
{
#include libavcodec/avcodec.h
#include libavformat/avformat.h
#include libswscale/swscale.h
}
#includeiostream
#include SDL.h
#include SDL_thread.h
#include SDL_opengl.h
#include gl/gl.h
#include gl/glu.h
//#ifdef __MINGW32__
#undef main /* Prevents SDL from overriding main() */
//#endif
#include stdio.h
//
// These functions should not be used except from pointers in a RWops
static int myseekfunc(SDL_RWops *context, int offset, int whence) {
SDL_SetError(Can't seek in this kind of RWops); return(-1); }
static int myreadfunc(SDL_RWops *context, void *ptr, int size, int
maxnum) { memset(ptr,0,size*maxnum); return(maxnum); }
static int mywritefunc(SDL_RWops *context, const void *ptr, int size,
int num) { return(num); }
static int myclosefunc(SDL_RWops *context)
{
if(context-type != 0xdeadbeef) { SDL_SetError(Wrong kind of RWops
for myclosefunc()); return(-1); }
free(context-hidden.unknown.data1);
SDL_FreeRW(context);
return(0);
}
// Note that this function is NOT static -- we want it directly callable
from other source files
SDL_RWops *MyCustomRWop()
{
SDL_RWops *c=SDL_AllocRW();
if(c==NULL) return(NULL);
c-seek =myseekfunc;
c-read =myreadfunc;
c-write=mywritefunc;
c-close=myclosefunc;
c-type =0xdeadbeef;
printf(deadbeef=%d\n,c-type);
c-hidden.unknown.data1=malloc(10);
return(c);
}
int main(int argc, char *argv[])
{
AVFormatContext *pFormatCtx = NULL;
int i, videoStream;
AVCodecContext *pCodecCtx = NULL;
AVCodec *pCodec = NULL;
AVFrame *pFrame = NULL;
AVPacketpacket;
int frameFinished;
//float aspect_ratio;
AVDictionary*optionsDict = NULL;
struct SwsContext *sws_ctx = NULL;
SDL_Overlay *bmp = NULL;
SDL_Surface *screen = NULL;
SDL_Surface *screen2 = NULL;
SDL_Surface *rgbscreen = NULL;
SDL_Rectrect;
SDL_Event event;
if(argc 2) { fprintf(stderr, Usage: test.exe videofile\n);
exit(1); }
// Register all formats and codecs
av_register_all();
if(SDL_Init(SDL_INIT_VIDEO | SDL_INIT_AUDIO | SDL_INIT_TIMER)) {
fprintf(stderr, Could not initialize SDL - %s\n, SDL_GetError());
exit(1); }
// Open video file
if(avformat_open_input(pFormatCtx, argv[1], NULL, NULL)!=0)
return -1; // Couldn't open file
// Retrieve stream information
if(avformat_find_stream_info(pFormatCtx, NULL)0)return -1; //
Couldn't find stream information
// Dump information about file onto standard error
//av_dump_format(pFormatCtx, 0, argv[1], 0);
// Find the first video stream
videoStream=-1;
for(i=0; ipFormatCtx-nb_streams; i++)
if(pFormatCtx-streams[i]-codec-codec_type==AVMEDIA_TYPE_VIDEO) {
videoStream=i; break; }
if(videoStream==-1)return -1; // Didn't find a video stream
// Get a pointer to the codec context for the video stream
pCodecCtx=pFormatCtx-streams[videoStream]-codec;
// Find the decoder for the video stream
pCodec=avcodec_find_decoder(pCodecCtx-codec_id);
if(pCodec==NULL) {fprintf(stderr, Unsupported codec!\n);
return -1;} // Codec not found
// Open codec
if(avcodec_open2(pCodecCtx, pCodec, optionsDict) 0)return -1;
// Could not open codec
// Allocate video frame
pFrame = avcodec_alloc_frame();
#if defined(TRIAL)
int width1 = pCodecCtx-width, height1 = pCodecCtx-height, width2
= pCodecCtx-width, height2 = pCodecCtx-height;
struct SwsContext *resize = sws_getContext(width1, height1,
PIX_FMT_YUV420P, width2, height2, PIX_FMT_RGB24, SWS_BICUBIC, NULL,
NULL, NULL);
AVFrame* frame1 = avcodec_alloc_frame(); // this is your original frame
AVFrame* frame2 = avcodec_alloc_frame();
int num_bytes = avpicture_get_size(PIX_FMT_RGB24, width2, height2);
uint8_t* frame2_buffer = (uint8_t *) av_malloc(num_bytes *
sizeof(uint8_t));
#endif
// Make a screen to put our video
#ifndef __DARWIN__
screen =
Re: [Libav-user] get RGB values from ffmpeg frame
, outfile1);
//decode VC-1 image
av_init_packet(pkt);
pkt.data = pVidData;
pkt.size = vidBytesXfered;
len = avcodec_decode_video2(c, picture, got_picture, pkt);
//len = avcodec_decode_video(c, picture, got_picture,
pVidData, vidBytesXfered);
if (len 0)
{
fprintf(stderr, Error while decoding frame\n);
exit(1);
}
#if 0
if (got_picture)
{
for(i=0; ic-height; i++)
fwrite(picture-data[0] + i *
picture-linesize[0], 1, c-width, outfile );
for(i=0; ic-height/2; i++)
fwrite(picture-data[1] + i *
picture-linesize[1], 1, c-width/2, outfile );
for(i=0; ic-height/2; i++)
fwrite(picture-data[2] + i *
picture-linesize[2], 1, c-width/2, outfile );
}
#endif
#if 1
// Did we get a video frame?
if(got_picture)
{
SDL_LockYUVOverlay(bmp);
AVPicture pict;
pict.data[0] = bmp-pixels[0];
pict.data[1] = bmp-pixels[2];
pict.data[2] = bmp-pixels[1];
pict.linesize[0] = bmp-pitches[0];
pict.linesize[1] = bmp-pitches[2];
pict.linesize[2] = bmp-pitches[1];
SwsContext * convert_ctx = sws_getContext ( c-width,
c-height, c-pix_fmt,c-width, c-height, PIX_FMT_YUV420P, SWS_BICUBIC, NULL,
NULL, NULL );
sws_scale( convert_ctx, picture-data, picture-linesize, 0,
c-height, pict.data, pict.linesize );
sws_freeContext ( convert_ctx );
/*
// Convert the image into YUV format that SDL uses
img_convert(pict, PIX_FMT_YUV420P,
(AVPicture *)picture,
c-pix_fmt,
c-width, c-height);
*/
SDL_UnlockYUVOverlay(bmp);
rect.x = 0;
rect.y = 0;
rect.w = c-width;
rect.h = c-height;
SDL_DisplayYUVOverlay(bmp, rect);
//av_free_packet(packet);
}
#endif
#if 0//only works with VC-1
// display
//av_free_packet(pkt);
// Did we get a video frame?
if(got_picture)
{
SDL_LockYUVOverlay(bmp);
memcpy(bmp-pixels[0], picture-data[0], c-width *
c-height);//y
memcpy(bmp-pixels[2], picture-data[1], (c-width *
c-height)/4);//v
memcpy(bmp-pixels[1], picture-data[2], (c-width *
c-height)/4);//u
SDL_UnlockYUVOverlay(bmp);
rect.x = 0;
rect.y = 0;
rect.w = c-width;
rect.h = c-height;
SDL_DisplayYUVOverlay(bmp, rect);//display this image
}
#endif
//should probably not be in the loop
SDL_PollEvent(event);
switch(event.type)
{
//printf();
case SDL_QUIT:
running = FALSE;
SDL_Quit();
exit(0);
break;
case SDL_KEYDOWN:
switch(event.key.keysym.sym)
{
case SDLK_ESCAPE:
case SDLK_q:
running = FALSE;
SDL_Quit();
exit(0);
break;
case SDLK_f:
toggle = !toggle;
break;
default:
break;
}
default:
break;
}
}
running = FALSE;
// Close dev handle
//..
FreeLtDll(m_ltDll);
printf(-- CloseDevice\n\n);
fclose(outfile);
fclose(outfile1);
avcodec_close(c);
av_free(c);
av_free(picture);
SDL_FreeYUVOverlay( bmp );
SDL_FreeSurface( screen );
SDL_CloseAudio();
SDL_Quit();
printf(\n);
}
-Original Message-
From: libav-user-boun...@ffmpeg.org [mailto:libav-user-boun...@ffmpeg.org] On
Behalf Of Navin
Sent: 20 November 2012 12:09
To: libav-user@ffmpeg.org
Subject: Re: [Libav-user] get RGB values from
Re: [Libav-user] get RGB values from ffmpeg frame
Actually, Malik; the idea was not to use the ffmpeg.exe. I wanted the bitmap/RGB values to be stored in memory (a datastructure) in my program and then I'd pass the pointer of the datastructure to a client program which is running at the same time. The client program would then take the info from the datastructure and use it to display the video frame. Thanks for the code you posted. Macros and datatypes like LT_SUCCESS, MAX_FR_SZ, metaD_t, LtHandle, ASF_IDX_SZ etc. seem new. Am a bit puzzled. Navin On 11/20/2012 5:00 PM, Malik Cissé wrote: Hi Navin, Also YUV4:2:0 in BMP does not seem feasible (sorry for my previous statement). One think you can do too is use ffmpeg in a system call in your code. This is very versatile (you need to replace 422 by 420 though) char ffmpegString[256];//should be plenty char fmt[] = bmp; //bmp, jpg, png, pgm etc... sprintf(ffmpegString, ffmpeg.exe -s %dx%d -vcodec rawvideo -f rawvideo -pix_fmt uyvy422 -i raw_uyvy422.yuv -f image2 raw_uyvy422.%s,RawWidth, RawHeight, fmt); system(ffmpegString); Malik Cisse ___ Libav-user mailing list Libav-user@ffmpeg.org http://ffmpeg.org/mailman/listinfo/libav-user
Re: [Libav-user] get RGB values from ffmpeg frame
Navin, Macros and datatypes like LT_SUCCESS, MAX_FR_SZ, metaD_t, LtHandle, ASF_IDX_SZ etc. seem new. Am a bit puzzled. The code is taken from a proprietary stuff and I did not want to disclose everything but the main idea is there. -Original Message- From: libav-user-boun...@ffmpeg.org [mailto:libav-user-boun...@ffmpeg.org] On Behalf Of Navin Sent: 20 November 2012 12:39 To: libav-user@ffmpeg.org Subject: Re: [Libav-user] get RGB values from ffmpeg frame Actually, Malik; the idea was not to use the ffmpeg.exe. I wanted the bitmap/RGB values to be stored in memory (a datastructure) in my program and then I'd pass the pointer of the datastructure to a client program which is running at the same time. The client program would then take the info from the datastructure and use it to display the video frame. Thanks for the code you posted. Macros and datatypes like LT_SUCCESS, MAX_FR_SZ, metaD_t, LtHandle, ASF_IDX_SZ etc. seem new. Am a bit puzzled. Navin On 11/20/2012 5:00 PM, Malik Cissé wrote: Hi Navin, Also YUV4:2:0 in BMP does not seem feasible (sorry for my previous statement). One think you can do too is use ffmpeg in a system call in your code. This is very versatile (you need to replace 422 by 420 though) char ffmpegString[256];//should be plenty char fmt[] = bmp; //bmp, jpg, png, pgm etc... sprintf(ffmpegString, ffmpeg.exe -s %dx%d -vcodec rawvideo -f rawvideo -pix_fmt uyvy422 -i raw_uyvy422.yuv -f image2 raw_uyvy422.%s,RawWidth, RawHeight, fmt); system(ffmpegString); Malik Cisse ___ Libav-user mailing list Libav-user@ffmpeg.org http://ffmpeg.org/mailman/listinfo/libav-user ___ Libav-user mailing list Libav-user@ffmpeg.org http://ffmpeg.org/mailman/listinfo/libav-user
Re: [Libav-user] get RGB values from ffmpeg frame
@Everyone: I had seen the doc/examples code and every other possible
source of information but there seems to be some problem causing the
crashes when I use RGB. Even for the below Tutorial01 code of ffmpeg for
example.
Am using the latest ffmpeg-win32-dev and I've tried with a wmv, many mp4
(from youtube, h264 (high) (avc1/0x31637661), yuv420p 1280x720,
1139kb/s, 24fps) files and avi too, but they're all crashing with
Unhandled exception, Access violation writing location 0x002a3000 at the
line sws_scale. Even in the previous complete program that I posted to
the mailing list, getting the RGB values were a problem because of a
similar crash while iterating through frame-data.
Could anyone please help out with at least this program? Is it working
on your end?
#include stdio.h
#include stdlib.h
extern C
{
#include libavcodec/avcodec.h
#include libavformat/avformat.h
#include libswscale/swscale.h
}
#includeiostream
void SaveFrame(AVFrame *pFrame, int width, int height, int iFrame)
{
FILE *pFile;
char szFilename[32];
int y;
sprintf(szFilename, frame%d.ppm, iFrame);// Open file
pFile=fopen(szFilename, wb);
if(pFile==NULL) return;
fprintf(pFile, P6\n%d %d\n255\n, width, height);// Write header
for(y=0; yheight; y++) fwrite(pFrame-data[0]+y*pFrame-linesize[0],
1, width*3, pFile);// Write pixel data
fclose(pFile);// Close file
}//SaveFrame
int main(int argc, char *argv[])
{
AVFormatContext *pFormatCtx = NULL;
int i, videoStream;
AVCodecContext *pCodecCtx = NULL;
AVCodec *pCodec = NULL;
AVFrame *pFrame = NULL;
AVFrame *pFrameRGB = NULL;
AVPacketpacket;
int frameFinished;
int numBytes;
uint8_t *buffer = NULL;
AVDictionary*optionsDict = NULL;
struct SwsContext *sws_ctx = NULL;
if(argc 2) { printf(Please provide a movie file\n); return -1; }
av_register_all();// Register all formats and codecs
if(avformat_open_input(pFormatCtx, argv[1], NULL, NULL)!=0)
return -1; // Couldn't open file
if(avformat_find_stream_info(pFormatCtx, NULL)0) return -1; //
Couldn't find stream information
// Dump information about file onto standard error
av_dump_format(pFormatCtx, 0, argv[1], 0);
// Find the first video stream
videoStream=-1;
for(i=0; ipFormatCtx-nb_streams; i++)
if(pFormatCtx-streams[i]-codec-codec_type==AVMEDIA_TYPE_VIDEO) {
videoStream=i; break; }
if(videoStream==-1) return -1; // Didn't find a video stream
// Get a pointer to the codec context for the video stream
pCodecCtx=pFormatCtx-streams[videoStream]-codec;
// Find the decoder for the video stream
pCodec=avcodec_find_decoder(pCodecCtx-codec_id);
if(pCodec==NULL) { fprintf(stderr, Unsupported codec!\n); return
-1; }// Codec not found
if(avcodec_open2(pCodecCtx, pCodec, optionsDict)0) return -1; //
Could not open codec
pFrame=avcodec_alloc_frame();// Allocate video frame
// Allocate an AVFrame structure
pFrameRGB=avcodec_alloc_frame();
if(pFrameRGB==NULL) return -1;
// Determine required buffer size and allocate buffer
numBytes=avpicture_get_size(PIX_FMT_RGB24, pCodecCtx-width,
pCodecCtx-height);
buffer=(uint8_t *)av_malloc(numBytes*sizeof(uint8_t));
sws_ctx = sws_getContext(pCodecCtx-width, pCodecCtx-height,
pCodecCtx-pix_fmt, pCodecCtx-width, pCodecCtx-height, PIX_FMT_RGB24,
SWS_BILINEAR, NULL, NULL, NULL );
// Assign appropriate parts of buffer to image planes in pFrameRGB.
Note that pFrameRGB is an AVFrame, but AVFrame is a superset of AVPicture
avpicture_fill((AVPicture *)pFrameRGB, buffer, PIX_FMT_RGB24,
pCodecCtx-width, pCodecCtx-height);
// Read frames and save first five frames to disk
i=0;
while(av_read_frame(pFormatCtx, packet)=0)
{
// Is this a packet from the video stream?
if(packet.stream_index==videoStream)
{
// Decode video frame
avcodec_decode_video2(pCodecCtx, pFrame, frameFinished, packet);
// Did we get a video frame?
if(frameFinished)
{
// Convert the image from its native format to RGB
sws_scale(sws_ctx, (uint8_t const * const *)pFrame-data,
pFrame-linesize, 0, pCodecCtx-height, pFrameRGB-data,
pFrameRGB-linesize );
// Save the frame to disk
if(++i=5) SaveFrame(pFrameRGB, pCodecCtx-width,
pCodecCtx-height, i);
}
}
// Free the packet that was allocated by av_read_frame
av_free_packet(packet);
}//while
av_free(buffer);av_free(pFrameRGB);// Free the RGB image
av_free(pFrame);// Free the YUV frame
avcodec_close(pCodecCtx);// Close the codec
avformat_close_input(pFormatCtx);// Close the video file
return 0;
}
Navin
On 11/20/2012 6:16 PM, Malik Cissé wrote:
Navin,
Macros and datatypes like LT_SUCCESS, MAX_FR_SZ, metaD_t, LtHandle, ASF_IDX_SZ
etc. seem new. Am a bit puzzled.
The code is taken from a proprietary stuff and I did not want to disclose
everything but the main idea is there.
[Libav-user] get RGB values from ffmpeg frame
Hi! Glad to be part of this mailing list.
What I wanted to create, was a program which would receive a streaming
video, and when it decodes a frame of the video into either a bitmap
format or just pure RGB (perhaps stored in a char array), it would
notify another program that it has received a frame, and the other
program would take the RGB values and display it.
I've already asked this question here:
http://ffmpeg.zeranoe.com/forum/viewtopic.php?f=15t=805
and rogerdpack told me to post my question on the libav mailing list.
I have been through many websites, but they either use img_convert
(which doesn't work) or sws_scale, which crashes when I try to use it
with RGB.
Could anyone help with a complete piece of code which can give me the
RGB values of a frame?
This is a part of the YUV conversion that I tried initially.
i=0;
while(av_read_frame(pFormatCtx, packet) = 0)
{
// Is this a packet from the video stream?
if(packet.stream_index==videoStream)
{
// Decode video frame
avcodec_decode_video2(pCodecCtx, pFrame, frameFinished, packet);
// Did we get a video frame?
if(frameFinished)
{
// Convert the image into YUV format that SDL uses
sws_scale( sws_ctx, (uint8_t const * const *)pFrame-data,
pFrame-linesize, 0, pCodecCtx-height, pict.data, pict.linesize );
--
Nav
___
Libav-user mailing list
Libav-user@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/libav-user
