ffmpeg video to opengl texture
来源:互联网 发布:淘宝网德国过滤水壶 编辑:程序博客网 时间:2024/05/18 21:41
I'm trying to render frames grabbed and converted from a video using ffmpeg to an opengl texture to be put on a quad. I've pretty much exhausted google and not found an answer, well I've found answers but none of them seem to have worked.
Basically, I am using avcodec_decode_video2() to decode the frame and then sws_scale() to convert the frame to RGB and then glTexSubImage2D() to create an openGL texture from it but can't seem to get anything to work.
I've made sure the "destination" AVFrame has power of 2 dimensions in the SWS Context setup. Here is my code:
SwsContext *img_convert_ctx = sws_getContext(pCodecCtx->width, pCodecCtx->height, pCodecCtx->pix_fmt, 512, 256, PIX_FMT_RGB24, SWS_BICUBIC, NULL, NULL, NULL);//While still frames to readwhile(av_read_frame(pFormatCtx, &packet)>=0) { glClear(GL_COLOR_BUFFER_BIT); //If the packet is from the video stream if(packet.stream_index == videoStream) { //Decode the video avcodec_decode_video2(pCodecCtx, pFrame, &frameFinished, &packet); //If we got a frame then convert it and put it into RGB buffer if(frameFinished) { printf("frame finished: %i\n", number); sws_scale(img_convert_ctx, pFrame->data, pFrame->linesize, 0, pCodecCtx->height, pFrameRGB->data, pFrameRGB->linesize); glBindTexture(GL_TEXTURE_2D, texture); //gluBuild2DMipmaps(GL_TEXTURE_2D, 3, pCodecCtx->width, pCodecCtx->height, GL_RGB, GL_UNSIGNED_INT, pFrameRGB->data); glTexSubImage2D(GL_TEXTURE_2D, 0, 0,0, 512, 256, GL_RGB, GL_UNSIGNED_BYTE, pFrameRGB->data[0]); SaveFrame(pFrameRGB, pCodecCtx->width, pCodecCtx->height, number); number++; } } glColor3f(1,1,1); glBindTexture(GL_TEXTURE_2D, texture); glBegin(GL_QUADS); glTexCoord2f(0,1); glVertex3f(0,0,0); glTexCoord2f(1,1); glVertex3f(pCodecCtx->width,0,0); glTexCoord2f(1,0); glVertex3f(pCodecCtx->width, pCodecCtx->height,0); glTexCoord2f(0,0); glVertex3f(0,pCodecCtx->height,0); glEnd();
As you can see in that code, I am also saving the frames to .ppm files just to make sure they are actually rendering, which they are.
The file being used is a .wmv at 854x480, could this be the problem? The fact I'm just telling it to go 512x256?
P.S. I've looked at this stackoverflow question but it didn't help.
Also, I have glEnable(GL_TEXTURE_2D); as well and have tested it by just loading in a normal bmp.
EDIT
I'm getting an image on the screen now but it is a garbled mess, I'm guessing something to do with changing things to a power of 2 (in the decode, swscontext and gluBuild2DMipmaps as shown in my code). I'm usually nearly exactly the same code as shown above, only I've changed glTexSubImage2D to gluBuild2DMipmaps and changed the types to GL_RGBA.
Here is what the frame looks like:
EDIT AGAIN
Just realised I haven't showed the code for how pFrameRGB is set up:
//Allocate video frame for 24bit RGB that we convert to.AVFrame *pFrameRGB;pFrameRGB = avcodec_alloc_frame();if(pFrameRGB == NULL) { return -1;}//Allocate memory for the raw data we get when converting.uint8_t *buffer;int numBytes;numBytes = avpicture_get_size(PIX_FMT_RGB24, pCodecCtx->width, pCodecCtx->height);buffer = (uint8_t *) av_malloc(numBytes*sizeof(uint8_t));//Associate frame with our bufferavpicture_fill((AVPicture *) pFrameRGB, buffer, PIX_FMT_RGB24, pCodecCtx->width, pCodecCtx->height);
Now that I ahve changed the PixelFormat in avgpicture_get_size to PIX_FMT_RGB24, I've done that in SwsContext as well and changed GluBuild2DMipmaps to GL_RGB and I get a slightly better image but it looks like I'm still missing lines and it's still a bit stretched:
Another Edit
After following Macke's advice and passing the actual resolution to OpenGL I get the frames nearly proper but still a bit skewed and in black and white, also it's only getting 6fps now rather than 110fps:
I'm trying to render frames grabbed and converted from a video using ffmpeg to an opengl texture to be put on a quad. I've pretty much exhausted google and not found an answer, well I've found answers but none of them seem to have worked.
Basically, I am using avcodec_decode_video2() to decode the frame and then sws_scale() to convert the frame to RGB and then glTexSubImage2D() to create an openGL texture from it but can't seem to get anything to work.
I've made sure the "destination" AVFrame has power of 2 dimensions in the SWS Context setup. Here is my code:
SwsContext *img_convert_ctx = sws_getContext(pCodecCtx->width, pCodecCtx->height, pCodecCtx->pix_fmt, 512, 256, PIX_FMT_RGB24, SWS_BICUBIC, NULL, NULL, NULL);//While still frames to readwhile(av_read_frame(pFormatCtx, &packet)>=0) { glClear(GL_COLOR_BUFFER_BIT); //If the packet is from the video stream if(packet.stream_index == videoStream) { //Decode the video avcodec_decode_video2(pCodecCtx, pFrame, &frameFinished, &packet); //If we got a frame then convert it and put it into RGB buffer if(frameFinished) { printf("frame finished: %i\n", number); sws_scale(img_convert_ctx, pFrame->data, pFrame->linesize, 0, pCodecCtx->height, pFrameRGB->data, pFrameRGB->linesize); glBindTexture(GL_TEXTURE_2D, texture); //gluBuild2DMipmaps(GL_TEXTURE_2D, 3, pCodecCtx->width, pCodecCtx->height, GL_RGB, GL_UNSIGNED_INT, pFrameRGB->data); glTexSubImage2D(GL_TEXTURE_2D, 0, 0,0, 512, 256, GL_RGB, GL_UNSIGNED_BYTE, pFrameRGB->data[0]); SaveFrame(pFrameRGB, pCodecCtx->width, pCodecCtx->height, number); number++; } } glColor3f(1,1,1); glBindTexture(GL_TEXTURE_2D, texture); glBegin(GL_QUADS); glTexCoord2f(0,1); glVertex3f(0,0,0); glTexCoord2f(1,1); glVertex3f(pCodecCtx->width,0,0); glTexCoord2f(1,0); glVertex3f(pCodecCtx->width, pCodecCtx->height,0); glTexCoord2f(0,0); glVertex3f(0,pCodecCtx->height,0); glEnd();
As you can see in that code, I am also saving the frames to .ppm files just to make sure they are actually rendering, which they are.
The file being used is a .wmv at 854x480, could this be the problem? The fact I'm just telling it to go 512x256?
P.S. I've looked at this stackoverflow question but it didn't help.
Also, I have glEnable(GL_TEXTURE_2D); as well and have tested it by just loading in a normal bmp.
EDIT
I'm getting an image on the screen now but it is a garbled mess, I'm guessing something to do with changing things to a power of 2 (in the decode, swscontext and gluBuild2DMipmaps as shown in my code). I'm usually nearly exactly the same code as shown above, only I've changed glTexSubImage2D to gluBuild2DMipmaps and changed the types to GL_RGBA.
Here is what the frame looks like:
EDIT AGAIN
Just realised I haven't showed the code for how pFrameRGB is set up:
//Allocate video frame for 24bit RGB that we convert to.AVFrame *pFrameRGB;pFrameRGB = avcodec_alloc_frame();if(pFrameRGB == NULL) { return -1;}//Allocate memory for the raw data we get when converting.uint8_t *buffer;int numBytes;numBytes = avpicture_get_size(PIX_FMT_RGB24, pCodecCtx->width, pCodecCtx->height);buffer = (uint8_t *) av_malloc(numBytes*sizeof(uint8_t));//Associate frame with our bufferavpicture_fill((AVPicture *) pFrameRGB, buffer, PIX_FMT_RGB24, pCodecCtx->width, pCodecCtx->height);
Now that I ahve changed the PixelFormat in avgpicture_get_size to PIX_FMT_RGB24, I've done that in SwsContext as well and changed GluBuild2DMipmaps to GL_RGB and I get a slightly better image but it looks like I'm still missing lines and it's still a bit stretched:
Another Edit
After following Macke's advice and passing the actual resolution to OpenGL I get the frames nearly proper but still a bit skewed and in black and white, also it's only getting 6fps now rather than 110fps:
P.S.
I've got a function to save the frames to image after sws_scale() and they are coming out fine as colour and everything so something in OGL is making it B&W.
LAST EDIT
Working! Okay I have it working now, basically I am not padding out the texture to a power of 2 and just using the resolution the video is.
I got the texture showing up properly with a lucky guess at the correct glPixelStorei()
glPixelStorei(GL_UNPACK_ALIGNMENT, 2);
Also, if anyone else has the subimage() showing blank problem like me, you have to fill the texture at least once with glTexImage2D() and so I use it once in the loop and then use glTexSubImage2D() after that.
Thanks Macke and datenwolf for all your help.
转帖:http://stackoverflow.com/questions/6495523/ffmpeg-video-to-opengl-texture
- ffmpeg video to opengl texture
- Converting FFmpeg frame to OpenGL ES texture
- OpenGL: Render to Texture
- OpenGL 中的 Render To Texture
- opengl es 2.0 Reder to texture iOS
- OpenGL&CG技术之Render To Texture
- Encoding images to video with ffmpeg
- OpenGL ES 帧缓冲对象(FBO):Render to texture
- Video Cropping with Texture View
- OpenGL ES texture fonts
- Skia OpenGL Texture Helloworld
- opengl texture map
- jpeg转换OpenGL Texture
- png转换OpenGL Texture
- openGL CG4 - Lighting + Texture
- OpenGL 练习12 texture
- Opengl备忘4---Texture
- OpenGL Texture 纹理
- 最常用的也是最容易忘记的Shell知识
- android样式学习(二) android给listview设置分割线Divider样式
- 座右铭!!
- 追求卓越!
- myEclipse使用(一)MyEclipse不编译的解决办法总结
- ffmpeg video to opengl texture
- 二叉树的基本功能实现(c++)
- 配置
- android与数据库交互,传递实体类
- windows程序设计学习笔记(第三章) 窗口与消息
- 你选择了你的记忆,和生活。
- 营销的本质
- Converting FFmpeg frame to OpenGL ES texture
- 現實不等於真相