convert sample rate from AV_SAMPLE_FMT_FLTP to AV_SAMPLE_FMT_S16

https://stackoverflow.com/questions/14989397/how-to-convert-sample-rate-from-av-sample-fmt-fltp-to-av-sample-fmt-s16

Ask Question

up vote6down votefavorite

11

I am decoding aac to pcm with ffmpeg with avcodec_decode_audio3. However it decodes into AV_SAMPLE_FMT_FLTP sample format (PCM 32bit Float Planar) and i need AV_SAMPLE_FMT_S16 (PCM 16 bit signed - S16LE).

I know that ffmpeg can do this easily with -sample_fmt. I want to do the same with the code but i still couldn't figure it out.

audio_resample did not work for: it fails with error message: .... conversion failed.

android-ndk ffmpeg sample pcm libav

shareimprove this question

asked Feb 20 '13 at 20:35

frankish

2,89143264

 

 

Did you ever work out the answer to this? Am facing exact same problem – Reuben Scratton Mar 12 '13 at 17:01

add a comment

3 Answers

activeoldestvotes

up vote33down voteaccepted

EDIT 9th April 2013: Worked out how to use libswresample to do this... much faster!

At some point in the last 2-3 years FFmpeg's AAC decoder's output format changed from AV_SAMPLE_FMT_S16 to AV_SAMPLE_FMT_FLTP. This means that each audio channel has it's own buffer, and each sample value is a 32-bit floating point value scaled from -1.0 to +1.0.

Whereas with AV_SAMPLE_FMT_S16 the data is in a single buffer, with the samples interleaved, and each sample is a signed integer from -32767 to +32767.

And if you really need your audio as AV_SAMPLE_FMT_S16, then you have to do the conversion yourself. I figured out two ways to do it:

1. Use libswresample (recommended)

#include "libswresample/swresample.h"...SwrContext *swr;...// Set up SWR context once you've got codec informationswr = swr_alloc();av_opt_set_int(swr, "in_channel_layout",  audioCodec->channel_layout, 0);av_opt_set_int(swr, "out_channel_layout", audioCodec->channel_layout,  0);av_opt_set_int(swr, "in_sample_rate",     audioCodec->sample_rate, 0);av_opt_set_int(swr, "out_sample_rate",    audioCodec->sample_rate, 0);av_opt_set_sample_fmt(swr, "in_sample_fmt",  AV_SAMPLE_FMT_FLTP, 0);av_opt_set_sample_fmt(swr, "out_sample_fmt", AV_SAMPLE_FMT_S16,  0);swr_init(swr);...// In your decoder loop, after decoding an audio frame:AVFrame *audioFrame = ...;int16_t* outputBuffer = ...;swr_convert(&outputBuffer, audioFrame->nb_samples, audioFrame->extended_data, audioFrame->nb_samples);   

And that's all you have to do!

2. Do it by hand in C (original answer, not recommended)

So in your decode loop, when you've got an audio packet you decode it like this:

AVCodecContext *audioCodec;   // init'd elsewhereAVFrame *audioFrame;          // init'd elsewhereAVPacket packet;              // init'd elsewhereint16_t* outputBuffer;        // init'd elsewhereint out_size = 0;...int len = avcodec_decode_audio4(audioCodec, audioFrame, &out_size, &packet);

And then, if you've got a full frame of audio, you can convert it fairly easily:

    // Convert from AV_SAMPLE_FMT_FLTP to AV_SAMPLE_FMT_S16    int in_samples = audioFrame->nb_samples;    int in_linesize = audioFrame->linesize[0];    int i=0;    float* inputChannel0 = (float*)audioFrame->extended_data[0];    // Mono    if (audioFrame->channels==1) {        for (i=0 ; i<in_samples ; i++) {            float sample = *inputChannel0++;            if (sample<-1.0f) sample=-1.0f; else if (sample>1.0f) sample=1.0f;            outputBuffer[i] = (int16_t) (sample * 32767.0f);        }    }    // Stereo    else {        float* inputChannel1 = (float*)audioFrame->extended_data[1];        for (i=0 ; i<in_samples ; i++) {             outputBuffer[i*2] = (int16_t) ((*inputChannel0++) * 32767.0f);             outputBuffer[i*2+1] = (int16_t) ((*inputChannel1++) * 32767.0f);        }    }    // outputBuffer now contains 16-bit PCM!

I've left a couple of things out for clarity... the clamping in the mono path should ideally be duplicated in the stereo path. And the code can be easily optimized.

shareimprove this answer

edited Apr 9 '13 at 21:11

answered Mar 12 '13 at 21:23

Reuben Scratton

32.3k85975

 

 

I have a related problem, this time, I need to convert S16 to S16P. Because latest ffmpeg needs S16P for libmp3lame encoding. I'll be glad if you take a look at: stackoverflow.com/questions/18131389/… – frankishAug 8 '13 at 18:01

 

Reuben, would you happen to have this code still? I'm trying to get this conversion working but I'm having some issues. I would like to see the complete working solution if you could post a link. Thanks in advance. – William Seemann Dec 24 '13 at 6:34

 

I no longer have the code for option 2... using libswresample is the only sane way to solve this problem. What are the issues you have? – Reuben Scratton Dec 24 '13 at 18:52

 

Using the code you posted above for option #1 "swr_init(swr);" fails with a -1 return code using FFmpeg 2.1. Did you encounter this? – William Seemann Dec 25 '13 at 2:49

 

No I didn't get that. Try using the 1.2 branch of FFmpeg instead, cos thats what I used. If that's not an option then look at the implementation of swr_init() in swresample.c... you'll see it logs quite a lot of error info (which allegedly goes to stderr by default). – Reuben Scratton Dec 25 '13 at 19:33

show 1 more comment

up vote4down vote

I found 2 resample function from FFMPEG. The performance maybe better.

1. avresample_convert() http://libav.org/doxygen/master/group__lavr.html
2. swr_convert() http://spirton.com/svn/MPlayer-SB/ffmpeg/libswresample/swresample_test.c

shareimprove this answer

answered Apr 8 '13 at 7:43

Albert Liao

512

 

 

You were definitely on the right lines here Albert... I had a performance complaint earlier today so had to look into finding an optimized method to do this conversion and libswresample is my new best friend. My answer above has been updated with the necessary code. – Reuben Scratton Apr 9 '13 at 21:14

add a comment

up vote1down vote

Thanks Reuben for a solution to this. I did find that some of the sample values were slightly off when compared with a straight ffmpeg -i file.wav. It seems that in the conversion, they use a round() on the value.

To do the conversion, I did what you did with a bid of modification to work for any amount of channels:

if (audioCodecContext->sample_fmt == AV_SAMPLE_FMT_FLTP){    int nb_samples = decoded_frame->nb_samples;    int channels = decoded_frame->channels;    int outputBufferLen = nb_samples & channels * 2;    short* outputBuffer = new short[outputBufferLen/2];    for (int i = 0; i < nb_samples; i++)    {         for (int c = 0; c < channels; c++)         {             float* extended_data = (float*)decoded_frame->extended_data[c];             float sample = extended_data[i];             if (sample < -1.0f) sample = -1.0f;             else if (sample > 1.0f) sample = 1.0f;             outputBuffer[i * channels + c] = (short)round(sample * 32767.0f);         }    }    // Do what you want with the data etc.}

I went from ffmpeg 0.11.1 -> 1.1.3 and found the change of sample format annoying. I looked at setting the request_sample_fmt to AV_SAMPLE_FMT_S16 but it seems the aac decoder doesn't support anything other than AV_SAMPLE_FMT_FLTP anyway.

shareimprove this answer

answered Mar 21 '13 at 22:29

Brad Mitchell

1251110

 

 

great addition, thank you – frankish Mar 22 '13 at 9:14

 

I updated my answer with a better way using libswresample. It's surprisingly easy to do. – Reuben ScrattonApr 9 '13 at 21:12

 

@BradMitchell How can we do the opposite of this one? Would you mind taking a look at stackoverflow.com/questions/18131389/… ? – frankish Aug 9 '13 at 8:30

add a comment