iPhone/iPad开发札记 2012/03/08 音频

1. MixerHost 

* setupAudioSession 

这个方法可以 commented out !

Q: Why 不影响使用playing过程 ？比如sample rate/catetory之类？

* understanding ASBD 

/*!

    @struct         AudioStreamBasicDescription

    @abstract       This structure encapsulates all the information for describing the basic

                    format properties of a stream of audio data.

    @discussion     This structure is sufficient to describe any constant bit rate format that  has

                    channels that are the same size. Extensions are required for variable bit rate

                    data and for constant bit rate data where the channels have unequal sizes.

                    However, where applicable, the appropriate fields will be filled out correctly

                    for these kinds of formats (the extra data is provided via separate properties).

                    In all fields, a value of 0 indicates that the field is either unknown, not

                    applicable or otherwise is inapproprate for the format and should be ignored.

                    Note that 0 is still a valid value for most formats in the mFormatFlags field.

                    In audio data a frame is one sample across allchannels. In non-interleaved

                    audio, the per frame fields identify one channel. In interleaved audio, the per

                    frame fields identify the set of n channels. In uncompressed audio, aPacket is

                    one frame, (mFramesPerPacket == 1). In compressed audio, a Packet is an

                    indivisible chunk of compressed data, for example an AAC packet will contain

                    1024 sample frames.

    @field          mSampleRate

                        The number of sample frames per second of the data in the stream.

    @field          mFormatID

                        A four char code indicating the general kind of data in the stream.

    @field         mFormatFlags

                        Flags specific to each format.

    @field          mBytesPerPacket

                        The number of bytes in a packet of data.

    @field          mFramesPerPacket

                        The number of sample frames in each packet of data.

    @field          mBytesPerFrame

                        The number of bytes in a single sample frame of data.

    @field          mChannelsPerFrame

                        The number of channels in each frame of data.

    @field          mBitsPerChannel

                        The number of bits of sample data for each channel in a frame of data.

    @field          mReserved

                        Pads the structure out to force an even 8 byte alignment.

*/

* Understanding readAudioFilesIntoMemory

* soundStructPtr

// Data structure for mono or stereo sound, to pass to the application's render callback function, 

//    which gets invoked by a Mixer unit input bus when it needs more audio to play.

typedefstruct {

   BOOL                 isStereo;          // set to true if there is data in the audioDataRight member

   UInt32               frameCount;        // the total number of frames in the audio data

    UInt32               sampleNumber;       // the next audio sample to play

   AudioUnitSampleType  *audioDataLeft;    // the complete left (or mono) channel of audio data read from an audio file

   AudioUnitSampleType  *audioDataRight;   // the complete right channel of audio data read from an audio file

    

} soundStruct, *soundStructPtr;

Q:为什么这个结构会传递给 render callback function ? 谁决定的？ 谁传的？

  A:在configureAndInitializeAudioProcessingGraph方法中，通过AUGraphSetNodeInputCallback设置，代码如下：2012/03/11 13:30.

        // Setup the struture that contains the input render callback 

        AURenderCallbackStruct inputCallbackStruct;

        inputCallbackStruct.inputProc        = &inputRenderCallback;

        inputCallbackStruct.inputProcRefCon  =soundStructArray;

        

       NSLog (@"Registering the render callback with mixer unit input bus %u", busNumber);

       // Set a callback for the specified node's specified input

        result =AUGraphSetNodeInputCallback (

                    processingGraph,

                     mixerNode,

                     busNumber,

                     &inputCallbackStruct

                 );

2. audio for conference 

* compile errors 

Build test555 of project test555 with configuration Debug

CompileC build/test555.build/Debug-iphonesimulator/test555.build/Objects-normal/i386/test555AppDelegate.o Classes/test555AppDelegate.m normal i386 objective-c com.apple.compilers.gcc.4_2

cd /Users/frankzou/conf-ios/test555

setenv LANG en_US.US-ASCII

setenv PATH "/Developer/Platforms/iPhoneSimulator.platform/Developer/usr/bin:/Developer/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin"

/Developer/Platforms/iPhoneSimulator.platform/Developer/usr/bin/gcc-4.2 -x objective-c -arch i386 -fmessage-length=0 -pipe -std=c99 -Wno-trigraphs -fpascal-strings -fasm-blocks -O0 -Wreturn-type -Wunused-variable -isysroot /Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator4.3.sdk -fexceptions -fvisibility=hidden -mmacosx-version-min=10.6 -gdwarf-2 -fobjc-abi-version=2 -fobjc-legacy-dispatch -D__IPHONE_OS_VERSION_MIN_REQUIRED=40300 -iquote /Users/frankzou/conf-ios/test555/build/test555.build/Debug-iphonesimulator/test555.build/test555-generated-files.hmap -I/Users/frankzou/conf-ios/test555/build/test555.build/Debug-iphonesimulator/test555.build/test555-own-target-headers.hmap -I/Users/frankzou/conf-ios/test555/build/test555.build/Debug-iphonesimulator/test555.build/test555-all-target-headers.hmap -iquote /Users/frankzou/conf-ios/test555/build/test555.build/Debug-iphonesimulator/test555.build/test555-project-headers.hmap -F/Users/frankzou/conf-ios/test555/build/Debug-iphonesimulator -I/Users/frankzou/conf-ios/test555/build/Debug-iphonesimulator/include -I/Users/frankzou/conf-ios/test555/build/test555.build/Debug-iphonesimulator/test555.build/DerivedSources/i386 -I/Users/frankzou/conf-ios/test555/build/test555.build/Debug-iphonesimulator/test555.build/DerivedSources -include /var/folders/ZO/ZO5VlmhdEja2NlA56riDYU+++TI/-Caches-/com.apple.Xcode.501/SharedPrecompiledHeaders/test555_Prefix-grzrqdfqypqwbsgcbqnvjgspkftx/test555_Prefix.pch -c /Users/frankzou/conf-ios/test555/Classes/test555AppDelegate.m -o /Users/frankzou/conf-ios/test555/build/test555.build/Debug-iphonesimulator/test555.build/Objects-normal/i386/test555AppDelegate.o

In file included from /Users/frankzou/conf-ios/test555/Classes/AQPlayer.h:62,

                 from /Users/frankzou/conf-ios/test555/Classes/test555ViewController.h:10,

                 from /Users/frankzou/conf-ios/test555/Classes/test555AppDelegate.m:10:

/Users/frankzou/conf-ios/test555/../../study/xcode-doc/SpeakHere/iPublicUtility/CAStreamBasicDescription.h:91: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'CAStreamBasicDescription'

/Users/frankzou/conf-ios/test555/../../study/xcode-doc/SpeakHere/iPublicUtility/CAStreamBasicDescription.h:298: error: expected '=', ',', ';', 'asm' or '__attribute__' before '<' token

/Users/frankzou/conf-ios/test555/../../study/xcode-doc/SpeakHere/iPublicUtility/CAStreamBasicDescription.h:299: error: expected '=', ',', ';', 'asm' or '__attribute__' before '==' token

/Users/frankzou/conf-ios/test555/../../study/xcode-doc/SpeakHere/iPublicUtility/CAStreamBasicDescription.h:301: error: expected '=', ',', ';', 'asm' or '__attribute__' before '!=' token

/Users/frankzou/conf-ios/test555/../../study/xcode-doc/SpeakHere/iPublicUtility/CAStreamBasicDescription.h:302: error: expected '=', ',', ';', 'asm' or '__attribute__' before '<=' token

/Users/frankzou/conf-ios/test555/../../study/xcode-doc/SpeakHere/iPublicUtility/CAStreamBasicDescription.h:303: error: expected '=', ',', ';', 'asm' or '__attribute__' before '>=' token

/Users/frankzou/conf-ios/test555/../../study/xcode-doc/SpeakHere/iPublicUtility/CAStreamBasicDescription.h:304: error: expected '=', ',', ';', 'asm' or '__attribute__' before '>' token

/Users/frankzou/conf-ios/test555/../../study/xcode-doc/SpeakHere/iPublicUtility/CAStreamBasicDescription.h:307: error: expected ';', ',' or ')' before '&' token

In file included from /Users/frankzou/conf-ios/test555/Classes/AQPlayer.h:63,

                 from /Users/frankzou/conf-ios/test555/Classes/test555ViewController.h:10,

                 from /Users/frankzou/conf-ios/test555/Classes/test555AppDelegate.m:10:

/Users/frankzou/conf-ios/test555/../../study/xcode-doc/SpeakHere/iPublicUtility/CAXException.h:65: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'CAX4CCString'

/Users/frankzou/conf-ios/test555/../../study/xcode-doc/SpeakHere/iPublicUtility/CAXException.h:87: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'CAXException'

In file included from /Users/frankzou/conf-ios/test555/Classes/test555ViewController.h:10,

                 from /Users/frankzou/conf-ios/test555/Classes/test555AppDelegate.m:10:

/Users/frankzou/conf-ios/test555/Classes/AQPlayer.h:67: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'AQPlayer'

In file included from /Users/frankzou/conf-ios/test555/Classes/test555AppDelegate.m:10:

/Users/frankzou/conf-ios/test555/Classes/test555ViewController.h:14: error: expected specifier-qualifier-list before 'AQPlayer'

ROOT CAUSE: 

all files referencing c++ code should use prefix *.mm

#import<UIKit/UIKit.h>

#include"AQPlayer.h"

            ^ This is a C++ code. 

* LINK error 

Ld build/Debug-iphonesimulator/test555.app/test555 normal i386

cd /Users/frankzou/conf-ios/test555

setenv MACOSX_DEPLOYMENT_TARGET 10.6

setenv PATH "/Developer/Platforms/iPhoneSimulator.platform/Developer/usr/bin:/Developer/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin"

/Developer/Platforms/iPhoneSimulator.platform/Developer/usr/bin/g++-4.2 -arch i386 -isysroot /Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator4.3.sdk -L/Users/frankzou/conf-ios/test555/build/Debug-iphonesimulator -L/Users/frankzou/conf-ios/test555/../share/static_libs -L/Users/frankzou/conf-ios/test555 -F/Users/frankzou/conf-ios/test555/build/Debug-iphonesimulator -filelist /Users/frankzou/conf-ios/test555/build/test555.build/Debug-iphonesimulator/test555.build/Objects-normal/i386/test555.LinkFileList -mmacosx-version-min=10.6 -Xlinker -objc_abi_version -Xlinker 2 -framework Foundation -framework UIKit -framework CoreGraphics -lspeexdsp -lspeex -lsecurity -lsdcommon -lsdclient -lrtppac -lnetwork -linfosessioncommon -lH264Decode -lGcClientPdu -lframework -lfilemanager -lconnmgr -lConfJson -lconfcli -lapppdu -lAnnotation -lconference -o /Users/frankzou/conf-ios/test555/build/Debug-iphonesimulator/test555.app/test555

ld: warning: directory '/Users/frankzou/conf-ios/test555/../share/static_libs' following -L not found

Undefined symbols:

  "AQPlayer::StartQueue(signed char)", referenced from:

      -[test555ViewController onHear] in test555ViewController.o

  "AQPlayer::InitNewPlaybackQueue()", referenced from:

      -[test555ViewController onHear] in test555ViewController.o

  "AQPlayer::AQPlayer()", referenced from:

      -[test555ViewController initPlayer] in test555ViewController.o

ld: symbol(s) not found

collect2: ld returned 1 exit status

ROOT CAUSE:  AQPlayer.mm没有纳入Targets/test555/compile source(s) 中

3. MixerHost continues 

* inputRenderCallBack method

static OSStatus inputRenderCallback (

   void                        *inRefCon,     // A pointer to a struct containing the complete audio data 

                                               //    to play, as well as state information such as the  

                                               //    first sample to play on this invocation of the callback.

   AudioUnitRenderActionFlags  *ioActionFlags,// Unused here. When generating audio, use ioActionFlags to indicate silence 

                                               //    between sounds; for silence, also memset the ioData buffers to 0.

    const AudioTimeStamp        *inTimeStamp,  // Unused here.

   UInt32                      inBusNumber,   // The mixer unit input bus that is requesting some new

                                               //        frames of audio data to play.

   UInt32                      inNumberFrames,// The number of frames of audio to provide to the buffer(s)

                                               //        pointed to by the ioData parameter.

   AudioBufferList             *ioData        // On output, the audio data to play. The callback's primary 

                                               //        responsibility is to fill the buffer(s) in the 

                                               //        AudioBufferList.

) {

Q: inNumberFrames: 如果需要的数量网络层还未传过来，如何处理？

 

Ｑ: inRefCon是在生成AU Grauph的时候注册的？不变吗？

    A: 是的，有完整的需要播放的audio data（已经从文件中读到内存中），也有一些状态信息（如当前正在播放的sample编号）。在其它程序中，可能有自已定义的其它意义。  2012/03/11

Ｑ：从网络传来的audio data应该通过什么东东传进来？inRefCon ?

    A: 不必，直接call下层API去获取即可。 2012/03/11