语音识别，语义理解一站式解决（android平台&olami sdk）

语音记账demo：http://blog.csdn.net/ls0609/article/details/72765789

olami sdk实现了把录音或者文字转化为用户可以理解的json字符串从而实现语义理解，用户可以定义自己的语义，是不是很强大？本文讲述怎么自定义语义，以及如何解析自定义语义。本文使用olami sdk做了一个在线听书的demo，用户只需类似“我想听***”就能实现听书的在线查找并播放。用的是喜马拉雅的在线听书sdk.基于eclipse开发环境，libs目录下jar和so文件如下：olami-android-sdk.jar //olami sdk 的jarafinal_0.5.1_bin.jarlitepal.jargson-2.2.4.jarokhttp-2.4.0.jarokhttp-urlconnection-2.2.0.jarokio-1.4.0.jaropensdk.jar          //上面这几个都是喜马拉雅需要的jarlibspeex.so          //olami sdk 需要用到speex压缩功能libxmediaplayer.so   // 喜马拉雅solibxmediaplayer_x.so // 喜马拉雅so

概述： 
VoiceSdkService中定义了OlamiVoiceRecognizer语音识别引擎，通过点击MusicActivity的开始button启动录音，录音结果在VoiceSdkService中的onResult()回调中拿到识别的Json字符串，在processServiceMessage()函数中处理后找到要听书的名称，然后进入BookUtil进行搜索，搜索到结果后通知VoiceSdkService进行播放，并通知MusicActivity更新播放进度等信息。

1.AndroidManifest.xml配置

<?xml version="1.0" encoding="utf-8"?>     <manifest xmlns:android="http://schemas.android.com/apk/res/android"        package="com.olami.musicdemo"        android:versionCode="1"        android:versionName="1.0" >        <uses-sdk            android:minSdkVersion="8"            android:targetSdkVersion="21" />        <uses-permission android:name="android.permission.RECORD_AUDIO"/>         <uses-permission android:name="android.permission.INTERNET"/>         <uses-permission android:name="android.permission.ACCESS_NETWORK_STATE"/>         <uses-permission android:name="android.permission.ACCESS_WIFI_STATE"/>         <uses-permission android:name="android.permission.READ_PHONE_STATE"/>        <uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />        <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />        <application            android:name="com.olami.musicdemo.OlamiApplication"            android:allowBackup="true"            android:icon="@drawable/ic_launcher"            android:label="@string/app_name"            android:theme="@style/AppTheme" >            <!--喜马拉雅听书测试账号app_key-->            <meta-data                android:name="app_key"                android:value="b617866c20482d133d5de66fceb37da3" />            <!--喜马拉雅听书测试账号包名-->            <meta-data                android:name="pack_id"                android:value="com.app.test.android" />            <activity                android:name=".MusicActivity"                android:label="@string/app_name" >                <intent-filter>                    <action android:name="android.intent.action.MAIN" />                    <category android:name="android.intent.category.LAUNCHER" />                </intent-filter>            </activity>             <!--注册olami sdk service-->            <service                android:name=".VoiceSdkService"                android:exported="true" >            </service>            <!--注册喜马拉雅听书service-->             <service                              android:name=             "com.ximalaya.ting.android.opensdk.player.service.XmPlayerService"            />        </application>    </manifest>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59

2.layout布局文件

layout_musicview.xml

TextView 有两个，tv_name显示听书的名称, tv_totoal_time显示听书的总时间。 
ProgressBar 实时刷新显示听书的进度

<RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"  xmlns:tools="http://schemas.android.com/tools"  android:layout_width="fill_parent"  android:layout_height="wrap_content"  android:background="@android:color/transparent">    <TextView        android:text="name"        android:layout_width="wrap_content"        android:layout_height="wrap_content"        android:layout_marginTop="5dp"        android:layout_centerHorizontal="true"        android:id="@+id/tv_name"/>    <ProgressBar        style="?android:attr/progressBarStyleHorizontal"        android:layout_width="fill_parent"        android:layout_height="wrap_content"        android:layout_below="@+id/tv_name"        android:layout_marginTop="10dp"        android:layout_marginLeft="20dp"        android:layout_marginRight="20dp"        android:id="@+id/progressbar_music"/>    <TextView        android:text="total_time"        android:layout_width="wrap_content"        android:layout_height="wrap_content"        android:layout_below="@+id/progressbar_music"        android:layout_marginTop="10dp"        android:layout_centerHorizontal="true"        android:id="@+id/tv_total_time"/></RelativeLayout>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

activity_music.xml

<RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"    xmlns:tools="http://schemas.android.com/tools"    android:layout_width="match_parent"    android:layout_height="match_parent"    android:paddingLeft="@dimen/activity_horizontal_margin"    android:paddingRight="@dimen/activity_horizontal_margin"    tools:context="com.olami.musicdemo.MusicActivity" >    <TextView        android:id="@+id/tv_inputText"        android:layout_width="wrap_content"        android:layout_height="wrap_content"        android:layout_marginTop="5dp"        android:text="输入：" />    <TextView        android:id="@+id/tv_volume"        android:layout_width="wrap_content"        android:layout_height="wrap_content"        android:layout_alignLeft="@+id/tv_inputText"        android:layout_below="@+id/tv_inputText"        android:layout_marginTop="40dp"        android:text="音量：" />    <TextView        android:id="@+id/tv_result"        android:layout_width="wrap_content"        android:layout_height="wrap_content"        android:layout_below="@+id/tv_volume"        android:layout_marginTop="20dp"        android:maxLines="15"        android:ellipsize="end"        android:text="服务器返回sentence："        android:visibility="visible" />    <com.olami.musicdemo.MusicView        android:id="@+id/music_view"        android:layout_width="fill_parent"        android:layout_height="80dp"        android:layout_centerInParent="true"        >    </com.olami.musicdemo.MusicView>    <EditText        android:id="@+id/et_content"        android:layout_width="wrap_content"        android:layout_height="40dp"        android:layout_above="@+id/btn_stop"        android:layout_alignLeft="@+id/tv_inputText"        android:layout_marginBottom="10dp"        android:layout_toLeftOf="@+id/btn_send"        android:background="#E7E7E7"        android:singleLine="true"        android:text="上海的天气" />         <Button        android:id="@+id/btn_send"        android:layout_width="wrap_content"        android:layout_height="wrap_content"        android:layout_alignBaseline="@+id/et_content"        android:layout_alignBottom="@+id/et_content"        android:layout_alignParentRight="true"        android:text="提交" />    <Button        android:id="@+id/btn_start"        android:layout_width="wrap_content"        android:layout_height="wrap_content"          android:layout_alignParentBottom="true"        android:layout_centerHorizontal="true"        android:text="开始" />    <Button        android:id="@+id/btn_stop"        android:layout_width="wrap_content"        android:layout_height="wrap_content"        android:layout_alignLeft="@+id/et_content"        android:layout_alignParentBottom="true"        android:text="停止" />    <Button        android:id="@+id/btn_cancel"        android:layout_width="wrap_content"        android:layout_height="wrap_content"        android:layout_alignParentBottom="true"        android:layout_toRightOf="@+id/et_content"        android:text="取消" /></RelativeLayout>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89

自定义MusicView比较简单，代码如下：

public class MusicView extends RelativeLayout{    private Context mContext;    private Handler mHandler;    private TextView mTextViewName;    private TextView mTextViewTotalTime;    private ProgressBar mProgressBar;    public MusicView(Context context,AttributeSet attrs) {        super(context,attrs);        LayoutInflater inflater =(LayoutInflater) context.getSystemService(                                            context.LAYOUT_INFLATER_SERVICE);        RelativeLayout view = (RelativeLayout) inflater.inflate(R.layout.layout_musicview, this,true);        mTextViewName = (TextView) view.findViewById(R.id.tv_name);        mTextViewTotalTime = (TextView) view.findViewById(R.id.tv_total_time);        mProgressBar = (ProgressBar)view.findViewById(R.id.progressbar_music);    }    public void initMusicView(Context context,Handler handler)    {        mContext = context;        mHandler = handler;    }    public void setMusicName(String name)    {//设置播放名称        mTextViewName.setText(name);    }    public void setProgress(int progress)    {//设置播放进度        mProgressBar.setProgress(progress);    }    public void setTotalTime(String time)    {//设置播放总时间        mTextViewTotalTime.setText(time);    }}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

布局效果图如下： 

3.MusicActivity和VoiceSdkService通信

本文没有用bind service的方式实现activity和service的消息通信。 
MusicAcitity 和 VoiceSdkService中分别实现了一个CommunicationAssist的接口

public interface CommunicationAssist {    public void callBack(int what, int arg1, int arg2,Bundle data, Object obj);}
1
2
3
1
2
3

然后把他们分别实现CommunicationAssist接口的变量注册到OlamiApplication，这样通过OlamiApplication实现了MusicAcitity 和 VoiceSdkService桥接。

3.1OlamiApplication

1) 注册MusicActivity到VoiceSdkService的回调

public void setActivityToServiceListener(CommunicationAssist listener){   ActivityToServiceListener = listener;}
1
2
3
4
1
2
3
4

这个是在VoiceSdkService中调用setActivityToServiceListener(),把VoiceSdkService中的VoiceSdkComAssist注册到application中，MusicActivity中可以通过getActivityToServiceListener 
这个函数回调向VoiceSdkService发送消息。

2) 注册 VoiceSdkService到MusicActivity回调

public void setServiceToActivityListener(CommunicationAssist listener){  mServiceToActivityListener = listener;}
1
2
3
4
1
2
3
4

这个是在MusicAcitivity中调用setServiceToActivityListener(),这样在VoiceSdkService中就可以通过getServiceToActivityListener()获得回调向MusciActivity发送消息。

3.2 MusicActivity

public class MusicActivity extends Activity {    private Handler mHandler;    private Handler mInComingHandler;    private ActivityComAssist mActivityComAssist;    private Button mBtnStart;    private Button mBtnStop;    private Button mBtnCancel;    private Button mBtnSend;    private EditText mEditText;    private TextView mTextView;    private TextView mInputTextView;    private TextView mTextViewVolume;    private BookUtil mBookUtil = null;    private MusicView mMusicView = null;    @Override    protected void onCreate(Bundle savedInstanceState) {        super.onCreate(savedInstanceState);        setContentView(R.layout.activity_music);        initHandler();//初始化handler用于内部消息处理        initInComingHandler();//用于处理来自VoiceSdkService的消息        initCommunicationAssist();//向application注册消息回调，VoiceSdkSerive可以                //通过getServiceToActivityListener()获得回调向MusicActivity发送消息        initView();//初始化view控件        Intent intent = new Intent();        intent.setClass(MusicActivity.this, VoiceSdkService.class);        startService(intent);//启动后台服务    }    private void initView()    {        mBtnStart = (Button) findViewById(R.id.btn_start);        mBtnStop = (Button) findViewById(R.id.btn_stop);        mBtnCancel = (Button) findViewById(R.id.btn_cancel);        mBtnSend = (Button) findViewById(R.id.btn_send);        mInputTextView = (TextView) findViewById(R.id.tv_inputText);        mEditText = (EditText) findViewById(R.id.et_content);        mTextView = (TextView) findViewById(R.id.tv_result);        mTextViewVolume = (TextView) findViewById(R.id.tv_volume);        mBtnStart.setOnClickListener(new OnClickListener(){            @Override            public void onClick(View v) {                sendMessageToService(MessageConst.CLIENT_ACTION_START_RECORED,0,0,null,null);            }                   });        mBtnStop.setOnClickListener(new OnClickListener(){            @Override            public void onClick(View v) {                sendMessageToService(MessageConst.CLIENT_ACTION_STOP_RECORED,0,0,null,null);                mBtnStart.setText("开始");                Log.i("led","MusicActivity mBtnStop onclick 开始");            }                   });        mBtnCancel.setOnClickListener(new OnClickListener(){            @Override            public void onClick(View v) {                sendMessageToService(MessageConst.CLIENT_ACTION_CANCEL_RECORED,0,0,null,null);            }                   });        mBtnSend.setOnClickListener(new OnClickListener(){            @Override                public void onClick(View v) {            sendMessageToService(            MessageConst.CLIENT_ACTION_SENT_TEXT,0,0,null,mEditText.getText());                mInputTextView.setText("文字: "+mEditText.getText());            }                   });        mMusicView = (MusicView) findViewById(R.id.music_view);        //if(mMusicView != null)            //mMusicView.initMusicView(MusicActivity.this,mHandler);    }    private void initHandler()    {        mHandler = new Handler(){            @Override            public void handleMessage(Message msg)            {                switch (msg.what){                case MessageConst.CLIENT_ACTION_START_RECORED:                    break;                default:                    break;                  }            }        };    }    //InComingHandler 收到来自VoiceSdkService的消息用于更新界面，    //包括开始录音，结束录音，播放的书的名称和进度，音量等信息。    private void initInComingHandler()    {        mInComingHandler = new Handler(){            @Override            public void handleMessage(Message msg)            {                switch (msg.what){                case MessageConst.CLIENT_ACTION_START_RECORED:                    mBtnStart.setText("录音中");                    Log.i("led","MusicActivity 录音中");                    break;                case MessageConst.CLIENT_ACTION_STOP_RECORED:                    mBtnStart.setText("识别中");                    Log.i("led","MusicActivity 识别中");                    break;                case MessageConst.CLIENT_ACTION_CANCEL_RECORED:                    mBtnStart.setText("开始");                    mTextView.setText("已取消");                    break;                case MessageConst.CLIENT_ACTION_ON_ERROR:                    mTextView.setText("错误代码："+msg.arg1);                    mBtnStart.setText("开始");                    break;                case MessageConst.CLIENT_ACTION_UPDATA_VOLUME:                    mTextViewVolume.setText("音量: "+msg.arg1);                    break;                case MessageConst.SERVER_ACTION_RETURN_RESULT:                    //mTextView.setText(msg.obj.toString());                    mBtnStart.setText("开始");                    break;                case MessageConst.CLIENT_ACTION_PLAY_BOOK_AFTER_SEARCH:                    mBtnStart.setText("开始");                    mBookUtil = BookUtil.getInstance();                    mBookUtil.play(msg.arg1);                    break;                case MessageConst.CLIENT_ACTION_UPDATA_PLAYING_BOOK_NAME:                    mMusicView.setMusicName(msg.obj.toString());                    break;                case MessageConst.CLIENT_ACTION_UPDATE_BOOK_PROGRESS:                    int current = msg.arg1;                    int duration = msg.arg2;                    mMusicView.setProgress(current*100/duration);                    float time = duration/1000/60;                    mMusicView.setTotalTime("总时间："+time);                    break;                case MessageConst.CLIENT_ACTION_UPDATA_INPUT_TEXT:                    if(msg.obj != null)                       mInputTextView.setText("文字: "+msg.obj.toString());                    break;                case MessageConst.CLIENT_ACTION_UPDATA_SERVER_MESSAGE:                    if(msg.obj != null)                        mTextView.setText("服务器返回sentence: "+msg.obj.toString());                    break;                default:                    break;                }            }        };    }    private void initCommunicationAssist()    {//向Application注册VoiceSdkService到MusicActivity的回调        mActivityComAssist = new ActivityComAssist();        OlamiApplication.getInstance().setServiceToActivityListener(mActivityComAssist);    }    private void sendMessageToService(int what, int arg1, int arg2, Bundle data, Object obj)    {//向VoiceSdkService发送消息        if(OlamiApplication.getInstance().getActivityToServiceListener() != null)            OlamiApplication.getInstance().getActivityToServiceListener().callBack            (what, arg1, arg2, data, obj);    }    private class ActivityComAssist implements CommunicationAssist{    //实现CommunicationAssist借口，用于回调VoiceSdkService发送过来的消息        @Override        public void callBack(int what, int arg1, int arg2, Bundle data,Object obj) {            Message msg = Message.obtain(null, what);            msg.arg1 = arg1;            msg.arg2 = arg2;            if (data != null)                msg.setData(data);            if (obj != null)                msg.obj = obj;            mInComingHandler.sendMessage(msg);        }           }    @Override    public void onDestroy() {        //退出应用，停止VoiceSdkService，会进行资源的释放        super.onDestroy();        Intent intent = new Intent();        intent.setClass(MusicActivity.this, VoiceSdkService.class);        stopService(intent);    }}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198

3.3 VoiceSdkService

@Override    public void onCreate() {        initHandler();//用于内部消息处理        initInComingHandler();//用于处理来自MusicActivity的消息        initCommunicationAssist();//向application注册消息回调，这样MusicActivity可        //以通过getActivityToServiceListener()回调向VoiceSdkService发送消息        initViaVoiceRecognizerListener();//初始化录音识别回调listener        init();//olami录音识别引擎初始化        initXmly();//喜马拉雅初始化    }
1
2
3
4
5
6
7
8
9
10
1
2
3
4
5
6
7
8
9
10

public void init(){    initHandler();    mOlamiVoiceRecognizer = new OlamiVoiceRecognizer(VoiceSdkService.this);    TelephonyManager telephonyManager=(TelephonyManager) this.getSystemService(    (this.getBaseContext().TELEPHONY_SERVICE);    String imei=telephonyManager.getDeviceId();    mOlamiVoiceRecognizer.init(imei);//设置身份标识，可以填null    mOlamiVoiceRecognizer.setListener(mOlamiVoiceRecognizerListener);//设置识别结果回调listener    mOlamiVoiceRecognizer.setLocalization(    OlamiVoiceRecognizer.LANGUAGE_SIMPLIFIED_CHINESE);//设置支持的语音类型，优先选择中文简体    mOlamiVoiceRecognizer.setAuthorization("51a4bb56ba954655a4fc834bfdc46af1",                            "asr","68bff251789b426896e70e888f919a6d","nli");      //注册Appkey，在olami官网注册应用后生成的appkey    //注册api，请直接填写“asr”，标识语音识别类型    //注册secret，在olami官网注册应用后生成的secret    //注册seq ，请填写“nli”    mOlamiVoiceRecognizer.setVADTailTimeout(2000);//录音时尾音结束时间，建议填//2000ms    //设置经纬度信息，不愿上传位置信息，可以填0     mOlamiVoiceRecognizer.setLatitudeAndLongitude(31.155364678184498,121.34882432933009); }
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

定义OlamiVoiceRecognizerListener

onError(int errCode)//出错回调，可以对比官方文档错误码看是什么错误 
onEndOfSpeech()//录音结束 
onBeginningOfSpeech()//录音开始 
onResult(String result, int type)//result是识别结果JSON字符串 
onCancel()//取消识别，不会再返回识别结果 
onUpdateVolume(int volume)//录音时的音量，1-12个级别大小音量

下面是VoiceSdkService完整代码：

public class VoiceSdkService extends Service{    private Handler mHandler;    private Handler mInComingHandler;    private VoiceSdkComAssist mVoiceSdkComAssist;    private OlamiVoiceRecognizer mOlamiVoiceRecognizer;    private OlamiVoiceRecognizerListener mOlamiVoiceRecognizerListener;    private BookUtil mBookUtil = null;    private boolean mIsRecordPause = false;    @Override    public void onCreate() {        initHandler();        initInComingHandler();        initCommunicationAssist();        initViaVoiceRecognizerListener();        init();        initXmly();    }    @Override    public IBinder onBind(Intent intent) {        // TODO Auto-generated method stub        return null;    }    public void init()    {        initHandler();        mOlamiVoiceRecognizer = new OlamiVoiceRecognizer(VoiceSdkService.this);        TelephonyManager telephonyManager=(TelephonyManager) this.getSystemService(        this.getBaseContext().TELEPHONY_SERVICE);        String imei=telephonyManager.getDeviceId();        mOlamiVoiceRecognizer.init(imei);//set null if you do not want to notify olami server.        mOlamiVoiceRecognizer.setListener(mOlamiVoiceRecognizerListener);        mOlamiVoiceRecognizer.setLocalization(                                         OlamiVoiceRecognizer.LANGUAGE_SIMPLIFIED_CHINESE);        mOlamiVoiceRecognizer.setAuthorization(            "51a4bb56ba954655a4fc834bfdc46af1",            "asr","68bff251789b426896e70e888f919a6d","nli");                mOlamiVoiceRecognizer.setVADTailTimeout(2000);        mOlamiVoiceRecognizer.setLatitudeAndLongitude(31.155364678184498,121.34882432933009);     }    private void initHandler()    {        mHandler = new Handler(){            @Override            public void handleMessage(Message msg)            {                switch (msg.what){                case MessageConst.CLIENT_ACTION_START_RECORED:                    sendMessageToActivity(MessageConst.CLIENT_ACTION_START_RECORED,0,0,null,null);                    break;                case MessageConst.CLIENT_ACTION_STOP_RECORED:                    sendMessageToActivity(MessageConst.CLIENT_ACTION_STOP_RECORED,0,0,null,null);                    break;                case MessageConst.CLIENT_ACTION_ON_ERROR:                    sendMessageToActivity(MessageConst.CLIENT_ACTION_ON_ERROR,msg.arg1,0,null,null);                    break;                case MessageConst.CLIENT_ACTION_PLAY_BOOK_AFTER_SEARCH:                    sendMessageToActivity(MessageConst.                    CLIENT_ACTION_PLAY_BOOK_AFTER_SEARCH, msg.arg1, 0, null, msg.obj);                    break;                case MessageConst.CLIENT_ACTION_UPDATA_PLAYING_BOOK_NAME:                    sendMessageToActivity(MessageConst.                    CLIENT_ACTION_UPDATA_PLAYING_BOOK_NAME, msg.arg1, 0, null, msg.obj);                    break;                case MessageConst.CLIENT_ACTION_UPDATE_BOOK_PROGRESS:                    sendMessageToActivity(MessageConst.                    CLIENT_ACTION_UPDATE_BOOK_PROGRESS, msg.arg1, msg.arg2, null, null);                    break;                case MessageConst.CLIENT_ACTION_CANCEL_RECORED:                    sendMessageToActivity(MessageConst.                    CLIENT_ACTION_CANCEL_RECORED, msg.arg1, msg.arg2, null, null);                    break;                default:                    break;                }            }        };    }    private void initInComingHandler()    {        mInComingHandler = new Handler(){            @Override            public void handleMessage(Message msg)            {                switch (msg.what){                case MessageConst.CLIENT_ACTION_START_RECORED:                    if(mOlamiVoiceRecognizer != null)                        mOlamiVoiceRecognizer.start();                      break;                case MessageConst.CLIENT_ACTION_STOP_RECORED:                    if(mOlamiVoiceRecognizer != null)                        mOlamiVoiceRecognizer.stop();                       break;                case MessageConst.CLIENT_ACTION_CANCEL_RECORED:                    if(mOlamiVoiceRecognizer != null)                        mOlamiVoiceRecognizer.cancel();                     break;                case MessageConst.CLIENT_ACTION_SENT_TEXT:                    if(mOlamiVoiceRecognizer != null)                        mOlamiVoiceRecognizer.sendText(msg.obj.toString());                                     break;                }            }        };    }    private void initViaVoiceRecognizerListener()    {        mOlamiVoiceRecognizerListener = new OlamiVoiceRecognizerListener();    }    private class OlamiVoiceRecognizerListener implements IOlamiVoiceRecognizerListener{        @Override        public void onError(int errCode) {            mHandler.sendMessage(mHandler.obtainMessage(            MessageConst.CLIENT_ACTION_ON_ERROR,errCode,0));        }        @Override        public void onEndOfSpeech() {            mHandler.sendEmptyMessage(MessageConst.CLIENT_ACTION_STOP_RECORED);            if(mIsRecordPause)            {                mIsRecordPause = false;                mBookUtil.resumePlay();            }        }        @Override        public void onBeginningOfSpeech() {            if(mBookUtil.isPlaying())            {                mBookUtil.pause();                mIsRecordPause = true;            }            mHandler.sendEmptyMessage(MessageConst.CLIENT_ACTION_START_RECORED);        }        @Override        public void onResult(String result, int type) {                 sendMessageToActivity(MessageConst.SERVER_ACTION_RETURN_RESULT,type,0,null,result);            processServiceMessage(result);        }        @Override        public void onCancel() {            mHandler.sendEmptyMessage(MessageConst.CLIENT_ACTION_CANCEL_RECORED);        }        @Override        public void onUpdateVolume(int volume) {        sendMessageToActivity(MessageConst.CLIENT_ACTION_UPDATA_VOLUME,volume,0,null,null);        }    }    private void initCommunicationAssist()    {        mVoiceSdkComAssist = new VoiceSdkComAssist();        OlamiApplication.getInstance().setActivityToServiceListener(mVoiceSdkComAssist);    }    private void initXmly()    {        if(mBookUtil == null)        {            mBookUtil = BookUtil.getInstance();            mBookUtil.init(VoiceSdkService.this);            mBookUtil.setHandler(mHandler);        }    }    private void processServiceMessage(String message)    {        String input = null;        String serverMessage = null;        try{            JSONObject jsonObject = new JSONObject(message);            JSONArray jArrayNli = jsonObject.optJSONObject("data").optJSONArray("nli");            JSONObject jObj = jArrayNli.optJSONObject(0);            JSONArray jArraySemantic = null;            if(message.contains("semantic"))              jArraySemantic = jObj.getJSONArray("semantic");            else{                input = jsonObject.optJSONObject("data").optJSONObject("asr").                optString("result");                sendMessageToActivity(MessageConst.                                     CLIENT_ACTION_UPDATA_INPUT_TEXT, 0, 0, null, input);                serverMessage = jObj.optJSONObject("desc_obj").opt("result").toString();                sendMessageToActivity(MessageConst.                        CLIENT_ACTION_UPDATA_SERVER_MESSAGE, 0, 0, null, serverMessage);                return;            }            JSONObject jObjSemantic;            JSONArray jArraySlots;            JSONArray jArrayModifier;            String type = null;            String songName = null;            String singer = null;            if(jObj != null) {                type = jObj.optString("type");                if("musiccontrol".equals(type))                {                    jObjSemantic = jArraySemantic.optJSONObject(0);                    input = jObjSemantic.optString("input");                    jArraySlots = jObjSemantic.optJSONArray("slots");                    jArrayModifier = jObjSemantic.optJSONArray("modifier");                    String modifier = (String)jArrayModifier.opt(0);                    if((jArrayModifier != null) && ("play".equals(modifier)))                    {                        if(jArraySlots != null)                           for(int i=0,k=jArraySlots.length(); i<k; i++)                           {                               JSONObject obj = jArraySlots.getJSONObject(i);                               String name = obj.optString("name");                               if("singer".equals(name))                                   singer = obj.optString("value");                               else if("songname".equals(name))                                   songName = obj.optString("value");                           }                    }else if((modifier != null) && ("stop".equals(modifier)))                    {                        if(mBookUtil != null)                            if(mBookUtil.isPlaying())                                mBookUtil.stop();                    }else if((modifier != null) && ("pause".equals(modifier)))                    {                        if(mBookUtil != null)                            if(mBookUtil.isPlaying())                                mBookUtil.pause();                    }else if((modifier != null) && ("resume_play".equals(modifier)))                    {                        if(mBookUtil != null)                            mBookUtil.resumePlay();                    }else if((modifier != null) && ("add_volume".equals(modifier)))                    {                        if(mBookUtil != null)                            mBookUtil.addVolume();                    }else if((modifier != null) && ("del_volume".equals(modifier)))                    {                        if(mBookUtil != null)                            mBookUtil.delVolume();                    }else if((modifier != null) && ("next".equals(modifier)))                    {                        if(mBookUtil != null)                            mBookUtil.next();                    }else if((modifier != null) && ("previous".equals(modifier)))                    {                        if(mBookUtil != null)                            mBookUtil.prev();                    }else if((modifier != null) && ("play_index".equals(modifier)))                    {                        int position = 0;                        if(jArraySlots != null)                               for(int i=0,k=jArraySlots.length(); i<k; i++)                               {                                   JSONObject obj = jArraySlots.getJSONObject(i);                                   JSONObject jNumDetial = obj.getJSONObject("num_detail");                                   String index = jNumDetial.optString("recommend_value");                                   position = Integer.parseInt(index) - 1;                               }                        if(mBookUtil != null)                            mBookUtil.skipTo(position);                    }                }            }            if(songName != null)            {                if(singer != null)                {                }else{                    mBookUtil.searchBookAndPlay(songName,0,0);                }            }else if(singer != null)            {                mBookUtil.searchBookAndPlay(songName,0,0);            }            serverMessage = jObj.optJSONObject("desc_obj").opt("result").toString();        }        catch (Exception e)        {            e.printStackTrace();        }        //发送消息更新语音识别的文字        sendMessageToActivity(MessageConst.CLIENT_ACTION_UPDATA_INPUT_TEXT, 0, 0, null, input);        //发送消息更新服务器返回的结果字符串        sendMessageToActivity(MessageConst.CLIENT_ACTION_UPDATA_SERVER_MESSAGE,                                                     0, 0, null, serverMessage);    }    private void sendMessageToActivity(int what, int arg1, int arg2, Bundle data, Object obj)    {        if(OlamiApplication.getInstance().getServiceToActivityListener() != null)            OlamiApplication.getInstance().getServiceToActivityListener().                                             callBack(what, arg1, arg2, data, obj);    }    private class VoiceSdkComAssist implements CommunicationAssist{        @Override        public void callBack(int what, int arg1, int arg2, Bundle data,Object obj) {            Message msg = Message.obtain(null, what);            msg.arg1 = arg1;            msg.arg2 = arg2;            if (data != null)                msg.setData(data);            if (obj != null)                msg.obj = obj;            mInComingHandler.sendMessage(msg);        }           }    @Override    public void onDestroy(){        super.onDestroy();        if(mOlamiVoiceRecognizer != null)            mOlamiVoiceRecognizer.destroy();        if(mBookUtil != null)        {            mBookUtil.destroy();        }    }}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344

3.4 VoiceSdkService中onResult的回调处理

在VoiceSdkService.java中processServiceMessage(String message)用于处理onResult的回调数据。例如“我要听三国演义”返回如下数据：

{    "data": {        "asr": {            "result": "我要听三国演义",            "speech_status": 0,            "final": true,            "status": 0        },        "nli": [            {                "desc_obj": {                    "result": "正在努力搜索中，请稍等",                    "status": 0                },                "semantic": [                    {                        "app": "musiccontrol",                        "input": "我要听三国演义",                        "slots": [                            {                                "name": "songname",                                "value": "三国演义"                            }                        ],                        "modifier": [                            "play"                        ],                        "customer": "58df512384ae11f0bb7b487e"                    }                ],                "type": "musiccontrol"            }        ]    },    "status": "ok"}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

1）解析出nli中type类型是musiccontrol,这是语法返回app的类型，而这个在线听书的demo只关心musiccontrol这 个app类型，其他的忽略。

2）用户说的话转成文字是在asr中的result中获取 
3）在nli中的semantic中，input值是用户说的话，同asr中的result。 
modifier代表返回的行为动作，此处可以看到是play就是要求播放，slots中的数据表示歌曲名称是三国演义。 
那么动作是play，内容是歌曲名称是三国演义，在这个demo中调用 
mBookUtil.searchBookAndPlay(songName,0,0);会先查询，查询到结果会再发播放消息要求播放，我要听三国演义这个流程就走完了。

4.BookUtil

说一下搜索听书的实现过程public void searchBookInfo(String bookName,final int index,final boolean isNeedPlay){    mBookName = bookName;    Map<String, String> param = new HashMap<String, String>();    param.put(DTransferConstants.SEARCH_KEY, bookName);    param.put(DTransferConstants.CATEGORY_ID, "" + 3);//此处3代表搜索的是听书    //param.put(DTransferConstants.PAGE, "" + mPageId);    param.put(DTransferConstants.SORT, "asc");//返回列表的排序是正序还是逆序    param.put(DTransferConstants.PAGE_SIZE, "" + PAGE_SIZE);//每页能返回多少个查询结果    mPage = (index/PAGE_SIZE)+1;//当前在第几页    mPlayerManager = XmPlayerManager.getInstance(mContext);//喜马拉雅初始化部分    mPlayerManager.init(mNotificationId, null);    mPlayerManager.addPlayerStatusListener(mPlayerStatusListener);    mPlayerManager.addAdsStatusListener(mAdsListener);    CommonRequest.getSearchedAlbums(param, new IDataCallBack<SearchAlbumList>()    {        @Override        public void onSuccess(SearchAlbumList object)           {                               if (object != null && object.getAlbums() != null                    && object.getAlbums().size() != 0)            {                if (mSearchAlbumList == null)                {                    mSearchAlbumList = object;                }                else                {                    mSearchAlbumList.getAlbums().addAll(object.getAlbums());                }                //mTrackAdapter.notifyDataSetChanged();                Map<String, String> map = new HashMap<String, String>();                map.put(                DTransferConstants.ALBUM_ID, ""+object.getAlbums().get(0).getId());                map.put(DTransferConstants.SORT, "asc");                map.put(DTransferConstants.PAGE, "" + mPage);                map.put(DTransferConstants.PAGE_SIZE,  "" + PAGE_SIZE);                CommonRequest.getTracks(map, new IDataCallBack<TrackList>()                {                        @Override                        public void onSuccess(TrackList object)                        {                            mTrackList = object;                            mTotalCount = mTrackList.getTotalCount();                            if(mTrackList.getTracks().size() <= 0)                                return;                            String str = "专辑："+mTrackList.getAlbumTitle()+                                            get(0).getTrackTitle().toString();                            if(isNeedPlay)                            {                                mPosition = index % PAGE_SIZE;                                mHandler.sendMessage(mHandler.obtainMessage(                                MessageConst.CLIENT_ACTION_PLAY_BOOK_AFTER_SEARCH,                                 index % PAGE_SIZE,0));//此处mTrackList中已经查询出结果                                //向VoiceSdkService发送消息进行播放                            }                            else                                sendBookInfoToServer();                                                         }                        @Override                        public void onError(int code, String message)                        {                            Log.i("ppp","error: "+message);                            sendBookInfoToServer();                        }                });            }        }        @Override        public void onError(int code, String message)        {              Log.i("ppp","error: "+message);              sendBookInfoToServer();        }    });}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86

5.demo中支持的说法

我想听西游记 
我要听西游记 
播放西游记 
听西游记 
我想听西游记这本书 
上一首 
上一回 
下一首 
下一回 
暂停/暂停播放 
继续/继续播放 
声音大一点 
声音小一点 
关闭/关闭播放

用的是喜马拉雅测试账号，只支持听书的功能，查找歌曲的结果返回为空。

6.源码下载链接

用olamisdk语音识别引擎做在线听书demo

7.相关链接

语音记账demo：http://blog.csdn.net/ls0609/article/details/72765789

olami开放平台语法编写简介：http://blog.csdn.net/ls0609/article/details/71624340

olami开放平台语法官方介绍：https://cn.olami.ai/wiki/?mp=nli&content=nli2.html

转载请注明CSDN博文地址：http://blog.csdn.net/ls0609/article/details/71519203