Android使用OpenCV和FFMpeg的简单方法-开源项目javacv的使用
来源：互联网 发布：transmit mac 编辑：程序博客网 时间：2024/06/05 10:06
转自：http://doandroid.info/android%E4%BD%BF%E7%94%A8opencv%E5%92%8Cffmpeg%E7%9A%84%E7%AE%80%E5%8D%95%E6%96%B9%E6%B3%95-%E5%BC%80%E6%BA%90%E9%A1%B9%E7%9B%AEjavacv%E7%9A%84%E4%BD%BF%E7%94%A8/
首先感谢javacv开源项目的工作组.
0 使用说明
JavaCV first provides wrappers to commonly used libraries by researchers in the field of computer vision: OpenCV, FFmpeg, libdc1394, PGR FlyCapture, OpenKinect, videoInput, and ARToolKitPlus. The following classes, found under the com.googlecode.javacv.cpp package namespace, expose their complete APIs: opencv_core, opencv_imgproc, opencv_video, opencv_flann, opencv_features2d, opencv_calib3d, opencv_objdetect, opencv_highgui, opencv_legacy, opencv_ml, opencv_contrib, avutil, avcodec, avformat, avdevice, avfilter, postproc, swscale, dc1394, PGRFlyCapture, freenect, videoInputLib, and ARToolKitPlus, respectively. Moreover, utility classes make it easy to use their functionality on the Java platform, including Android.
javacv的项目最早是为java平台封装了机器视觉领域的开源库，后提供Android支持。其中的库包含OpenCV, FFmpeg, libdc1394, PGR FlyCapture, OpenKinect, videoInput, and ARToolKitPlus等等。在命名空间com.googlecode.javacv.cpp下包含了所有的类对象。并暴露出了他们完整的API:opencv_legacy, opencv_ml, opencv_contrib, avutil, avcodec, avformat, avdevice, avfilter, postproc, swscale, dc1394, PGRFlyCapture, freenect, videoInputLib, and ARToolKitPlus.工具类的使用使得我们可以非常容易的使用他的功能。
1 Follow the instructions on this page: http://developer.android.com/resources/tutorials/hello-world.html
2 Go to File > New > Folder, select your project as parent folder, type “libs/armeabi” as Folder name, and click Finish.
3 Copy javacpp.jar and javacv.jar in the newly created “libs” folder.
4 Extract directly all the *.so files from javacv-android-arm.jar as well as the ones from OpenCV-2.3.1-android-arm.zip and ffmpeg-0.7.11-android-arm.zip in the newly created “libs/armeabi” folder, without creating any new subdirectories.
5 Navigate to Project > Properties > Java Build Path > Libraries and click “Add JARs…”
6 Select both javacpp.jar and javacv.jar from the newly created “libs” folder.
本文将介绍如何在Android中使用这些库。
1 创建一个Android项目
2 在新建的项目上，右击选择Folder，输入libs/armeabi作为文件夹名称。这里解释一下，我们在使用jni的方式开发Android项目的时候，使用ndk-build编译项目后，一般会自动生成libs/armeabi文件夹，并将我们的库.so或者.a文件在文件夹中生成。本文也是这个目的。后面将会拷贝很多文件到这个目录里面来。
3 在 http://code.google.com/p/javacv/downloads/list 下载几个文件。
3.1 javacv-bin-20120329.zip
3.2 ffmpeg-0.7.11-android-arm.zip
3.3 OpenCV-2.3.1-android-arm.zip
解压所有的文件，并将javacv-bin-20120329.zip中的javacpp.jar和javacv.jar拷贝到刚才建立的/libs目录。
将javacv-bin-20120329.zip目录的javacv-android-arm.jar文件打开，进入最里面的目录，将所有的.so文件拷贝到libs/armeabi目录。
将ffmpeg-0.7.11-android-arm.zip里面的所有.so文件拷贝到libs/armeabi目录。
将OpenCV-2.3.1-android-arm.zip里面的所有.so文件拷贝到libs/armeabi目录。
4 选择Project->Properties->Java Build Path->Libraries,点击Add JARs…并选择libs目录的javacpp.jar 和 javacv.jar文件。
1 文件下载
1.1 javacv-bin-20120329.zip
1.2 ffmpeg-0.7.11-android-arm.zip
1.3 OpenCV-2.3.1-android-arm.zip
2 创建项目
新建一个Android项目。
3 加入lib库文件
将javacv-bin-20120329.zip中的javacpp.jar和javacv.jar拷贝到刚才建立的/libs目录。
将javacv-bin-20120329.zip目录的javacv-android-arm.jar文件打开，进入最里面的目录，将所有的.so文件拷贝到libs/armeabi目录。
将ffmpeg-0.7.11-android-arm.zip里面的所有.so文件拷贝到libs/armeabi目录。
将OpenCV-2.3.1-android-arm.zip里面的所有.so文件拷贝到libs/armeabi目录。
4 编译例子代码FacePreview
本代码在javacv-bin-20120329.zip压缩文件中，进入/javacv-bin/samples/目录，将FacePreview.java加入到刚刚建立项目的src目录。
本代码不需要layout文件。
下面的代码含部分注释：
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
* Copyright (C) 2010,2011,2012 Samuel Audet
*
* FacePreview - A fusion of OpenCV's facedetect and Android's CameraPreview samples,
* with JavaCV + JavaCPP as the glue in between.
*
* This file was based on CameraPreview.java that came with the Samples for
* Android SDK API 8, revision 1 and contained the following copyright notice:
*
* Copyright (C) 2007 The Android Open Source Project
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*
* IMPORTANT - Make sure the AndroidManifest.xml file looks like this:
*
* <?xml version="1.0" encoding="utf-8"?>
* <manifest xmlns:android="http://schemas.android.com/apk/res/android"
* package="com.googlecode.javacv.facepreview"
* android:versionCode="1"
* android:versionName="1.0" >
* <uses-sdk android:minSdkVersion="4" />
* <uses-permission android:name="android.permission.CAMERA" />
* <uses-feature android:name="android.hardware.camera" />
* <application android:label="@string/app_name">
* <activity
* android:name="FacePreview"
* android:label="@string/app_name"
* android:screenOrientation="landscape">
* <intent-filter>
* <action android:name="android.intent.action.MAIN" />
* <category android:name="android.intent.category.LAUNCHER" />
* </intent-filter>
* </activity>
* </application>
* </manifest>
*/
package com.jouhu.opencvffmpeg;
import android.app.Activity;
import android.app.AlertDialog;
import android.content.Context;
import android.graphics.Canvas;
import android.graphics.Color;
import android.graphics.ImageFormat;
import android.graphics.Paint;
import android.hardware.Camera;
import android.hardware.Camera.Size;
import android.os.Bundle;
import android.util.Log;
import android.view.SurfaceHolder;
import android.view.SurfaceView;
import android.view.View;
import android.view.Window;
import android.view.WindowManager;
import android.widget.FrameLayout;
import java.io.File;
import java.io.IOException;
import java.nio.ByteBuffer;
import java.util.List;
import com.googlecode.javacpp.Loader;
import com.googlecode.javacv.cpp.opencv_objdetect;
import static com.googlecode.javacv.cpp.opencv_core.*;
import static com.googlecode.javacv.cpp.opencv_imgproc.*;
import static com.googlecode.javacv.cpp.opencv_objdetect.*;
import static com.googlecode.javacv.cpp.opencv_highgui.*;
// ----------------------------------------------------------------------
public class FacePreview extends Activity {
private FrameLayout layout;
private FaceView faceView;
private Preview mPreview;
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
//设置全屏
getWindow().addFlags(WindowManager.LayoutParams.FLAG_FULLSCREEN);
// Hide the window title.
//隐藏标题栏
requestWindowFeature(Window.FEATURE_NO_TITLE);
// Create our Preview view and set it as the content of our activity.
try {
//代码创建FrameLayout 对象，并将FaceView加入Layout其中
layout = new FrameLayout(this);
faceView = new FaceView(this);
mPreview = new Preview(this, faceView);
layout.addView(mPreview);
layout.addView(faceView);
setContentView(layout);
} catch (IOException e) {
e.printStackTrace();
new AlertDialog.Builder(this).setMessage(e.getMessage()).create().show();
}
}
}
// ----------------------------------------------------------------------
//基于View继承并实现Camera.PreviewCallback接口的FaceView对象
class FaceView extends View implements Camera.PreviewCallback {
public static final int SUBSAMPLING_FACTOR = 4;
//定义opencv中的IplImage对象
private IplImage grayImage;
//Haar选择子定义
private CvHaarClassifierCascade classifier;
//opencv的内存对象定义
private CvMemStorage storage;
private CvSeq faces;
public FaceView(FacePreview context) throws IOException {
super(context);
// Load the classifier file from Java resources.
//加载面部识别haar选择子的xml文件
File classifierFile = Loader.extractResource(getClass(),
"/com/jouhu/opencvffmpeg/haarcascade_frontalface_alt.xml",
context.getCacheDir(), "classifier", ".xml");
if (classifierFile == null || classifierFile.length() <= 0) {
throw new IOException("Could not extract the classifier file from Java resource.");
}
//Log.v("FaceView", "1111111111111111111111111111");
// Preload the opencv_objdetect module to work around a known bug.
//加载opencv_objdetect.so文件
Loader.load(opencv_objdetect.class);
//Log.v("FaceView", "1111111111111111111111111111");
//创建Haar对象
classifier = new CvHaarClassifierCascade(cvLoad(classifierFile.getAbsolutePath()));
//Log.v("FaceView", "1111111111111111111111111111");
classifierFile.delete();
if (classifier.isNull()) {
throw new IOException("Could not load the classifier file.");
}
//Log.v("FaceView", "1111111111111111111111111111");
storage = CvMemStorage.create();
}
public void onPreviewFrame(final byte[] data, final Camera camera) {
try {
//获取到每帧图像的时候，调用函数processImage对数据进行处理，并将处理后的数据给camera并显示出来
Camera.Size size = camera.getParameters().getPreviewSize();
processImage(data, size.width, size.height);
camera.addCallbackBuffer(data);
} catch (RuntimeException e) {
// The camera has probably just been released, ignore.
}
}
protected void processImage(byte[] data, int width, int height) {
// First, downsample our image and convert it into a grayscale IplImage
int f = SUBSAMPLING_FACTOR;
if (grayImage == null || grayImage.width() != width/f || grayImage.height() != height/f) {
grayImage = IplImage.create(width/f, height/f, IPL_DEPTH_8U, 1);
}
int imageWidth = grayImage.width();
int imageHeight = grayImage.height();
int dataStride = f*width;
int imageStride = grayImage.widthStep();
ByteBuffer imageBuffer = grayImage.getByteBuffer();
for (int y = 0; y < imageHeight; y++) {
int dataLine = y*dataStride;
int imageLine = y*imageStride;
for (int x = 0; x < imageWidth; x++) {
imageBuffer.put(imageLine + x, data[dataLine + f*x]);
}
}
//Haar 脸部检测函数
faces = cvHaarDetectObjects(grayImage, classifier, storage, 1.1, 3, CV_HAAR_DO_CANNY_PRUNING);
//重绘 触发onDraw
postInvalidate();
cvClearMemStorage(storage);
}
@Override
protected void onDraw(Canvas canvas) {
Paint paint = new Paint();
paint.setColor(Color.RED);
paint.setTextSize(20);
String s = "FacePreview - This side up.";
float textWidth = paint.measureText(s);
canvas.drawText(s, (getWidth()-textWidth)/2, 20, paint);
if (faces != null) {
paint.setStrokeWidth(2);
paint.setStyle(Paint.Style.STROKE);
float scaleX = (float)getWidth()/grayImage.width();
float scaleY = (float)getHeight()/grayImage.height();
int total = faces.total();
for (int i = 0; i < total; i++) {
CvRect r = new CvRect(cvGetSeqElem(faces, i));
int x = r.x(), y = r.y(), w = r.width(), h = r.height();
canvas.drawRect(x*scaleX, y*scaleY, (x+w)*scaleX, (y+h)*scaleY, paint);
}
}
}
}
// ----------------------------------------------------------------------
class Preview extends SurfaceView implements SurfaceHolder.Callback {
SurfaceHolder mHolder;
Camera mCamera;
Camera.PreviewCallback previewCallback;
Preview(Context context, Camera.PreviewCallback previewCallback) {
super(context);
this.previewCallback = previewCallback;
// Install a SurfaceHolder.Callback so we get notified when the
// underlying surface is created and destroyed.
mHolder = getHolder();
mHolder.addCallback(this);
mHolder.setType(SurfaceHolder.SURFACE_TYPE_PUSH_BUFFERS);
}
public void surfaceCreated(SurfaceHolder holder) {
// The Surface has been created, acquire the camera and tell it where
// to draw.
mCamera = Camera.open();
try {
mCamera.setPreviewDisplay(holder);
} catch (IOException exception) {
mCamera.release();
mCamera = null;
// TODO: add more exception handling logic here
}
}
public void surfaceDestroyed(SurfaceHolder holder) {
// Surface will be destroyed when we return, so stop the preview.
// Because the CameraDevice object is not a shared resource, it's very
// important to release it when the activity is paused.
mCamera.stopPreview();
mCamera.release();
mCamera = null;
}
private Size getOptimalPreviewSize(List<Size> sizes, int w, int h) {
final double ASPECT_TOLERANCE = 0.05;
double targetRatio = (double) w / h;
if (sizes == null) return null;
Size optimalSize = null;
double minDiff = Double.MAX_VALUE;
int targetHeight = h;
// Try to find an size match aspect ratio and size
for (Size size : sizes) {
double ratio = (double) size.width / size.height;
if (Math.abs(ratio - targetRatio) > ASPECT_TOLERANCE) continue;
if (Math.abs(size.height - targetHeight) < minDiff) {
optimalSize = size;
minDiff = Math.abs(size.height - targetHeight);
}
}
// Cannot find the one match the aspect ratio, ignore the requirement
if (optimalSize == null) {
minDiff = Double.MAX_VALUE;
for (Size size : sizes) {
if (Math.abs(size.height - targetHeight) < minDiff) {
optimalSize = size;
minDiff = Math.abs(size.height - targetHeight);
}
}
}
return optimalSize;
}
public void surfaceChanged(SurfaceHolder holder, int format, int w, int h) {
// Now that the size is known, set up the camera parameters and begin
// the preview.
Camera.Parameters parameters = mCamera.getParameters();
List<Size> sizes = parameters.getSupportedPreviewSizes();
Size optimalSize = getOptimalPreviewSize(sizes, w, h);
parameters.setPreviewSize(optimalSize.width, optimalSize.height);
mCamera.setParameters(parameters);
if (previewCallback != null) {
mCamera.setPreviewCallbackWithBuffer(previewCallback);
Camera.Size size = parameters.getPreviewSize();
byte[] data = new byte[size.width*size.height*
ImageFormat.getBitsPerPixel(parameters.getPreviewFormat())/8];
mCamera.addCallbackBuffer(data);
}
mCamera.startPreview();
}
}
5 注意事项
5.1 项目里面需要一个haarcascade_frontalface_alt.xml文件，请务必放到src目录，跟FacePreview.java在同一个目录下，如果找不到，可以下载本项目。包含了这个代码。
5.2 本代码使用的javacv版本为2012年4月11日版本，请随时关注javacv的代码更新。
5.3 务必加入权限设置，如果不加入会出错，错误如下。
04-09 22:03:55.748: E/AndroidRuntime(4089): FATAL EXCEPTION: main
04-09 22:03:55.748: E/AndroidRuntime(4089): java.lang.RuntimeException: Fail to connect to camera service
04-09 22:03:55.748: E/AndroidRuntime(4089): at android.hardware.Camera.native_setup(Native Method)
04-09 22:03:55.748: E/AndroidRuntime(4089): at android.hardware.Camera.(Camera.java:185)
04-09 22:03:55.748: E/AndroidRuntime(4089): at android.hardware.Camera.open(Camera.java:165)
04-09 22:03:55.748: E/AndroidRuntime(4089): at com.jouhu.opencvffmpeg.Preview.surfaceCreated(FacePreview.java:225)
04-09 22:03:55.748: E/AndroidRuntime(4089): at android.view.SurfaceView.updateWindow(SurfaceView.java:536)
04-09 22:03:55.748: E/AndroidRuntime(4089): at android.view.SurfaceView.dispatchDraw(SurfaceView.java:339)
04-09 22:03:55.748: E/AndroidRuntime(4089): at android.view.ViewGroup.drawChild(ViewGroup.java:1638)
04-09 22:03:55.748: E/AndroidRuntime(4089): at android.view.ViewGroup.dispatchDraw(ViewGroup.java:1367)
04-09 22:03:55.748: E/AndroidRuntime(4089): at android.view.ViewGroup.drawChild(ViewGroup.java:1638)
04-09 22:03:55.748: E/AndroidRuntime(4089): at android.view.ViewGroup.dispatchDraw(ViewGroup.java:1367)
04-09 22:03:55.748: E/AndroidRuntime(4089): at android.view.View.draw(View.java:6832)
04-09 22:03:55.748: E/AndroidRuntime(4089): at android.widget.FrameLayout.draw(FrameLayout.java:352)
04-09 22:03:55.748: E/AndroidRuntime(4089): at android.view.ViewGroup.drawChild(ViewGroup.java:1640)
04-09 22:03:55.748: E/AndroidRuntime(4089): at android.view.ViewGroup.dispatchDraw(ViewGroup.java:1367)
04-09 22:03:55.748: E/AndroidRuntime(4089): at android.view.View.draw(View.java:6832)
04-09 22:03:55.748: E/AndroidRuntime(4089): at android.widget.FrameLayout.draw(FrameLayout.java:352)
04-09 22:03:55.748: E/AndroidRuntime(4089): at com.android.internal.policy.impl.PhoneWindow$DecorView.draw(PhoneWindow.java:1895)
04-09 22:03:55.748: E/AndroidRuntime(4089): at android.view.ViewRoot.draw(ViewRoot.java:1407)
04-09 22:03:55.748: E/AndroidRuntime(4089): at android.view.ViewRoot.performTraversals(ViewRoot.java:1163)
04-09 22:03:55.748: E/AndroidRuntime(4089): at android.view.ViewRoot.handleMessage(ViewRoot.java:1727)
04-09 22:03:55.748: E/AndroidRuntime(4089): at android.os.Handler.dispatchMessage(Handler.java:99)
04-09 22:03:55.748: E/AndroidRuntime(4089): at android.os.Looper.loop(Looper.java:123)
04-09 22:03:55.748: E/AndroidRuntime(4089): at android.app.ActivityThread.main(ActivityThread.java:4627)
04-09 22:03:55.748: E/AndroidRuntime(4089): at java.lang.reflect.Method.invokeNative(Native Method)
04-09 22:03:55.748: E/AndroidRuntime(4089): at java.lang.reflect.Method.invoke(Method.java:521)
04-09 22:03:55.748: E/AndroidRuntime(4089): at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:858)
04-09 22:03:55.748: E/AndroidRuntime(4089): at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:616)
04-09 22:03:55.748: E/AndroidRuntime(4089): at dalvik.system.NativeStart.main(Native Method)
6 代码下载
由于里面包含了很多so文件，所有代码非常大。我将所有的.so去掉附上代码。
OpenCVFFmpegTest.tar
7 参考文章
7.1 http://code.google.com/p/javacv/
7.2 http://code.google.com/p/javacv/issues/detail?id=183&can=1&start=100
- Android使用OpenCV和FFMpeg的简单方法-开源项目javacv的使用
- Android项目中使用javacv和javacpp的代码混淆配置
- javaCV开发详解之技术杂烩：javaCV能帮我们做什么？能实现什么功能？ffmpeg和openCV能实现功能，javaCV如何做到更快、更简单的实现相应的功能？等等一堆实用话题
- JavaCV的配置及使用
- JavaCV的配置及使用
- JavaCV的配置及使用
- 关于在Android中使用JavaCV的问题
- 使用JavaCV（ffmpeg）录制视频
- JavaCV的初次使用（在Eclipse）
- 【Android】Android使用OpenCV实现人脸识别（OpenCV+JavaCV）
- Android Studio使用OpenCV的配置方法
- Android Studio使用OpenCV的配置方法
- Android Studio使用OpenCV的配置方法
- Android Studio使用OpenCV的配置方法
- Android Studio 使用 OpenCV 的配置方法
- Android Studio 使用 OpenCV 的配置方法
- Android Studio使用OpenCV的配置方法
- javacv-ffmpeg播放视频里的音频
- 用PDB库调试Python程序
- ACdream群原创群赛(2) A~F J 解题报告
- 编译android源码前的准备
- 认识店面桌面图标的作用和意义
- test
- Android使用OpenCV和FFMpeg的简单方法-开源项目javacv的使用
- ctags
- 设计模式的解析和实现(C++)之九-Decorator模式
- （算法题）几道算法题
- 设计C++回调模型（二）：线程安全
- 软件测试用例的设计心得
- 关于json对象和json字符串的转换以及操作
- Linux-2.6.25 TCPIP函数调用大致流程
- 从客户端检测到有潜在危险的Request.Form 值
