谈谈fresco的bitmap内存分配
来源:互联网 发布:发票数据导出软件 编辑:程序博客网 时间:2024/06/05 03:56
bitmap的内存分配主要有两个含义:
- 应用程序实现时针对bitmap的内存缓存;
- 从原始数据(byte[])经过decode生成bitmap过程中的内存分配问题;
其中第一个含义,在Fresco中实现对应的就是interface MemoryCache,这里略有提及,不会大费章节,主要关注decode过程中的内存分配问题。
WHAT?
从http://frescolib.org/关于memory的介绍可以知道,在4.x及以下的系统上,Fresco的bitmap decode会把bitmap的pixel data(像素数据)放到一个“特殊的内存区域’”,这个特殊的内存区域其实就是ashmem,至于ashmem是什么,详细的可以参考罗升阳的博客(http://blog.csdn.net/luoshengyang/article/details/6651971),这里不详述。
WHY?
那为什么在4.x及以下系统需要这样做?原因是如果Bitmap数量很多时会占用大量的内存(这里内存特指Java Heap),必然就会更加频繁的触发虚拟机进行GC,GC会导致stop the world,就会出现卡顿,而5.0之后采用了art模式,对GC进行了优化,情况比较乐观(为什么这么说?另说),而在4.x及以下的手机就比较糟糕了,所以,在4.x及以下系统上Fresco选择了ashmem作为pixel data的存储区域。(对于想深究GC的问题,可自行分析dalvik和art的相关代码)
另外,Java Heap的GC问题既然会导致freeze的问题,那么我们也可以采用native heap,这个不受GC的影响,而且内存很大(基本可以认为只受物理内存大小影响),需要的只是特别注意去回收内存。那为什么不用native heap,而去选择ashmem呢?这个问题后面会讲解。
HOW?
Bitmap在Fresco这个框架是在decode时,内存被分配到ashmem中呢?接下来从代码上,慢慢分析这个过程。
PlatformDecoder
在Fresco中,提供了PlatformDecoder这个接口用于处理bitmap的decode过程,下面包括了该接口的具体实现类;
从类名就可以推测,Fresco根据系统版本不同,使用不同的decoder,从下面的代码也大致证实了这个推测(但有所差别):
/** * * source : com/facebook/imagepipeline/core/ImagePipelineFactory.java * */
/**
* Provide the implementation of the PlatformDecoder for the current platform using the
* provided PoolFactory
*
* @param poolFactory The PoolFactory
* @return The PlatformDecoder implementation
*/
public static PlatformDecoder buildPlatformDecoder(
PoolFactory poolFactory,
boolean directWebpDirectDecodingEnabled,
BitmapCounterTracker bitmapCounterTracker) {
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.LOLLIPOP) {
int maxNumThreads = poolFactory.getFlexByteArrayPoolMaxNumThreads();
return new ArtDecoder(
poolFactory.getBitmapPool(),
maxNumThreads,
new Pools.SynchronizedPool<>(maxNumThreads));
} else {
if (directWebpDirectDecodingEnabled
&& Build.VERSION.SDK_INT < Build.VERSION_CODES.KITKAT) {
return new GingerbreadPurgeableDecoder(bitmapCounterTracker);
} else {
return new KitKatPurgeableDecoder(
poolFactory.getFlexByteArrayPool(), bitmapCounterTracker);
}
}
}
1.1
分别是,5.0以上都使用ArtDecoder,4.4以下如果directWebpDirectDecodingEnabled为true,则使用GingerbreadPurgeableDecoder,不然其他都使用KitKatPurgeableDecoder。这些decoder主要包含两个方法对原始数据进行decode:decodeFromEncodedImage和decodeJPEGFromEncodedImage。由于在decode环节上,这两个方法并无二致,为了方便,所以就选择decodeFromEncodedImage进行深入分析。
DalvikPurgeableDecoder
先看DalvikPurgeableDecoder的decodeFromEncodedImage方法:
/** * * source : com/facebook/imagepipeline/platform/DalvikPurgeableDecoder.java * */
@Override
public CloseableReference<Bitmap> decodeFromEncodedImage(
final EncodedImage encodedImage,
Bitmap.Config bitmapConfig) {
BitmapFactory.Options options = getBitmapFactoryOptions(
encodedImage.getSampleSize(),
bitmapConfig);
CloseableReference<PooledByteBuffer> bytesRef = encodedImage.getByteBufferRef();
Preconditions.checkNotNull(bytesRef);
try {
Bitmap bitmap = decodeByteArrayAsPurgeable(bytesRef, options);
return pinBitmap(bitmap);
} finally {
CloseableReference.closeSafely(bytesRef);
}
}
private static BitmapFactory.Options getBitmapFactoryOptions(
int sampleSize,
Bitmap.Config bitmapConfig) {
BitmapFactory.Options options = new BitmapFactory.Options();
options.inDither = true; // known to improve picture quality at low cost
options.inPreferredConfig = bitmapConfig;
// Decode the image into a 'purgeable' bitmap that lives on the ashmem heap
options.inPurgeable = true;
// Enable copy of of bitmap to enable purgeable decoding by filedescriptor
options.inInputShareable = true;
// Sample size should ONLY be different than 1 when downsampling is enabled in the pipeline
options.inSampleSize = sampleSize;
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.HONEYCOMB) {
options.inMutable = true; // no known perf difference; allows postprocessing to work
}
return options;
}
2.1
着重关注和内存分配相关的两个设置:inPurgeable和inInputShareable。上面注释看,inPurgeable能使bitmap的内存分配到ashmem上,对于通过filedescriptor去decode的方式,还要设置inInputShareable为true,只能够使内存分配到ashmem上。先关注inPurgeable
/**
* If this is set to true, then the resulting bitmap will allocate its
* pixels such that they can be purged if the system needs to reclaim
* memory. In that instance, when the pixels need to be accessed again
* (e.g. the bitmap is drawn, getPixels() is called), they will be
* automatically re-decoded.
*
* <p>For the re-decode to happen, the bitmap must have access to the
* encoded data, either by sharing a reference to the input
* or by making a copy of it. This distinction is controlled by
* inInputShareable. If this is true, then the bitmap may keep a shallow
* reference to the input. If this is false, then the bitmap will
* explicitly make a copy of the input data, and keep that. Even if
* sharing is allowed, the implementation may still decide to make a
* deep copy of the input data.</p>
*
* <p>While inPurgeable can help avoid big Dalvik heap allocations (from
* API level 11 onward), it sacrifices performance predictability since any
* image that the view system tries to draw may incur a decode delay which
* can lead to dropped frames. Therefore, most apps should avoid using
* inPurgeable to allow for a fast and fluid UI. To minimize Dalvik heap
* allocations use the {@link #inBitmap} flag instead.</p>
*
* <p class="note"><strong>Note:</strong> This flag is ignored when used
* with {@link #decodeResource(Resources, int,
* android.graphics.BitmapFactory.Options)} or {@link #decodeFile(String,
* android.graphics.BitmapFactory.Options)}.</p>
*/
public boolean inPurgeable;
2.2
从注释看inPurgeable的作用是,在KITKAT及以下,使用该参数的话,当系统需要回收内存的时候,bitmap的pixels可以被清除。好在的是,当pixels需要被重新访问的时候(例如bitmap draw或者调用getPixels()的时候),它们又可以重新被decode出来。要重新decode出来的话,自然需要encoded data,encdoded data可能来源于对原始那份encoded data的引用,或者是对原始数据的拷贝。具体是引用或者拷贝,就是根据inInputShareable来决定的,如果是true那就是引用,不然就是deep copy,但是inInputShareable即使设置为true,不同的实现也可能是直接进行deep copy。而大的问题是,inPurgeable虽然能够避免在Java heap中分配内存,但是牺牲点了UI的流畅性,为什么?因为重新decode的过程是在UI线程进行的,这会导致一定的掉帧问题。
inPurgeable可以引出了两个问题:
- Decode过程是如何根据它来把pixels分配到ashmem?sdk文档中并没有提及到这一点,是不是真的如此?
- Decode为什么会导致drop frames,又该如何解决?
Pixels到ashmem过程
回到decodeFromEncodedImage,它主要是调用decodeByteArrayAsPurgeable进行decode的过程,为了方便(因为GingerbreadPurgeableDecoder的decode过程较为复杂,而且存在一定问题,留待后面解析),我们分析KitKatPurgeableDecoder的该方法:
/**
* Decodes a byteArray into a purgeable bitmap
*
* @param bytesRef the byte buffer that contains the encoded bytes
* @return
*/
@Override
protected Bitmap decodeByteArrayAsPurgeable(
CloseableReference<PooledByteBuffer> bytesRef,
BitmapFactory.Options options) {
final PooledByteBuffer pooledByteBuffer = bytesRef.get();
final int length = pooledByteBuffer.size();
final CloseableReference<byte[]> encodedBytesArrayRef = mFlexByteArrayPool.get(length);
try {
final byte[] encodedBytesArray = encodedBytesArrayRef.get();
pooledByteBuffer.read(0, encodedBytesArray, 0, length);
Bitmap bitmap = BitmapFactory.decodeByteArray(
encodedBytesArray,
0,
length,
options);
return Preconditions.checkNotNull(bitmap, "BitmapFactory returned null");
} finally {
CloseableReference.closeSafely(encodedBytesArrayRef);
}
}
3.1
很简单,就是就是copy数据到byte[],然后调用BitmapFactory的decodeByteArray进行decode。BitmapFactory的decodeByteArray最终通过jni的方式,调用BitmapFactory.cpp的nativeDecodeByteArray方法:
static jobject nativeDecodeByteArray(JNIEnv* env, jobject, jbyteArray byteArray,
int offset, int length, jobject options) {
/* If optionsShareable() we could decide to just wrap the java array and
share it, but that means adding a globalref to the java array object
and managing its lifetime. For now we just always copy the array's data
if optionsPurgeable(), unless we're just decoding bounds.
*/
bool purgeable = optionsPurgeable(env, options) && !optionsJustBounds(env, options);
AutoJavaByteArray ar(env, byteArray);
SkMemoryStream* stream = new SkMemoryStream(ar.ptr() + offset, length, purgeable);
SkAutoUnref aur(stream);
return doDecode(env, stream, NULL, options, purgeable);
}
3.2
从上面代码看,除非BitmapOptions设置了inJustDecodeBounds为true,不然由于inPurgeable为true,所以临时变量purgeable都为true。purgeable的值影响的是SkMemoryStream的区别:
SkMemoryStream::SkMemoryStream(const void* src, size_t size, bool copyData) {
fData = newFromParams(src, size, copyData);
fOffset = 0;
}
static SkData* newFromParams(const void* src, size_t size, bool copyData) {
if (copyData) {
return SkData::NewWithCopy(src, size);
} else {
return SkData::NewWithProc(src, size, NULL, NULL);
}
}
SkData* SkData::NewWithCopy(const void* data, size_t length) {
if (0 == length) {
return SkData::NewEmpty();
}
void* copy = sk_malloc_throw(length); // balanced in sk_free_releaseproc
memcpy(copy, data, length);
return new SkData(copy, length, sk_free_releaseproc, NULL);
}
SkData* SkData::NewWithProc(const void* data, size_t length,
ReleaseProc proc, void* context) {
return new SkData(data, length, proc, context);
}
SkData::SkData(const void* ptr, size_t size, ReleaseProc proc, void* context) {
fPtr = ptr;
fSize = size;
fReleaseProc = proc;
fReleaseProcContext = context;
}
3.3
从上面代码看,区别就是如果是true,则copy一份数据,不然纯粹是引用。我们这里肯定是true,所以固定会copy一份数据,也就是copy一份encoded data。这内存的分配是在native heap上做的。
另外这个刚好反应上面说的,“但是inInputShareable即使设置为true,不同的实现也可能是直接进行deep copy”。
再看关键的doDecode方法,由于这个方法很长,只截取相关的一部分去看:
// since we "may" create a purgeable imageref, we require the stream be ref'able
// i.e. dynamically allocated, since its lifetime may exceed the current stack
// frame.
static jobject doDecode(JNIEnv* env, SkStreamRewindable* stream, jobject padding,
jobject options, bool allowPurgeable, bool forcePurgeable = false) {
int sampleSize = 1;
SkImageDecoder::Mode mode = SkImageDecoder::kDecodePixels_Mode;
SkBitmap::Config prefConfig = SkBitmap::kARGB_8888_Config;
bool doDither = true;
bool isMutable = false;
float scale = 1.0f;
bool isPurgeable = forcePurgeable || (allowPurgeable && optionsPurgeable(env, options));
bool preferQualityOverSpeed = false;
bool requireUnpremultiplied = false;
jobject javaBitmap = NULL;//其实就是BitmapOptions的inBitmap
if (options != NULL) {
sampleSize = env->GetIntField(options, gOptions_sampleSizeFieldID);
if (optionsJustBounds(env, options)) {
mode = SkImageDecoder::kDecodeBounds_Mode;
}
- ........................................
//根据部分参数计算scale的的值(我们这里scale设置为1)
if (env->GetBooleanField(options, gOptions_scaledFieldID)) {
const int density = env->GetIntField(options, gOptions_densityFieldID);
const int targetDensity = env->GetIntField(options, gOptions_targetDensityFieldID);
const int screenDensity = env->GetIntField(options, gOptions_screenDensityFieldID);
if (density != 0 && targetDensity != 0 && density != screenDensity) {
scale = (float) targetDensity / density;
}
}
}
const bool willScale = scale != 1.0f;
isPurgeable &= !willScale;//如果bitmap需要scale那么isPurgeable必然为false
SkImageDecoder* decoder = SkImageDecoder::Factory(stream);
if (decoder == NULL) {
return nullObjectReturn("SkImageDecoder::Factory returned null");
}
- ...........................................................
SkAutoTDelete<SkBitmap> adb(outputBitmap == NULL ? new SkBitmap : NULL);
if (outputBitmap == NULL) outputBitmap = adb.get();
NinePatchPeeker peeker(decoder);
decoder->setPeeker(&peeker);
//如果isPurgeable,使用kDecodeBounds_Mode,用于先计算出图片的宽高等,而不是直接decode出pixels来
SkImageDecoder::Mode decodeMode = isPurgeable ? SkImageDecoder::kDecodeBounds_Mode : mode;
- ...........................................................................
// Only setup the decoder to be deleted after its stack-based, refcounted
// components (allocators, peekers, etc) are declared. This prevents RefCnt
// asserts from firing due to the order objects are deleted from the stack.
SkAutoTDelete<SkImageDecoder> add(decoder);
AutoDecoderCancel adc(options, decoder);
// To fix the race condition in case "requestCancelDecode"
// happens earlier than AutoDecoderCancel object is added
// to the gAutoDecoderCancelMutex linked list.
if (options != NULL && env->GetBooleanField(options, gOptions_mCancelID)) {
return nullObjectReturn("gOptions_mCancelID");
}
SkBitmap decodingBitmap;
//使用kDecodeBounds_Mode,先计算出图片的宽高等,而不是直接decode出pixels来
if (!decoder->decode(stream, &decodingBitmap, prefConfig, decodeMode)) {
return nullObjectReturn("decoder->decode returned false");
}
- .............................................
if (willScale) {
.............
.............
} else {
//把outputBitmap设置为刚才decode出的decodingBitmap
outputBitmap->swap(decodingBitmap);
}
SkPixelRef* pr;
if (isPurgeable) {
pr = installPixelRef(outputBitmap, stream, sampleSize, doDither);
} else {
// if we get here, we're in kDecodePixels_Mode and will therefore
// already have a pixelref installed.
pr = outputBitmap->pixelRef();
}
if (pr == NULL) {
return nullObjectReturn("Got null SkPixelRef");
}
if (!isMutable && javaBitmap == NULL) {
// promise we will never change our pixels (great for sharing and pictures)
pr->setImmutable();
}
// detach bitmap from its autodeleter, since we want to own it now
adb.detach();
if (javaBitmap != NULL) {
bool isPremultiplied = !requireUnpremultiplied;
GraphicsJNI::reinitBitmap(env, javaBitmap, outputBitmap, isPremultiplied);
outputBitmap->notifyPixelsChanged();
// If a java bitmap was passed in for reuse, pass it back
return javaBitmap;
}
int bitmapCreateFlags = 0x0;
if (isMutable) bitmapCreateFlags |= GraphicsJNI::kBitmapCreateFlag_Mutable;
if (!requireUnpremultiplied) bitmapCreateFlags |= GraphicsJNI::kBitmapCreateFlag_Premultiplied;
// now create the java bitmap
return GraphicsJNI::createBitmap(env, outputBitmap, javaAllocator.getStorageObj(),
bitmapCreateFlags, ninePatchChunk, layoutBounds, -1);
}
static SkPixelRef* installPixelRef(SkBitmap* bitmap, SkStreamRewindable* stream,
int sampleSize, bool ditherImage) {
SkImageRef* pr;
// only use ashmem for large images, since mmaps come at a price
if (bitmap->getSize() >= 32 * 1024) {
pr = new SkImageRef_ashmem(stream, bitmap->config(), sampleSize);
} else {
pr = new SkImageRef_GlobalPool(stream, bitmap->config(), sampleSize);
}
pr->setDitherImage(ditherImage);
bitmap->setPixelRef(pr)->unref();
pr->isOpaque(bitmap);
return pr;
}
3.4
从上面红色标注可以大概知道(根据名字),当Bitmap的size大于32k时,bitmap会被分配到ashmem中,不然被分配到一个叫global pool的地方。所以前面提到,“当inPurgeable为ture时,bitmap的pixels会被分配到ashmem上”这个说法过于绝对,不太准确(至少官方kitkat-release这个分支上),准确上应该说“当inPurgeable为ture且bitmap占用内存超过32k时,bitmap的pixels会被分配到ashmem上”。
而且,由于inPurgeable 为ture,所以decoder的SkImageDecoder::Mode decodeMode是kDecodeBounds_Mode,看decode函数:
bool SkImageDecoder::decode(SkStream* stream, SkBitmap* bm,
SkBitmap::Config pref, Mode mode) {
// we reset this to false before calling onDecode
fShouldCancelDecode = false;
// assign this, for use by getPrefConfig(), in case fUsePrefTable is false
fDefaultPref = pref;
// pass a temporary bitmap, so that if we return false, we are assured of
// leaving the caller's bitmap untouched.
SkBitmap tmp;
if (!this->onDecode(stream, &tmp, mode)) {
return false;
}
bm->swap(tmp);
return true;
}
3.5
假设decoder是SkPNGImageDecoder,那么
bool SkPNGImageDecoder::onDecode(SkStream* sk_stream, SkBitmap* decodedBitmap,
Mode mode) {
png_structp png_ptr;
png_infop info_ptr;
if (!onDecodeInit(sk_stream, &png_ptr, &info_ptr)) {
return false;
}
if (setjmp(png_jmpbuf(png_ptr))) {
return false;
}
PNGAutoClean autoClean(png_ptr, info_ptr);
png_uint_32 origWidth, origHeight;
int bitDepth, colorType, interlaceType;
png_get_IHDR(png_ptr, info_ptr, &origWidth, &origHeight, &bitDepth,
&colorType, &interlaceType, int_p_NULL, int_p_NULL);
SkBitmap::Config config;
bool hasAlpha = false;
SkPMColor theTranspColor = 0; // 0 tells us not to try to match
if (!this->getBitmapConfig(png_ptr, info_ptr, &config, &hasAlpha, &theTranspColor)) {
return false;
}
const int sampleSize = this->getSampleSize();
SkScaledBitmapSampler sampler(origWidth, origHeight, sampleSize);
decodedBitmap->setConfig(config, sampler.scaledWidth(), sampler.scaledHeight());
if (SkImageDecoder::kDecodeBounds_Mode == mode) {
return true;
}
................................
}
3.6
可见,如果decode mode是kDecodeBounds_Mode,则不会分配实际内存,只是计算出诸如宽高等信息。而且上面的描述我们只是从函数名推断pixels会被分配到ashmem中,但是哪里会分配到实际的内存呢?但看完整个deDecode的过程,都没有发现有内存分配的地方,那究竟内存什么时候分配,在哪里分配?
回到前面注释说的那段话:当pixels需要被重新访问的时候(例如bitmap draw或者调用getPixels()的时候),它们又可以重新被decode出来。那么猜想会不会在draw bitmap的时候bitmap的内存进行分配的呢?找到Canvas的drawBitmap方法,由此走上开始一段代码copy的路,看:
public void drawBitmap(@NonNull Bitmap bitmap, float left, float top, @Nullable Paint paint) {
throwIfCannotDraw(bitmap);
native_drawBitmap(mNativeCanvasWrapper, bitmap, left, top,
paint != null ? paint.getNativeInstance() : 0, mDensity, mScreenDensity, bitmap.mDensity);
}
{"native_drawBitmap","(IIFFIIII)V",
(void*) SkCanvasGlue::drawBitmap__BitmapFFPaint},
static void drawBitmap__BitmapFFPaint(JNIEnv* env, jobject jcanvas,
SkCanvas* canvas, SkBitmap* bitmap,
jfloat left, jfloat top,
SkPaint* paint, jint canvasDensity,
jint screenDensity, jint bitmapDensity) {
SkScalar left_ = SkFloatToScalar(left);
SkScalar top_ = SkFloatToScalar(top);
if (canvasDensity == bitmapDensity || canvasDensity == 0
|| bitmapDensity == 0) {
if (screenDensity != 0 && screenDensity != bitmapDensity) {
SkPaint filteredPaint;
if (paint) {
filteredPaint = *paint;
}
filteredPaint.setFilterBitmap(true);
canvas->drawBitmap(*bitmap, left_, top_, &filteredPaint);
} else {
canvas->drawBitmap(*bitmap, left_, top_, paint);
}
} else {
canvas->save();
SkScalar scale = SkFloatToScalar(canvasDensity / (float)bitmapDensity);
canvas->translate(left_, top_);
canvas->scale(scale, scale);
SkPaint filteredPaint;
if (paint) {
filteredPaint = *paint;
}
filteredPaint.setFilterBitmap(true);
canvas->drawBitmap(*bitmap, 0, 0, &filteredPaint);
canvas->restore();
}
}
void SkCanvas::drawBitmap(const SkBitmap& bitmap, SkScalar x, SkScalar y,
const SkPaint* paint) {
SkDEBUGCODE(bitmap.validate();)
if (NULL == paint || paint->canComputeFastBounds()) {
SkRect bounds = {
x, y,
x + SkIntToScalar(bitmap.width()),
y + SkIntToScalar(bitmap.height())
};
if (paint) {
(void)paint->computeFastBounds(bounds, &bounds);
}
if (this->quickReject(bounds)) {
return;
}
}
SkMatrix matrix;
matrix.setTranslate(x, y);
this->internalDrawBitmap(bitmap, matrix, paint);
}
void SkCanvas::internalDrawBitmap(const SkBitmap& bitmap,
const SkMatrix& matrix, const SkPaint* paint) {
if (reject_bitmap(bitmap)) {
return;
}
SkLazyPaint lazy;
if (NULL == paint) {
paint = lazy.init();
}
SkDEBUGCODE(bitmap.validate();)
CHECK_LOCKCOUNT_BALANCE(bitmap);
LOOPER_BEGIN(*paint, SkDrawFilter::kBitmap_Type)
while (iter.next()) {
iter.fDevice->drawBitmap(iter, bitmap, matrix, looper.paint());
}
LOOPER_END
}
void SkDevice::drawBitmap(const SkDraw& draw, const SkBitmap& bitmap,
const SkMatrix& matrix, const SkPaint& paint) {
draw.drawBitmap(bitmap, matrix, paint);
}
void SkDraw::drawBitmap(const SkBitmap& bitmap, const SkMatrix& prematrix,
const SkPaint& origPaint) const {
SkDEBUGCODE(this->validate();)
// nothing to draw
if (fRC->isEmpty() ||
bitmap.width() == 0 || bitmap.height() == 0 ||
bitmap.getConfig() == SkBitmap::kNo_Config) {
return;
}
SkPaint paint(origPaint);
paint.setStyle(SkPaint::kFill_Style);
SkMatrix matrix;
if (!matrix.setConcat(*fMatrix, prematrix)) {
return;
}
if (clipped_out(matrix, *fRC, bitmap.width(), bitmap.height())) {
return;
}
if (fBounder && just_translate(matrix, bitmap)) {
SkIRect ir;
int32_t ix = SkScalarRound(matrix.getTranslateX());
int32_t iy = SkScalarRound(matrix.getTranslateY());
ir.set(ix, iy, ix + bitmap.width(), iy + bitmap.height());
if (!fBounder->doIRect(ir)) {
return;
}
}
if (bitmap.getConfig() != SkBitmap::kA8_Config &&
just_translate(matrix, bitmap)) {
//
// It is safe to call lock pixels now, since we know the matrix is
// (more or less) identity.
//
SkAutoLockPixels alp(bitmap);
if (!bitmap.readyToDraw()) {
return;
}
int ix = SkScalarRound(matrix.getTranslateX());
int iy = SkScalarRound(matrix.getTranslateY());
if (clipHandlesSprite(*fRC, ix, iy, bitmap)) {
uint32_t storage[kBlitterStorageLongCount];
SkBlitter* blitter = SkBlitter::ChooseSprite(*fBitmap, paint, bitmap,
ix, iy, storage, sizeof(storage));
if (blitter) {
SkAutoTPlacementDelete<SkBlitter> ad(blitter, storage);
SkIRect ir;
ir.set(ix, iy, ix + bitmap.width(), iy + bitmap.height());
SkScan::FillIRect(ir, *fRC, blitter);
return;
}
}
}
................................
................................
}
class SkAutoLockPixels : public SkNoncopyable {
public:
SkAutoLockPixels(const SkBitmap& bm, bool doLock = true) : fBitmap(bm) {
fDidLock = doLock;
if (doLock) {
bm.lockPixels();
}
}
~SkAutoLockPixels() {
if (fDidLock) {
fBitmap.unlockPixels();
}
}
private:
const SkBitmap& fBitmap;
bool fDidLock;
};
void SkBitmap::lockPixels() const {
if (NULL != fPixelRef && 0 == sk_atomic_inc(&fPixelLockCount)) {
fPixelRef->lockPixels();
this->updatePixelsFromRef();
}
SkDEBUGCODE(this->validate();)
}
void SkPixelRef::lockPixels() {
SkASSERT(!fPreLocked || SKPIXELREF_PRELOCKED_LOCKCOUNT == fLockCount);
if (!fPreLocked) {
SkAutoMutexAcquire ac(*fMutex);
if (1 == ++fLockCount) {
fPixels = this->onLockPixels(&fColorTable);
}
}
}
3.7
最后一段代码,this->onLockPixels这里的this就是前面SkImageRef_ashmem对象,那么:
void* SkImageRef_ashmem::onLockPixels(SkColorTable** ct) {
SkASSERT(fBitmap.getPixels() == NULL);
SkASSERT(fBitmap.getColorTable() == NULL);
// fast case: check if we can just pin and get the cached data
if (-1 != fRec.fFD) {//由于未分配内存,fFD为-1
SkASSERT(fRec.fAddr);
SkASSERT(!fRec.fPinned);
int pin = ashmem_pin_region(fRec.fFD, 0, 0);
if (ASHMEM_NOT_PURGED == pin) { // yea, fast case!
fBitmap.setPixels(fRec.fAddr, fCT);
fRec.fPinned = true;
} else if (ASHMEM_WAS_PURGED == pin) {
ashmem_unpin_region(fRec.fFD, 0, 0);
// let go of our colortable if we lost the pixels. Well get it back
// again when we re-decode
if (fCT) {
fCT->unref();
fCT = NULL;
}
#if defined(DUMP_ASHMEM_LIFECYCLE) || defined(TRACE_ASH_PURGE)
SkDebugf("===== ashmem purged %d\n", fBitmap.getSize());
#endif
} else {
SkDebugf("===== ashmem pin_region(%d) returned %d\n", fRec.fFD, pin);
// return null result for failure
if (ct) {
*ct = NULL;
}
return NULL;
}
} else {
// no FD, will create an ashmem region in allocator
}
return this->INHERITED::onLockPixels(ct);
}
void* SkImageRef::onLockPixels(SkColorTable** ct) {
if (NULL == fBitmap.getPixels()) {
(void)this->prepareBitmap(SkImageDecoder::kDecodePixels_Mode);
}
if (ct) {
*ct = fBitmap.getColorTable();
}
return fBitmap.getPixels();
}
bool SkImageRef::prepareBitmap(SkImageDecoder::Mode mode) {
if (fErrorInDecoding) {
return false;
}
/* As soon as we really know our config, we record it, so that on
subsequent calls to the codec, we are sure we will always get the same
result.
*/
if (SkBitmap::kNo_Config != fBitmap.config()) {
fConfig = fBitmap.config();
}
if (NULL != fBitmap.getPixels() ||
(SkBitmap::kNo_Config != fBitmap.config() &&
SkImageDecoder::kDecodeBounds_Mode == mode)) {
return true;
}
SkASSERT(fBitmap.getPixels() == NULL);
if (!fStream->rewind()) {
SkDEBUGF(("Failed to rewind SkImageRef stream!"));
return false;
}
SkImageDecoder* codec;
if (fFactory) {
codec = fFactory->newDecoder(fStream);
} else {
codec = SkImageDecoder::Factory(fStream);
}
if (codec) {
SkAutoTDelete<SkImageDecoder> ad(codec);
codec->setSampleSize(fSampleSize);
codec->setDitherImage(fDoDither);
if (this->onDecode(codec, fStream, &fBitmap, fConfig, mode)) {
return true;
}
}
#ifdef DUMP_IMAGEREF_LIFECYCLE
if (NULL == codec) {
SkDebugf("--- ImageRef: <%s> failed to find codec\n", this->getURI());
} else {
SkDebugf("--- ImageRef: <%s> failed in codec for %d mode\n",
this->getURI(), mode);
}
#endif
fErrorInDecoding = true;
fBitmap.reset();
return false;
}
bool SkImageRef_ashmem::onDecode(SkImageDecoder* codec, SkStream* stream,
SkBitmap* bitmap, SkBitmap::Config config,
SkImageDecoder::Mode mode) {
if (SkImageDecoder::kDecodeBounds_Mode == mode) {
return this->INHERITED::onDecode(codec, stream, bitmap, config, mode);
}
// Ashmem memory is guaranteed to be initialized to 0.
codec->setSkipWritingZeroes(true);
AshmemAllocator alloc(&fRec, this->getURI());
codec->setAllocator(&alloc);
bool success = this->INHERITED::onDecode(codec, stream, bitmap, config,
mode);
// remove the allocator, since its on the stack
codec->setAllocator(NULL);
if (success) {
// remember the colortable (if any)
SkRefCnt_SafeAssign(fCT, bitmap->getColorTable());
return true;
} else {
if (fRec.fPinned) {
ashmem_unpin_region(fRec.fFD, 0, 0);
fRec.fPinned = false;
}
this->closeFD();
return false;
}
}
bool SkImageRef::onDecode(SkImageDecoder* codec, SkStream* stream,
SkBitmap* bitmap, SkBitmap::Config config,
SkImageDecoder::Mode mode) {
return codec->decode(stream, bitmap, config, mode);
}
bool SkImageDecoder::decode(SkStream* stream, SkBitmap* bm,
SkBitmap::Config pref, Mode mode) {
// we reset this to false before calling onDecode
fShouldCancelDecode = false;
// assign this, for use by getPrefConfig(), in case fUsePrefTable is false
fDefaultPref = pref;
// pass a temporary bitmap, so that if we return false, we are assured of
// leaving the caller's bitmap untouched.
SkBitmap tmp;
if (!this->onDecode(stream, &tmp, mode)) {
return false;
}
bm->swap(tmp);
return true;
}
3.8
假设图片是png图片,那么使用的decoder就是SkPNGImageDecoder:
bool SkPNGImageDecoder::onDecode(SkStream* sk_stream, SkBitmap* decodedBitmap,
Mode mode) {
png_structp png_ptr;
png_infop info_ptr;
if (!onDecodeInit(sk_stream, &png_ptr, &info_ptr)) {
return false;
}
if (setjmp(png_jmpbuf(png_ptr))) {
return false;
}
PNGAutoClean autoClean(png_ptr, info_ptr);
png_uint_32 origWidth, origHeight;
int bitDepth, colorType, interlaceType;
png_get_IHDR(png_ptr, info_ptr, &origWidth, &origHeight, &bitDepth,
&colorType, &interlaceType, int_p_NULL, int_p_NULL);
SkBitmap::Config config;
bool hasAlpha = false;
SkPMColor theTranspColor = 0; // 0 tells us not to try to match
if (!this->getBitmapConfig(png_ptr, info_ptr, &config, &hasAlpha, &theTranspColor)) {
return false;
}
const int sampleSize = this->getSampleSize();
SkScaledBitmapSampler sampler(origWidth, origHeight, sampleSize);
decodedBitmap->setConfig(config, sampler.scaledWidth(), sampler.scaledHeight());
if (SkImageDecoder::kDecodeBounds_Mode == mode) {
return true;
}
// from here down we are concerned with colortables and pixels
// we track if we actually see a non-opaque pixels, since sometimes a PNG sets its colortype
// to |= PNG_COLOR_MASK_ALPHA, but all of its pixels are in fact opaque. We care, since we
// draw lots faster if we can flag the bitmap has being opaque
bool reallyHasAlpha = false;
SkColorTable* colorTable = NULL;
if (colorType == PNG_COLOR_TYPE_PALETTE) {
decodePalette(png_ptr, info_ptr, &hasAlpha, &reallyHasAlpha, &colorTable);
}
SkAutoUnref aur(colorTable);
if (!this->allocPixelRef(decodedBitmap,
SkBitmap::kIndex8_Config == config ? colorTable : NULL)) {
return false;
}
.......................................
.......................................
}
bool SkImageDecoder::allocPixelRef(SkBitmap* bitmap,
SkColorTable* ctable) const {
return bitmap->allocPixels(fAllocator, ctable);
}
bool SkBitmap::allocPixels(Allocator* allocator, SkColorTable* ctable) {
HeapAllocator stdalloc;
if (NULL == allocator) {
allocator = &stdalloc;
}
return allocator->allocPixelRef(this, ctable);
}
3.9
这里的allocator就是AshmemAllocator:
class AshmemAllocator : public SkBitmap::Allocator {
public:
AshmemAllocator(SkAshmemRec* rec, const char name[])
: fRec(rec), fName(name) {}
virtual bool allocPixelRef(SkBitmap* bm, SkColorTable* ct) {
const size_t size = roundToPageSize(bm->getSize());
int fd = fRec->fFD;
void* addr = fRec->fAddr;
SkASSERT(!fRec->fPinned);
if (-1 == fd) {
SkASSERT(NULL == addr);
SkASSERT(0 == fRec->fSize);
fd = ashmem_create_region(fName, size);
#ifdef DUMP_ASHMEM_LIFECYCLE
SkDebugf("=== ashmem_create_region %s size=%d fd=%d\n", fName, size, fd);
#endif
if (-1 == fd) {
SkDebugf("------- imageref_ashmem create failed <%s> %d\n",
fName, size);
return false;
}
int err = ashmem_set_prot_region(fd, PROT_READ | PROT_WRITE);
if (err) {
SkDebugf("------ ashmem_set_prot_region(%d) failed %d\n",
fd, err);
close(fd);
return false;
}
addr = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_PRIVATE, fd, 0);
if (-1 == (long)addr) {
SkDebugf("---------- mmap failed for imageref_ashmem size=%d\n",
size);
close(fd);
return false;
}
fRec->fFD = fd;
fRec->fAddr = addr;
fRec->fSize = size;
} else {
SkASSERT(addr);
SkASSERT(size == fRec->fSize);
(void)ashmem_pin_region(fd, 0, 0);
}
bm->setPixels(addr, ct);
fRec->fPinned = true;
return true;
}
private:
// we just point to our caller's memory, these are not copies
SkAshmemRec* fRec;
const char* fName;
};
3.10
好了,到此为止,我们终于找到对于isPurgeable为true的图片,内存分配的时机和流程,只有当bitmap进行draw或者getPixels(这个大抵一致)的时候,bitmap 的pixels才进行实际的内存分配,而且也发现了前面说的当bitmap重新进行decode的时候会导致drop frames的原因,因为从上面代码看,bitmap draw的整个流程看,操作都处于UI线程,由于存在耗时的操作,自然就会导致不流畅的问题。
Why drop frames?
isPurgeable为ture的图片在UI线程上decode会出现掉帧问题,那怎么办?Fresco的思路是(个人推测),既然不管是初始decode还是被清除掉后re-decode,都会有卡顿问题,那么可以提前把该内存区域锁定住,既做到自行预先进行pixels的内存分配解决初始decode的卡顿,又可以防止该区域内存在系统内存资源紧张下被回收后re-decode时卡顿问题。恰恰,系统确实提供了这个方法,就是AndroidBitmap_lockPixels,把这个行为称作pin。
在Fresco的DalvikPurgeableDecoder中,
/**
* Creates a bitmap from encoded bytes.
*
* @param encodedImage the encoded image with reference to the encoded bytes
* @param bitmapConfig the {@link android.graphics.Bitmap.Config}
* used to create the decoded Bitmap
* @return the bitmap
* @throws TooManyBitmapsException if the pool is full
* @throws java.lang.OutOfMemoryError if the Bitmap cannot be allocated
*/
@Override
public CloseableReference<Bitmap> decodeFromEncodedImage(
final EncodedImage encodedImage,
Bitmap.Config bitmapConfig) {
BitmapFactory.Options options = getBitmapFactoryOptions(
encodedImage.getSampleSize(),
bitmapConfig);
CloseableReference<PooledByteBuffer> bytesRef = encodedImage.getByteBufferRef();
Preconditions.checkNotNull(bytesRef);
try {
Bitmap bitmap = decodeByteArrayAsPurgeable(bytesRef, options);
return pinBitmap(bitmap);
} finally {
CloseableReference.closeSafely(bytesRef);
}
}
4.1
每次decode完后,都会进行pinBitmap的操作,那么pinBitmap干什么?
/**
* Pins the bitmap
*/
public CloseableReference<Bitmap> pinBitmap(Bitmap bitmap) {
try {
// Real decoding happens here - if the image was corrupted, this will throw an exception
Bitmaps.pinBitmap(bitmap);
} catch (Exception e) {
bitmap.recycle();
throw Throwables.propagate(e);
}
if (!mUnpooledBitmapsCounter.increase(bitmap)) {
bitmap.recycle();
throw new TooManyBitmapsException();
}
return CloseableReference.of(bitmap, mUnpooledBitmapsCounter.getReleaser());
}
/**
* Pin the bitmap so that it cannot be 'purged'. Only makes sense for purgeable bitmaps
* WARNING: Use with caution. Make sure that the pinned bitmap is recycled eventually. Otherwise,
* this will simply eat up ashmem memory and eventually lead to unfortunate crashes.
* We *may* eventually provide an unpin method - but we don't yet have a compelling use case for
* that.
* @param bitmap the purgeable bitmap to pin
*/
public static void pinBitmap(Bitmap bitmap) {
Preconditions.checkNotNull(bitmap);
nativePinBitmap(bitmap);
}
{ "nativePinBitmap",
"(Landroid/graphics/Bitmap;)V",
(void*) Bitmaps_pinBitmap },
/**
* Pins bitmap's pixels.
*
* <p> Throws RuntimeException if unable to pin.
*/
static void Bitmaps_pinBitmap(
JNIEnv* env,
jclass clazz,
jobject bitmap) {
UNUSED(clazz);
int rc = AndroidBitmap_lockPixels(env, bitmap, 0);
if (rc != ANDROID_BITMAP_RESULT_SUCCESS) {
safe_throw_exception(env, "Failed to pin Bitmap");
}
}
int AndroidBitmap_lockPixels(JNIEnv* env, jobject jbitmap, void** addrPtr) {
if (NULL == env || NULL == jbitmap) {
return ANDROID_BITMAP_RESULT_BAD_PARAMETER;
}
SkBitmap* bm = GraphicsJNI::getNativeBitmap(env, jbitmap);
if (NULL == bm) {
return ANDROID_BITMAP_RESULT_JNI_EXCEPTION;
}
bm->lockPixels();
void* addr = bm->getPixels();
if (NULL == addr) {
bm->unlockPixels();
return ANDROID_BITMAP_RESULT_ALLOCATION_FAILED;
}
if (addrPtr) {
*addrPtr = addr;
}
return ANDROID_BITMAP_RESULT_SUCCESS;
}
4.2
lockPixels的代码前面已经分析过,就是实际进行内存分配的操作,所以pin的操作实际上就是预先分配好pixels的内存,避免在UI线程进行decode,另外再看一下lockPixels函数:
void SkBitmap::lockPixels() const {
if (NULL != fPixelRef && 0 == sk_atomic_inc(&fPixelLockCount)) {
fPixelRef->lockPixels();
this->updatePixelsFromRef();
}
SkDEBUGCODE(this->validate();)
}
4.3
该内存区域会采用引用计数的方法去管理,所以每一次lockPixels,fPixelLockCount都会自增,这样能够保证该内存区域不会被系统自动回收掉,这样就可以避免出现re-decode导致的卡顿问题,但是这就需要自己主动管理该内存区域。
讲到这里终于可以回顾之前的两个问题,并得出结论:
- Q:Decode过程是如何根据它来把pixels分配到ashmem?sdk文档中并没有提及到这一点,是不是真的如此?
A:isPurgeable为ture的情况下,bitmap的pixels如果超过32k情况下,会被分配到ashmem,分配的时机可以是在bitmap draw或者手动调用getPixels等方法时,而Fresco采用的是,主动执行“pin”操作去让pixels预先分配到ashmem。32k以下的bitmap会分配到global pool中(这里没有做详细分析,有兴趣的另行分析)。所以说,Fresco的bitmap在4.4及以下系统会被分配到ashmem的说法过于绝对。 - Q:Decode为什么会导致drop frames,又该如何解决?
A:导致drop frames的原因是,isPurgeable为ture的bitmap会在UI线程中进行decode,过于耗时,所以自然可能存在卡顿问题。解决的办法就是,预先进行“pin”操作去预先decode出bitmap的pixels到内存(ashmem或者global pool),并可以锁定该内存区域,防止系统回收掉该内存区域,导致re-decode。
GingerbreadPurgeableDecoder
前面对KitKatPurgeableDecoder进行了详细的分析,至于GingerbreadPurgeableDecoder,由于BitmapOptions的isPurgeable都为true,所以在使用系统API(主要是BitmapFactory的decode系列方法)进行decode的过程基本和KitKatPurgeableDecoder是完全一致的。但是,GingerbreadPurgeableDecoder当前master分支上的代码非常让人费解(可能是issue或者可能已经deprecated),其中有这样的一个函数:
/** * * source : com/facebook/imagepipeline/platform/GingerbreadPurgeableDecoder.java * */
protected Bitmap decodeFileDescriptorAsPurgeable(
CloseableReference<PooledByteBuffer> bytesRef,
int inputLength,
byte[] suffix,
BitmapFactory.Options options) {
MemoryFile memoryFile = null;
try {
memoryFile = copyToMemoryFile(bytesRef, inputLength, suffix);
FileDescriptor fd = getMemoryFileDescriptor(memoryFile);
Bitmap bitmap = sWebpBitmapFactory.decodeFileDescriptor(fd, null, options);
return Preconditions.checkNotNull(bitmap, "BitmapFactory returned null");
} catch (IOException e) {
throw Throwables.propagate(e);
} finally {
if (memoryFile != null) {
memoryFile.close();
}
}
}
5.1
它的decode方法依赖于sWebpBitmapFactory这样的一个WebpBitmapFactory对象。为什么说费解?前面说道,4.4及以下如果directWebpDirectDecodingEnabled为true,则使用GingerbreadPurgeableDecoder。而directWebpDirectDecodingEnabled == mConfig.getExperiments().isWebpSupportEnabled()这个值,是对ImagePipelineConfig初始化的时候应用设置的,含义是是否支持Webp格式的图片,假设希望支持,那么设置为true。那么我们再看下面在ImagePipelineConfig的构造函数的部分代码:
/** * * source : com/facebook/imagepipeline/core/ImagePipelineConfig.java * */
// Here we manage the WebpBitmapFactory implementation if any
WebpBitmapFactory webpBitmapFactory = mImagePipelineExperiments.getWebpBitmapFactory();
if (webpBitmapFactory != null) {
BitmapCreator bitmapCreator = new HoneycombBitmapCreator(getPoolFactory());
setWebpBitmapFactory(webpBitmapFactory, mImagePipelineExperiments, bitmapCreator);
} else {
// We check using introspection only if the experiment is enabled
if (mImagePipelineExperiments.isWebpSupportEnabled() &&
WebpSupportStatus.sIsWebpSupportRequired) {
webpBitmapFactory = WebpSupportStatus.loadWebpBitmapFactoryIfExists();
if (webpBitmapFactory != null) {
BitmapCreator bitmapCreator = new HoneycombBitmapCreator(getPoolFactory());
setWebpBitmapFactory(webpBitmapFactory, mImagePipelineExperiments, bitmapCreator);
}
}
}
5.2
/** * * source : com/facebook/common/webp/WebpSupportStatus.java * */
public static final boolean sIsWebpSupportRequired =
Build.VERSION.SDK_INT <= Build.VERSION_CODES.JELLY_BEAN_MR1;
5.3
粗体部分意思是,如果应用设置了isWebpSupportEnabled而且SDK_INT <= 17那么会初始化一个WebpBitmapFactory实例,SDK_INT>=18该实例为NULL。但是回过头去看前面说的“4.4以下如果directWebpDirectDecodingEnabled为true,则使用GingerbreadPurgeableDecoder”,那问题就出现了,SDK_INT=18的虽然会使用GingerbreadPurgeableDecoder去decode bitmap,但是sWebpBitmapFactory却为NULL,然后就会出现NullPointerException。拿了一部API 18的手机实测,该情况确实如此,只不过可能由于webp support部分还是处于“实验”阶段,出现issue也正常。
抛开issue回到GingerbreadPurgeableDecoder上继续说。从2.1代码可知道,GingerbreadPurgeableDecoderdecode前会先把原始数据(encoded data)会copy到MemoryFile中去:
private static MemoryFile copyToMemoryFile(
CloseableReference<PooledByteBuffer> bytesRef,
int inputLength,
@Nullable byte[] suffix) throws IOException {
int outputLength = inputLength + (suffix == null ? 0 : suffix.length);
MemoryFile memoryFile = new MemoryFile(null, outputLength);
memoryFile.allowPurging(false);
PooledByteBufferInputStream pbbIs = null;
LimitedInputStream is = null;
OutputStream os = null;
try {
pbbIs = new PooledByteBufferInputStream(bytesRef.get());
is = new LimitedInputStream(pbbIs, inputLength);
os = memoryFile.getOutputStream();
ByteStreams.copy(is, os);
if (suffix != null) {
memoryFile.writeBytes(suffix, 0, inputLength, suffix.length);
}
return memoryFile;
} finally {
CloseableReference.closeSafely(bytesRef);
Closeables.closeQuietly(pbbIs);
Closeables.closeQuietly(is);
Closeables.close(os, true);
}
}
5.4
从中的好处就是,借助MemoryFile把encoded data拷贝到ashmem中去,尽量避免在Java Heap上分配内存而造成频繁GC的问题,这相对于KitKatPurgeableDecoder直接把encoded data拷贝到byte array,也就是Java Heap上是一个优势。(至于MemoryFile是如何能够映射数据到ashmem的细节,可以参看前面说的罗升阳说的那边文章。)
详细看一下GingerbreadPurgeableDecoder的decode过程:
@Override
public Bitmap decodeFileDescriptor(
FileDescriptor fd,
Rect outPadding,
BitmapFactory.Options opts) {
return hookDecodeFileDescriptor(fd, outPadding, opts);
}
@DoNotStrip
public static Bitmap hookDecodeFileDescriptor(
FileDescriptor fd,
Rect outPadding,
BitmapFactory.Options opts) {
StaticWebpNativeLoader.ensure();
Bitmap bitmap;
boolean isWebp = false;
long originalSeekPosition = nativeSeek(fd, 0, false);
if (originalSeekPosition != -1) {
InputStream inputStream = wrapToMarkSupportedStream(new FileInputStream(fd));
try {
byte[] header = getWebpHeader(inputStream, opts);
isWebp = isWebpHeader(header, 0, HEADER_SIZE);
boolean isWebpSupported = isWebpSupportedByPlatform(header, 0, HEADER_SIZE);
if (isWebp && !isWebpSupported) {
bitmap = nativeDecodeStream(
inputStream,
opts,
getScaleFromOptions(opts),
getInTempStorageFromOptions(opts));
// We send error if the direct decode failed
sendWebpErrorLog("webp_direct_decode_fd", bitmap);
setPaddingDefaultValues(outPadding);
setWebpBitmapOptions(bitmap, opts);
} else {
nativeSeek(fd, originalSeekPosition, true);
bitmap = originalDecodeFileDescriptor(fd, outPadding, opts);
if (bitmap == null && isWebp) {
// Notify that the native decode has failed and that we're trying to decode directly
sendWebpErrorLog("webp_native_decode_fd_fallback", bitmap);
// We fallback into our code for decoding
bitmap = nativeDecodeStream(
new FileInputStream(fd),
opts,
getScaleFromOptions(opts),
getInTempStorageFromOptions(opts));
// Notify that the direct decoder failed after native decoder
sendWebpErrorLog("webp_direct_decode_fd_fallback", bitmap);
setWebpBitmapOptions(bitmap, opts);
}
}
} finally {
try {
inputStream.close();
} catch (Throwable t) {
/* ignore */
}
}
} else {
bitmap = hookDecodeStream(new FileInputStream(fd), outPadding, opts);
if (bitmap == null && isWebp) {
// Notify that the native decoding was wrong
sendWebpErrorLog("webp_native_decode_out_seek_fd_fallback", bitmap);
// We fallback into our code for decoding
bitmap = nativeDecodeStream(
new FileInputStream(fd),
opts,
getScaleFromOptions(opts),
getInTempStorageFromOptions(opts));
// Notify if the direct decoding has failed after native decoder
sendWebpErrorLog("webp_direct_decode_out_seek_fd_fallback", bitmap);
setWebpBitmapOptions(bitmap, opts);
}
setPaddingDefaultValues(outPadding);
}
return bitmap;
}
5.5
hookDecodeFileDescriptor函数,判断如果是webp格式的图片,而且判断当前系统不支持decode该webp图片格式(android系统对多媒体格式支持的可以看https://developer.android.com/guide/topics/media/media-formats.html)
不支持的话,就调用自己的nativeDecodeStream方法去解析webp图片,如果系统本身就支持本webp图片,那么直接调用originalDecodeFileDescriptor方法:
@DoNotStrip
private static Bitmap originalDecodeFileDescriptor(
FileDescriptor fd,
Rect outPadding,
BitmapFactory.Options opts) {
return BitmapFactory.decodeFileDescriptor(fd, outPadding, opts);
}
5.6
直接调用BitmapFactory.decodeFileDescriptor的decode过程,就和上面KitKatPurgeableDecoder的decode过程没有太大差异了,唯一不同就是inInputShareable的处理:
static jobject nativeDecodeFileDescriptor(JNIEnv* env, jobject clazz, jobject fileDescriptor,
jobject padding, jobject bitmapFactoryOptions) {
NPE_CHECK_RETURN_ZERO(env, fileDescriptor);
jint descriptor = jniGetFDFromFileDescriptor(env, fileDescriptor);
struct stat fdStat;
if (fstat(descriptor, &fdStat) == -1) {
doThrowIOE(env, "broken file descriptor");
return nullObjectReturn("fstat return -1");
}
bool isPurgeable = optionsPurgeable(env, bitmapFactoryOptions);
bool isShareable = optionsShareable(env, bitmapFactoryOptions);
bool weOwnTheFD = false;
if (isPurgeable && isShareable) {
int newFD = ::dup(descriptor);
if (-1 != newFD) {
weOwnTheFD = true;
descriptor = newFD;
}
}
SkAutoTUnref<SkData> data(SkData::NewFromFD(descriptor));
if (data.get() == NULL) {
return nullObjectReturn("NewFromFD failed in nativeDecodeFileDescriptor");
}
SkAutoTUnref<SkMemoryStream> stream(new SkMemoryStream(data));
/* Allow purgeable iff we own the FD, i.e., in the puregeable and
shareable case.
*/
return doDecode(env, stream, padding, bitmapFactoryOptions, weOwnTheFD);
}
只有isPurgeable和isShareable都为true时,才能去dup fd,不然就是引用旧的fd。
至于调用Fresco的nativeDecodeStream方法去解析webp图片的过程,这里就不在赘述了,感兴趣可以自行分析(依然是把pixels放到ashmem中去,通过BitmapCreator)。
ArtDecoder
最后简单的看看ArtDecoder,该decoder没有使用什么“trick”去做一些黑科技,直接使用BitmapFactory的decode方法:
protected CloseableReference<Bitmap> decodeStaticImageFromStream(
InputStream inputStream,
BitmapFactory.Options options) {
Preconditions.checkNotNull(inputStream);
int sizeInBytes = BitmapUtil.getSizeInByteForBitmap(
options.outWidth,
options.outHeight,
options.inPreferredConfig);
final Bitmap bitmapToReuse = mBitmapPool.get(sizeInBytes);
if (bitmapToReuse == null) {
throw new NullPointerException("BitmapPool.get returned null");
}
options.inBitmap = bitmapToReuse;
Bitmap decodedBitmap;
ByteBuffer byteBuffer = mDecodeBuffers.acquire();
if (byteBuffer == null) {
byteBuffer = ByteBuffer.allocate(DECODE_BUFFER_SIZE);
}
try {
options.inTempStorage = byteBuffer.array();
decodedBitmap = BitmapFactory.decodeStream(inputStream, null, options);
} catch (RuntimeException re) {
mBitmapPool.release(bitmapToReuse);
throw re;
} finally {
mDecodeBuffers.release(byteBuffer);
}
if (bitmapToReuse != decodedBitmap) {
mBitmapPool.release(bitmapToReuse);
decodedBitmap.recycle();
throw new IllegalStateException();
}
return CloseableReference.of(decodedBitmap, mBitmapPool);
}
/**
* Options returned by this method are configured with mDecodeBuffer which is GuardedBy("this")
*/
private static BitmapFactory.Options getDecodeOptionsForStream(
EncodedImage encodedImage,
Bitmap.Config bitmapConfig) {
final BitmapFactory.Options options = new BitmapFactory.Options();
// Sample size should ONLY be different than 1 when downsampling is enabled in the pipeline
options.inSampleSize = encodedImage.getSampleSize();
options.inJustDecodeBounds = true;
// fill outWidth and outHeight
BitmapFactory.decodeStream(encodedImage.getInputStream(), null, options);
if (options.outWidth == -1 || options.outHeight == -1) {
throw new IllegalArgumentException();
}
options.inJustDecodeBounds = false;
options.inDither = true;
options.inPreferredConfig = bitmapConfig;
options.inMutable = true;
return options;
}
6.1
使用了BitmapOptions的inBitmap和inTempStorage去优化内存使用。inBitmap是由上层的BitmapPool去分配内存,inTempStorage是由SynchronizedPool分配内存,都是用缓存池的方式分配和回收内存,做到对这些区域的内存可管理,减少各个不同地方自行分配内存。
Unpin
既然存在“pin”操作,自然就应该会有unpin的操作,因为pin操作导致了内存区域的引用计数增加,只有通过unpin操作,才能把相应的引用计数减少,保证内存能够被回收,不然就会导致oom。
Fresco中确实提供了这样的方法,进行“unpin”操作:
public static void releaseByteBuffer(Bitmap bitmap) {
Preconditions.checkNotNull(bitmap);
nativeReleaseByteBuffer(bitmap);
}
6.2
对应JNI的方法就是:
static void Bitmaps_releaseByteBuffer(
JNIEnv* env,
jclass clazz,
jobject bitmap) {
UNUSED(clazz);
int rc = AndroidBitmap_unlockPixels(env, bitmap);
if (rc != ANDROID_BITMAP_RESULT_SUCCESS) {
safe_throw_exception(env, "Failed to unlock Bitmap pixels");
}
}
6.3
其实就是AndroidBitmap_unlockPixels
int AndroidBitmap_unlockPixels(JNIEnv* env, jobject jbitmap) {
if (NULL == env || NULL == jbitmap) {
return ANDROID_BITMAP_RESULT_BAD_PARAMETER;
}
SkBitmap* bm = GraphicsJNI::getNativeBitmap(env, jbitmap);
if (NULL == bm) {
return ANDROID_BITMAP_RESULT_JNI_EXCEPTION;
}
// notifyPixelsChanged() needs be called to apply writes to GL-backed
// bitmaps. Note that this will slow down read-only accesses to the
// bitmaps, but the NDK methods are primarily intended to be used for
// writes.
bm->notifyPixelsChanged();
bm->unlockPixels();
return ANDROID_BITMAP_RESULT_SUCCESS;
}
void SkBitmap::unlockPixels() const {
SkASSERT(NULL == fPixelRef || fPixelLockCount > 0);
if (NULL != fPixelRef && 1 == sk_atomic_dec(&fPixelLockCount)) {
fPixelRef->unlockPixels();
this->updatePixelsFromRef();
}
SkDEBUGCODE(this->validate();)
}
void SkPixelRef::unlockPixels() {
SkASSERT(!fPreLocked || SKPIXELREF_PRELOCKED_LOCKCOUNT == fLockCount);
if (!fPreLocked) {
SkAutoMutexAcquire ac(*fMutex);
SkASSERT(fLockCount > 0);
if (0 == --fLockCount) {
this->onUnlockPixels();
fPixels = NULL;
fColorTable = NULL;
}
}
}
6.4
如果是SkPixelRef是SkImageRef_ashmem,则:
void SkImageRef_ashmem::onUnlockPixels() {
this->INHERITED::onUnlockPixels();
if (-1 != fRec.fFD) {
SkASSERT(fRec.fAddr);
SkASSERT(fRec.fPinned);
//回收bitmap在ashmem的内存
ashmem_unpin_region(fRec.fFD, 0, 0);
fRec.fPinned = false;
}
// we clear this with or without an error, since we've either closed or
// unpinned the region
//设置pixels为NULL
fBitmap.setPixels(NULL, NULL);
}
6.5
可见,调用releaseByteBuffer用就能够回收掉bitmap占用的内存,但前提是没有别的地方对这块内存进行过lock操作,因为lock操作会导致引用计数增加。但是,奇怪的是,在Fresco中,没有任何一个地方会调用该方法,那不会导致OOM吗?
找了很久,终于发现,我们不一定需要执行unpin操作进行内存回收,直接对Bitmap调用recycle方法也是可以的,这也是我们最常用的回收Bitmap的方法,而且是系统提供的API。看Bitmap的recycle方法:
/**
* Free the native object associated with this bitmap, and clear the
* reference to the pixel data. This will not free the pixel data synchronously;
* it simply allows it to be garbage collected if there are no other references.
* The bitmap is marked as "dead", meaning it will throw an exception if
* getPixels() or setPixels() is called, and will draw nothing. This operation
* cannot be reversed, so it should only be called if you are sure there are no
* further uses for the bitmap. This is an advanced call, and normally need
* not be called, since the normal GC process will free up this memory when
* there are no more references to this bitmap.
*/
public void recycle() {
if (!mRecycled) {
if (nativeRecycle(mNativeBitmap)) {
// return value indicates whether native pixel object was actually recycled.
// false indicates that it is still in use at the native level and these
// objects should not be collected now. They will be collected later when the
// Bitmap itself is collected.
mBuffer = null;
mNinePatchChunk = null;
}
mRecycled = true;
}
}
6.6
nativeRecycle对应的JNI方法是:
static jboolean Bitmap_recycle(JNIEnv* env, jobject, SkBitmap* bitmap) {
#ifdef USE_OPENGL_RENDERER
if (android::uirenderer::Caches::hasInstance()) {
return android::uirenderer::Caches::getInstance().resourceCache.recycle(bitmap);
}
#endif // USE_OPENGL_RENDERER
bitmap->setPixels(NULL, NULL);
return true;
}
void SkBitmap::setPixels(void* p, SkColorTable* ctable) {
if (NULL == p) {
this->setPixelRef(NULL, 0);
return;
}
Sk64 size = this->getSize64();
SkASSERT(!size.isNeg() && size.is32());
this->setPixelRef(new SkMallocPixelRef(p, size.get32(), ctable, false))->unref();
// since we're already allocated, we lockPixels right away
this->lockPixels();
SkDEBUGCODE(this->validate();)
}
SkPixelRef* SkBitmap::setPixelRef(SkPixelRef* pr, size_t offset) {
// do this first, we that we never have a non-zero offset with a null ref
if (NULL == pr) {
offset = 0;
}
if (fPixelRef != pr || fPixelRefOffset != offset) {
if (fPixelRef != pr) {
this->freePixels();
SkASSERT(NULL == fPixelRef);
SkSafeRef(pr);
fPixelRef = pr;
}
fPixelRefOffset = offset;
this->updatePixelsFromRef();
}
SkDEBUGCODE(this->validate();)
return pr;
}
void SkBitmap::freePixels() {
// if we're gonna free the pixels, we certainly need to free the mipmap
this->freeMipMap();
if (fColorTable) {
fColorTable->unref();
fColorTable = NULL;
}
if (NULL != fPixelRef) {
if (fPixelLockCount > 0) {
fPixelRef->unlockPixels();
}
fPixelRef->unref();
fPixelRef = NULL;
fPixelRefOffset = 0;
}
fPixelLockCount = 0;
fPixels = NULL;
}
6.7
调用recycle方法,不管是引用计数的多少,直接把pixels回收掉,引用计数也置为0。所以Fresco是用recycle的方法进行bitmap的内存回收:
public BitmapCounter(int maxCount, int maxSize) {
Preconditions.checkArgument(maxCount > 0);
Preconditions.checkArgument(maxSize > 0);
mMaxCount = maxCount;
mMaxSize = maxSize;
mUnpooledBitmapsReleaser = new ResourceReleaser<Bitmap>() {
@Override
public void release(Bitmap value) {
try {
decrease(value);
} finally {
value.recycle();
}
}
};
}
6.8
结束语
到此为止,Fresco对bitmap decode的内存分配问题已经算是比较深入分析了,但是还留待一点思考的空间:
- Art模式下,究竟对GC做了什么优化的地方,不会导致stop-the-world的问题呢?
- Fresco的开发人员是有多了解Bitmap的decode和android平台特性,才能够找到以isPurgeable为切入点,让Bitmap的pixels分配到ashmem上,并知道drop frames问题并提出解决方法,OMG。
0 0
- 谈谈fresco的bitmap内存分配
- android bitmap的内存分配和优化
- bitmap 内存分配
- Android性能优化:谈谈Bitmap的内存管理与优化
- Android性能优化:谈谈Bitmap的内存管理与优化
- 谈谈C#中的内存分配
- 谈谈C#中的内存分配
- Fresco 缓存自定的 Bitmap
- 谈谈jvm内存分配和管理
- jemalloc横向分析(四) tcache分配内存中使用到的位图bitmap
- Bitmap的内存缓存
- Bitmap 的内存优化
- Bitmap的内存占用
- Bitmap的内存优化
- 谈谈内存的使用
- Bitmap详解与Bitmap的内存优化
- Fresco 由缓存转Bitmap
- weblogic的内存分配
- JDK版本报错:override a superclass method和java compiler level does not match the version
- OC代码规范总结
- 一些常用的CSS兼容性写法
- Docker
- Jump Game
- 谈谈fresco的bitmap内存分配
- 水题 模板
- sed命令详解
- Nginx源码分析 - Nginx启动以及IOCP模型
- linux shell
- bzoj4383
- 【Linux】将Oracle安装目录从根目录下迁移到逻辑卷
- 2016新财富最佳分析师榜单全揭晓
- The need for a POP POP RET instruction sequence