Android MultiDex机制杂谈

来源：互联网发布：ubuntu安装eclipse c 编辑：程序博客网时间：2024/05/22 06:47

0x00 为什么需要MultiDex

如果你是一名android开发者，随着app功能复杂度的增加，代码量的增多和库的不断引入，你迟早会在5.0以下的某款设备上遇到：

Conversion to Dalvik format failed:Unable to execute dex: method ID not in [0, 0xffff]: 65536

或者

trouble writing output:Too many field references: 131000; max is 65536.You may try using --multi-dex option.

这说明你的app的main dex方法数已经超过65535，如果打算继续兼容5.0以下手机，你可以采用google提供的MultiDex方案，但main dex方法数为什么不能超过65535呢？

其实在Dalvik的invoke-kind指令集中，method reference index只留了16bits，最多能调用65535个方法。

invoke-kind {vC, vD, vE, vF, vG}, meth@BBBBB: method reference index (16 bits)

所以在生成dex文件的过程中，当方法数超过65535就会报错。我们可以在dx工具源码中找到一些线索：

dalvik/dx/src/com/android/dx/merge/IndexMap.java

/** * Maps the index offsets from one dex file to those in another. For example, if * you have string #5 in the old dex file, its position in the new dex file is * {@code strings[5]}. */public final class IndexMap {    private final Dex target;    public final int[] stringIds;    public final short[] typeIds;    public final short[] protoIds;    public final short[] fieldIds;    public final short[] methodIds;

可以看到，methodIds，typeIds，protoIds，fieldIds都是short[]类型，对于每个具体的method来说都是限制在16bits。google dalvik开发者在这上面挖了个坑，MultiDex就是来填坑的。

0x01 使用MultiDex

MultiDex出现在google官方提供的support包里面，使用的时候需要在build.gradle中加上依赖：

compile 'com.android.support:multidex:1.0.0'

同时让app的Application继承MultiDexApplication

public class MultiDexApplication extends Application {    public MultiDexApplication() {    }    protected void attachBaseContext(Context base) {        super.attachBaseContext(base);        MultiDex.install(this);    }}

也别忘了在build.gradle中修改：

multiDexEnabled true

0x02 MultiDex.install过程

先从MultiDex.install开始分析，传入的context是MultiDexApplication，loader是dalvik.system.PathClassLoader，运行时会先去app的dexDir /data/data/pkgname/code_cache/secondary-dexes下找secondary dex(除了main dex，其他都叫secondary dex，一个apk中可能存在多个secondary dex)，找到后先校验zip格式，没问题就直接installSecondaryDexes，否则会去强制从apk中重新extract secondary dex。

MultiDex.java

public static void install(Context context) {...     ClassLoader loader;     try {         loader = context.getClassLoader();         ...         File dexDir = new File(applicationInfo.dataDir, SECONDARY_FOLDER_NAME);         List<File> files = MultiDexExtractor.load(context, applicationInfo, dexDir, false);            if (checkValidZipFiles(files)) {                installSecondaryDexes(loader, dexDir, files);            } else {                Log.w(TAG, "Files were not valid zip files.  Forcing a reload.");                // Try again, but this time force a reload of the zip file.                files = MultiDexExtractor.load(context, applicationInfo, dexDir, true);                if (checkValidZipFiles(files)) {                    installSecondaryDexes(loader, dexDir, files);                } else {                    // Second time didn't work, give up                    throw new RuntimeException("Zip files were not valid.");                }            }

MultiDexExtractor.load方法中，sourceApk指向的是/data/app/pkgname.apk，然后通过getZipCrc获取apk的CRC校验码，去和最后一次CRC校验码对比，若一致或者不是forceReload，那么直接loadExistingExtractions，loadExistingExtractions直接为/data/data/pkgname/code_cache/secondary-dexes/下已经存在的.dex创建File对象；如果不一致说明apk已经被修改了，dex需要重新从apk中抽取，此时执行performExtractions。

MultiDexExtractor.java

static List<File> load(Context context, ApplicationInfo applicationInfo, File dexDir,        boolean forceReload) throws IOException {    Log.i(TAG, "MultiDexExtractor.load(" + applicationInfo.sourceDir + ", " + forceReload + ")");    final File sourceApk = new File(applicationInfo.sourceDir);    long currentCrc = getZipCrc(sourceApk);    List<File> files;    if (!forceReload && !isModified(context, sourceApk, currentCrc)) {        try {            files = loadExistingExtractions(context, sourceApk, dexDir);        } catch (IOException ioe) {            Log.w(TAG, "Failed to reload existing extracted secondary dex files,"                    + " falling back to fresh extraction", ioe);            files = performExtractions(sourceApk, dexDir);            putStoredApkInfo(context, getTimeStamp(sourceApk), currentCrc, files.size() + 1);        }    } else {        Log.i(TAG, "Detected that extraction must be performed.");        files = performExtractions(sourceApk, dexDir);        putStoredApkInfo(context, getTimeStamp(sourceApk), currentCrc, files.size() + 1);    }    Log.i(TAG, "load found " + files.size() + " secondary dex files");    return files;}

performExtractions中真正抽取在extract方法中，输入参数分别是：

apk(ZipFile) 指向/data/app/pkgname.apk
dexFile(ZipEntry) 指向classes2.dex
extractTo(File) 指向/data/data/pkgname/code_cache/secondary-dexes/pkgname.apk.classes2.zip
extractedFilePrefix(String) pkgname.apk.classes

extract把dexFile写入到extractTo指向的一个entry，其中InputStream读的是apk中的classes2.apk，ZipOutputStream指向的是一个tmpFile，具体步骤为：

在/data/data/pkgname/code_cache/secondary-dexes/创建pkgname.apk.classes12345.zip的tmpFile；
对步骤1的tmpFile建立ZipOutputStream；
创建一个指向classes.dex的ZipEntry对象；
向tmpFile写入这个entry；
将tmpFile重命名为pkgname.apk.classes2.zip，tmpFile此时还存在；
删除tmpFile；

extract的实际效果就是在/data/data/pkgname/code_cache/secondary-dexes下创建了：

/data/data/pkgname/code_cache/secondary-dexes/pkgname.apk.classes2.zip

/data/data/pkgname/code_cache/secondary-dexes/pkgname.apk.classes3.zip

…

/data/data/pkgname/code_cache/secondary-dexes/pkgname.apk.classesN.zip

N是MultiDex拆分后dex的个数

private static void extract(ZipFile apk, ZipEntry dexFile, File extractTo,        String extractedFilePrefix) throws IOException, FileNotFoundException {    InputStream in = apk.getInputStream(dexFile);    ZipOutputStream out = null;    File tmp = File.createTempFile(extractedFilePrefix, EXTRACTED_SUFFIX,            extractTo.getParentFile());    Log.i(TAG, "Extracting " + tmp.getPath());    try {        out = new ZipOutputStream(new BufferedOutputStream(new FileOutputStream(tmp)));        try {            ZipEntry classesDex = new ZipEntry("classes.dex");            // keep zip entry time since it is the criteria used by Dalvik            classesDex.setTime(dexFile.getTime());            out.putNextEntry(classesDex);            byte[] buffer = new byte[BUFFER_SIZE];            int length = in.read(buffer);            while (length != -1) {                out.write(buffer, 0, length);                length = in.read(buffer);            }            out.closeEntry();        } finally {            out.close();        }        Log.i(TAG, "Renaming to " + extractTo.getPath());        if (!tmp.renameTo(extractTo)) {            throw new IOException("Failed to rename \"" + tmp.getAbsolutePath() +                    "\" to \"" + extractTo.getAbsolutePath() + "\"");        }    } finally {        closeQuietly(in);        tmp.delete(); // return status ignored    }}

最后到真正的install阶段，在MultiDex中有三个私有嵌套类V19,V14和V4来负责具体的系统版本，分别对应android 4.4以上，4.0以上和4.0以下系统。

private static void installSecondaryDexes(ClassLoader loader, File dexDir, List<File> files)        throws IllegalArgumentException, IllegalAccessException, NoSuchFieldException,        InvocationTargetException, NoSuchMethodException, IOException {    if (!files.isEmpty()) {        if (Build.VERSION.SDK_INT >= 19) {            V19.install(loader, files, dexDir);        } else if (Build.VERSION.SDK_INT >= 14) {            V14.install(loader, files, dexDir);        } else {            V4.install(loader, files);        }    }}

以4.4为例，V19.install其实就是对输入additionalClassPathEntries反射调用makeDexElements创建Element[]对象，再去修改dalvik.system.BaseDexClassLoader的pathList字段表示的DexPathList类中的dexElements字段内容，把Element[]对象添加进去，这样以后dalvik.system.PathClassLoader就可以找到在secondary dex中的class了。

private static final class V19 {    private static void install(ClassLoader loader, List<File> additionalClassPathEntries,            File optimizedDirectory)                    throws IllegalArgumentException, IllegalAccessException,                    NoSuchFieldException, InvocationTargetException, NoSuchMethodException {        /* The patched class loader is expected to be a descendant of         * dalvik.system.BaseDexClassLoader. We modify its         * dalvik.system.DexPathList pathList field to append additional DEX         * file entries.         */        Field pathListField = findField(loader, "pathList");        Object dexPathList = pathListField.get(loader);        ArrayList<IOException> suppressedExceptions = new ArrayList<IOException>();        expandFieldArray(dexPathList, "dexElements", makeDexElements(dexPathList,                new ArrayList<File>(additionalClassPathEntries), optimizedDirectory,                suppressedExceptions));        if (suppressedExceptions.size() > 0) {            for (IOException e : suppressedExceptions) {                Log.w(TAG, "Exception in makeDexElement", e);            }            Field suppressedExceptionsField =                    findField(loader, "dexElementsSuppressedExceptions");            IOException[] dexElementsSuppressedExceptions =                    (IOException[]) suppressedExceptionsField.get(loader);            if (dexElementsSuppressedExceptions == null) {                dexElementsSuppressedExceptions =                        suppressedExceptions.toArray(                                new IOException[suppressedExceptions.size()]);            } else {                IOException[] combined =                        new IOException[suppressedExceptions.size() +                                        dexElementsSuppressedExceptions.length];                suppressedExceptions.toArray(combined);                System.arraycopy(dexElementsSuppressedExceptions, 0, combined,                        suppressedExceptions.size(), dexElementsSuppressedExceptions.length);                dexElementsSuppressedExceptions = combined;            }            suppressedExceptionsField.set(loader, dexElementsSuppressedExceptions);        }    }

还有一点需要注意的是，DexPathList:makeDexElements最终会去做dex2opt，其中optimizedDirectory就是之前的dexDir，优化后的dex文件是pkgname.apk.classes2.dex，然而dex2opt会消耗较多的cpu时间，如果全部放在main线程去处理的话，比较影响用户体验，甚至可能引起ANR。

DexPathList.java

private static Element[] makeDexElements(ArrayList<File> files, File optimizedDirectory,                                         ArrayList<IOException> suppressedExceptions) {    ArrayList<Element> elements = new ArrayList<Element>();    for (File file : files) {        File zip = null;        DexFile dex = null;        String name = file.getName();        if (file.isDirectory()) {        } else if (file.isFile()){            if (name.endsWith(DEX_SUFFIX)) {                // Raw dex file (not inside a zip/jar).                try {                    dex = loadDexFile(file, optimizedDirectory);                } catch (IOException ex) {                    System.logE("Unable to load dex file: " + file, ex);                }            } else {...            }        } else {            System.logW("ClassLoader referenced unknown path: " + file);        }        if ((zip != null) || (dex != null)) {            elements.add(new Element(file, false, zip, dex));        }    }    return elements.toArray(new Element[elements.size()]);}

0x03 探明的坑

坑1 如果secondary dex文件太大，可能导致应用在安装过程中出现ANR，这个在0x02 MultiDex.install的最后也提到过，规避方法以后将继续介绍。
坑2 Dalvik linearAlloc bug (Issue 22586)：采用MutilDex方案的app在Android4.0以前的设备上可能会启动失败。
坑N-1 Dalvik linearAlloc limit (Issue 78035)：使用MultiDex的app需要申请一个大内存，运行时可能导致程序crash，这个Issue在Android4.0已经修复了, 不过还是有可能在低于Android5.0的设备上出现。
坑N main dex capacity exceeded，一边愉快地编译apk，一边写着代码，突然出现“main dex capacity exceeded”，build failed了… 这个时候怎么办，一种看似有效的办法是指定dex中的method数，例如：

android.applicationVariants.all {    variant ->        dex.doFirst{            dex->            if (dex.additionalParameters == null) {                dex.additionalParameters = []            }                dex.additionalParameters += '--set-max-idx-number=55000'       }}

然并卵，虽然编译没问题了，但是运行时会大概率出现ClassNotFoundException和NoClassDefFoundError导致crash，原因很简单，MultiDex.install之前依赖的所有类必须在main dex中，暴力指定main dex数量，可能导致这些类被划分到了secondary dex，系统的PathClassLoader并不能在main dex中找到全部需要加载的类！好在5.0之后，安装app时ART会对apk中所有的classes(..N).dex预编译输出为一个.oat文件，因此找不到类的情况会彻底解决，但是编译时dex过程中main dex capacity exceeded的问题却仍然存在。

一个解决办法是build时指定maindexlist.txt，具体可以参考本博客的另一篇文章MultiDex中出现的main dex capacity exceeded解决之道。

0 0