判断文本文件编码的有用工具:juniversalchardet

来源：互联网发布：2017网络流行歌曲编辑：程序博客网时间：2024/05/29 07:17

在java的项目中,为了对付多个平台,有时候需要判断要出来的文本文件时什么编码.

如果不知道编码的话有可能就产生乱码了.

当然有一中方法就是事先约定是什么编码.比如只能是UTF-8等,这样就稍微有一点不方便.

最近在发现了一个判断的小工具.比较好用.随便推广一下.

juniversalchardet:http://code.google.com/p/juniversalchardet/

能够识别的编码如下:

Chinese
- ISO-2022-CN
- BIG5
- EUC-TW
- GB18030
- HZ-GB-2312¹

Cyrillic
- ISO-8859-5
- KOI8-R
- WINDOWS-1251
- MACCYRILLIC
- IBM866
- IBM855

Greek
- ISO-8859-7
- WINDOWS-1253

Hebrew
- ISO-8859-8
- WINDOWS-1255

Japanese
- ISO-2022-JP
- SHIFT_JIS
- EUC-JP

Korean
- ISO-2022-KR
- EUC-KR

Unicode
- UTF-8
- UTF-16BE / UTF-16LE
- UTF-32BE / UTF-32LE / X-ISO-10646-UCS-4-3412¹ / X-ISO-10646-UCS-4-2143¹

Others
- WINDOWS-1252

测试代码:

    public static void main(String[] args) throws IOException {        // TODO Auto-generated method stub        byte[] buf = new byte[4096];       //String fileName = args[0];        String fileName = "d:/test.txt";        java.io.FileInputStream fis = new java.io.FileInputStream(fileName);        // (1)        UniversalDetector detector = new UniversalDetector(null);        // (2)        int nread;        while ((nread = fis.read(buf)) > 0 && !detector.isDone()) {          detector.handleData(buf, 0, nread);        }        // (3)        detector.dataEnd();        // (4)        String encoding = detector.getDetectedCharset();        if (encoding != null) {          System.out.println("Detected encoding = " + encoding);        } else {          System.out.println("No encoding detected.");        }        // (5)        detector.reset();                      String cread;        StringBuffer content = new StringBuffer();               InputStreamReader r = new InputStreamReader(new FileInputStream(fileName), encoding);        BufferedReader   in = new BufferedReader(r);        while ((cread = in.readLine()) != null) {            content.append(cread);        }        System.out.print(content.toString());    }