Stanford - Cryptography I - Week 3 Programming Assignment

题注

Week2是没有Programming Assignment的，我们只能直接看Week3的题目了。Week3的问题是有关Hash函数碰撞的问题。这方面世界上研究最为透彻的当然是我们清华大学的王小云教授了。她破解了世界上注明的MD5和SHA1 Hash Function，即能在远小于给定安全常数所对应的时间内，找到两个不相等的值x和y，使得

对于MD5算法：MD5(x) = MD5(y)

对于SHA1算法：SHA1(x) = SHA1(y)

我自己作为另两位国内知名信息安全专家的学生（具体是谁就不方便透露了，不合适…），有幸也见过很多次王小云教授。不得不说，王小云老师绝对是一个一心学术的好老师，与她交流时没有一点架子，非常和蔼可亲，但是工作起来绝对属于不要命的… 这一点和我的两位导师很像，在我写这篇博客的时候，我的一位导师应该还在办公室工作呢吧~ 另一位导师每天早上8:00一定会到办公室，然后工作到晚上6:30左右，周末周日也是如此。成功没有捷径，两位老师都自诩自己并不是聪明的人，但是两位老师都用勤奋弥补了所谓的智力落差（当然了，我认为两位老师是绝顶聪明的，相比我来说…）。

作为他们的博士生，我也需要继续努力呀，虽然我可能做不到其中一位导师那样，在Eurocrypt和Asiacrypt上面发表论文，但是如ICDCS等的B类顶级安全会议，应该可以冲击一下！希望自己能够达到两位老师的期待吧。

题目

In this assignment your task is to find hash function collisions using the birthday attack.

Consider the hash function obtained by truncating the output of SHA256 to 50 bits, sayH(x)=LSB50(SHA256(x)), that is we drop all but the right most 50 bits of the output. Your goal is to find a collision on this hash function. Find two stringsx≠y such that LSB50(SHA256(x))=LSB50(SHA256(y)) and then enter the hex encoding of these strings in the fields below.

For an implementation of SHA256 use an existing crypto library such as PyCrypto (Python), Crypto++ (C++), or any other.

分析

这道题的思路很简单，使用birthday attack攻击算法。birthday attack本身的原理我就不在这里阐述了，朋友们可以在阅读代码的时候理解birthday attack。如果使用birthday attack，对于一个安全常数为\lambda的Hash算法，我们有1/2的概率，使用2^{\lambda / 2}的时间找到一对碰撞。因此，寻找SHA256 LSB50的collision的算法复杂度为O(2^25)，在这里我们计算2^26。然而，细细想来这就没有那么简单了，虽然collision本身的算法复杂度为O(2^25)，但真正找到这对碰撞，如果用普通的方法，需要进行2^25*（2^25-1）/ 2 = O(2^50)比较，这个是普通台式机所没有办法在短时间内比较的。

解决这一问题的方案就是用空间复杂度换时间复杂度，即利用HashMap来解决这一问题。参考在讨论区中student MikhailEgorov的帖子，原文如下：

Week 3 PA - Tips to find collision usingJava

Helloeverybody! I want to share some useful tips about week 3programming problem. I think I've found good way to solve thisproblem, which is pretty fast and not memory consuming. The messagespace we have to try to find a collision is less than 2^26. I usedinteger numbers (4 byte) as messages. For storing message hash(which is 50 bit) I used long type (8 byte). To find collision Iused hashmap where hash is a key and message is a value.java.util.HashMap<Long, Integer> hm;Naive approach. 1. In a loop compute hash value for each message(from 0 to 2^26) and check whether hashmap already contains keywhich is equal to hash of the current message. If so we found acollision. Great.

Naive approach requires a lot of time and memory. I failed to find acollision even with 3GB java heap! Then I used more cleverapproach.

1. Igrouped 2^26 messages by last 10 bits of a hash. So if two messagehashes have last 10 bits equal then those messages belong to onegroup. Each group is a separate file with integers on disk and wehave 2^10=1024 such files. We can find 50-bit collision only in onefile, because there is no collisions between messages from distinctfiles. For my notebook it took about 2 minutes to separate 2^26integers to 1024 files.

2. Then Isequentially processed each file with loop and hasmap like in naiveapproach. But here we have to process only messages in one file ata time. It took less than 512MB heap and about 1-2 minutes to findcollision for 50-bit hash. Really good!

Finally I submitted two messages and got points) Good luck!

这一思路，可以很好地通过空间复杂度来减少时间复杂度。另外，在Java实现的时候特别要注意数组中哪边是高50位，哪边是低50位。在Java中，假定给定数组为byte[32]，那么byte[31]是最低位，byte[0]是最高位。

另外，如果我们选择的遍历输入是从0到2^26的话，是找不到碰撞的… 也就是说碰到了另外1/2的概率。不过，如果从2^26到2^27，就能成功地找到碰撞了。这一点还挺好玩的，也可能是Boneh他们团队弄出的一个小陷阱吧~ 大家也可以试试构造其他的遍历输入，看看能不能找到其他的碰撞值。

代码

Util.java，主要是一些功能函数。

import java.security.MessageDigest;import java.security.NoSuchAlgorithmException;public class Util {//将整数转换为byte数组public static byte[] intToByteArray(int value){return new byte[] { (byte)(value >>> 24), (byte)(value >>> 16), (byte)(value >>> 8), (byte)value};}//计算给定值的SHA256值，并返回其LSB50位public static byte[] calculateSHA256(byte[] data){MessageDigest md = null;try {md = MessageDigest.getInstance("SHA-256");} catch (NoSuchAlgorithmException e) {e.printStackTrace();}md.update(data);byte[] b = md.digest();byte[] result = {(byte) (b[25]&0x03), b[26], b[27], b[28], b[29], b[30], b[31]};return result;}//将byte数组转换为十六进制String形式，这是为了方便显示和提交答案public static String bytesToHexString(byte[] bArray) {StringBuffer sb = new StringBuffer(bArray.length);String sTemp;for (int i = 0; i < bArray.length; i++) {sTemp = Integer.toHexString(0xFF & bArray[i]);if (sTemp.length() < 2)sb.append(0);sb.append(sTemp.toUpperCase());}return sb.toString();}}

Solution.java

public class Solution {//整个计算过程所用到的最大循环次数private static final int MAX_INT = (int) Math.pow(2, 26);//建立1024个文件public void createSaveHashFiles() {for (int i = 0; i < 1024; i++) {//输出到result文件夹下File file = new File("result/" + i + ".txt");if (!file.exists()) {try {file.createNewFile();} catch (IOException e) {e.printStackTrace();}} else {file.delete();try {file.createNewFile();} catch (IOException e) {e.printStackTrace();}}}}public void createAndSaveHash() {//开启1024个文件留，用于存储以0到1023开头的hash值File[] fileArray = new File[1024];FileOutputStream[] fosArray = new FileOutputStream[1024];PrintStream[] psArray = new PrintStream[1024];for (int i = 0; i < 1024; i++) {fileArray[i] = new File("result/" + i + ".txt");try {fosArray[i] = new FileOutputStream(fileArray[i]);psArray[i] = new PrintStream(fosArray[i], true);} catch (FileNotFoundException e) {e.printStackTrace();}}for (int i = MAX_INT; i < MAX_INT * 2; i++) {byte[] result = Util.calculateSHA256(Util.intToByteArray(i));String resultstring = Util.bytesToHexString(result);//判断计算结果的前10位，确定将结果写入到哪一个对应的文件中int filename = (result[0] & 0xff) * 256 + (result[1] & 0xff);if (i % (MAX_INT/100) == 0){System.out.println((i / (MAX_INT/100) - 100) + "% complete.");}psArray[filename].println(i + ":" + resultstring);psArray[filename].flush();}for (int i = 0; i < 1024; i++) {psArray[i].close();try {fosArray[i].close();} catch (IOException e) {e.printStackTrace();}}}public boolean detectCollision(int i) {HashMap<String, String> hashMapFori = new HashMap<String, String>();System.out.println("Detecting file " + i + ".txt");File file = new File("result/" + i+ ".txt");try {FileInputStream fis = new FileInputStream(file);BufferedReader br = new BufferedReader(new InputStreamReader(fis));String line;while ((line = br.readLine()) != null) {String[] temp = line.split(":");if (hashMapFori.containsKey(temp[1])) {//找到碰撞，输出并返回System.out.println("collision: " + hashMapFori.get(temp[1]) + " " + temp[0]);br.close();fis.close();return true;} else {hashMapFori.put(temp[1], temp[0]);}}br.close();fis.close();return false;} catch (IOException e) {e.printStackTrace();return false;}}public static void main(String[] args) {Solution solution = new Solution();System.out.println("Start creating files...");solution.createSaveHashFiles();System.out.println("Start save hash values...");solution.createAndSaveHash();System.out.println("Start detecting collision...");for (int i = 0; i < 1024; i++) {if (solution.detectCollision(i)) {break;}}}}

答案

生成的result文件夹下的文件如图。注意到每个文件的大小都超过1M，因此整个存储开销是很大的。不过，时间开销就很短了，整个运行时间大概在30分钟左右。

Detect Collection的结果如图。

因此，我们可以知道答案为x = 70058362，y = 127786176。上传答案时需要以十六进制表示，即为x = 042D017A，y = 079DDCC0。