文字记录，对话吴恩达（Andrew Ng）：超级大咖深度解析人工智能的发展现状与未来

---

转自：微信订阅号  机器学习研究会，人工智能交流群文字记录

---

 

雷鸣: 首先介绍嘉宾

嘉宾：

-吴恩达（Andrew Ng）：百度首席科学家，“百度大脑”、“谷歌大脑”负责人，斯坦福大学计算机科学系和电子工程系终身教授，人工智能实验室主任，Coursera联合创始人

-徐伟：百度IDL杰出科学家，前Facebook大规模推荐平台负责人、NEClab高级研究员

主持人雷鸣：北大大数据与机器学习中心联合主任，百度创始七剑客，酷我创始人

雷鸣: 另外，我们还邀请了韩旭：密苏里大学教授现在百度硅谷人工智能实验室任principal scientist

 

徐伟:

Hi， 大家好

Andrew Ng 吴恩达: Hello everyone.

雷鸣: 首先我们探讨第一个问题：深度学习的技术现状和未来发展，是否会成为通用基础技术？

 

雷鸣：现在深度学习非常火热，被吹得神乎其神。这个技术是否会成为某种意义的终极技术？

 

徐伟：深度学习会是为人工智能的一个重要部分

 

Andrew Ng 吴恩达: Regarding thesecond question, yes there's been a lot of hype about Deep Learning. I think itis creating tremendous value today—it is letting usturn the huge amounts of data we have into huge amounts of value.

 

Andrew Ng 吴恩达: I'm alsoconfident that deep learning will keep on creating a lot of value in the nextfew years—we still have far too many ideas, and too few people to do them.

 

徐伟：未来人工智能会是非常惯用的技术。所以深度学习也会是被广泛使用。

 

Andrew Ng 吴恩达:  But we're also very far from "humanlevel intelligence," and do not yet see any clear path to get there. Ithink some of the 炒作 has been a bit irresponsible. 人工智能会对整个社会有很大的影响力。SoI think it's important that all of us have a clear understanding of what'scoming, but also what is not, so that we can plan appropriately.

 

雷鸣: 一个问题，如果在有比较充分的数据和运算能力下，是否深度学习可以超越几乎所有的经典的数据挖掘算法？

徐伟: 可以说对大多数监督式学习问题，深度学习会超越传统的算法，对于一些非监督学习的问题，似乎还不是很明确。

 

雷鸣: @Andrew What's your opinion about this question?

Andrew Ng 吴恩达: I agree with XuWei. Despite all the value created by Deep Learning, most of it is currentlysupervised learning, meaning learning relatively simple A-->B mappings.

 

Andrew Ng 吴恩达: For example,perhaps A is an email, and B whether or not it is spam. That's a spam filter.Or perhaps A is an image, and B is an object label. That's object recognition.

 

Andrew Ng 吴恩达: With a lot oflabeled data (i.e., (A,B) pairs) and a big enough network, you can prove that adeep learning algorithm can learn almost any function to a very high level ofaccuracy.

 

雷鸣：@徐伟，那对于半监督学习呢？

徐伟：目前为止一个比较成功的非监督的例子是word embedding。

Andrew Ng 吴恩达:With a lot oflabeled data (i.e., (A,B) pairs) and a big enough network, you can prove that adeep learning algorithm can learn almost any function to a very high level ofaccuracy.

徐伟: 不过也有很多人不认为word embedding 是深度学习。

Andrew Ng 吴恩达: one of the mostexciting recent breakthroughs is Deep Learning algorithms can now learnA-->B mapping where B isn't just 0/1 or an integer (like the examplesabove), but can be very complex things like a sentence.

雷鸣: 看起来深度学习是非常有前景的

 

雷鸣: @Andrew，你在说自然语言吗，是指的翻译，还是语音识别？还是语义处理级别的？

Andrew Ng 吴恩达: For example, ifA is an image, and B is a caption, then that's image captioning. (The firstpaper to do this was by Xu Wei and his colleagues. :-)) Or if A is an englishsentence, and B is a french sentence, then that's machine translation, whichwas done by Ilya Suskevar and others. Or if A is an (Image,Question) pair and Bis an answer, that's Image Question Answering (also by Xu Wei!).

 

雷鸣：看起来不仅仅是语言，还包括了图像，这个进展是非常大的

 

徐伟：另外word embedding 和传统的非监督学习也并不完全一样，实际上运用了上下文作为监督。我认为非监督学习要取得成功，实际上更会使类似这样的弱监督学习

 

Andrew Ng 吴恩达: Supervisedlearning has been very sucessful for both 翻译 and 语音识别。Our mostsuccessful approach at Baidu on speech recognition has been to use a very largeneural network, and to learn an A-->B mapping directly where A is an audioclip and B is the text transcript. Tony who's next to me had led the teamworking on the mandarin version of this, and we believe this is now the world'sbest mandarin speech recognition system.

 

雷鸣：下面我们聊一下在一些领域，深度学习带来的一些新的进展

 

雷鸣：比如说语音识别，我前些时间到百度美国研究院，看到Andrew给我演示的语音识别，在非常大的噪音环境下，我基本上听不出来里面的人说什么，但是语音识别出来了。之后我仔细听，发现语音识别时对的。

 

雷鸣：Andrew，方便介绍一下全球在语音识别方面的最新进展吗？

 

雷鸣：Gotta，看来我们想到一起了。你先回答了我问的

 

雷鸣: 对于语音识别这块儿，随着技术进步，我们做出超过人对人声识别的系统，是否是一个必然结果？

Andrew Ng 吴恩达: I think speechrecognition will move toward end-to-end learning. We are finding that thedataset size is one of the biggest drivers of performance. I find some of therecent work on attention models also promising. We were also very heavily influencedby Alex Grave's work on CTC.

 

Andrew Ng 吴恩达:  Two challenges that remain: Transcribing longutterances. We surpass human-level performance for short phrases, out ofcontext. But we are still much worse than human-level performance when there'smore context, such as long conversations. A second major challenge isunderstanding the content of the text, rather than only transcribing it.

 

Andrew Ng 吴恩达: But I'm excitedabout building a speech-enabled world. Just as (thanks to Steve Jobs) thesmartphone touchscreen fundamentally changed how we interact with computers, Ithink that speech will also fundamentally transform how we interact withcomputers in the next few years.

 

徐伟: 目前的语音识别系统还没有像人那人能够很好的利用上下文信息。

雷鸣: 另外一个问题时关于图片识别，以及视频内容识别这块儿，这快儿的全球的最新进展如何？微软最近的一个图像识别的进展，用了100多层的深度神经网络，得到了不错的效果。是否超深的神经网络，也是一个发展方向？

 

徐伟：对象ImageNet这样的问题上，目前的deep residue net的结果已经非常好了。目前我们也看到很深的模型在一些NLP也有很好的效果

 

徐伟：目前对深度学习模型一个最大的难点之一还是记忆以及知识表示

 

雷鸣：对于计算机视觉这块儿？尤其是运动图像（视频）处理，当前的进展和挑战如何？

徐伟: 对于视频，运算能力还是一个重大的瓶颈

 

雷鸣: 在自然语言对话领域，现在深度学习应用的效果如何？

徐伟: ImageNet只有百万级别的数据，训练需要数天。

雷鸣: @徐伟，是否面临的自大问题，正如你上面说的，记忆和知识表达？

 

徐伟：还有推理也是一个难点

 

徐伟：深度学习模型还不能很好地利用已有知识库

 

雷鸣: 下面我们聊一下一些实用的领域，比如现在百度大力发展的无人驾驶，这块的挑战和机遇是什么？对于无人驾驶，在5年内，我们真的能够在大街上乘坐商用的无人驾驶汽车吗？

Andrew Ng 吴恩达: I'm fortunate tohave gotten to work on several different cars before. The Baidu one is the 4thcar I've helped build. But this is the first time in my life that I've felt wesee a clear path to making autonomous vehicles a reality.

 

Andrew Ng 吴恩达: Just for fun,here're pictures of some other cars I had worked on!

 

Andrew Ng 吴恩达：Here in China,500 people a day die from car accidents. If we can make autonomous driving areality only one day sooner, that's an extra 500 people whose lives wesave.  This will be one of the mostimportant applications of AI in this decade.

 

Andrew Ng 吴恩达: Why do I thinkwe now have a clear shot toward making this a reality? We now have verysophisticated deep learning algorithms that are performing far better than everbefore. We were fortunate that Lin Yuanqing had joined Baidu a few months agoand is now leading a big part of this effort.

 

Andrew Ng 吴恩达: But in additionto that, we have also developed a unique strategy that is different than mostothers that have been working on autonomous driving. We call this strategyTRAIN TERRAIN (铁轨战略).

 

雷鸣: @andrew，那我们五年内有可能在大街上乘坐无人驾驶车吗？

Andrew Ng 吴恩达: We hope to havecommercial autonomous driving services by 2018, and be in mass production by2020.

 

Andrew Ng 吴恩达: Here're the keyelements of the TRAIN TERRAIN strategy: 1. Don't try to roll out autonomousvehicles everywhere all at once. Instead, start from a small region (such as ashuttle route or small city), and grow from there.

 

Andrew Ng 吴恩达: 2. Realize thatcomputer-driven cars are not the same thing as a human-driven car. They behavedifferently: They never drive drunk. But, they don’t understanda policeman’s hand gestures. Make sure people in the “autonomyenabled” regions have realistic expectations.

3. Design autonomous cars to be clearlyrecognizable, so that people can immediately spot them for what they are.

4. Make the behavior of the autonomous carshighly predictable. Predictability, even more than cleverness, leads to safety.

5. Implement modest infrastructure changesin the autonomy enabled regions to make sure the cars understand what they needto do. For example, give emergency workers a clear way (such as a wirelessbecon) to communicate with the car. Make sure the roads are well maintained,with clearly painted lines. And so on.

 

Andrew Ng 吴恩达: With thesechanges, I think we can safely put autonomous cars on the roads soon.

 

雷鸣：自动驾驶，一个挺科幻的概念，再有几年就满大街都是，真让人感慨技术发展速度啊

Andrew Ng 吴恩达: Yes I agree! Iwant to say something to all the young people reading this. I think we're in aunque moment in history where AI can really change the world.

 

雷鸣: @Andrew，Agree！

Andrew Ng 吴恩达: If you know howto use or apply AI, you can be a position where the decision you make todaywill really change how the world is in 10 years. There will be thousands ormillions whose would have lost their lives but for your efforts. Or you cantransform entire industries, and help countless people.

Andrew Ng 吴恩达: That's why I'mreally excited about AI.

 

Andrew Ng 吴恩达: If you are youngand considering what career path to pursue, I hope that you will considerlearning about AI, and joining the AI research and development community atBaidu or elsewhere, so that we can all work together to make the world a betterplace!

 

雷鸣：@Andrew，这确实非常激动人心！现在已经有无数的人为AI痴迷。现在的理科学生们确实都在学习机器学习

 

雷鸣：我们再谈一下机器人，机器人真的离我们还非常遥远吗？是否近5年都没有巨大的商业化机会？                                                                                                徐伟：非常专用功能的机器人，应该还是会有很快的发展和应用。但像人那样的机器人，还比较遥远。

 

Andrew Ng 吴恩达: 我很高兴听到有很多人对机器学习感兴趣。现在的问题是机器学习的想法和机会太多，不过能做的人太少。The world needs more AI people!

 

雷鸣：@Andrew，你也多做一些好的在线课程，让大家多学习！这次活动，一共有20个大群，近10,000人参加，可见现在大家对AI是多么感兴趣。

 

雷鸣：现在销售量最大的应该是扫地机器人，不过很多人都觉得他们不像机器人。那么在未来几年，会有那些机器人领域大放异彩？工业机器人？家庭机器人？服务机器人？       

 

Andrew Ng 吴恩达: There's a lot ofexciting work in robotic applications right now that focuses on specificnarrow/vertical applications. Other than autonomous cars, I see exciting workin precision agriculture, automated power plant inspection, automated picking(for ecommerce fulfillment), automated security robots, and so on. Most ofthese robots have hardware and software designed to carry out that particulartask, and so do not look like humanoid robots. I think truly general-purposerobots (other than robot arms in factory automation) are still a little furtheraway.

 

徐伟：self-driving car可能是一个最重要的一种。未来几年类，家用还很难真正帮人做事，

 

雷鸣: What do you think about Amazon Echo.  It's pretty popular in US now.

Andrew Ng 吴恩达: I have an AmazonEcho in my home. I think it is a nice start to home automation using voicecommands, but it is still the very early days of a new industry. I've beenimpressed by Amazon's work, but it's still too early to say whether this willbe the right design in the long term.

 

Andrew Ng 吴恩达: But I do thinkthat in the future, we should be able to talk to all the devices in our homesand have them understand and respond to us.

 

Andrew Ng 吴恩达:  I hope that a few decades from now, I willhave grandchildren who are mystified as to how, back in 2016, if you were to gohome and say something to your microwave oven, it would just sit there andrudely ignore you!

 

雷鸣：对于AI从业的人士，Andrew有什么职业发展建议？

 

Andrew Ng 吴恩达: AI is changingso rapidly, I think all of us that work in AI have to keep on learning.

Andrew Ng 吴恩达: Once again, Iwant to say something to the young people reading this. Every Saturday, youwill have a choice: You can either watch TV, or you can study. If you study, itturns out that there will be almost no short term reward. The following Monday,you won't be that much better at your job, and your boss almost certainly won'tknow you spent all day studying nor tell you "good job." So, you havealmost nothing to show for your day of hard work.

 

Andrew Ng 吴恩达: But here's thesecret: If you study hard not just for one weekend, but do so weekend afterweekend… for a year… then you will become great at it. I think studying has almost noshort term rewards. But the long term rewards are huge!

 

Andrew Ng 吴恩达: So one of thechallenges—which I hope many readers will rise to—is to keep ourselvesmotivated and to keep learning and studying, week after week, year after year.

 

雷鸣:非常鼓励的话，大家共勉！

 

徐伟: 说道reward, 延迟很久的reward目前也是deep reinforcement learning很难处理的一点

雷鸣: @徐伟，对人类也相当复杂的问题，对机器学习也一样啊。。。

Andrew Ng 吴恩达: 说的对！

 

雷鸣：那么我们下面进入到问答环节，提问开始

 

黄伟：end to end的方法，对数据的依赖有多高，机器学习如何提高对数据的学习和处理能力，而不是所有的数据必须人工标注才能使用？ 

徐伟：如果不能解决这个问题，AI如何真正的进化？ -- 所以需要Agent在环境中能主动地获取数据，向人那样深度学习应用在自然语言处理方面感觉还是不怎么理想，有这么一种说法，图像是你直接看到的，语言却是人类抽象之后的。特别是将深度学习应用在贴吧的帖子，或者电商网站上的评论数据，貌似效果都不如人为的规则+传统的模型，你们怎么看呢？（工程师2）徐伟：人类对话还涉及了大量的知识的运用。目前深度学习还很缺乏这一点。在较少依赖语言的一些NLP问题上（比如机器翻译），我们的试验显示深度学习目前已经可以超越传统模型了

 

Andrew Ng 吴恩达:【2】在传统的机器学习任务上，模型选择一直是个问题。现在深度学习给人的感觉是要比传统的svm，决策树，或者boosting等都要好，但是实际应用中真是这样吗？像搜索排序，推荐等问题，业界已经成熟运用深度学习了吗？（工程师2）谢谢你的问题。Yes, Deep Learning is the best algorithm for a lot of the mostimportant tasks that we use in production systems, including search,recommendations, and others. The general pattern is as follows: If you have arelatively small training set, then the performance depends more on your skillat hand-engineering features, and deep learning won't have a significantadvantage over SVMs, Boosting, Decision Trees. But in the regime of bigdata—where you have a massive labeled training set—the supervised deep learningis more likely to do well. This is partially because deep learning algorithmsare very "high capacity" (say high VC dimension, if you know whatthat is). This lets it exploit very large datasets better than most otheralgorithms. They are also more scalable than say an SVM with a non-linearkernel. This lets us build the systems needed to train them on huge datasets.To help visualize all this, here is a cartoon plot that explains how I think ofthe overall trends of the performance of DL vs. more traditional algorithms:

 

【1】请问两位老师，第一个问题，怎样用深度学习做点击率预测CTR？其中一个难点在于输入特征维度高而且sparse，深度学习针对这类问题怎么解决？第二个问题，深度学习在推荐方面有什么好的方法？谢谢（学生群2）--    徐伟：其实这是深度学习的一个优势，把高位稀疏直接作为输入，它学到的是类似embedding一样的东西。

 

Thus, my question is whether it is possibleto apply some methods in NLP to deal with biology problems, and which onesmight be most possible? -- 徐伟：The most widely used model for NLP is recurrent models. Perhaps theyare already used for biology problems.

 

Andrew Ng 吴恩达:  Q: Do you know any work is being done whereAI actually helps extending and training human intelligence? take the exampleof google alphago. would it be nice if  professional GO players canbenefit from AlphaGo's reasoning of a game? A: We saw this happen after GaryKasparov lost to Deep Blue. Human chess players are now far better throughlearning from and also partnering with computer chess players. I've heard ofthis starting in Go as well, but that feels like it's in an earlier stage. Butmore generally, I see a lot of opportunities for computers to supplement thehuman brain. I am especially excited about online education. I think MOOCs likeCoursera and open.163.com have been a great start. I hope that online educationbecomes more adaptive and flexible over time, and that computers can reallyhelp customize our learning experiences, the way a personal tutor might.

 

Andrew Ng 吴恩达: Q: Prof Ng. andXu. For medical imaging, it's very difficult to collect large scale, accurate,well labeled data. How can we get a better performance. A: There's a lot oflow-hanging fruit today in deep learning in problems with a lot of data. If youdon't have a lot of data, in the short term you might end up have to just relyon more traditional engineering methods (including careful feature design). Butlooking slightly further out, I'm excited about other forms of learning,including transfer learning, semi-supervised learning and unsupervisedlearning—and quite possibly ones we haven't imagined yet—that would help us dowell even on small amounts of data. There's a lot of active research on thesetopics in Baidu and elsewhere. I don't think any of us feel like we have theright algorithms yet, but I'm seeing a lot of progress each year.

 

徐伟：交流群13:光：1.Will deep learning networks evolve to develop logical thinking? Orlogical thinking is completely different from deep learning methods in natureso we need different method to compensate deep learning network? -- Right now,there is no good way for to evolve a large deep learning model (there's work onevolving small models). So whether we will have deep learning model capable ofhandling logical reasoning will depends on the new models designed byresearchers. But I do believe it is possible to achieved by deep learningmodel, as evidenced by the rapid progress in the area of NLP using deeplearning.

 

雷鸣：由于时间关系，请@Andrew，@徐伟再回答最后一个问题。结束后请告诉我一下

 

Andrew Ng 吴恩达: 谢谢大家的热情，这么晚的时间还在和我与徐伟交流。我们也希望未来会有更多的机会和中国的人工智能人员交流，也希望会有机会来支持中国的人工智能发展！

 

徐伟：谢谢大家的非常好的问题。时间有限，不能一一解答

 

徐伟:

答

雷鸣:

@Andrew，我们应该感谢你给大家带来如此精彩的内容！

Andrew Ng 吴恩达:

非常感谢雷鸣安排这个论坛！

雷鸣:

@徐伟，非常感谢！

徐伟:

谢谢雷鸣的组织

 

雷鸣：那我们今天的沟通到此结束！谢谢大家！