What is Facebook's architecture?
来源:互联网 发布:imap.126.com 端口 编辑:程序博客网 时间:2024/04/20 04:40
很早之前就在quora上看见这个介绍了,当时只是简单瞄了几眼,仔细看了几处兴趣的地方,今天又回过头去看看,可以根据它深入学习一些东西啊!
原文地址:http://www.quora.com/What-is-Facebooks-architecture
From various readings and conversations I had, my understanding of Facebook's current architecture is:
- Web front-end written in PHP. Facebook's HipHop [1] then converts it to C++ and compiles it using g++, thus providing a high performance templating and Web logic execution layer
- Business logic is exposed as services using Thrift [2]. Some of these services are implemented in PHP, C++ or Java depending on service requirements (some other languages are probably used...)
- Services implemented in Java don't use any usual enterprise application server but rather use Facebook's custom application server. At first this can look as wheel reinvented but as these services are exposed and consumed only (or mostly) using Thrift, the overhead of Tomcat, or even Jetty was probably too high with no significant added value for their need.
- Persistence is done using MySQL, Memcached [3], Facebook's Cassandra [4], Hadoop's HBase [5]. Memcached is used as a cache for MySQL as well as a general purpose cache. Facebook engineers admit that their use of Cassandra is currently decreasing as they now prefer HBase for its simpler consistency model and its MapReduce ability.
- Offline processing is done using Hadoop and Hive
- Data such as logging, clicks and feeds transit using Scribe [6] and are aggregating and stored in HDFS using Scribe-HDFS [7], thus allowing extended analysis using MapReduce
- BigPipe [8] is their custom technology to accelerate page rendering using a pipelining logic
- Varnish Cache [9] is used for HTTP proxying. They've prefered it for its high performance and efficiency [10].
- The storage of the billions of photos posted by the users is handled by Haystack, an ad-hoc storage solution developed by Facebook which brings low level optimizations and append-only writes [11].
- Facebook Messages is using its own architecture which is notably based on infrastructure sharding and dynamic cluster management. Business logic and persistence is encapsulated in so-called 'Cell'. Each Cell handles a part of users ; new Cells can be added as popularity grows [12]. Persistence is achieved using HBase [13].
- Facebook Messages' search engine is built with an inverted index stored in HBase [14]
- Facebook Search Engine's implementation details are unknown as far as I know
- The typeahead search uses a custom storage and retrieval logic [15]
- Chat is based on an Epoll server developed in Erlang and accessed using Thrift [16]
- Facebook is estimated to own more than 60,000 servers [17]. Their recent datacenter in Prineville, Oregon is based on entirely self-designed hardware [18] that was recently unveiled as Open Compute Project [19].
- 300 TB of data is stored in Memcached processes [20]
- Their Hadoop and Hive cluster is made of 3000 servers with 8 cores, 32 GB RAM, 12 TB disks that is a total of 24k cores, 96 TB RAM and 36 PB disks [20]
- 100 billion hits per day, 50 billion photos, 3 trillion objects cached, 130 TB of logs per day as of july 2010 [21]
[1] HipHop for PHP: http://developers.facebook.com/b...
[2] Thrift: http://thrift.apache.org/
[3] Memcached: http://memcached.org/
[4] Cassandra: http://cassandra.apache.org/
[5] HBase: http://hbase.apache.org/
[6] Scribe: https://github.com/facebook/scribe
[7] Scribe-HDFS: http://hadoopblog.blogspot.com/2...
[8] BigPipe: http://www.facebook.com/notes/fa...
[9] Varnish Cache: http://www.varnish-cache.org/
[10] Facebook goes for Varnish: http://www.varnish-software.com/...
[11] Needle in a haystack: efficient storage of billions of photos:http://www.facebook.com/note.php...
[12] Scaling the Messages Application Back End: http://www.facebook.com/note.php...
[13] The Underlying Technology of Messages: https://www.facebook.com/note.ph...
[14] The Underlying Technology of Messages Tech Talk:http://www.facebook.com/video/vi...
[15] Facebook's typeahead search architecture: http://www.facebook.com/video/vi...
[16] Facebook Chat: http://www.facebook.com/note.php...
[17] Who has the most Web Servers?: http://www.datacenterknowledge.c...
[18] Building Efficient Data Centers with the Open Compute Project: http://www.facebook.com/note.php...
[19] Open Compute Project: http://opencompute.org/
[20] Facebook's architecture presentation at Devoxx 2010:http://www.devoxx.com
[21] Scaling Facebook to 500 millions users and beyond:http://www.facebook.com/note.php...
- What is Facebook's architecture?
- What is Facebook's architecture?
- Facebook’s architecture(转)
- Architecture Design - 1 What is Architecture
- What Is Service-Oriented Architecture
- What Is Service-Oriented Architecture
- What is a software architecture?
- what's is build?
- What is the Best Multi-Stage Architecture for Object Recognition?
- 《What is the Best Multi-Stage Architecture for Object Recognition?》
- Perf Feedback - What's slowing down Mobile Facebook
- What have been Facebook's greatest technical accomplishments?
- What is McAfee's refund policy?
- What is NetApp's Cluster File System?
- What is the difference between a von Neumann architecture and a Harvard architecture?
- what's annotations is in java? what's the relationship between annotation and reflection?
- Gmail is different. Here's what you need to know.
- What is Favicon.ico? Personalise Your Site's Bookmark(zt)
- __FILE__, __LINE__,
- Java Socket编程
- 回来啦
- RHEL在VBOX和XP下上网设置问题?
- 开始吧
- What is Facebook's architecture?
- vector基本操作|随机生成乱序字符串
- 算式计算器C++实现代码(顺序栈结构 增加了一些功能 比如三角函数 求指教)
- Linux 驱动开发人员所需要做的工作内容
- MFC程序最小化到托盘
- 银行家算法
- 为什么会这样
- VC连接SQL SERVER数据库
- Android 怎样画视图