YouTube技术架构

来源:互联网 发布:重置linux root密码 编辑:程序博客网 时间:2024/04/27 14:44

谈不上翻译,就是摘录


1 billion video views per day


1.Apache
2.Python
3.Linux (SuSe)
4.MySQL
5.psyco, a dynamic python->C compiler
6.lighttpd for video instead of Apache

       -->NetScaler-->Apache plus Lighttpd(for video/multiple process)-->Python-app-server(format html)->databases(MySQL to ...)
       |
Client |->CDN (popular content)-->Youtube Server(less popular)

Keep it simple and cheap
Keep a simple network path
Use commodity hardware
Use Simple common tools provided by os


MySQL -->database partioning with users assigned to different shards
spreads writes and reads
replica lag(备份延迟?) 0

colocation arrangement
Use 5 or 6 data centers plus the CDN
8.Images are replicated to different data centers using BigTable

Stall for time. Creative and risky tricks can help you cope in the short term while you work out longer term solutions
做最重要的事情
outsource service for short time to market
Keep it simple
Shard or partition
Constant interation on bottlenecks
Succeed as a team

FastCGI
1) CGI是WebServer与其他独立进程交互的工具,FastCGI常驻内存,不必每次都fork一个子进程,从而提升了效率


I found this information interesting:

    The application code is written mostly in Python (the web app is not the bottleneck… the database RPC is)
    They use Apache for page content and lighttpd for serving video
    Thumbnails are now served by Google’s BigTable
    They’re running SuSE Linux with MySQL
    HW RAID-10 across multiple disks was too slow. HW RAID-1 with SW RAID-0 was faster because the Linux I/O scheduler could see the multiple volumes and would therefore schedule more I/O
    

Citrix 05年收购了NetScaler (http server/load balancer/发布平台)


原文:http://highscalability.com/youtube-architecture