A Distributed Systems Reading List
来源:互联网 发布:win10网络连接图标不见 编辑:程序博客网 时间:2024/03/29 20:20
A Distributed Systems Reading List
Introduction
I often argue that the toughest thing about distributed systems is changing the way you think. The below is a collection of material I've found useful for motivating these changes.
Thought Provokers
Ramblings that make you think about the way you design. Not everything can be solved with big servers, databases and transactions.
- Harvest, Yield and Scalable Tolerant Systems - Real world applications of CAP from Brewer et al
- On Designing and Deploying Internet Scale Services - James Hamilton
- Latency Exists, Cope! - Commentary on coping with latency and it's architectural impacts
- Latency - the new web performance bottleneck - not at all new (seePatterson), but noteworthy
- The Perils of Good Abstractions - Building the perfect API/interface is difficult
- Chaotic Perspectives - Large scale systems are everything developers dislike - unpredictable, unordered and parallel
- Website Architecture - A collection of scalable architecture papers from various of the large websites
- Data on the Outside versus Data on the Inside - Pat Helland
- Memories, Guesses and Apologies - Pat Helland
- SOA and Newton's Universe - Pat Helland
- Building on Quicksand - Pat Helland
- Why Distributed Computing? - Jim Waldo
- A Note on Distributed Computing - Waldo, Wollrath et al
- Stevey's Google Platforms Rant - Yegge's SOA platform experience
Amazon
Somewhat about the technology but more interesting is the culture and organization they've created to work with it.
- A Conversation with Werner Vogels - Coverage of Amazon's transition to a service-based architecture
- Discipline and Focus - Additional coverage of Amazon's transition to a service-based architecture
- Vogels on Scalability
- SOA creates order out of chaos @ Amazon
Current "rocket science" in distributed systems.
- MapReduce
- Chubby Lock Manager
- Google File System
- BigTable
- Data Management for Internet-Scale Single-Sign-On
- Dremel: Interactive Analysis of Web-Scale Datasets
- Large-scale Incremental Processing Using Distributed Transactions and Notifications
- Megastore: Providing Scalable, Highly Available Storage for Interactive Services - Smart design for low latency Paxos implementation across datacentres.
Consistency Models
Key to building systems that suit their environments is finding the right tradeoff between consistency and availability.
- CAP Conjecture - Consistency, Availability, Parition Tolerance cannot all be satisfied at once
- Consistency, Availability, and Convergence - Proves the upper bound for consistency possible in a typical system
- CAP Twelve Years Later: How the "Rules" Have Changed - Eric Brewer expands on the original tradeoff description
- Consistency and Availability - Vogels
- Eventual Consistency - Vogels
- Avoiding Two-Phase Commit - Two phase commit avoidance approaches
- 2PC or not 2PC, Wherefore Art Thou XA? - Two phase commit isn't a silver bullet
- Life Beyond Distributed Transactions - Helland
- If you have too much data, then 'good enough' is good enough - NoSQL, Future of data theory - Pat Helland
- Starbucks doesn't do two phase commit - Asynchronous mechanisms at work
- You Can't Sacrifice Partition Tolerance - Additional CAP commentary
- Optimistic Replication - Relaxed consistency approaches for data replication
Theory
Papers that describe various important elements of distributed systems design.
- Distributed Computing Economics - Jim Gray
- Rules of Thumb in Data Engineering - Jim Gray and Prashant Shenoy
- Fallacies of Distributed Computing - Peter Deutsch
- Impossibility of distributed consensus with one faulty process - also known as FLP [access requires account and/or payment, a free version can be foundhere]
- Unreliable Failure Detectors for Reliable Distributed Systems. A method for handling the challenges of FLP
- Lamport Clocks - How do you establish a global view of time when each computer's clock is independent
- The Byzantine Generals Problem
- Lazy Replication: Exploiting the Semantics of Distributed Services
- Scalable Agreement - Towards Ordering as a Service
Languages and Tools
Issues of distributed systems construction with specific technologies.
- Programming Distributed Erlang Applications: Pitfalls and Recipes - Building reliable distributed applications isn't as simple as merely choosing Erlang and OTP.
Infrastructure
- Principles of Robust Timing over the Internet - Managing clocks is essential for even basics such as debugging
Storage
- Consistent Hashing and Random Trees
- Amazon's Dynamo Storage Service
Paxos Consensus
Understanding this algorithm is the challenge. I would suggest reading "Paxos Made Simple" before the other papers and again afterward.
- The Part-Time Parliament - Leslie Lamport
- Paxos Made Simple - Leslie Lamport
- Paxos Made Live - An Engineering Perspective - Chandra et al
- Revisiting the Paxos Algorithm - Lynch et al
- How to build a highly available system with consensus - Butler Lampson
- Reconfiguring a State Machine - Lamport et al - changing cluster membership
- Implementing Fault-Tolerant Services Using the State Machine Approach: a Tutorial - Fred Schneider
Other Consensus Papers
- Mencius: Building Efficient Replicated State Machines for WANs - consensus algorithm for wide-area network
Gossip Protocols (Epidemic Behaviours)
- How robust are gossip-based communication protocols?
- Astrolabe: A Robust and Scalable Technology For Distributed Systems Monitoring, Management, and Data Mining
- Epidemic Computing at Cornell
- Fighting Fire With Fire: Using Randomized Gossip To Combat Stochastic Scalability Limits
- Bi-Modal Multicast
P2P
- Chord: A Scalable Peer-to-peer Lookup Protocol for Internet Applications
- Kademlia: A Peer-to-peer Information System Based on the XOR Metric
- Pastry: Scalable, decentralized object location and routing for large-scale peer-to-peer systems
- PAST: A large-scale, persistent peer-to-peer storage utility - storage system atop Pastry
- SCRIBE: A large-scale and decentralised application-level multicast infrastructure - wide area messaging atop Pastry
Experience at MySpace
One of the larger websites out there with a high write load which is not the norm (most are read dominated).
- Inside MySpace
- Mix06 - Running a Megasite on Microsoft Technologies - Mix 06 Sessions/
- MySpace Storage Challenges
eBay
Interesting they dumped most of J2EE and use a lot of db partitioning. Check out their site upgrade tool as well.
- SD Forum 2006
关闭提示 关闭
确 认取 消- A Distributed Systems Reading List
- A Distributed Systems Reading List
- 转 -A Distributed Systems Reading List
- Book reading: Distributed Systems(Principles and Paradigms)----Chapter 8
- Distributed Systems Architecture: A Middleware Approach
- A Filesystems reading list
- Introduction to Distributed Systems
- Distributed Systems Topologies
- Distributed Systems with ZeroMQ
- distributed cache systems summary
- Distributed Systems-Basics
- Distributed Systems-Paxos
- A Reading List in Computer Networking
- Replication models in distributed systems
- Distributed Systems笔记-middlewares
- Designing Distributed Systems - A Conversation with Ken Arnold, Part III 阅读笔记
- Reading Papers about Distributed Replication
- Reading List
- Mindjet.MindManager.v9.0“参数错误”解决办法
- DB2 Merge Into语句的使用
- DB2 向表中写数据的几种方法
- 浅谈MongoDB之二 备份与修复(13/04 补充)
- Mysql中使用UNION语句进行多表连接查询
- A Distributed Systems Reading List
- HTML5开源游戏引擎lufylegend1.7.1发布
- 【转】mini2440裸机开启MMU实验
- 虚函数与虚继承寻踪
- 数据库驱动下载及连接字符串
- 随笔
- 2013-4-10
- Delphi 文件处理(2)
- 对象的传值与返回