geth结构解析和源码分析

第一部分 看看geth客户端的整体结构
创建私链的时候已经指定所有的信息都放在private-geth目录下，现在是已经有过挖矿的目录。

当时我们把创世文件genesis.json放在该目录下了、

root@i-5tthrr8u:/home/ubuntu/private-geth# lltotal 16drwxr-xr-x 3 root   root   4096 Jul  2 17:02 ./drwxr-xr-x 6 ubuntu ubuntu 4096 Jul  4 14:07 ../drwx------ 5 root   root   4096 Jul  2 17:41 data/-rw-r--r-- 1 root   root    529 Jul  2 16:29 genesis.json

进入真正的存放数据的目录private-geth/data/00
geth中保存的是区块链的相关数据
keystore中保存的是该链条中的用户信息

root@i-5tthrr8u:/home/ubuntu/private-geth/data/00# lltotal 20drwx------ 4 root root 4096 Jul  2 17:23 ./drwx------ 5 root root 4096 Jul  2 17:41 ../drwxr-xr-x 5 root root 4096 Jul  2 17:02 geth/-rw------- 1 root root 1391 Jul  2 17:58 historydrwx------ 2 root root 4096 Jul  2 17:10 keystore/

之前我们这个节点已经创建了两个账户，现在我们可以看到keystore里面有两个账户信息的文件

root@i-5tthrr8u:/home/ubuntu/private-geth/data/00/keystore# lltotal 16drwx------ 2 root root 4096 Jul  2 17:10 ./drwx------ 4 root root 4096 Jul  2 17:23 ../-rw------- 1 root root  491 Jul  2 17:02 UTC--2017-07-02T09-02-56.470592674Z--28b769b3b9109afd1e9e50a9312c5a3bfae8a699-rw------- 1 root root  491 Jul  2 17:10 UTC--2017-07-02T09-10-28.087401309Z--b4e2e2514eae3684157bf34a0cee2c07c431cf92

每个账户都由一对钥匙定义，一个私钥和一个公钥。 账户以地址为索引，地址由公钥衍生而来，取公钥的最后 20个字节。每对私钥 /地址都编码在一个钥匙文件里。钥匙文件是JSON文本文件，可以用任何文本编辑器打开和浏览。钥匙文件的关键部分，账户私钥，通常用你创建帐户时设置的密码进行加密。钥匙文件的文件名格式为UTC。账号列出时是按字母顺序排列，但是由于时间戳格式，实际上它是按创建顺序排列。如果把秘钥丢了钥匙文件可以在以太坊节点数据目录的keystore子目录下找到，接下来我们进入一个keystore目录文件看看他的信息：

root@i-5tthrr8u:/home/ubuntu/private-geth/data/00/keystore# vim UTC--2017-07-02T09-02-56.470592674Z--28b769b3b9109afd1e9e50a9312c5a3bfae8a699 {"address":"28b769b3b9109afd1e9e50a9312c5a3bfae8a699","crypto":{"cipher":"aes-128-ctr","ciphertext":"89ce1513b4b5a325735891b559c361ce696bb2c173a7a1b290549e79dad8f847","cipherparams":{"iv":"982c86418fae2dd39e04d1e51528cffa"},"kdf":"scrypt","kdfparams":{"dklen":32,"n":262144,"p":1,"r":8,"salt":"4227384ea0e3d15af1bac190f7e01d392543d0a5ca1ec931c1d340f87845f771"},"mac":"46cffc6e4f57fa27b69e53dc4ae43a03ce1b93f24c132aa4655f53ddf215f112"},"id":"e516b9d4-2161-4648-b3db-fc2ef1c3739c","version":3}

警告：记住密码并”备份钥匙文件”。为了从账号发送交易，包括发送以太币，你必须同时有钥匙文件和密码。确保钥匙文件有个备份并牢记密码，尽可能安全地存储它们。这里没有逃亡路径，如果钥匙文件丢失或忘记密码，就会丢失所有的以太币。没有密码不可能进入账号，也没有忘记密码选项。所以一定不要忘记密码。

接下来进入geth可以看到chaindata，lightchaindata，nodes目录

root@i-5tthrr8u:/home/ubuntu/private-geth/data/00/geth# lltotal 24drwxr-xr-x 5 root root 4096 Jul  2 17:02 ./drwx------ 4 root root 4096 Jul  2 17:23 ../drwxr-xr-x 2 root root 4096 Jul  4 14:12 chaindata/drwxr-xr-x 2 root root 4096 Jul  2 17:02 lightchaindata/-rw-r--r-- 1 root root    0 Jul  2 17:02 LOCK-rw------- 1 root root   64 Jul  2 17:02 nodekeydrwxr-xr-x 2 root root 4096 Jul  4 15:55 nodes/

进入nodes（我们这条私链有三个节点，所以这里有三个ldb文件）

root@i-5tthrr8u:/home/ubuntu/private-geth/data/00/geth/nodes# lltotal 5316drwxr-xr-x 2 root root    4096 Jul  4 15:55 ./drwxr-xr-x 5 root root    4096 Jul  2 17:02 ../-rw-r--r-- 1 root root  405250 Jul  4 15:57 000033.log-rw-r--r-- 1 root root 2132979 Jul  4 15:55 000035.ldb-rw-r--r-- 1 root root 2131238 Jul  4 15:55 000036.ldb-rw-r--r-- 1 root root  739354 Jul  4 15:55 000037.ldb-rw-r--r-- 1 root root      16 Jul  4 14:12 CURRENT-rw-r--r-- 1 root root       0 Jul  2 17:02 LOCK-rw-r--r-- 1 root root    8187 Jul  4 15:55 LOG-rw-r--r-- 1 root root    4557 Jul  4 15:55 MANIFEST-000013

进入chaindata，区块链最后的本地存储都是以ldb文件的形势（但这里是不是应该每个区块一个ldb文件呢？）

root@i-5tthrr8u:/home/ubuntu/private-geth/data/00/geth/chaindata# lltotal 52drwxr-xr-x 2 root root  4096 Jul  5 09:51 ./drwxr-xr-x 5 root root  4096 Jul  2 17:02 ../-rw-r--r-- 1 root root  5288 Jul  2 17:56 000008.ldb-rw-r--r-- 1 root root 11681 Jul  4 14:12 000009.ldb-rw-r--r-- 1 root root  8921 Jul  4 14:13 000010.log-rw-r--r-- 1 root root    16 Jul  4 14:12 CURRENT-rw-r--r-- 1 root root     0 Jul  2 17:02 LOCK-rw-r--r-- 1 root root  2807 Jul  4 14:12 LOG-rw-r--r-- 1 root root   346 Jul  4 14:12 MANIFEST-000011

进入Lightchaindata

root@i-5tthrr8u:/home/ubuntu/private-geth/data/00/geth/lightchaindata# lltotal 24drwxr-xr-x 2 root root 4096 Jul  2 17:02 ./drwxr-xr-x 5 root root 4096 Jul  2 17:02 ../-rw-r--r-- 1 root root 1237 Jul  2 17:02 000001.log-rw-r--r-- 1 root root   16 Jul  2 17:02 CURRENT-rw-r--r-- 1 root root    0 Jul  2 17:02 LOCK-rw-r--r-- 1 root root  358 Jul  2 17:02 LOG-rw-r--r-- 1 root root   54 Jul  2 17:02 MANIFEST-000000

第二部分 看看源码的结构

1 Core/types/block.go
首先看到的是一个区块的结构

// Block represents an entire block in the Ethereum blockchain.type Block struct {    header       *Header    uncles       []*Header    transactions Transactions    // caches    hash和size字段是cache之用，避免多次 hash/sign导致性能损失    hash atomic.Value    size atomic.Value    // Td is used by package core to store the total difficulty    // of the chain up to and including the block.挖矿难度    td *big.Int    // These fields are used by package eth to track    // inter-peer block relay.    ReceivedAt   time.Time    ReceivedFrom interface{}}

这是一个区块体的结构，区块体是动态的存储数据的，主要包含了交易列表和uncle列表// Body is a simple (mutable, non-safe) data container for storing and moving// a block's data contents (transactions and uncles) together.type Body struct {    Transactions []*Transaction    Uncles       []*Header}

区块头的结构体，里面的参数我们都很熟悉就不解释了// Header represents a block header in the Ethereum blockchain.type Header struct {    ParentHash  common.Hash    `json:"parentHash"       gencodec:"required"`    UncleHash   common.Hash    `json:"sha3Uncles"       gencodec:"required"`    Coinbase    common.Address `json:"miner"            gencodec:"required"`    Root        common.Hash    `json:"stateRoot"        gencodec:"required"`    TxHash      common.Hash    `json:"transactionsRoot" gencodec:"required"`    ReceiptHash common.Hash    `json:"receiptsRoot"     gencodec:"required"`    Bloom       Bloom          `json:"logsBloom"        gencodec:"required"`    Difficulty  *big.Int       `json:"difficulty"       gencodec:"required"`    Number      *big.Int       `json:"number"           gencodec:"required"`    GasLimit    *big.Int       `json:"gasLimit"         gencodec:"required"`    GasUsed     *big.Int       `json:"gasUsed"          gencodec:"required"`    Time        *big.Int       `json:"timestamp"        gencodec:"required"`    Extra       []byte         `json:"extraData"        gencodec:"required"`    MixDigest   common.Hash    `json:"mixHash"          gencodec:"required"`    Nonce       BlockNonce     `json:"nonce"            gencodec:"required"`}

2 这是一个交易的结构体
Core/types/transaction.go

1ContractTransaction的区别在于：Recipient == nil ； 2. Transaction能以RLP算法进行Encode和Decode； 3. hash/size/from字段是cache之用，避免多次 hash/sign导致性能损失；type Transaction struct {    data txdata    // caches    hash atomic.Value    size atomic.Value    from atomic.Value}type txdata struct {    AccountNonce uint64          `json:"nonce"    gencodec:"required"`    Price        *big.Int        `json:"gasPrice" gencodec:"required"`    GasLimit     *big.Int        `json:"gas"      gencodec:"required"`    Recipient    *common.Address `json:"to"       rlp:"nil"` // nil means contract creation    Amount       *big.Int        `json:"value"    gencodec:"required"`    Payload      []byte          `json:"input"    gencodec:"required"`    // Signature values 签名    V *big.Int `json:"v" gencodec:"required"`    R *big.Int `json:"r" gencodec:"required"`    S *big.Int `json:"s" gencodec:"required"`    // This is only used when marshaling to JSON.    Hash *common.Hash `json:"hash" rlp:"-"`}

3 Receiptroot我们刚刚在区块头有看到，那他具体包含的是什么呢？它是一个交易的结果，主要包括了poststate,交易所花费的gas,bloom和logs

// Receipt represents the results of a transaction.type Receipt struct {    // Consensus fields    PostState         []byte   `json:"root"              gencodec:"required"`    CumulativeGasUsed *big.Int `json:"cumulativeGasUsed" gencodec:"required"`    Bloom             Bloom    `json:"logsBloom"         gencodec:"required"`    Logs              []*Log   `json:"logs"              gencodec:"required"`    // Implementation fields (don't reorder!)    TxHash          common.Hash    `json:"transactionHash" gencodec:"required"`    ContractAddress common.Address `json:"contractAddress"`    GasUsed         *big.Int       `json:"gasUsed" gencodec:"required"`}

4 一个个交易被打包到区块上面，那区块又是怎么变成去快链的呢？
Core/blockchain.go

// BlockChain represents the canonical chain given a database with a genesis block. The Blockchain manages chain imports, reverts, chain reorganisations.// Importing blocks in to the block chain happens according to the set of rules defined by the two stage Validator. （需要两个阶段的验证）Processing of blocks is done using the Processor which processes the included transaction.（第一阶段交易的验证） The validation of the state is done in the second part of the Validator.（第二阶段state的验证） Failing results in aborting of the import.// The BlockChain also helps in returning blocks from **any** chain included in the database as well as blocks that represents the canonical chain. It's important to note that GetBlock can return any block and does not need to be included in the canonical one where as GetBlockByNumber always represents the canonical chain.type BlockChain struct {    config *params.ChainConfig // chain & network configuration    hc           *HeaderChain    chainDb      **ethdb**.Database 本地数据库    eventMux     *event.TypeMux    genesisBlock *types.Block    mu      sync.RWMutex // global mutex for locking chain operations    chainmu sync.RWMutex // blockchain insertion lock    procmu  sync.RWMutex // block processor lock    checkpoint       int          // checkpoint counts towards the new checkpoint    currentBlock     *types.Block // Current head of the block chain    currentFastBlock *types.Block // Current head of the fast-sync chain (may be above the block chain!)    stateCache   *state.StateDB // State database to reuse between imports (contains state cache)    bodyCache    *lru.Cache     // Cache for the most recent block bodies    bodyRLPCache *lru.Cache     // Cache for the most recent block bodies in RLP encoded format    blockCache   *lru.Cache     // Cache for the most recent entire blocks    futureBlocks *lru.Cache     // future blocks are blocks added for later processing    quit    chan struct{} // blockchain quit channel    running int32         // running must be called atomically    // procInterrupt must be atomically called    procInterrupt int32          // interrupt signaler for block processing    wg            sync.WaitGroup // chain processing wait group for shutting down    engine    consensus.Engine    processor Processor // block processor interface    validator Validator // block and state validator interface    vmConfig  vm.Config    badBlocks *lru.Cache // Bad block cache}

注意：1. BlockChain无结构化查询需求，仅Hash查询， Key/Value数据库最方便； 2. 低层用LevelDB存储，性能好

5 stateDB用来存储世界状态
Core/state/statedb.go

// StateDBs within the ethereum protocol are used to store anything// within the merkle trie. StateDBs take care of caching and storing// nested states. It's the general query interface to retrieve:// * Contracts// * Accountstype StateDB struct {    db            ethdb.Database //本地数据库    trie          *trie.SecureTrie    pastTries     []*trie.SecureTrie    codeSizeCache *lru.Cache    // This map holds 'live' objects, which will get modified while processing a state transition.    stateObjects           map[common.Address]*stateObject    stateObjectsDirty      map[common.Address]struct{}    stateObjectsDestructed map[common.Address]struct{}    // The refund counter, also used by state transitioning.    refund *big.Int    thash, bhash common.Hash    txIndex      int    logs         map[common.Hash][]*types.Log    logSize      uint    preimages map[common.Hash][]byte    // Journal of state modifications. This is the backbone of    // Snapshot and RevertToSnapshot.    journal        journal    validRevisions []revision    nextRevisionId int    lock sync.Mutex}

注意：1. StateDB完整记录Transaction的执行情况； 2. StateDB的重点是StateObjects； 3. StateDB中的 stateObjects，Account的Address为 key，记录其Balance、nonce、code、codeHash ，以及tire中的 {string:Hash}等信息；

那我们接下来看看stateObject结构体
Core/state/state_object.go

// stateObject represents an Ethereum account which is being modified.//// The usage pattern is as follows:// First you need to obtain a state object.// Account values can be accessed and modified through the object.// Finally, call CommitTrie to write the modified storage trie into a database.type stateObject struct {    address common.Address // Ethereum address of this account    data    Account    db      *StateDB    // DB error.    // State objects are used by the consensus core and VM which are    // unable to deal with database-level errors. Any error that occurs    // during a database read is memoized here and will eventually be returned    // by StateDB.Commit.    dbErr error    // Write caches.    trie *trie.SecureTrie // storage trie, which becomes non-nil on first access    code Code             // contract bytecode, which gets set when code is loaded    cachedStorage Storage // Storage entry cache to avoid duplicate reads    dirtyStorage  Storage // Storage entries that need to be flushed to disk    // Cache flags.    // When an object is marked suicided it will be delete from the trie    // during the "update" phase of the state transition.    dirtyCode bool // true if the code was updated    suicided  bool    touched   bool    deleted   bool    onDirty   func(addr common.Address) // Callback method to mark a state object newly dirty}

再看看state的一个接口，可以查看账户的余额，nonce，代码和storage

// ChainStateReader wraps access to the state trie of the canonical blockchain. Note that implementations of the interface may be unable to return state values for old blocks.// In many cases, using CallContract can be preferable to reading raw contract storage.type ChainStateReader interface {    BalanceAt(ctx context.Context, account common.Address, blockNumber *big.Int) (*big.Int, error)    StorageAt(ctx context.Context, account common.Address, key common.Hash, blockNumber *big.Int) ([]byte, error)    CodeAt(ctx context.Context, account common.Address, blockNumber *big.Int) ([]byte, error)    NonceAt(ctx context.Context, account common.Address, blockNumber *big.Int) (uint64, error)}

所有的结构凑明朗了，那具体的验证过程是怎么样的呢
Core/state_processor.go
Core/state_transition.go
Core/block_validator.go

StateProcessor 1. 调用StateTransition，验证（执行）Transaction； 2. 计算Gas、Recipt、Uncle Reward

// StateProcessor is a basic Processor, which takes care of transitioning// state from one point to another.//// StateProcessor implements Processor.type StateProcessor struct {    config *params.ChainConfig // Chain configuration options    bc     *BlockChain         // Canonical block chain    engine consensus.Engine    // Consensus engine used for block rewards}

StateTransition
1. 验证（执行）Transaction；
3. 扣除transaction.data.payload计算数据所需要消耗的gas；
4. 在vm中执行code（生成contract or 执行contract）；vm执 行过程中，其gas会被自动消耗。如果gas不足，vm会自 选退出；
5. 将多余的gas退回到sender.balance中；
6. 将消耗的gas换成balance加到当前env.Coinbase()中；

/*The State Transitioning ModelA state transition is a change made when a transaction is applied to the current world stateThe state transitioning model does all all the necessary work to work out a valid new state root.1) Nonce handling2) Pre pay gas3) Create a new state object if the recipient is \0*324) Value transfer== If contract creation ==  4a) Attempt to run transaction data  4b) If valid, use result as code for the new state object== end ==5) Run Script section6) Derive new state root*/type StateTransition struct {    gp         *GasPool    msg        Message    gas        uint64    gasPrice   *big.Int    initialGas *big.Int    value      *big.Int    data       []byte    state      vm.StateDB    evm *vm.EVM}

BlockValidator
1. 验证UsedGas
2. 验证Bloom
3. 验证receiptSha
4. 验证stateDB.IntermediateRoot

// BlockValidator is responsible for validating block headers, uncles and// processed state.//// BlockValidator implements Validator.type BlockValidator struct {    config *params.ChainConfig // Chain configuration options    bc     *BlockChain         // Canonical block chain    engine consensus.Engine    // Consensus engine used for validating}

可以注意到刚才的state和block都是写进db数据库的，那我们看一下leveldb数据库结构

type LDBDatabase struct {    fn string      // filename for reporting    db *leveldb.DB // LevelDB instance    getTimer       gometrics.Timer // Timer for measuring the database get request counts and latencies    putTimer       gometrics.Timer // Timer for measuring the database put request counts and latencies    delTimer       gometrics.Timer // Timer for measuring the database delete request counts and latencies    missMeter      gometrics.Meter // Meter for measuring the missed database get requests    readMeter      gometrics.Meter // Meter for measuring the database get request data usage    writeMeter     gometrics.Meter // Meter for measuring the database put request data usage    compTimeMeter  gometrics.Meter // Meter for measuring the total time spent in database compaction    compReadMeter  gometrics.Meter // Meter for measuring the data read during compaction    compWriteMeter gometrics.Meter // Meter for measuring the data written during compaction    quitLock sync.Mutex      // Mutex protecting the quit channel access    quitChan chan chan error // Quit channel to stop the metrics collection before closing the database    log log.Logger // Contextual logger tracking the database path}

type Database interface {    Put(key []byte, value []byte) error    Get(key []byte) ([]byte, error)    Delete(key []byte) error    Close()    NewBatch() Batch}