程序博客网 > 淘宝官网电脑版

hbase 源代码分析 （8） delete 过程 详解

来源：互联网 发布：淘宝官网电脑版 编辑：程序博客网 时间：2024/06/05 06:20
上一章节：hbase 源代码分析 （7） put 过程 详解
http://blog.csdn.net/chenfenggang/article/details/75094161


delete的过程和put相似。

具体可以参考put过程

delete很直接，直接到第一步就发送了请求，相比put，get，scan，已经getRegionLocator等过程，这简直就是奇迹。所以我也很直接把所有都贴出来了。
  1.   @Override
  2.   public void delete(final Delete delete)
  3.   throws IOException {
  4.     RegionServerCallable<Boolean> callable = new RegionServerCallable<Boolean>(connection,
  5.         tableName, delete.getRow()) {
  6.       @Override
  7.       public Boolean call(int callTimeout) throws IOException {
  8.         PayloadCarryingRpcController controller = rpcControllerFactory.newController();
  9.         controller.setPriority(tableName);
  10.         controller.setCallTimeout(callTimeout);
  12.         try {
  13.           MutateRequest request = RequestConverter.buildMutateRequest(
  14.             getLocation().getRegionInfo().getRegionName(), delete);
  15.           MutateResponse response = getStub().mutate(controller, request);
  16.           return Boolean.valueOf(response.getProcessed());
  17.         } catch (ServiceException se) {
  18.           throw ProtobufUtil.getRemoteException(se);
  19.         }
  20.       }
  21.     };
  22.     rpcCallerFactory.<Boolean> newCaller(rpcTimeout).callWithRetries(callable,
  23.         this.operationTimeout);
  24.   }


服务端，装转几回合之后调用了这主要的方法。长的要命。400行，我不删除了，留着你们细细看吧。

1）转到后面的类的几个重要过程。
  1.      region.delete(delete);
  1.   OperationStatus[] batchMutate = this.batchMutate(new Mutation[]{mutation});
这里面的几个转转主要是，
1）钩子，很多协处理要处理。
2）认证，判断正确性
3）分情况。
然后到这个类。

这个大类主要就看看里面红色的注解就行了。什么都帮我做了。我就轻松了。
  1.  private long doMiniBatchMutation(BatchOperationInProgress<?> batchOp) throws IOException {
  2.     boolean isInReplay = batchOp.isInReplay();
  3.     // variable to note if all Put items are for the same CF -- metrics related
  4.     boolean putsCfSetConsistent = true;
  5.     //The set of columnFamilies first seen for Put.
  6.     Set<byte[]> putsCfSet = null;
  7.     // variable to note if all Delete items are for the same CF -- metrics related
  8.     boolean deletesCfSetConsistent = true;
  9.     //The set of columnFamilies first seen for Delete.
  10.     Set<byte[]> deletesCfSet = null;
  12.     long currentNonceGroup = HConstants.NO_NONCE, currentNonce = HConstants.NO_NONCE;
  13.     WALEdit walEdit = new WALEdit(isInReplay);
  14.     MultiVersionConcurrencyControl.WriteEntry writeEntry = null;
  15.     long txid = 0;
  16.     boolean doRollBackMemstore = false;
  17.     boolean locked = false;
  19.     /** Keep track of the locks we hold so we can release them in finally clause */
  20.     List<RowLock> acquiredRowLocks = Lists.newArrayListWithCapacity(batchOp.operations.length);
  21.     // reference family maps directly so coprocessors can mutate them if desired
  22.     Map<byte[], List<Cell>>[] familyMaps = new Map[batchOp.operations.length];
  23.     // We try to set up a batch in the range [firstIndex,lastIndexExclusive)
  24.     int firstIndex = batchOp.nextIndexToProcess;
  25.     int lastIndexExclusive = firstIndex;
  26.     boolean success = false;
  27.     int noOfPuts = 0, noOfDeletes = 0;
  28.     WALKey walKey = null;
  29.     long mvccNum = 0;
  30.     long addedSize = 0;
  31.     try {
  32.       // ------------------------------------
  33.       // STEP 1. Try to acquire as many locks as we can, and ensure
  34.       // we acquire at least one.
  35.       // ----------------------------------
  36.       int numReadyToWrite = 0;
  37.       long now = EnvironmentEdgeManager.currentTime();
  38.       while (lastIndexExclusive < batchOp.operations.length) {
  39.         Mutation mutation = batchOp.getMutation(lastIndexExclusive);
  40.         boolean isPutMutation = mutation instanceof Put;
  42.         Map<byte[], List<Cell>> familyMap = mutation.getFamilyCellMap();
  43.         // store the family map reference to allow for mutations
  44.         familyMaps[lastIndexExclusive] = familyMap;
  46.         // skip anything that "ran" already
  47.         if (batchOp.retCodeDetails[lastIndexExclusive].getOperationStatusCode()
  48.             != OperationStatusCode.NOT_RUN) {
  49.           lastIndexExclusive++;
  50.           continue;
  51.         }
  53.         try {
  54.           if (isPutMutation) {
  55.             // Check the families in the put. If bad, skip this one.
  56.             if (isInReplay) {
  57.               removeNonExistentColumnFamilyForReplay(familyMap);
  58.             } else {
  59.               checkFamilies(familyMap.keySet());
  60.             }
  61.             checkTimestamps(mutation.getFamilyCellMap(), now);
  62.           } else {
  63.             prepareDelete((Delete) mutation);
  64.           }
  65.           checkRow(mutation.getRow(), "doMiniBatchMutation");
  66.         } catch (NoSuchColumnFamilyException nscf) {
  67.           LOG.warn("No such column family in batch mutation", nscf);
  68.           batchOp.retCodeDetails[lastIndexExclusive] = new OperationStatus(
  69.               OperationStatusCode.BAD_FAMILY, nscf.getMessage());
  70.           lastIndexExclusive++;
  71.           continue;
  72.         } catch (FailedSanityCheckException fsce) {
  73.           LOG.warn("Batch Mutation did not pass sanity check", fsce);
  74.           batchOp.retCodeDetails[lastIndexExclusive] = new OperationStatus(
  75.               OperationStatusCode.SANITY_CHECK_FAILURE, fsce.getMessage());
  76.           lastIndexExclusive++;
  77.           continue;
  78.         } catch (WrongRegionException we) {
  79.           LOG.warn("Batch mutation had a row that does not belong to this region", we);
  80.           batchOp.retCodeDetails[lastIndexExclusive] = new OperationStatus(
  81.               OperationStatusCode.SANITY_CHECK_FAILURE, we.getMessage());
  82.           lastIndexExclusive++;
  83.           continue;
  84.         }
  86.         // If we haven't got any rows in our batch, we should block to
  87.         // get the next one.
  88.         RowLock rowLock = null;
  89.         try {
  90.           rowLock = getRowLock(mutation.getRow(), true);
  91.         } catch (IOException ioe) {
  92.           LOG.warn("Failed getting lock in batch put, row="
  93.             + Bytes.toStringBinary(mutation.getRow()), ioe);
  94.         }
  95.         if (rowLock == null) {
  96.           // We failed to grab another lock
  97.           break; // stop acquiring more rows for this batch
  98.         } else {
  99.           acquiredRowLocks.add(rowLock);
  100.         }
  102.         lastIndexExclusive++;
  103.         numReadyToWrite++;
  105.         if (isPutMutation) {
  106.           // If Column Families stay consistent through out all of the
  107.           // individual puts then metrics can be reported as a mutliput across
  108.           // column families in the first put.
  109.           if (putsCfSet == null) {
  110.             putsCfSet = mutation.getFamilyCellMap().keySet();
  111.           } else {
  112.             putsCfSetConsistent = putsCfSetConsistent
  113.                 && mutation.getFamilyCellMap().keySet().equals(putsCfSet);
  114.           }
  115.         } else {
  116.           if (deletesCfSet == null) {
  117.             deletesCfSet = mutation.getFamilyCellMap().keySet();
  118.           } else {
  119.             deletesCfSetConsistent = deletesCfSetConsistent
  120.                 && mutation.getFamilyCellMap().keySet().equals(deletesCfSet);
  121.           }
  122.         }
  123.       }
  125.       // we should record the timestamp only after we have acquired the rowLock,
  126.       // otherwise, newer puts/deletes are not guaranteed to have a newer timestamp
  127.       now = EnvironmentEdgeManager.currentTime();
  128.       byte[] byteNow = Bytes.toBytes(now);
  130.       // Nothing to put/delete -- an exception in the above such as NoSuchColumnFamily?
  131.       if (numReadyToWrite <= 0) return 0L;
  133.       // We've now grabbed as many mutations off the list as we can
  135.       // ------------------------------------
  136.       // STEP 2. Update any LATEST_TIMESTAMP timestamps
  137.       // ----------------------------------
  138.       for (int i = firstIndex; !isInReplay && i < lastIndexExclusive; i++) {
  139.         // skip invalid
  140.         if (batchOp.retCodeDetails[i].getOperationStatusCode()
  141.             != OperationStatusCode.NOT_RUN) continue;
  143.         Mutation mutation = batchOp.getMutation(i);
  144.         if (mutation instanceof Put) {
  145.           updateCellTimestamps(familyMaps[i].values(), byteNow);
  146.           noOfPuts++;
  147.         } else {
  148.           prepareDeleteTimestamps(mutation, familyMaps[i], byteNow);
  149.           noOfDeletes++;
  150.         }
  151.         rewriteCellTags(familyMaps[i], mutation);
  152.       }
  154.       lock(this.updatesLock.readLock(), numReadyToWrite);
  155.       locked = true;
  157.       // calling the pre CP hook for batch mutation
  158.       if (!isInReplay && coprocessorHost != null) {
  159.         MiniBatchOperationInProgress<Mutation> miniBatchOp =
  160.           new MiniBatchOperationInProgress<Mutation>(batchOp.getMutationsForCoprocs(),
  161.           batchOp.retCodeDetails, batchOp.walEditsFromCoprocessors, firstIndex, lastIndexExclusive);
  162.         if (coprocessorHost.preBatchMutate(miniBatchOp)) return 0L;
  163.       }
  165.       // ------------------------------------
  166.       // STEP 3. Build WAL edit
  167.       // ----------------------------------
  168.       Durability durability = Durability.USE_DEFAULT;
  169.       for (int i = firstIndex; i < lastIndexExclusive; i++) {
  170.         // Skip puts that were determined to be invalid during preprocessing
  171.         if (batchOp.retCodeDetails[i].getOperationStatusCode() != OperationStatusCode.NOT_RUN) {
  172.           continue;
  173.         }
  175.         Mutation m = batchOp.getMutation(i);
  176.         Durability tmpDur = getEffectiveDurability(m.getDurability());
  177.         if (tmpDur.ordinal() > durability.ordinal()) {
  178.           durability = tmpDur;
  179.         }
  180.         if (tmpDur == Durability.SKIP_WAL) {
  181.           recordMutationWithoutWal(m.getFamilyCellMap());
  182.           continue;
  183.         }
  185.         long nonceGroup = batchOp.getNonceGroup(i), nonce = batchOp.getNonce(i);
  186.         // In replay, the batch may contain multiple nonces. If so, write WALEdit for each.
  187.         // Given how nonces are originally written, these should be contiguous.
  188.         // They don't have to be, it will still work, just write more WALEdits than needed.
  189.         if (nonceGroup != currentNonceGroup || nonce != currentNonce) {
  190.           if (walEdit.size() > 0) {
  191.             assert isInReplay;
  192.             if (!isInReplay) {
  193.               throw new IOException("Multiple nonces per batch and not in replay");
  194.             }
  195.             // txid should always increase, so having the one from the last call is ok.
  196.             // we use HLogKey here instead of WALKey directly to support legacy coprocessors.
  197.             walKey = new ReplayHLogKey(this.getRegionInfo().getEncodedNameAsBytes(),
  198.               this.htableDescriptor.getTableName(), now, m.getClusterIds(),
  199.               currentNonceGroup, currentNonce, mvcc);
  200.             txid = this.wal.append(this.htableDescriptor,  this.getRegionInfo(),  walKey,
  201.               walEdit, true);
  202.             walEdit = new WALEdit(isInReplay);
  203.             walKey = null;
  204.           }
  205.           currentNonceGroup = nonceGroup;
  206.           currentNonce = nonce;
  207.         }
  209.         // Add WAL edits by CP
  210.         WALEdit fromCP = batchOp.walEditsFromCoprocessors[i];
  211.         if (fromCP != null) {
  212.           for (Cell cell : fromCP.getCells()) {
  213.             walEdit.add(cell);
  214.           }
  215.         }
  216.         addFamilyMapToWALEdit(familyMaps[i], walEdit);
  217.       }
  219.       // -------------------------
  220.       // STEP 4. Append the final edit to WAL. Do not sync wal.
  221.       // -------------------------
  222.       Mutation mutation = batchOp.getMutation(firstIndex);
  223.       if (isInReplay) {
  224.         // use wal key from the original
  225.         walKey = new ReplayHLogKey(this.getRegionInfo().getEncodedNameAsBytes(),
  226.           this.htableDescriptor.getTableName(), WALKey.NO_SEQUENCE_ID, now,
  227.           mutation.getClusterIds(), currentNonceGroup, currentNonce, mvcc);
  228.         long replaySeqId = batchOp.getReplaySequenceId();
  229.         walKey.setOrigLogSeqNum(replaySeqId);
  230.       }
  231.       if (walEdit.size() > 0) {
  232.         if (!isInReplay) {
  233.         // we use HLogKey here instead of WALKey directly to support legacy coprocessors.
  234.         walKey = new HLogKey(this.getRegionInfo().getEncodedNameAsBytes(),
  235.             this.htableDescriptor.getTableName(), WALKey.NO_SEQUENCE_ID, now,
  236.             mutation.getClusterIds(), currentNonceGroup, currentNonce, mvcc);
  237.         }
  238.         txid = this.wal.append(this.htableDescriptor, this.getRegionInfo(), walKey, walEdit, true);
  239.       }
  240.       // ------------------------------------
  241.       // Acquire the latest mvcc number
  242.       // ----------------------------------
  243.       if (walKey == null) {
  244.         // If this is a skip wal operation just get the read point from mvcc
  245.         walKey = this.appendEmptyEdit(this.wal);
  246.       }
  247.       if (!isInReplay) {
  248.         writeEntry = walKey.getWriteEntry();
  249.         mvccNum = writeEntry.getWriteNumber();
  250.       } else {
  251.         mvccNum = batchOp.getReplaySequenceId();
  252.       }
  254.       // ------------------------------------
  255.       // STEP 5. Write back to memstore
  256.       // Write to memstore. It is ok to write to memstore
  257.       // first without syncing the WAL because we do not roll
  258.       // forward the memstore MVCC. The MVCC will be moved up when
  259.       // the complete operation is done. These changes are not yet
  260.       // visible to scanners till we update the MVCC. The MVCC is
  261.       // moved only when the sync is complete.
  262.       // ----------------------------------
  263.       for (int i = firstIndex; i < lastIndexExclusive; i++) {
  264.         if (batchOp.retCodeDetails[i].getOperationStatusCode()
  265.             != OperationStatusCode.NOT_RUN) {
  266.           continue;
  267.         }
  268.         doRollBackMemstore = true; // If we have a failure, we need to clean what we wrote
  269.         addedSize += applyFamilyMapToMemstore(familyMaps[i], mvccNum, isInReplay);
  270.       }
  272.       // -------------------------------
  273.       // STEP 6. Release row locks, etc.
  274.       // -------------------------------
  275.       if (locked) {
  276.         this.updatesLock.readLock().unlock();
  277.         locked = false;
  278.       }
  279.       releaseRowLocks(acquiredRowLocks);
  281.       // -------------------------
  282.       // STEP 7. Sync wal.
  283.       // -------------------------
  284.       if (txid != 0) {
  285.         syncOrDefer(txid, durability);
  286.       }
  288.       doRollBackMemstore = false;
  289.       // calling the post CP hook for batch mutation
  290.       if (!isInReplay && coprocessorHost != null) {
  291.         MiniBatchOperationInProgress<Mutation> miniBatchOp =
  292.           new MiniBatchOperationInProgress<Mutation>(batchOp.getMutationsForCoprocs(),
  293.           batchOp.retCodeDetails, batchOp.walEditsFromCoprocessors, firstIndex, lastIndexExclusive);
  294.         coprocessorHost.postBatchMutate(miniBatchOp);
  295.       }
  297.       // ------------------------------------------------------------------
  298.       // STEP 8. Advance mvcc. This will make this put visible to scanners and getters.
  299.       // ------------------------------------------------------------------
  300.       if (writeEntry != null) {
  301.         mvcc.completeAndWait(writeEntry);
  302.         writeEntry = null;
  303.       } else if (isInReplay) {
  304.         // ensure that the sequence id of the region is at least as big as orig log seq id
  305.         mvcc.advanceTo(mvccNum);
  306.       }
  308.       for (int i = firstIndex; i < lastIndexExclusive; i ++) {
  309.         if (batchOp.retCodeDetails[i] == OperationStatus.NOT_RUN) {
  310.           batchOp.retCodeDetails[i] = OperationStatus.SUCCESS;
  311.         }
  312.       }
  314.       // ------------------------------------
  315.       // STEP 9. Run coprocessor post hooks. This should be done after the wal is
  316.       // synced so that the coprocessor contract is adhered to.
  317.       // ------------------------------------
  318.       if (!isInReplay && coprocessorHost != null) {
  319.         for (int i = firstIndex; i < lastIndexExclusive; i++) {
  320.           // only for successful puts
  321.           if (batchOp.retCodeDetails[i].getOperationStatusCode()
  322.               != OperationStatusCode.SUCCESS) {
  323.             continue;
  324.           }
  325.           Mutation m = batchOp.getMutation(i);
  326.           if (m instanceof Put) {
  327.             coprocessorHost.postPut((Put) m, walEdit, m.getDurability());
  328.           } else {
  329.             coprocessorHost.postDelete((Delete) m, walEdit, m.getDurability());
  330.           }
  331.         }
  332.       }
  334.       success = true;
  335.       return addedSize;
  336.     } finally {
  337.       // if the wal sync was unsuccessful, remove keys from memstore
  338.       if (doRollBackMemstore) {
  339.         for (int j = 0; j < familyMaps.length; j++) {
  340.           for(List<Cell> cells:familyMaps[j].values()) {
  341.             rollbackMemstore(cells);
  342.           }
  343.         }
  344.         if (writeEntry != null) mvcc.complete(writeEntry);
  345.       } else {
  346.         this.addAndGetGlobalMemstoreSize(addedSize);
  347.         if (writeEntry != null) {
  348.           mvcc.completeAndWait(writeEntry);
  349.         }
  350.       }
  352.       if (locked) {
  353.         this.updatesLock.readLock().unlock();
  354.       }
  355.       releaseRowLocks(acquiredRowLocks);
  357.       // See if the column families were consistent through the whole thing.
  358.       // if they were then keep them. If they were not then pass a null.
  359.       // null will be treated as unknown.
  360.       // Total time taken might be involving Puts and Deletes.
  361.       // Split the time for puts and deletes based on the total number of Puts and Deletes.
  363.       if (noOfPuts > 0) {
  364.         // There were some Puts in the batch.
  365.         if (this.metricsRegion != null) {
  366.           this.metricsRegion.updatePut();
  367.         }
  368.       }
  369.       if (noOfDeletes > 0) {
  370.         // There were some Deletes in the batch.
  371.         if (this.metricsRegion != null) {
  372.           this.metricsRegion.updateDelete();
  373.         }
  374.       }
  375.       if (!success) {
  376.         for (int i = firstIndex; i < lastIndexExclusive; i++) {
  377.           if (batchOp.retCodeDetails[i].getOperationStatusCode() == OperationStatusCode.NOT_RUN) {
  378.             batchOp.retCodeDetails[i] = OperationStatus.FAILURE;
  379.           }
  380.         }
  381.       }
  382.       if (coprocessorHost != null && !batchOp.isInReplay()) {
  383.         // call the coprocessor hook to do any finalization steps
  384.         // after the put is done
  385.         MiniBatchOperationInProgress<Mutation> miniBatchOp =
  386.             new MiniBatchOperationInProgress<Mutation>(batchOp.getMutationsForCoprocs(),
  387.                 batchOp.retCodeDetails, batchOp.walEditsFromCoprocessors, firstIndex,
  388.                 lastIndexExclusive);
  389.         coprocessorHost.postBatchMutateIndispensably(miniBatchOp, success);
  390.       }
  392.       batchOp.nextIndexToProcess = lastIndexExclusive;
  393.     }
  394.   }

很轻松完成了。

未完待续。。。














阅读全文
0 0
淘宝官网电脑版 淘宝官网电脑版
原创粉丝点击
热门IT博客
幻想神域辅助软件幻想神域辅助软件
java数组的长度怎么算java数组的长度怎么算
周扬青 淘宝直播
网络机顶盒的软件
政府网站域名
ubuntu进桌面后无unity
js判断不等于
在线录音机软件
微信抢红包脚本源码js
黑马就业班java单词表
ubuntu设置任务栏位置
淘宝怎么设置折扣
淘宝关注量最高的店铺
淘宝可以自助组装电脑
intent传递二进制数据
vms监控软件
汽车电子控制软件
sql 修改字段值
冯大辉 知乎匿名
net与php的区别
热门问题 老师的惩罚 人脸识别 我在镇武司摸鱼那些年 重生之率土为王 我在大康的咸鱼生活 盘龙之生命进化 天生仙种 凡人之先天五行 春回大明朝 姑娘不必设防，我是瞎子 一个木一个乐 陈乐一资料 一乐也 啵乐一腐味漫漫野画集 陈一乐 乐一番 东鹏特饮一元乐享 舒一乐 啵乐腐味漫漫野画集 金字旁一个乐 啵乐一腐味满满下载 及时行乐下一句 一人向隅满坐不乐 济南欧乐堡一日攻略 安居乐业打一生肖 寓教于乐下一句 啵乐一腐味满满野画集 平安喜乐下一句 乐堡啤酒多少钱一瓶 乐茜乐茹18一35 一个金字旁一个乐 一个火一个乐 陈乐一梦里花 乐一乐 乐一 陈一 昆明一成驾校 空乘工资多少钱一个月 一个山一个乘 忽复乘舟梦日边上一句 一个山一个乘读什么 宝贝乘涨开一点 三个5一个1加减乘除等于24 一乘寺贤 2018男空乘一个月多少钱 4乘120电缆多少钱一米 十八乘六打一字谜底 昆明一乘驾校电话 云南一乘驾校 一乘驾校学费是多少 昆明一乘驾校

程序博客网，程序员的互联网技术博客家园。csdn论坛精品 msdn技术资料都在这里