HBase客户端访问超时的多个因素及参数

在一个需要低延时响应的hbase集群中,使用hbase默认的客户端超时配置简直就是灾难。

但是我们可以考虑在客户端上加上如下几个参数,去改变这种状况:

1. hbase.rpc.timeout: RPC timeout, The default 60s, 可以修改为5000(5s)

2. ipc.socket.timeout: Socket link timeout, should be less than or equal to RPC timeout, the default is 20s

3. hbase.client.retries.number: The number of retries, default is 14, 可以配置为1

4. hbase.client.pause: Sleep time again, the default is 1s, can be reduced, such as 100ms(在1.1版本的hbase已经变为100ms,请对照你使用的hbase版本)

5. zookeeper.recovery.retry: The number of retries ZK, Can be adjusted to 3 times, ZK is not easy to hang, And if HBase cluster problem, Each retry retry the operation of ZK will be, The total number of retry ZK is: hbase.client.retries.number * zookeeper.recovery.retry, And sleep time each retry will have exponential growth of 2, Every time you access the HBase will try again, In a HBase operation if it involves multiple ZK access, If ZK is not available, There will be many times the ZK retry, Is a waste of time.

6. zookeeper.recovery.retry.intervalmill: Sleep time ZK retries, the default is 1s, can be reduced, for example: 200ms

7. hbase.regionserver.lease.period: A scan query when interacting with server timeout, the default is 60s.(在1.1版本,该参数名为hbase.client.scanner.timeout.period)

 

Retry interval strategy RPC:

public static long getPauseTime(final long pause, final int tries) {

int ntries = tries;

// RETRY_BACKOFF[] = { 1, 1, 1, 2, 2, 4, 4, 8, 16, 32, 64 }

    if (ntries >= HConstants.RETRY_BACKOFF.length) {

      ntries = HConstants.RETRY_BACKOFF.length - 1;

    }

    long normalPause = pause * HConstants.RETRY_BACKOFF[ntries];

    long jitter =  (long)(normalPause * RANDOM.nextFloat() * 0.01f); // 1% possible jitter

    return normalPause + jitter;

  }

 

 

Retry interval strategy ZK:

// RetryCounter类

//Sleep time 指数级增长

public void sleepUntilNextRetry() throws InterruptedException {

    int attempts = getAttemptTimes();

    long sleepTime = (long) (retryIntervalMillis * Math.pow(2, attempts));

    timeUnit.sleep(sleepTime);

       }

      

// retriesRemaining, The default value ismaxReties, Each retry after reduction1

       public int getAttemptTimes() {

          return maxRetries-retriesRemaining+1;

       }

 

已标记关键词 清除标记
相关推荐
<p> <strong><span style="font-size:20px;color:#FF0000;">本课程主要针对计算机相关专业的正在做毕设的学生与需要项目实战练习的Java学习者</span></strong> </p> <p> <span style="color:#FF0000;"><strong><span style="font-size:18px;">1. 包含:<span style="color:#FFFF00;background-color:#FF0000;">项目源码、</span><span style="color:#FFFF00;background-color:#FF0000;">项目文档、数据库脚本、软件工具</span>等所有资料</span></strong></span> </p> <p> <span style="color:#FF0000;"><strong><span style="font-size:18px;">2. 手把手的带你从零开始部署运行本套系统</span></strong></span> </p> <p> <span style="color:#FF0000;"><strong><span style="font-size:18px;">3. 该项目附带的源码资料可作为毕设使用</span></strong></span> </p> <p> <span style="color:#FF0000;"><strong><span style="font-size:18px;">4. 提供技术答疑和远程协助指导</span></strong></span><strong><span style="font-size:18px;"></span></strong> </p> <p> <br /> </p> <p> <span style="font-size:18px;"><strong>项目运行截图:</strong></span> </p> <p> <strong><span style="font-size:18px;">1)系统登陆界面</span></strong> </p> <p> <strong><span style="font-size:18px;"><img src="https://img-bss.csdn.net/202002241015433522.png" alt="" /><br /> </span></strong> </p> <p> <strong><span style="font-size:18px;"><strong><span style="font-size:18px;">2)学生模块</span></strong></span></strong> </p> <p> <strong><span style="font-size:18px;"><img src="https://img-bss.csdn.net/202002241015575966.png" alt="" /></span></strong> </p> <p> <strong><span style="font-size:18px;"><strong><span style="font-size:18px;">3)教师模块</span></strong></span></strong> </p> <p> <strong><span style="font-size:18px;"><img src="https://img-bss.csdn.net/202002241016127898.png" alt="" /></span></strong> </p> <p> <strong><span style="font-size:18px;"><strong><span style="font-size:18px;">4)系统管理员</span></strong></span></strong> </p> <p> <strong><span style="font-size:18px;"><img src="https://img-bss.csdn.net/202002241016281177.png" alt="" /></span></strong> </p> <p> <strong><span style="font-size:18px;"><img src="https://img-bss.csdn.net/202002241016369884.png" alt="" /></span></strong> </p> <p> <strong><span style="font-size:18px;"><br /> </span></strong> </p> <p> <strong><span style="font-size:18px;"><strong><span style="font-size:18px;">更多Java毕设项目请关注我的毕设系列课程 <a href="https://edu.csdn.net/lecturer/2104">https://edu.csdn.net/lecturer/2104</a></span></strong></span></strong> </p> <p> <strong><span style="font-size:18px;"><br /> </span></strong> </p>
©️2020 CSDN 皮肤主题: 酷酷鲨 设计师:CSDN官方博客 返回首页