Adding/Removing RegionServers dynamically from a Fully-Distributed HBase cluster

Adding a RegionServer dynamically to a running HBase Cluster

Being able to add RegionServers dynamically without bringing down the entire cluster is quite a significant requirement and at the same time damn easy.

Step 1: Edit $HBASE_HOME/conf/regionservers on the Master node and add the new address.

Step 2: Setup the new node with needed software, permissions.

Step 3: On that node run

$HBASE_HOME/bin/hbase-daemon.sh start regionserver

Confirm it worked by looking at the Master's web UI or in that region server's log.

That’s it. You’re done.

Removing/Decommissioning a RegionServer dynamically from a running HBase Cluster

Just as significant as being able to add a RegionServer to an HBase Cluster dynamically is essential, just that important it is to be able to remove one.  

And you can be happy that it is just a matter of two steps.

Step 1: On the node to be removed run

$HBASE_HOME/bin/hbase-daemon.sh stop regionserver

Step 2: Edit $HBASE_HOME/conf/regionservers on the Master node and remove the address of the particular node being removed.

You can check the UI to ensure the node has been removed from the cluster.

Be careful about your data

Ensure to disable the Load Balancer before Decommissioning/Removing a node. This is because there might be contention between the Master recovering from the loss of the recently removed RegionServer and the running Load Balancer. Hence, it important to stop the load balancer(if running)before removing the regionserver from the cluster.

Also, in Apache HBase 0.90.2, there is an added facility for having a node gradually shed its load and then shutdown itself down by using the graceful_stop.sh script as follows:

$HBASE_HOME/bin/graceful_stop.sh HOSTNAME

where HOSTNAME is the host carrying the RegionServer you would remove/decommission.