hive通过nginx实现HA高可用

1. nginx的安装

1.1 下载上传
http://nginx.org/en/download.html

1.2 linux上安装c++编译环境

yum install -y zlibyum install gcc-c++ openssl openssl-devel zlib-devel pcre pcre-devel zlib zlib-devel

1.3 解压并编译
注意:一定要在这里使用–with-stream,因为我们将使用流模块[en]Note: be sure to bring– with-stream here, because we are going to use the stream module

./configure --prefix=/opt/local/nginx/ --with-stream
make && make install

1.4 启动
编译后的nginx在指定目录/opt/local/nginx中生成[en]The compiled nginx is generated in the specified directory / opt/local/nginx
启动命令:sbin/nginx[en]Start command: sbin/nginx

在启动非根帐号时,需要注意的是,普通用户没有权限启动1024以下的端口。在超级用户帐号下使用以下命令可以解决此问题:[en]When starting a non-root account, you should note that ordinary users do not have the permission to launch ports below 1024. Use the following command under the root account to solve the problem:

setcap cap_net_bind_service=+eip /opt/local/nginx/sbin/nginx

2. nginx的配置

打开conf/nginx.conf,添加stream配置,和http同层,并且注释掉http配置。
这是通过访问NGINX服务器的端口10010,然后连接到其他三个hiveserver2服务来完成的,以确保任何1-2节点的hiveServer服务在它们关闭时仍然可以使用。[en]This is done by accessing port 10010 of the nginx server and then connecting to the other three hiveserver2 services to ensure that the hiveServer services of any 1-2 nodes can still be used if they are down.

stream {

    log_format proxy '$remote_addr [$time_local] '
                 '$protocol $status $bytes_sent $bytes_received '
                 '$session_time "$upstream_addr" '
                 '"$upstream_bytes_sent" "$upstream_bytes_received" "$upstream_connect_time"';

    access_log  logs/access.log proxy ;
    open_log_file_cache off;

    upstream beeline {
         server hadoop200:10000;
         server hadoop201:10000;
         server hadoop202:10000;
    }

     server {
        listen 10010;
        proxy_pass beeline;
        proxy_connect_timeout 10s;
        proxy_timeout 24h;
     }

}

20220728优化:
在使用dbveaver进行连接时,您经常会发现,如果一段时间没有使用它,则需要手动重新连接。时间很短,可能只有几分钟。[en]When using dbveaver to connect, you often find that you need to reconnect manually if you don’t use it for a period of time. The time is very short, maybe in a few minutes.
因此,添加了以下两个要观察的参数:[en]So the following two parameters are added to be observed:
proxy_connect_timeout 10s; # 连接超时
proxy_timeout 24h; # 代理超时

3.连接测试

Hiveserver2服务分别由56和57的3个节点启动,然后通过dbeaver测试。[en]The hiveserver2 service is started by 3 nodes of 56 and 57 respectively, and then passed the dbeaver test.

hive通过nginx实现HA高可用

Original: https://www.cnblogs.com/30go/p/16257818.html
Author: 硅谷工具人
Title: hive通过nginx实现HA高可用

原创文章受到原创版权保护。转载请注明出处:https://www.johngo689.com/6000/

转载文章受原作者版权保护。转载请注明原作者出处!

(0)

大家都在看

发表回复

登录后才能评论
免费咨询
免费咨询
扫码关注
扫码关注
联系站长

站长Johngo!

大数据和算法重度研究者!

持续产出大数据、算法、LeetCode干货,以及业界好资源!

2022012703491714

微信来撩,免费咨询:xiaozhu_tec

分享本页
返回顶部