Go语言之高级篇beego框架之日志收集系统

2023年5月29日上午8:28 • Go语言 • 阅读 50

一、日志收集系统架构设计

图1

图2

二、开发环境

1、安装jdk

jdk-8u51-windows-x64.exe

安装目录：C:\Program Files\jdk8

2、安装zookeeper

zookeeper-3.3.6.tar.gz

安装目录： C:\Program Files\zookeeper-3.3.6

配置：

C:\Program Files\zookeeper-3.3.6\conf

把配置文件：zoo_sample.cfg，改名成：zoo.cfg

修改部署路径为你安装的路径。

配置环境变量：

csharp;gutter:false; C:\Program Files\zookeeper-3.3.6\bin</p> <pre><code> #启动服务：进行目录，直接启动服务：C:\Program Files\zookeeper-3.3.6\bin 直接运行服务名称: zkServer.cmd;gutter:false;
h had sessionid 0x16908b8a4920000
2019-02-20 10:52:39,153 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn$Factory@251] – Accepted socket connection from /127.0.0.1:53568
2019-02-20 10:52:39,154 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@770] – Client attempting to renew session 0x16908b8a4920000 at /127.0.0
.1:53568
2019-02-20 10:52:39,155 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1580] – Established session 0x16908b8a4920000 with negotiated timeout 6
000 for client /127.0.0.1:53568
2019-02-20 10:52:39,158 – WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe
eperServer@593] – Dropping packet at server of type 14
2019-02-20 10:52:39,161 – WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@639] – Exception causing close of session 0x16908b8a4920000 due to java
.io.IOException: 远程主机强迫关闭了一个现有的连接。
2019-02-20 10:52:39,163 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1435] – Closed socket connection for client /127.0.0.1:53568 which had
sessionid 0x16908b8a4920000
2019-02-20 10:52:41,214 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn$Factory@251] – Accepted socket connection from /127.0.0.1:53570
2019-02-20 10:52:41,216 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@770] – Client attempting to renew session 0x16908b8a4920000 at /127.0.0
.1:53570
2019-02-20 10:52:41,217 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1580] – Established session 0x16908b8a4920000 with negotiated timeout 6
000 for client /127.0.0.1:53570
2019-02-20 10:52:41,220 – WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe
eperServer@593] – Dropping packet at server of type 14
2019-02-20 10:52:41,224 – WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@639] – Exception causing close of session 0x16908b8a4920000 due to java
.io.IOException: 远程主机强迫关闭了一个现有的连接。
2019-02-20 10:52:41,226 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1435] – Closed socket connection for client /127.0.0.1:53570 which had
sessionid 0x16908b8a4920000
2019-02-20 10:52:42,438 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn$Factory@251] – Accepted socket connection from /127.0.0.1:53571
2019-02-20 10:52:42,439 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@770] – Client attempting to renew session 0x16908b8a4920000 at /127.0.0
.1:53571
2019-02-20 10:52:42,441 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1580] – Established session 0x16908b8a4920000 with negotiated timeout 6
000 for client /127.0.0.1:53571
2019-02-20 10:52:42,444 – WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe
eperServer@593] – Dropping packet at server of type 14
2019-02-20 10:52:42,448 – WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@639] – Exception causing close of session 0x16908b8a4920000 due to java
.io.IOException: 远程主机强迫关闭了一个现有的连接。
2019-02-20 10:52:42,449 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1435] – Closed socket connection for client /127.0.0.1:53571 which had
sessionid 0x16908b8a4920000
2019-02-20 10:52:44,219 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn$Factory@251] – Accepted socket connection from /127.0.0.1:53572
2019-02-20 10:52:44,220 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@770] – Client attempting to renew session 0x16908b8a4920000 at /127.0.0
.1:53572
2019-02-20 10:52:44,221 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1580] – Established session 0x16908b8a4920000 with negotiated timeout 6
000 for client /127.0.0.1:53572
2019-02-20 10:52:44,223 – WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe
eperServer@593] – Dropping packet at server of type 14
2019-02-20 10:52:44,227 – WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@639] – Exception causing close of session 0x16908b8a4920000 due to java
.io.IOException: 远程主机强迫关闭了一个现有的连接。
2019-02-20 10:52:44,229 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1435] – Closed socket connection for client /127.0.0.1:53572 which had
sessionid 0x16908b8a4920000
2019-02-20 10:52:45,462 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn$Factory@251] – Accepted socket connection from /0:0:0:0:0:0:0:1:53573
2019-02-20 10:52:45,464 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@770] – Client attempting to renew session 0x16908b8a4920000 at /0:0:0:0
:0:0:0:1:53573
2019-02-20 10:52:45,465 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1580] – Established session 0x16908b8a4920000 with negotiated timeout 6
000 for client /0:0:0:0:0:0:0:1:53573
2019-02-20 10:52:45,468 – WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe
eperServer@593] – Dropping packet at server of type 14
2019-02-20 10:52:45,473 – WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@639] – Exception causing close of session 0x16908b8a4920000 due to java
.io.IOException: 远程主机强迫关闭了一个现有的连接。
2019-02-20 10:52:45,475 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1435] – Closed socket connection for client /0:0:0:0:0:0:0:1:53573 whic
h had sessionid 0x16908b8a4920000
2019-02-20 10:52:46,769 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn$Factory@251] – Accepted socket connection from /127.0.0.1:53574
2019-02-20 10:52:46,770 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@770] – Client attempting to renew session 0x16908b8a4920000 at /127.0.0
.1:53574
2019-02-20 10:52:46,772 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1580] – Established session 0x16908b8a4920000 with negotiated timeout 6
000 for client /127.0.0.1:53574
2019-02-20 10:52:46,775 – WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe
eperServer@593] – Dropping packet at server of type 14
2019-02-20 10:52:46,779 – WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@639] – Exception causing close of session 0x16908b8a4920000 due to java
.io.IOException: 远程主机强迫关闭了一个现有的连接。
2019-02-20 10:52:46,781 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1435] – Closed socket connection for client /127.0.0.1:53574 which had
sessionid 0x16908b8a4920000
2019-02-20 10:52:48,284 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn$Factory@251] – Accepted socket connection from /127.0.0.1:53575
2019-02-20 10:52:48,285 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@770] – Client attempting to renew session 0x16908b8a4920000 at /127.0.0
.1:53575
2019-02-20 10:52:48,286 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1580] – Established session 0x16908b8a4920000 with negotiated timeout 6
000 for client /127.0.0.1:53575
2019-02-20 10:52:48,289 – WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe
eperServer@593] – Dropping packet at server of type 14
2019-02-20 10:52:48,293 – WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@639] – Exception causing close of session 0x16908b8a4920000 due to java
.io.IOException: 远程主机强迫关闭了一个现有的连接。
2019-02-20 10:52:48,295 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1435] – Closed socket connection for client /127.0.0.1:53575 which had
sessionid 0x16908b8a4920000
2019-02-20 10:52:50,069 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn$Factory@251] – Accepted socket connection from /127.0.0.1:53576
2019-02-20 10:52:50,071 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@770] – Client attempting to renew session 0x16908b8a4920000 at /127.0.0
.1:53576
2019-02-20 10:52:50,072 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1580] – Established session 0x16908b8a4920000 with negotiated timeout 6
000 for client /127.0.0.1:53576
2019-02-20 10:52:50,075 – WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe
eperServer@593] – Dropping packet at server of type 14
2019-02-20 10:52:50,080 – WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@639] – Exception causing close of session 0x16908b8a4920000 due to java
.io.IOException: 远程主机强迫关闭了一个现有的连接。
2019-02-20 10:52:50,081 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1435] – Closed socket connection for client /127.0.0.1:53576 which had
sessionid 0x16908b8a4920000
2019-02-20 10:52:51,547 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn$Factory@251] – Accepted socket connection from /127.0.0.1:53577
2019-02-20 10:52:51,548 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@770] – Client attempting to renew session 0x16908b8a4920000 at /127.0.0
.1:53577
2019-02-20 10:52:51,550 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1580] – Established session 0x16908b8a4920000 with negotiated timeout 6
000 for client /127.0.0.1:53577
2019-02-20 10:52:51,552 – WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe
eperServer@593] – Dropping packet at server of type 14
2019-02-20 10:52:51,557 – WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@639] – Exception causing close of session 0x16908b8a4920000 due to java
.io.IOException: 远程主机强迫关闭了一个现有的连接。
2019-02-20 10:52:51,559 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1435] – Closed socket connection for client /127.0.0.1:53577 which had
sessionid 0x16908b8a4920000
2019-02-20 10:52:53,315 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn$Factory@251] – Accepted socket connection from /0:0:0:0:0:0:0:1:53579
2019-02-20 10:52:53,316 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@770] – Client attempting to renew session 0x16908b8a4920000 at /0:0:0:0
:0:0:0:1:53579
2019-02-20 10:52:53,318 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1580] – Established session 0x16908b8a4920000 with negotiated timeout 6
000 for client /0:0:0:0:0:0:0:1:53579
2019-02-20 10:52:53,321 – WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe
eperServer@593] – Dropping packet at server of type 14
2019-02-20 10:52:53,325 – WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@639] – Exception causing close of session 0x16908b8a4920000 due to java
.io.IOException: 远程主机强迫关闭了一个现有的连接。
2019-02-20 10:52:53,326 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1435] – Closed socket connection for client /0:0:0:0:0:0:0:1:53579 whic
h had sessionid 0x16908b8a4920000
2019-02-20 10:52:54,537 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn$Factory@251] – Accepted socket connection from /0:0:0:0:0:0:0:1:53580
2019-02-20 10:52:54,539 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@770] – Client attempting to renew session 0x16908b8a4920000 at /0:0:0:0
:0:0:0:1:53580
2019-02-20 10:52:54,540 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1580] – Established session 0x16908b8a4920000 with negotiated timeout 6
000 for client /0:0:0:0:0:0:0:1:53580
2019-02-20 10:52:54,545 – WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe
eperServer@593] – Dropping packet at server of type 14
2019-02-20 10:52:54,551 – WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@634] – EndOfStreamException: Unable to read additional data from client
sessionid 0x16908b8a4920000, likely client has closed socket
2019-02-20 10:52:54,552 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1435] – Closed socket connection for client /0:0:0:0:0:0:0:1:53580 whic
h had sessionid 0x16908b8a4920000
2019-02-20 10:52:56,153 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn$Factory@251] – Accepted socket connection from /127.0.0.1:53581
2019-02-20 10:52:56,154 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@770] – Client attempting to renew session 0x16908b8a4920000 at /127.0.0
.1:53581
2019-02-20 10:52:56,156 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1580] – Established session 0x16908b8a4920000 with negotiated timeout 6
000 for client /127.0.0.1:53581
2019-02-20 10:52:56,159 – WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe
eperServer@593] – Dropping packet at server of type 14
2019-02-20 10:52:56,163 – WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@634] – EndOfStreamException: Unable to read additional data from client
sessionid 0x16908b8a4920000, likely client has closed socket
2019-02-20 10:52:56,164 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1435] – Closed socket connection for client /127.0.0.1:53581 which had
sessionid 0x16908b8a4920000
2019-02-20 10:52:58,023 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn$Factory@251] – Accepted socket connection from /0:0:0:0:0:0:0:1:53582
2019-02-20 10:52:58,024 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@770] – Client attempting to renew session 0x16908b8a4920000 at /0:0:0:0
:0:0:0:1:53582
2019-02-20 10:52:58,026 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1580] – Established session 0x16908b8a4920000 with negotiated timeout 6
000 for client /0:0:0:0:0:0:0:1:53582
2019-02-20 10:52:58,028 – WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe
eperServer@593] – Dropping packet at server of type 14
2019-02-20 10:52:58,032 – WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@634] – EndOfStreamException: Unable to read additional data from client
sessionid 0x16908b8a4920000, likely client has closed socket
2019-02-20 10:52:58,034 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1435] – Closed socket connection for client /0:0:0:0:0:0:0:1:53582 whic
h had sessionid 0x16908b8a4920000
2019-02-20 10:52:59,424 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn$Factory@251] – Accepted socket connection from /127.0.0.1:53584
2019-02-20 10:52:59,425 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@770] – Client attempting to renew session 0x16908b8a4920000 at /127.0.0
.1:53584
2019-02-20 10:52:59,425 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1580] – Established session 0x16908b8a4920000 with negotiated timeout 6
000 for client /127.0.0.1:53584
2019-02-20 10:52:59,428 – WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe
eperServer@593] – Dropping packet at server of type 14
2019-02-20 10:52:59,431 – WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@634] – EndOfStreamException: Unable to read additional data from client
sessionid 0x16908b8a4920000, likely client has closed socket
2019-02-20 10:52:59,431 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1435] – Closed socket connection for client /127.0.0.1:53584 which had
sessionid 0x16908b8a4920000
2019-02-20 10:53:01,246 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn$Factory@251] – Accepted socket connection from /127.0.0.1:53585
2019-02-20 10:53:01,247 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@770] – Client attempting to renew session 0x16908b8a4920000 at /127.0.0
.1:53585
2019-02-20 10:53:01,249 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1580] – Established session 0x16908b8a4920000 with negotiated timeout 6
000 for client /127.0.0.1:53585
2019-02-20 10:53:01,252 – WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe
eperServer@593] – Dropping packet at server of type 14
2019-02-20 10:53:01,259 – WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@634] – EndOfStreamException: Unable to read additional data from client
sessionid 0x16908b8a4920000, likely client has closed socket
2019-02-20 10:53:01,260 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1435] – Closed socket connection for client /127.0.0.1:53585 which had
sessionid 0x16908b8a4920000
2019-02-20 10:53:02,677 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn$Factory@251] – Accepted socket connection from /0:0:0:0:0:0:0:1:53587
2019-02-20 10:53:02,678 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@770] – Client attempting to renew session 0x16908b8a4920000 at /0:0:0:0
:0:0:0:1:53587
2019-02-20 10:53:02,679 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1580] – Established session 0x16908b8a4920000 with negotiated timeout 6
000 for client /0:0:0:0:0:0:0:1:53587
2019-02-20 10:53:02,682 – WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe
eperServer@593] – Dropping packet at server of type 14
2019-02-20 10:53:02,685 – WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@634] – EndOfStreamException: Unable to read additional data from client
sessionid 0x16908b8a4920000, likely client has closed socket
2019-02-20 10:53:02,686 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1435] – Closed socket connection for client /0:0:0:0:0:0:0:1:53587 whic
h had sessionid 0x16908b8a4920000
2019-02-20 10:53:04,384 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn$Factory@251] – Accepted socket connection from /127.0.0.1:53588
2019-02-20 10:53:04,385 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@770] – Client attempting to renew session 0x16908b8a4920000 at /127.0.0
.1:53588
2019-02-20 10:53:04,387 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1580] – Established session 0x16908b8a4920000 with negotiated timeout 6
000 for client /127.0.0.1:53588
2019-02-20 10:53:04,390 – WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe
eperServer@593] – Dropping packet at server of type 14
2019-02-20 10:53:04,399 – WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@634] – EndOfStreamException: Unable to read additional data from client
sessionid 0x16908b8a4920000, likely client has closed socket
2019-02-20 10:53:04,400 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1435] – Closed socket connection for client /127.0.0.1:53588 which had
sessionid 0x16908b8a4920000
2019-02-20 10:53:06,078 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn$Factory@251] – Accepted socket connection from /0:0:0:0:0:0:0:1:53589
2019-02-20 10:53:06,079 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@770] – Client attempting to renew session 0x16908b8a4920000 at /0:0:0:0
:0:0:0:1:53589
2019-02-20 10:53:06,081 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1580] – Established session 0x16908b8a4920000 with negotiated timeout 6
000 for client /0:0:0:0:0:0:0:1:53589
2019-02-20 10:53:06,084 – WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe
eperServer@593] – Dropping packet at server of type 14
2019-02-20 10:53:06,089 – WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@634] – EndOfStreamException: Unable to read additional data from client
sessionid 0x16908b8a4920000, likely client has closed socket
2019-02-20 10:53:06,090 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1435] – Closed socket connection for client /0:0:0:0:0:0:0:1:53589 whic
h had sessionid 0x16908b8a4920000
2019-02-20 10:53:07,834 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn$Factory@251] – Accepted socket connection from /127.0.0.1:53590
2019-02-20 10:53:07,835 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@770] – Client attempting to renew session 0x16908b8a4920000 at /127.0.0
.1:53590
2019-02-20 10:53:07,836 – INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1580] – Established session 0x16908b8a4920000 with negotiated timeout 6
000 for client /127.0.0.1:53590
2019-02-20 10:53:07,838 – WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe
eperServer@593] – Dropping packet at server of type 14

3、kafka

kafka_2.12-2.1.1.tgz

安装目录：D:\run_file\kafka_2.12-2.1.1 （不能这在C:\Program Files\ 下面启动服务会报错。）

修改配置文件

D:\run_file\kafka_2.12-2.1.1\config

修改为本机IP地址：

csharp;gutter:false; advertised.host.name=172.28.5.25</p> <pre><code> #配置日志地址：;gutter:false;
########### Log Basics #############

A comma separated list of directories under which to store log files
#log.dirs=/tmp/kafka-logs
log.dirs=D:/run_file/kafka_2.12-2.1.1/kafka-logs #增加这行代码

启动服务：

进入安装目录中：

csharp;gutter:false; 在命令行下： cd D:\run_file\kafka_2.12-2.1.1</p> <p>./bin/windows/kafka-server-start.bat ./config/server.preperties</p> <pre><code>;gutter:false;
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSock
NIO.java:366)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1145
[2019-02-20 10:53:06,087] INFO [ZooKeeperClient] Waiting until connected. (kaf
.zookeeper.ZooKeeperClient)
[2019-02-20 10:53:06,089] INFO [ZooKeeperClient] Connected. (kafka.zookeeper.Z
KeeperClient)
[2019-02-20 10:53:06,189] INFO [ZooKeeperClient] Waiting until connected. (kaf
.zookeeper.ZooKeeperClient)
[2019-02-20 10:53:06,190] INFO [ZooKeeperClient] Connected. (kafka.zookeeper.Z
KeeperClient)
[2019-02-20 10:53:06,190] INFO [ZooKeeperClient] Waiting until connected. (kaf
.zookeeper.ZooKeeperClient)
[2019-02-20 10:53:06,191] INFO [ZooKeeperClient] Connected. (kafka.zookeeper.Z
KeeperClient)
[2019-02-20 10:53:07,832] INFO Opening socket connection to server localhost/1
.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) (org.

ache.zookeeper.ClientCnxn)
[2019-02-20 10:53:07,834] INFO Socket connection established to localhost/127.

0.1:2181, initiating session (org.apache.zookeeper.ClientCnxn)
[2019-02-20 10:53:07,836] WARN Connected to an old server; r-o mode will be un
ailable (org.apache.zookeeper.ClientCnxnSocket)
[2019-02-20 10:53:07,837] INFO Session establishment complete on server localh
t/127.0.0.1:2181, sessionid = 0x16908b8a4920000, negotiated timeout = 6000 (or
apache.zookeeper.ClientCnxn)
[2019-02-20 10:53:07,844] INFO [SocketServer brokerId=0] Started processors fo
1 acceptors (kafka.network.SocketServer)
[2019-02-20 10:53:07,853] INFO Kafka version : 2.1.1 (org.apache.kafka.common.

ils.AppInfoParser)
[2019-02-20 10:53:07,853] INFO Kafka commitId : 21234bee31165527 (org.apache.k
ka.common.utils.AppInfoParser)
[2019-02-20 10:53:07,856] INFO [KafkaServer id=0] started (kafka.server.KafkaS
ver)
[2019-02-20 10:53:12,993] INFO [GroupMetadataManager brokerId=0] Removed 0 exp
ed offsets in 0 milliseconds. (kafka.coordinator.group.GroupMetadataManager)

二、tail

在cmd中安装：

csharp;gutter:false; go get github.com/hpcloud/tail</p> <pre><code> #代码： ![Go语言之高级篇beego框架之日志收集系统](https://johngo-pic.oss-cn-beijing.aliyuncs.com/articles/20230526/1053682-20190220171741788-865805339.png) 创建一个my.log的文件路径：;gutter:false;
D:\GoFiles\my.log

示例：

vi my.log

go;gutter:false;</p> <h6>####################### Producer Basics</h6> <p>list of brokers used for bootstrapping knowledge about the rest of the cluster format: host1:port1,host2:port2 ...</p> <p>bootstrap.servers=localhost:9092</p> <p>specify the compression codec for all data generated: none, gzip, snappy, lz4, zstd compression.type=none</p> <p>name of the partitioner class for partitioning events; default partition spreads data randomly</p> <h1>partitioner.class=</h1> <p>the maximum amount of time the client will wait for the response of a request</p> <h1>request.timeout.ms=</h1> <p>how long <code>KafkaProducer.send</code> and <code>KafkaProducer.partitionsFor</code> will block for</p> <h1>max.block.ms=</h1> <p>the producer will wait for up to the given delay to allow other records to be sent so that the sends can be batched together</p> <h1>linger.ms=</h1> <p>the maximum size of a request in bytes</p> <h1>max.request.size=</h1> <p>the default batch size in bytes when batching multiple records sent to a partition</p> <h1>batch.size=</h1> <p>the total bytes of memory the producer can use to buffer records waiting to be sent to the server</p> <h1>buffer.memory=</h1> <pre><code> 示例：把文件中内容读出来 vi main.go;gutter:false;
package main

import (
"fmt"
"time"

"github.com/hpcloud/tail"
)

func main() {
filename := "D:\\GoFiles\\my.log"
tails, err := tail.TailFile(filename, tail.Config{
ReOpen: true,
Follow: true,
//Location: &tail.SeekInfo{Offset: 0, Whence: 2},
MustExist: false,
Poll: true,
})

if err != nil {
fmt.Println("tail file err:", err)
return
}

var msg *tail.Line
var ok bool
for true {
msg, ok =

执行结果：

go;gutter:false;</p> <h1>编译#在终端中输入：</h1> <p>D:\GoFiles\src\log_collect\tail>go build D:\GoFiles\src\log_collect\tail</p> <h1>运行</h1> <p>D:\GoFiles\src\log_collect\tail>tail.exe</p> <h1>返回结果：</h1> <h6>####################### Producer Basics</h6> <p>msg: msg: # list of brokers used for bootstrapping knowledge about the rest of the cl uster msg: # format: host1:port1,host2:port2 ...</p> <p>msg: bootstrap.servers=localhost:9092 msg: msg: # specify the compression codec for all data generated: none, gzip, snappy, lz4, zstd msg: compression.type=none msg: msg: # name of the partitioner class for partitioning events; default partition spreads data randomly msg: #partitioner.class= msg: msg: # the maximum amount of time the client will wait for the response of a req uest msg: #request.timeout.ms= msg: msg: # how long <code>KafkaProducer.send</code> and <code>KafkaProducer.partitionsFor</code> will bloc k for msg: #max.block.ms= msg: msg: # the producer will wait for up to the given delay to allow other records t o be sent so that the sends can be batched together msg: #linger.ms= msg: msg: # the maximum size of a request in bytes msg: #max.request.size= msg: msg: # the default batch size in bytes when batching multiple records sent to a partition msg: #batch.size= msg: msg: # the total bytes of memory the producer can use to buffer records waiting to be sent to the server msg: #buffer.memory=</p> <pre><code> # 二、els安装 1、安装Elasticsearch elasticsearch-5.5.0.msi 安装目录 C:\Program Files\Elastic\Elasticsearch #启动目录 C:\Program Files\Elastic\Elasticsearch\bin 运行： elasticsearch.exe ![Go语言之高级篇beego框架之日志收集系统](https://johngo-pic.oss-cn-beijing.aliyuncs.com/articles/20230526/1053682-20190222153441423-1888077137.png) #访问： http://127.0.0.1:9200/ ![Go语言之高级篇beego框架之日志收集系统](https://johngo-pic.oss-cn-beijing.aliyuncs.com/articles/20230526/1053682-20190222153525790-1793542590.png) 2、kafka（windows版本安装）软件部署： kibana-5.5.1-windows-x86.zip 解压到下面目录中： C:\Program Files (x86)\kibana #修改配置文件 elasticsearch.url: "http://127.0.0.1:9200" #指向es的启动页面 #启动kibana C:\Program Files (x86)\kibana\bin 运行： kibana.bat ![Go语言之高级篇beego框架之日志收集系统](https://johngo-pic.oss-cn-beijing.aliyuncs.com/articles/20230526/1053682-20190222152853059-749728098.png) 示例：运行程序的前提条件：先安装elastic.v2 安装包：;gutter:false;
go get gopkg.in/olivere/elastic.v2

使用方法：

go;gutter:false; import "gopkg.in/olivere/elastic.v2"</p> <pre><code> </code></pre> <p>目录：</p> <pre><code> ![Go语言之高级篇beego框架之日志收集系统](https://johngo-pic.oss-cn-beijing.aliyuncs.com/articles/20230526/1053682-20190222151957033-2093230296.png) </code></pre> <p>代码：vi main.go</p> <pre><code>;gutter:false;
package main

import (
"fmt"

elastic "gopkg.in/olivere/elastic.v2"
)

type Tweet struct {
User string
Message string
}

func main() {
client, err := elastic.NewClient(elastic.SetSniff(false), elastic.SetURL("http://127.0.0.1:9200/"))
if err != nil {
fmt.Println("connect es error", err)
return
}

fmt.Println("conn es succ")
for i := 0; i < 10000; i++ {
tweet := Tweet{User: "olivere", Message: "Take Five"}
_, err = client.Index().
Index("twitter").
Type("tweet").
Id(fmt.Sprintf("%d", i)).
BodyJson(tweet).
Do()
if err != nil {
// Handle error
panic(err)
return
}
}

fmt.Println("insert success")
}

打开页面：

http://localhost:5601

输入： twitter —>再点Discover就可以看到下面页面。

go;gutter:false;undefined

3、

Original: https://www.cnblogs.com/nulige/p/10405320.html
Author: 努力哥
Title: Go语言之高级篇beego框架之日志收集系统

原创文章受到原创版权保护。转载请注明出处：https://www.johngo689.com/535794/

转载文章受原作者版权保护。转载请注明原作者出处！

Go语言

【自取】最近整理的，有需要可以领取学习：

Linux核心资料大放送~

全栈面试题汇总（持续更新&可下载）

一个提高学习100%效率的工具！

【超详细】深度学习面试题目！

LeetCode Python刷题答案下载！

LeetCode Java版刷题答案下载！

LeetCode C++ 版本，抓紧保存！

LeetCode GO语言刷题答案下载！

gofs使用教程-基于golang的开源跨平台文件同步工具

gofs是基于golang开发的一款开箱即用的跨平台文件同步工具，支持在本地磁盘之间同步、从远程服务器同步变更到本地、将本地文件变更推送到远程服务器三种模式。开源地址如下：Gith…

Go语言 2023年5月25日
0051
Go Error 嵌套到底是怎么实现的？

原文链接： Go Error 嵌套到底是怎么实现的？ Go Error 的设计哲学是「Errors Are Values」。这句话应该怎么理解呢？翻译起来挺难的。不过从源码的角…

Go语言 2023年5月25日
0061
Go语言学习笔记1

1.Go语言环境搭建及基础知识 Go语言官方网站（http://golang.org）代码包文档网站（http://godoc.org）Go语言中文网（http://studygo…

Go语言 2023年5月29日
0057
Golang笔记

本文主要为go的学习过程笔记。一、基本介绍 1、开发环境安装-windows安装打开Golang官网，选择对应版本，进行安装。 2、环境变量配置 1）步骤（1）首先在环境变量…

Go语言 2023年5月25日
0047
Go语言基础之并发

并发是编程里面一个非常重要的概念，Go语言在语言层面天生支持并发，这也是Go语言流行的一个很重要的原因。 Go语言中的并发编程并发与并行并发：同一时间段内执行多个任务（你在用微…

Go语言 2023年5月29日
0046
Go语言中单个字符char rune

Go 语言的字符使用UTF-8 编码 *英文字母 1个 字节, 汉子 3个 字节 golang的字符称为rune，等价于C中的char，…

Go语言 2023年5月29日
0046
EbitenCookBook中文教程第一课:安装 Ebiten

本文实时更新原址:https://ebitencookbook.vercel.app/docs/CookBook_Start/class1 第一课安装 Ebiten 欢迎大家来到…

Go语言 2023年5月25日
0051
关于Golang的学习路线

基础安装golang环境Golang基础，流程控制，函数，方法，面向对象网络编程（自己做一个简单的tcp的聊天室，websocket，http，命令行工具）并发（可以看一下并发爬…

Go语言 2023年5月25日
0054
编译kubeadm使生成证书有效期为100年

问题编译检查结果问题当我使用kubeadm部署成功k8s集群时在想默认生成的证书有效期是多久，如下所示 /etc/kubernetes/pki/apiserver.crt …

Go语言 2023年5月25日
0050
go-micro集成RabbitMQ实战和原理

在go-micro中异步消息的收发是通过Broker这个组件来完成的，底层实现有RabbitMQ、Kafka、Redis等等很多种方式，这篇文章主要介绍go-micro使用Rabb…

Go语言 2023年5月25日
0058
B树-查找

B树系列文章 1. B树-介绍 2. B树-查找 3. B树-插入 4. B树-删除查找假设有一棵3阶B树，如下图所示。下面说明在该B树中查找 52的过程首先，从根结点出发…

Go语言 2023年5月25日
0058
Go语言之数组与切片基础

数组是同一类型元素的集合，可以放多个值，但是类型一致，内存中连续存储 Go 语言中不允许混合不同类型的元素，而且数组的大小，在定义阶段就确定了，不能更改 1、数组的定义 // 定义…

Go语言 2023年5月25日
0063
Golang项目的配置管理——Viper简易入门配置

Golang项目的配置管理——Viper简易入门配置 What is Viper？ From：https://github.com/spf13/viperViper is a co…

Go语言 2023年5月25日
0047
sync：二. 延迟初始化(once)

sync.Once 是 Go 标准库提供的使函数只执行一次的实现。作用与 init 函数类似，但有区别。在某些情况下预先初始化一个变量会增加函数的启动延迟，如果实际执行时可能用不上…

Go语言 2023年5月25日
0066
Go语言之网络编程

一、网络编程基础网络基础之TCP/IP协议族网络编程之socket 二、TCP Socket编程（一）流程首先应该了解服务端和客户端的处理流程： 1、服务端处理流程监听端…

Go语言 2023年5月29日
0057
go入门项目：(1) 基于命令行的图书的增删查改

Go 语言入门练手项目系列 01 基于命令行的图书的增删查改 02 文件管理持续更新中… 本文来自博客园，作者：Arway，转载请注明原文链接：https://www…

Go语言 2023年5月25日
0053

2024 年 5 月
一	二	三	四	五	六	日
		1	2	3	4	5
6	7	8	9	10	11	12
13	14	15	16	17	18	19
20	21	22	23	24	25	26
27	28	29	30	31

Go语言之高级篇beego框架之日志收集系统

一、日志收集系统架构设计

二、开发环境

修改配置文件

打开页面：

大家都在看