Yarn状态的查询,除了可以在hadoop103:8088页面查看以外,还可以通过命令操作。常见的命令操作如下所示。
启动集群
[hadoop@hadoop102 bin]$ myhadoop.sh start
运行wordcount案例
[hadoop@hadoop102 hadoop-3.1.3]$ hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-3.1.3.jar wordcount /input /output
1、列出所有Application:
[hadoop@hadoop102 hadoop-3.1.3]$ yarn application -list
2022-06-06 16:51:19,896 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032
Total number of applications (application-types: [], states: [SUBMITTED, ACCEPTED, RUNNING] and tags: []):0
Application-Id Application-Name Application-Type User Queue State Final-State Progress Tracking-URL
2、根据Application状态过滤: yarn application -list -appStates [ALL、NEW、NEW_SAVING、SUBMITTED、ACCEPTED、RUNNING、FINISHED、FAILED、KILLED]
[hadoop@hadoop102 hadoop-3.1.3]$ yarn application -list -appStates FINISHED
2022-06-06 16:53:49,886 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032
Total number of applications (application-types: [], states: [FINISHED] and tags: []):2
Application-Id Application-Name Application-Type User Queue State Final-State Progress Tracking-URL
application_1654496324557_0001 word count MAPREDUCE hadoop default FINISHED FAILED 100% http://hadoop102:19888/jobhistory/job/job_1654496324557_0001
application_1654496324557_0002 word count MAPREDUCE hadoop default FINISHED FAILED 100% http://hadoop102:19888/jobhistory/job/job_1654496324557_0002
2、Kill掉Application
[hadoop@hadoop102 hadoop-3.1.3]$ yarn application -kill application_1654496324557_0001
2022-06-06 16:57:34,082 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032
Application application_1654496324557_0001 has already finished
1、查询Application日志: yarn logs -applicationId <applicationid></applicationid>
[hadoop@hadoop102 hadoop-3.1.3]$ yarn logs -applicationId application_1654496324557_0001
2、查询Container日志: yarn logs -applicationId <applicationid> -containerId <containerid></containerid></applicationid>
[hadoop@hadoop102 hadoop-3.1.3]$ yarn logs -applicationId application_1654496324557_0001 -containerId container_1654496324557_0001_01_000001
1、列出所有Application尝试的列表: yarn applicationattempt -list <applicationid></applicationid>
[hadoop@hadoop102 hadoop-3.1.3]$ yarn applicationattempt -list application_1654496324557_0001
2022-06-06 17:05:10,914 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032
Total number of application attempts :1
ApplicationAttempt-Id State AM-Container-Id Tracking-URL
appattempt_1654496324557_0001_000001 FINISHED container_1654496324557_0001_01_000001 http://hadoop103:8088/proxy/application_1654496324557_0001/
2、打印ApplicationAttempt状态: yarn applicationattempt -status <applicationattemptid></applicationattemptid>
[hadoop@hadoop102 hadoop-3.1.3]$ yarn applicationattempt -status appattempt_1654496324557_0001_000001
2022-06-06 17:08:56,570 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032
Application Attempt Report :
ApplicationAttempt-Id : appattempt_1654496324557_0001_000001
State : FINISHED
AMContainer : container_1654496324557_0001_01_000001
Tracking-URL : http://hadoop103:8088/proxy/application_1654496324557_0001/
RPC Port : 32771
AM Host : hadoop103
Diagnostics : Task failed task_1654496324557_0001_m_000000
Job failed as tasks failed. failedMaps:1 failedReduces:0 killedMaps:0 killedReduces: 0
1、列出所有Container: yarn container -list <applicationattemptid></applicationattemptid>
[hadoop@hadoop103 ~]$ yarn container -list appattempt_1654496324557_0004_000001
2022-06-06 17:14:40,313 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032
Total number of containers :2
Container-Id Start Time Finish Time State Host Node Http Address LOG-URL
container_1654496324557_0004_01_000001 星期一 六月 06 17:13:55 +0800 2022 N/A RUNNING hadoop102:46450 http://hadoop102:8042 http://hadoop102:8042/node/containerlogs/container_1654496324557_0004_01_000001/hadoop
container_1654496324557_0004_01_000004 星期一 六月 06 17:14:38 +0800 2022 N/A RUNNING hadoop103:44058 http://hadoop103:8042 http://hadoop103:8042/node/containerlogs/container_1654496324557_0004_01_000004/hadoop
2、打印Container状态: yarn container -status <containerid></containerid>
[hadoop@hadoop103 ~]$ yarn container -status container_1654496324557_0006_01_000001
2022-06-06 17:20:51,357 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032
Container Report :
Container-Id : container_1654496324557_0006_01_000001
Start-Time : 1654507199188
Finish-Time : 0
State : RUNNING
Execution-Type : GUARANTEED
LOG-URL : http://hadoop103:8042/node/containerlogs/container_1654496324557_0006_01_000001/hadoop
Host : hadoop103:44058
NodeHttpAddress : http://hadoop103:8042
Diagnostics : null
注:只有任务跑的途中才能看到container的状态
[hadoop@hadoop103 ~]$ yarn node -list -all
2022-06-06 17:25:20,336 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032
Total Nodes:3
Node-Id Node-State Node-Http-Address Number-of-Running-Containers
hadoop104:38945 RUNNING hadoop104:8042 0
hadoop103:44058 RUNNING hadoop103:8042 0
hadoop102:46450 RUNNING hadoop102:8042 0
加载队列配置: yarn rmadmin -refreshQueues
[hadoop@hadoop103 ~]$ yarn rmadmin -refreshQueues
2022-06-06 17:27:10,830 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8033
打印队列信息: yarn queue -status <queuename></queuename>
[hadoop@hadoop103 ~]$ yarn queue -status default
2022-06-06 17:28:54,282 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032
Queue Information :
Queue Name : default
State : RUNNING
Capacity : 100.0%
Current Capacity : .0%
Maximum Capacity : 100.0%
Default Node Label expression :
Accessible Node Labels : *
Preemption : disabled
Intra-queue Preemption : disabled
Original: https://www.cnblogs.com/sw-code/p/16387038.html
Author: sw-code
Title: 大数据之Hadoop集群中Yarn常用命令
原创文章受到原创版权保护。转载请注明出处:https://www.johngo689.com/585572/
转载文章受原作者版权保护。转载请注明原作者出处!