大数据之Hadoop集群中Yarn常用命令

Yarn状态的查询，除了可以在hadoop103:8088页面查看以外，还可以通过命令操作。常见的命令操作如下所示。

启动集群
[hadoop@hadoop102 bin]$ myhadoop.sh start
运行wordcount案例
[hadoop@hadoop102 hadoop-3.1.3]$ hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-3.1.3.jar wordcount /input /output

1、列出所有Application：

[hadoop@hadoop102 hadoop-3.1.3]$ yarn application -list
2022-06-06 16:51:19,896 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032
Total number of applications (application-types: [], states: [SUBMITTED, ACCEPTED, RUNNING] and tags: []):0
                Application-Id      Application-Name        Application-Type          User           Queue                   State              Final-State             Progress                        Tracking-URL

2、根据Application状态过滤： yarn application -list -appStates [ALL、NEW、NEW_SAVING、SUBMITTED、ACCEPTED、RUNNING、FINISHED、FAILED、KILLED]

[hadoop@hadoop102 hadoop-3.1.3]$ yarn application -list -appStates FINISHED
2022-06-06 16:53:49,886 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032
Total number of applications (application-types: [], states: [FINISHED] and tags: []):2
                Application-Id      Application-Name        Application-Type          User           Queue                   State              Final-State             Progress                        Tracking-URL
application_1654496324557_0001            word count               MAPREDUCE        hadoop         default                FINISHED                   FAILED                 100% http://hadoop102:19888/jobhistory/job/job_1654496324557_0001
application_1654496324557_0002            word count               MAPREDUCE        hadoop         default                FINISHED                   FAILED                 100% http://hadoop102:19888/jobhistory/job/job_1654496324557_0002

2、Kill掉Application

[hadoop@hadoop102 hadoop-3.1.3]$ yarn application -kill application_1654496324557_0001
2022-06-06 16:57:34,082 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032
Application application_1654496324557_0001 has already finished

1、查询Application日志： yarn logs -applicationId <applicationid></applicationid>

[hadoop@hadoop102 hadoop-3.1.3]$ yarn logs -applicationId application_1654496324557_0001

2、查询Container日志： yarn logs -applicationId <applicationid> -containerId <containerid></containerid></applicationid>

[hadoop@hadoop102 hadoop-3.1.3]$ yarn logs -applicationId application_1654496324557_0001 -containerId container_1654496324557_0001_01_000001

1、列出所有Application尝试的列表： yarn applicationattempt -list <applicationid></applicationid>

[hadoop@hadoop102 hadoop-3.1.3]$ yarn applicationattempt -list application_1654496324557_0001
2022-06-06 17:05:10,914 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032
Total number of application attempts :1
         ApplicationAttempt-Id                 State                        AM-Container-Id                            Tracking-URL
appattempt_1654496324557_0001_000001                FINISHED    container_1654496324557_0001_01_000001  http://hadoop103:8088/proxy/application_1654496324557_0001/

2、打印ApplicationAttempt状态： yarn applicationattempt -status <applicationattemptid></applicationattemptid>

[hadoop@hadoop102 hadoop-3.1.3]$ yarn applicationattempt -status appattempt_1654496324557_0001_000001
2022-06-06 17:08:56,570 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032
Application Attempt Report :
        ApplicationAttempt-Id : appattempt_1654496324557_0001_000001
        State : FINISHED
        AMContainer : container_1654496324557_0001_01_000001
        Tracking-URL : http://hadoop103:8088/proxy/application_1654496324557_0001/
        RPC Port : 32771
        AM Host : hadoop103
        Diagnostics : Task failed task_1654496324557_0001_m_000000
Job failed as tasks failed. failedMaps:1 failedReduces:0 killedMaps:0 killedReduces: 0

1、列出所有Container： yarn container -list <applicationattemptid></applicationattemptid>

[hadoop@hadoop103 ~]$ yarn container -list appattempt_1654496324557_0004_000001
2022-06-06 17:14:40,313 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032
Total number of containers :2
                  Container-Id            Start Time             Finish Time                   State                    Host       Node Http Address                                 LOG-URL
container_1654496324557_0004_01_000001  星期一 六月 06 17:13:55 +0800 2022                       N/A                 RUNNING         hadoop102:46450     http://hadoop102:8042   http://hadoop102:8042/node/containerlogs/container_1654496324557_0004_01_000001/hadoop
container_1654496324557_0004_01_000004  星期一 六月 06 17:14:38 +0800 2022                       N/A                 RUNNING         hadoop103:44058     http://hadoop103:8042   http://hadoop103:8042/node/containerlogs/container_1654496324557_0004_01_000004/hadoop

2、打印Container状态： yarn container -status <containerid></containerid>

[hadoop@hadoop103 ~]$ yarn container -status container_1654496324557_0006_01_000001
2022-06-06 17:20:51,357 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032
Container Report :
        Container-Id : container_1654496324557_0006_01_000001
        Start-Time : 1654507199188
        Finish-Time : 0
        State : RUNNING
        Execution-Type : GUARANTEED
        LOG-URL : http://hadoop103:8042/node/containerlogs/container_1654496324557_0006_01_000001/hadoop
        Host : hadoop103:44058
        NodeHttpAddress : http://hadoop103:8042
        Diagnostics : null

注：只有任务跑的途中才能看到container的状态

[hadoop@hadoop103 ~]$ yarn node -list -all
2022-06-06 17:25:20,336 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032
Total Nodes:3
         Node-Id             Node-State Node-Http-Address       Number-of-Running-Containers
 hadoop104:38945                RUNNING    hadoop104:8042                                  0
 hadoop103:44058                RUNNING    hadoop103:8042                                  0
 hadoop102:46450                RUNNING    hadoop102:8042                                  0

加载队列配置： yarn rmadmin -refreshQueues

[hadoop@hadoop103 ~]$ yarn rmadmin -refreshQueues
2022-06-06 17:27:10,830 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8033

打印队列信息： yarn queue -status <queuename></queuename>

[hadoop@hadoop103 ~]$ yarn queue -status default
2022-06-06 17:28:54,282 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032
Queue Information :
Queue Name : default
        State : RUNNING
        Capacity : 100.0%
        Current Capacity : .0%
        Maximum Capacity : 100.0%
        Default Node Label expression :
        Accessible Node Labels : *
        Preemption : disabled
        Intra-queue Preemption : disabled

Original: https://www.cnblogs.com/sw-code/p/16387038.html
Author: sw-code
Title: 大数据之Hadoop集群中Yarn常用命令