HDFS文件损坏解决方法 HDFS文件损坏解决方法

文章图片
image.png 自己虚拟机搭建的伪分布Hadoop，没有停止进程，不小心点到关机。再次启动时就一直处于安全模式，而且说损坏了20个块

[hadoop@hadoop000 hadoop]$ hdfs fsck /Connecting to namenode via http://hadoop000:50070 FSCK started by hadoop (auth:SIMPLE) from /192.168.245.100 for path / at Thu May 30 22:25:35 CST 2019 ........ /spark_eventLog/application_1557764445673_0005: CORRUPT blockpool BP-1926021031-192.168.245.100-1557764272932 block blk_1073741882/spark_eventLog/application_1557764445673_0005: MISSING 1 blocks of total size 46746 B.. /spark_eventLog/application_1557764445673_0006: CORRUPT blockpool BP-1926021031-192.168.245.100-1557764272932 block blk_1073741888/spark_eventLog/application_1557764445673_0006: MISSING 1 blocks of total size 44885 B.. /spark_eventLog/application_1557764445673_0007: CORRUPT blockpool BP-1926021031-192.168.245.100-1557764272932 block blk_1073741893/spark_eventLog/application_1557764445673_0007: MISSING 1 blocks of total size 44932 B.. /spark_eventLog/application_1557764445673_0008: CORRUPT blockpool BP-1926021031-192.168.245.100-1557764272932 block blk_1073741898/spark_eventLog/application_1557764445673_0008: MISSING 1 blocks of total size 44935 B.. /spark_eventLog/application_1557764445673_0009: CORRUPT blockpool BP-1926021031-192.168.245.100-1557764272932 block blk_1073741903/spark_eventLog/application_1557764445673_0009: MISSING 1 blocks of total size 44931 B.. /spark_eventLog/application_1557764445673_0010: CORRUPT blockpool BP-1926021031-192.168.245.100-1557764272932 block blk_1073741908/spark_eventLog/application_1557764445673_0010: MISSING 1 blocks of total size 44935 B.. /spark_eventLog/application_1557764445673_0011: CORRUPT blockpool BP-1926021031-192.168.245.100-1557764272932 block blk_1073741913/spark_eventLog/application_1557764445673_0011: MISSING 1 blocks of total size 44931 B.. /spark_eventLog/application_1557764445673_0012: CORRUPT blockpool BP-1926021031-192.168.245.100-1557764272932 block blk_1073741918/spark_eventLog/application_1557764445673_0012: MISSING 1 blocks of total size 62124 B.. /spark_eventLog/application_1557764445673_0013: CORRUPT blockpool BP-1926021031-192.168.245.100-1557764272932 block blk_1073741923/spark_eventLog/application_1557764445673_0013: MISSING 1 blocks of total size 44932 B......................... /spark_eventLog/local-1558243986730: CORRUPT blockpool BP-1926021031-192.168.245.100-1557764272932 block blk_1073741876/spark_eventLog/local-1558243986730: MISSING 1 blocks of total size 38056 B.. /spark_eventLog/local-1558244355032: CORRUPT blockpool BP-1926021031-192.168.245.100-1557764272932 block blk_1073741877/spark_eventLog/local-1558244355032: MISSING 1 blocks of total size 38056 B.......... /tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1557764445673_0002-1558020903556-hadoop-select+domain+%2C+time+%2C+row_num...domain_info%28Stage-1558020926785-1-1-SUCCEEDED-root.hadoop-1558020910773.jhist: CORRUPT blockpool BP-1926021031-192.168.245.100-1557764272932 block blk_1073741843/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1557764445673_0002-1558020903556-hadoop-select+domain+%2C+time+%2C+row_num...domain_info%28Stage-1558020926785-1-1-SUCCEEDED-root.hadoop-1558020910773.jhist: MISSING 1 blocks of total size 35395 B.. /tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1557764445673_0002.summary: CORRUPT blockpool BP-1926021031-192.168.245.100-1557764272932 block blk_1073741842/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1557764445673_0002.summary: MISSING 1 blocks of total size 394 B.. /tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1557764445673_0002_conf.xml: CORRUPT blockpool BP-1926021031-192.168.245.100-1557764272932 block blk_1073741844/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1557764445673_0002_conf.xml: MISSING 1 blocks of total size 223006 B.. /tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1557764445673_0003-1558021068294-hadoop-select+%0Adomain%2C+%0Atime%2C+%0Arow_nu...domain_info%28Stage-1558021086767-1-1-SUCCEEDED-root.hadoop-1558021073405.jhist: CORRUPT blockpool BP-1926021031-192.168.245.100-1557764272932 block blk_1073741856/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1557764445673_0003-1558021068294-hadoop-select+%0Adomain%2C+%0Atime%2C+%0Arow_nu...domain_info%28Stage-1558021086767-1-1-SUCCEEDED-root.hadoop-1558021073405.jhist: MISSING 1 blocks of total size 35481 B.. /tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1557764445673_0003.summary: CORRUPT blockpool BP-1926021031-192.168.245.100-1557764272932 block blk_1073741855/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1557764445673_0003.summary: MISSING 1 blocks of total size 397 B.. /tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1557764445673_0003_conf.xml: CORRUPT blockpool BP-1926021031-192.168.245.100-1557764272932 block blk_1073741857/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1557764445673_0003_conf.xml: MISSING 1 blocks of total size 223167 B.. /tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1557764445673_0004-1558021143709-hadoop-select+%0Adomain%2C+%0Atime%2C+%0Arow_nu...domain_info%28Stage-1558021160715-1-1-SUCCEEDED-root.hadoop-1558021148130.jhist: CORRUPT blockpool BP-1926021031-192.168.245.100-1557764272932 block blk_1073741869/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1557764445673_0004-1558021143709-hadoop-select+%0Adomain%2C+%0Atime%2C+%0Arow_nu...domain_info%28Stage-1558021160715-1-1-SUCCEEDED-root.hadoop-1558021148130.jhist: MISSING 1 blocks of total size 35551 B.. /tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1557764445673_0004.summary: CORRUPT blockpool BP-1926021031-192.168.245.100-1557764272932 block blk_1073741868/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1557764445673_0004.summary: MISSING 1 blocks of total size 397 B.. /tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1557764445673_0004_conf.xml: CORRUPT blockpool BP-1926021031-192.168.245.100-1557764272932 block blk_1073741870/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1557764445673_0004_conf.xml: MISSING 1 blocks of total size 223322 B...Status: CORRUPT Total size:102274952 B Total dirs:29 Total files:60 Total symlinks:0 Total blocks (validated):57 (avg. block size 1794297 B) ******************************** CORRUPT FILES:20 MISSING BLOCKS:20 MISSING SIZE:1276573 B CORRUPT BLOCKS:20 ******************************** Minimally replicated blocks:37 (64.91228 %) Over-replicated blocks:0 (0.0 %) Under-replicated blocks:0 (0.0 %) Mis-replicated blocks:0 (0.0 %) Default replication factor:1 Average block replication:0.64912283 Corrupt blocks:20 Missing replicas:0 (0.0 %) Number of data-nodes:1 Number of racks:1 FSCK ended at Thu May 30 22:25:35 CST 2019 in 11 millisecondsThe filesystem under path '/' is CORRUPT

【HDFS文件损坏解决方法】解决方法：