一、HDFS之shell命令行客户端:Shell命令行操作HDFS
  1. ### --- 基本语法
  2. ~~~ bin/hadoop fs 具体命令 OR bin/hdfs dfs 具体命令
  1. ### --- 命令大全
  2. [root@linux121 hadoop-2.9.2]# bin/hdfs dfs
  3. Usage: hadoop fs [generic options]
  4. [-appendToFile <localsrc> ... <dst>]
  5. [-cat [-ignoreCrc] <src> ...]
  6. [-checksum <src> ...]
  7. [-chgrp [-R] GROUP PATH...]
  8. [-chmod [-R] <MODE[,MODE]... | OCTALMODE> PATH...]
  9. [-chown [-R] [OWNER][:[GROUP]] PATH...]
  10. [-copyFromLocal [-f] [-p] [-l] [-d] <localsrc> ... <dst>]
  11. [-copyToLocal [-f] [-p] [-ignoreCrc] [-crc] <src> ... <localdst>]
  12. [-count [-q] [-h] [-v] [-t [<storage type>]] [-u] [-x] <path> ...]
  13. [-cp [-f] [-p | -p[topax]] [-d] <src> ... <dst>]
  14. [-createSnapshot <snapshotDir> [<snapshotName>]]
  15. [-deleteSnapshot <snapshotDir> <snapshotName>]
  16. [-df [-h] [<path> ...]]
  17. [-du [-s] [-h] [-x] <path> ...]
  18. [-expunge]
  19. [-find <path> ... <expression> ...]
  20. [-get [-f] [-p] [-ignoreCrc] [-crc] <src> ... <localdst>]
  21. [-getfacl [-R] <path>]
  22. [-getfattr [-R] {-n name | -d} [-e en] <path>]
  23. [-getmerge [-nl] [-skip-empty-file] <src> <localdst>]
  24. [-help [cmd ...]]
  25. [-ls [-C] [-d] [-h] [-q] [-R] [-t] [-S] [-r] [-u] [<path> ...]]
  26. [-mkdir [-p] <path> ...]
  27. [-moveFromLocal <localsrc> ... <dst>]
  28. [-moveToLocal <src> <localdst>]
  29. [-mv <src> ... <dst>]
  30. [-put [-f] [-p] [-l] [-d] <localsrc> ... <dst>]
  31. [-renameSnapshot <snapshotDir> <oldName> <newName>]
  32. [-rm [-f] [-r|-R] [-skipTrash] [-safely] <src> ...]
  33. [-rmdir [--ignore-fail-on-non-empty] <dir> ...]
  34. [-setfacl [-R] [{-b|-k} {-m|-x <acl_spec>} <path>]|[--set <acl_spec> <path>]]
  35. [-setfattr {-n name [-v value] | -x name} <path>]
  36. [-setrep [-R] [-w] <rep> <path> ...]
  37. [-stat [format] <path> ...]
  38. [-tail [-f] <file>]
  39. [-test -[defsz] <path>]
  40. [-text [-ignoreCrc] <src> ...]
  41. [-touchz <path> ...]
  42. [-truncate [-w] <length> <path> ...]
  43. [-usage [cmd ...]]
  44. Generic options supported are:
  45. -conf <configuration file> specify an application configuration file
  46. -D <property=value> define a value for a given property
  47. -fs <file:///|hdfs://namenode:port> specify default filesystem URL to use, overrides 'fs.defaultFS' property from configurations.
  48. -jt <local|resourcemanager:port> specify a ResourceManager
  49. -files <file1,...> specify a comma-separated list of files to be copied to the map reduce cluster
  50. -libjars <jar1,...> specify a comma-separated list of jar files to be included in the classpath
  51. -archives <archive1,...> specify a comma-separated list of archives to be unarchived on the compute machines
  52. The general command line syntax is:
  53. command [genericOptions] [commandOptions]
二、HDFS命令演示
  1. ### --- 启动Hadoop集群(方便后续的测试)
  2. [root@linux121 hadoop-2.9.2]# sbin/start-dfs.sh
  3. [root@linux122 hadoop-2.9.2]# sbin/start-yarn.sh
  1. ### --- -help:输出这个命令参数
  2. [root@linux121 ~]# hadoop fs -help rm
  1. ### --- -ls: 显示目录信息
  2. [root@linux121 ~]# hadoop fs -ls /
  3. Found 4 items
  4. drwxr-xr-x - root supergroup 0 2021-08-08 19:40 /test
  5. drwx------ - root supergroup 0 2021-08-08 21:02 /tmp
  6. drwxr-xr-x - root supergroup 0 2021-08-08 19:46 /wcinput
  7. drwxr-xr-x - root supergroup 0 2021-08-08 21:36 /wcoutput
  1. ### --- -mkdir:在HDFS上创建目录
  2. [root@linux121 ~]# hadoop fs -mkdir -p /yanqi/bigdata
  1. ### --- -moveFromLocal:从本地剪切粘贴到HDFS
  2. [root@linux121 ~]# touch hadoop.txt
  3. [root@linux121 ~]# hadoop fs -moveFromLocal ./hadoop.txt /yanqi/bigdata
  1. ### --- -appendToFile:追加一个文件到已经存在的文件末尾
  2. [root@linux121 ~]# touch hdfs.txt
  3. [root@linux121 ~]# vim hdfs.txt
  4. namenode datanode block replication
  5. [root@linux121 ~]# hadoop fs -appendToFile hdfs.txt /yanqi/bigdata/hadoop.txt
  1. ### --- -cat:显示文件内容
  2. [root@linux121 ~]# hadoop fs -cat /yanqi/bigdata/hadoop.txt
  3. namenode datanode block replication
  1. ### --- -chgrp 、-chmod、-chown:Linux文件系统中的用法一样,修改文件所属权限
  2. [root@linux121 ~]# hadoop fs -chmod 666 /yanqi/bigdata/hadoop.txt
  3. [root@linux121 ~]# hadoop fs -chown root:root /yanqi/bigdata/hadoop.txt
  1. ### --- -copyFromLocal:从本地文件系统中拷贝文件到HDFS路径去
  2. [root@linux121 ~]# hadoop fs -copyFromLocal README.txt /
  1. ### --- -copyToLocal:从HDFS拷贝到本地
  2. [root@linux121 ~]# hadoop fs -copyToLocal /yanqi/bigdata/hadoop.txt ./
  3. [root@linux121 ~]# ls | grep hadoop.txt
  4. hadoop.txt
  1. ### --- -cp :从HDFS的一个路径拷贝到HDFS的另一个路径
  2. [root@linux121 ~]# hadoop fs -cp /yanqi/bigdata/hadoop.txt /hdfs.txt
  1. ### --- -mv:在HDFS目录中移动文件
  2. [root@linux121 ~]# hadoop fs -mv /hdfs.txt /yanqi/bigdata/
  1. ### --- -get:等同于copyToLocal,就是从HDFS下载文件到本地
  2. [root@linux121 ~]# hadoop fs -get /yanqi/bigdata/hadoop.txt ./
  3. [root@linux121 ~]# ls | grep hadoop.txt
  4. hadoop.txt
  1. ### --- -put:等同于copyFromLocal
  2. [root@linux121 ~]# hadoop fs -mkdir -p /user/root/test/
  1. ### --- 本地文件系统创建yarn.txt
  2. [root@linux121 ~]# vim yarn.txt
  3. resourcemanager nodemanager
  4. [root@linux121 ~]# hadoop fs -put ./yarn.txt /user/root/test/
  1. ### --- -tail:显示一个文件的末尾
  2. [root@linux121 ~]# hadoop fs -tail /user/root/test/yarn.txt
  3. resourcemanager nodemanager
  1. ### --- -rm:删除文件或文件夹
  2. [root@linux121 ~]# hadoop fs -rm /user/root/test/yarn.txt
  3. Deleted /user/root/test/yarn.txt
  1. ### --- -rmdir:删除空目录
  2. [root@linux121 ~]# hadoop fs -mkdir /test
  1. ### --- -du统计文件夹的大小信息
  2. [root@linux121 ~]# hadoop fs -du -s -h /user/root/test
  3. 0 /user/root/test
  4. [root@linux121 ~]# hadoop fs -du -h /user/root/test
  1. ### --- -setrep:设置HDFS中文件的副本数量
  2. [root@linux121 ~]# hadoop fs -setrep 10 /yanqi/bigdata/hadoop.txt
二、图3-3 HDFS副本数量
  1. ### --- 图3-3 HDFS副本数量
  2. ~~~ 这里设置的副本数只是记录在NameNode的元数据中,是否真的会有这么多副本,
  3. ~~~ 还得看DataNode的数量。因为目前只有3台设备,最多也就3个副本,
  4. ~~~ 只有节点数的增加到10台时,副本数才能达到10

 
 
 
 
 
 
 
 
 

Walter Savage Landor:strove with none,for none was worth my strife.Nature I loved and, next to Nature, Art:I warm’d both hands before the fire of life.It sinks, and I am ready to depart
                                                                                                                                                   ——W.S.Landor

 
版权声明:本文为yanqi_vip原创文章,遵循 CC 4.0 BY-SA 版权协议,转载请附上原文出处链接和本声明。
本文链接:https://www.cnblogs.com/yanqivip/p/16111899.html