集群版

[TOC] > [参考网址](https://blog.51cto.com/taoismli/2163097) ## 创建用户 ``` export username=im_user useradd -d /home/${username} -m ${username} passwd ${username} echo "${username} ALL = (root) NOPASSWD:ALL" | sudo tee /etc/sudoers.d/${username} ``` ## 设置免登陆 ``` cat >> /etc/hosts < hadoop.tmp.dir file:/usr/local/hadoop/tmp fs.defaultFS hdfs://192.168.0.110:9000 hadoop.http.staticuser.user im_user ``` ### 只修改 master 节点(h1节点) vim /usr/local/hadoop/etc/hadoop/hdfs-site.xml ``` dfs.webhdfs.enabled true dfs.namenode.http-address 0.0.0.0:50070 dfs.replication 2 dfs.namenode.name.dir file:/usr/local/hadoop/namenode dfs.datanode.data.dir file:/usr/local/hadoop/data dfs.datanode.http.address 0.0.0.0:9876 ``` vim /usr/local/hadoop/etc/hadoop/mapred-site.xml ``` mapreduce.framework.name yarn ``` vim /usr/local/hadoop/etc/hadoop/workers ``` h1 h2 h3 ``` vim /usr/local/hadoop/etc/hadoop/yarn-site.xml ``` yarn.resourcemanager.hostname h1 yarn.nodemanager.aux-services mapreduce_shuffle yarn.nodemanager.aux-services.mapred.shuffle.class org.apache.hadoop.mapred.ShuffleHandler ``` ### 初始化Hadoop系统 `/usr/local/hadoop/bin/hdfs namenode -format` 出现 `INFO common.Storage: Storage directory /usr/local/hadoop/hadoop-3.1.2/namenode has been successfully formatted. ` 表示成功 ### 启动 hadoop `/usr/local/hadoop/sbin/start-all.sh` ### 判断是否启动 主节点 ``` jps 39578 ResourceManager 39324 SecondaryNameNode 39933 Jps 39039 NameNode ``` 子节点 ``` jps 16000 Jps 15907 NodeManager 15780 DataNode ``` ### 停止hadoop `/usr/local/hadoop/sbin/stop-all.sh` ### 查看网址 `http://192.168.0.110:50070` ## curl 测试 ``` curl -L -i "http://192.168.0.110:50070/webhdfs/v1/input/hadoop-im_user-datanode-node-3.log?op=open" ```