hadoop|Hadoop（三） linux|java

目录
日志生成脚本
数据采集模块
xcall脚本
Hadoop安装

日志生成脚本 bin目录里

[doudou@hadoop101 ~]$ cd bin/ [doudou@hadoop101 bin]$ ll total 8 -rwxrwxrwx. 1 doudou root 161 Aug6 23:19 lq.sh -rwxrwxrwx. 1 doudou root 449 Aug5 08:30 xsync [doudou@hadoop101 bin]$ vim lq.sh

脚本里

#!/bin/bash for i in 192.168.149.132 192.168.149.133 do ssh $i "cd /opt/module/applog; java -jar gmall2020-mock-log-2021-01-22.jar >/dev/null 2>&1 &" done

数据采集模块 xcall脚本

[doudou@hadoop101 bin]$ vim xcall.sh [doudou@hadoop101 bin]$ chmod 777 xcall.sh [doudou@hadoop101 bin]$ ll total 12 -rwxrwxrwx. 1 doudou root 161 Aug6 23:19 lq.sh -rwxrwxrwx. 1 doudou root 129 Aug7 06:47 xcall.sh -rwxrwxrwx. 1 doudou root 449 Aug5 08:30 xsync

脚本里

#! /bin/bash for i in 192.168.149.129 192.168.149.132 192.168.149.133 do echo ---------- $i --------- ssh $i "$*" done

【hadoop|Hadoop（三）】执行

[doudou@hadoop101 bin]$ xcall.sh jps ---------- 192.168.149.129 --------- 3512 Jps ---------- 192.168.149.132 --------- 3528 Jps ---------- 192.168.149.133 --------- 3471 Jps

Hadoop安装

[doudou@hadoop101 ~]$ cd /opt [doudou@hadoop101 opt]$ ll total 0 drwxr-xr-x. 4 doudou doudou 40 Aug6 07:47 module drwxr-xr-x. 2 rootroot6 Oct 302018 rh drwxr-xr-x. 2 doudou doudou 40 Aug5 08:12 software [doudou@hadoop101 opt]$ cd software/ [doudou@hadoop101 software]$ rz -E

把对应的安装包拖到里面

文章图片

[doudou@hadoop101 software]$ ll total 740096 -rw-r--r--. 1 doudou root 224565455 Aug7 07:03 bigtable.lzo -rw-r--r--. 1 doudou root 338075860 Aug7 07:08 hadoop-3.1.3.tar.gz -rw-r--r--. 1 doudou root193831 Aug7 06:56 hadoop-lzo-0.4.20.jar -rw-r--r--. 1 doudou root 195013152 Aug5 08:08 jdk-8u212-linux-x64.tar.gz

配置

[doudou@hadoop101 software]$ tar -zxvf hadoop-3.1.3.tar.gz-C /opt/module/

[doudou@hadoop101 module]$ ll total 0 drwxr-xr-x. 3 doudou root 117 Aug7 06:36 applog drwxr-xr-x. 9 doudou root 149 Sep 112019 hadoop-3.1.3 drwxr-xr-x. 7 doudou root 245 Apr12019 jdk1.8.0_212

修改环境变量

[doudou@hadoop101 module]$ cd hadoop-3.1.3/ [doudou@hadoop101 hadoop-3.1.3]$ sudo vim /etc/profile.d/my_env.sh

export JAVA_HOME=/opt/module/jdk1.8.0_212 export PATH=$PATH:$JAVA_HOME/bin #HADOOP export HADOOP_HOME=/opt/module/hadoop-3.1.3 export PATH=$PATH:$HADOOP_HOME/bin export PATH=$PATH:$HADOOP_HOME/sbin

分发

[doudou@hadoop101 hadoop-3.1.3]$ source /etc/profile.d/my_env.sh [doudou@hadoop101 hadoop-3.1.3]$ cd .. [doudou@hadoop101 module]$ ll total 0 drwxr-xr-x. 3 doudou root 117 Aug7 06:36 applog drwxr-xr-x. 9 doudou root 149 Aug7 07:20 hadoop-3.1.3 drwxr-xr-x. 7 doudou root 245 Apr12019 jdk1.8.0_212 [doudou@hadoop101 module]$ xsync hadoop-3.1.3/

拷贝环境变量