CentOS 6.9 下GreenPlum源码编译安装及部署教程教程

文章图片

GreenPlum介绍 PostgreSql PostgreSQL is an advanced object-relational database management system that supports an extended subset of the SQL standard, including transactions, foreign keys, subqueries, triggers, user-defined types and functions. This distribution also contains C language bindings.
GreenPlum The Greenplum Database (GPDB) is an advanced, fully featured, open source data warehouse. It provides powerful and rapid analytics on petabyte scale data volumes. Uniquely geared toward big data analytics, Greenplum Database is powered by the world’s most advanced cost-based query optimizer delivering high analytical query performance on large data volumes.
Greenplum 是基于 PostgreSQL 开发的，语法与 PostgreSQL 几乎一样。
本质上，Greenplum 是一个关系型数据库集群，由多个独立的数据库服务组合成的逻辑数据库，对于用户来说 Greenplum 与 Oracle、PostgreSQL 并无不同，都可以通过标准 SQL 语句对数据进行访问。
GreenPlum源码编译安装以CentOS 6.9为例，下载地址为阿里云镜像。
CentOS-6.9-x86_64-bin-DVD1.iso
CentOS-6.9-x86_64-bin-DVD2.iso
强烈建议以gpadmin为用户名。
准备工作添加用户root权限
vim 打开 /etc/sudoers 文件（以root权限打开）
找到以下两行

## Allow root to run any commands anywhere rootALL=(ALL)ALL

并在下一行添加

gpadmin ALL=(ALL)ALL #注意对齐

保存并返回即可
安装gcc-c++

sudo yum install gcc-c++

安装git

sudo yum install git

安装cmake

cd /home/gpadmin wget https://cmake.org/files/v3.5/cmake-3.5.2.tar.gz tar -zxvf cmake-3.5.2.tar.gz cd cmake-3.5.2 ./configure --prefix=/usr/local make sudo make install

安装pip

sudo yum -y install epel-release sudo yum -y install python-pip pip install --upgrade #pip版本至少为 7.x.x

确保已安装必需的python模块

psutil
lockfile (>= 0.9.1)
paramiko
setuptools

确保动态链接库共享
添加 /usr/local/lib 和 /usr/local/lib64 到 /etc/ld.so.conf 中
并运行命令

sudo ldconfig

支持 c++11

sudo yum install -y centos-release-scl sudo yum install -y devtoolset-6-toolchain echo 'source scl_source enable devtoolset-6' >> ~/.bashrc source ~/.bashrc

安装依赖

sudo yum -y install gcc git apr bison flexreadline gcc-c++ curl-devel bzip2-devel python-devel readline-devel apr-devel libevent-devel openssl-devel perl-ExtUtils-Embed libxml2-devel openldap-devel pam pam-devel

安装GPORCA clone GPORCA

cd /home/gpadmin git clone https://github.com/greenplum-db/gporca.git

安装 gp-xerces

git clone https://github.com/greenplum-db/gp-xerces.git cd gp-xerces mkdir build cd build ../configure --prefix=/usr/local make sudo make install

安装ninja

cd /home/gpadmin git clone git://github.com/ninja-build/ninja.git && cd ninja ./configure.py --bootstrap sudo cp ninja /usr/bin

编译安装GPORCA

cd /home/gpadmin/gporca cmake -GNinja -H. -Bbuild sudo ninja install -C build

编译GreenPlum

cd /home/gpadmin git clone https://github.com/greenplum-db/gpdb.git cd gpdb ./configure --with-perl --with-python --with-libxml --with-gssapi --prefix=/usr/local/gpdb# Compile and install make -j8 sudo make -j8 install# Bring in greenplum environment into your running shell source /usr/local/gpdb/greenplum_path.sh

若出现

Greenplum Database installation complete.

则说明安装成功
部署以在同一台主机上的一个master节点和一个segment节点为例
配置host文件查看ip

ip addr show eth0 # 选择inet后的ip

修改hosts文件

sudo vim /etc/hosts

添加两行

mdw sdw1 #此处ip相同

修改network文件

sudo vim /etc/sysconfig/network #修改HOSTNAME=mdw

创建主机列表文件和segment文件

cd /home/gpadmin mkdir conf cd conf sudo vim hostlist

添加以下内容

mdw sdw1

sudo vim seg_hosts

添加以下内容

sdw1

共享内存与网络参数配置

sudo vim /etc/sysctl.conf

对照替换成以下内容

net.ipv4.ip_forward = 0 net.ipv4.conf.default.accept_source_route = 0 kernel.sysrq = 1 kernel.core_uses_pid = 1 net.ipv4.tcp_syncookies = 1 kernel.msgmnb = 65536 kernel.msgmax = 65536 kernel.sem = 250 64000 100 512 kernel.shmmax = 500000000 kernel.shmmni = 4096 kernel.shmall = 4000000000 kernel.sem = 250 64000 100 512 net.ipv4.tcp_tw_recycle = 1 net.ipv4.tcp_max_syn_backlog = 4096 net.core.netdev_max_backlog = 10000 vm.overcommit_memory = 2 net.ipv4.conf.all.arp_filter = 1

sudo sysctl -p

加入限制参数

sudo vim /etc/security/limits.conf

添加以下内容

* soft nofile 65536 * hard nofile 65536 * soft nproc 131072 * hard nproc 131072 #星号不能去掉

节点部署环境变量设置

source /usr/local/gpdb/greenplum_path.sh

配置免密码登陆

sudo pip install psutil gpssh-exkeys -f /home/gpadmin/conf/hostlist

创建数据目录

cd /home/gpadmin mkdir gpdata cd gpdata mkdir segmentdata1 masterdata

环境变量配置

vim ~/.bash_profile

添加以下内容

source /usr/local/gpdb/greenplum_path.sh export MASTER_DATA_DIRECTORY=/home/gpadmin/gpdata/masterdata/gpseg-1 export PGPORT=2345 export PGDATABASE=testDB

source ~/.bash_profile

编写数据库启动参数文件

cd /home/gpadmin/conf mkdir gpconfigs cd gpconfigs vim gpinitsystem_config

添加以下内容

#启动参数文件内容 ARRAY_NAME="Greenplum" SEG_PREFIX=gpseg PORT_BASE=40000 declare -a DATA_DIRECTORY=(/home/gpadmin/gpdata/segmentdata1 /home/gpadmin/gpdata/segmentdata1) MASTER_HOSTNAME=mdw MASTER_DIRECTORY=/home/gpadmin/gpdata/masterdata ##### Port number for the master instance. MASTER_PORT=2345 # #### Shell utility used to connect to remote hosts. TRUSTED_SHELL=/usr/bin/ssh ##### Maximum log file segments between automatic WAL checkpoints. CHECK_POINT_SEGMENTS=8 ENCODING=UNICODE #MIRROR_PORT_BASE=50000 REPLICATION_PORT_BASE=41000 #MIRROR_REPLICATION_PORT_BASE=51000 #declare -a MIRROR_DATA_DIRECTORY=(/home/gpadmin/gpdata/segmentmirror1 #/home/gpadmin/gpdata/segmentmirror) MACHINE_LIST_FILE=/home/gpadmin/conf/seg_hosts

初始化数据库

gpinitsystem -c /home/gpadmin/conf/gpconfigs/gpinitsystem_config –a

数据库使用
gpstart #开启数据库服务
gpstop #关闭数据库服务
gprecoverseg #恢复启动失败的节点
gpstate -m #查看节点状态
createdb testDB #创建数据库
psql #连接数据库 psql –help查看更多选项
TPC-H测试
运行Greenplum TPC-H测试
参考https://yq.aliyun.com/articles/93?commentId=29
【CentOS 6.9 下GreenPlum源码编译安装及部署教程】如有兴趣，请访问个人博客