presto-admin 安装及管理presto集群

rpm:https://repo1.maven.org/maven2/com/facebook/presto/presto-server-rpm/
https://prestodb.io/presto-admin/docs/current/presto-admin-commands.html#configuration-deploy

presto-admin:
https://docs.starburstdata.com/302-e/presto-admin/presto-admin-commands.html
https://repo1.maven.org/maven2/com/facebook/presto/presto-server-rpm/
https://prestodb.io/presto-admin/docs/current/presto-admin-commands.html#configuration-deploy
https://prestodb.io/presto-admin/docs/current/presto-admin-commands.html#server-install
安装presto-admin依赖的库文件
yum install postgresql-devel*
pip3 install psycopg2
pip3 install PyHive
pip3 install sqlalchemy
pip3 install pandas
pip3 install requests
pip3 install sasl
pip3 install thrift
pip3 install thrift-sasl


presto-admin 安装
python 2.6 2.7(需要安装pip)

curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py

ll get-pip.py
-rw-r--r--. 1 root root 1885433 Sep 25 15:38 get-pip.py

python get-pip.py

Collecting pip
Downloading pip-20.2.3-py2.py3-none-any.whl (1.5 MB)
|████████████████████████████████| 1.5 MB 126 kB/s
Collecting setuptools
Downloading setuptools-44.1.1-py2.py3-none-any.whl (583 kB)
|████████████████████████████████| 583 kB 67 kB/s
Collecting wheel
Downloading wheel-0.35.1-py2.py3-none-any.whl (33 kB)
Installing collected packages: pip, setuptools, wheel
Successfully installed pip-20.2.3 setuptools-44.1.1 wheel-0.35.1

clone presto-admin
git clone https://github.com/prestodb/presto-admin.git
presto-admin
cd presto-admin
pip install -r requirements.txt
python setup.py develop

Using /usr/lib/python2.7/site-packages
Searching for virtualenv==12.0.7
Best match: virtualenv 12.0.7
Adding virtualenv 12.0.7 to easy-install.pth file
Installing virtualenv script to /usr/bin
Installing virtualenv-3.4 script to /usr/bin

Using /usr/lib/python2.7/site-packages
Searching for pyflakes==1.0.0
Best match: pyflakes 1.0.0
Processing pyflakes-1.0.0-py2.7.egg
pyflakes 1.0.0 is already the active version in easy-install.pth
Installing pyflakes script to /usr/bin

Using /usr/lib/python2.7/site-packages/pyflakes-1.0.0-py2.7.egg
Searching for pep8==1.7.1
Best match: pep8 1.7.1
Processing pep8-1.7.1-py2.7.egg
pep8 1.7.1 is already the active version in easy-install.pth
Installing pep8 script to /usr/bin

Using /usr/lib/python2.7/site-packages/pep8-1.7.1-py2.7.egg
Searching for mccabe==0.4.0
Best match: mccabe 0.4.0
Processing mccabe-0.4.0-py2.7.egg
mccabe 0.4.0 is already the active version in easy-install.pth

Using /usr/lib/python2.7/site-packages/mccabe-0.4.0-py2.7.egg
Searching for ecdsa==0.16.0
Best match: ecdsa 0.16.0
Processing ecdsa-0.16.0-py2.7.egg
ecdsa 0.16.0 is already the active version in easy-install.pth

Using /usr/lib/python2.7/site-packages/ecdsa-0.16.0-py2.7.egg
Searching for typing==3.7.4.3
Best match: typing 3.7.4.3
Adding typing 3.7.4.3 to easy-install.pth file

Using /usr/lib/python2.7/site-packages
Searching for enum34==1.1.10
Best match: enum34 1.1.10
Adding enum34 1.1.10 to easy-install.pth file

Using /usr/lib/python2.7/site-packages
Finished processing dependencies for prestoadmin===2.6-SNAPSHOT

[root@DM02 presto-admin]# whereis presto-admin
presto-admin: /usr/bin/presto-admin

安装完之后会自动在/root/目录下产生一个目录/root/.prestoadmin:
[root@DM02 ~]# cd /root/.prestoadmin/
[root@DM02 .prestoadmin]# ls
catalog config.json coordinator log workers
catalog有用添加catalog,通过prestoadmin添加的catalog都会先写到该 目录下,
然后分发到集群中的其他节点。
修改config.json 指定coordinator,workers 相关信息
[root@DM02 .prestoadmin]# cat config.json
{
"username":"root",
"coordinator":"172.25.5.38",
"workers":[
"172.25.5.38","172.25.5.39","172.25.5.40"
],
"port":22
}
修改config.properties 指定discovery.uri=http://172.25.5.38:8080 节点
[root@DM02 .prestoadmin]# cd coordinator/
[root@DM02 coordinator]# ls
config.properties jvm.config node.properties
[root@DM02 coordinator]# cat config.properties
query.max-memory=50GB
node-scheduler.include-coordinator=true
discovery.uri=http://localhost:8080
discovery-server.enabled=true
http-server.http.port=8080
coordinator=true
query.max-memory-per-node=8GB
[root@DM02 coordinator]# cat jvm.config
-server
-Xmx16G
-XX:-UseBiasedLocking
-XX:+UseG1GC
-XX:G1HeapRegionSize=32M
-XX:+ExplicitGCInvokesConcurrent
-XX:+HeapDumpOnOutOfMemoryError
-XX:+UseGCOverheadLimit
-XX:+ExitOnOutOfMemoryError
-XX:ReservedCodeCacheSize=512M
-DHADOOP_USER_NAME=hive
[root@DM02 coordinator]# cat node.properties
node.launcher-log-file=/var/log/presto/launcher.log
node.server-log-file=/var/log/presto/server.log
catalog.config-dir=/etc/presto/catalog
node.data-dir=/var/lib/presto/data
node.environment=presto
plugin.dir=/usr/lib/presto/lib/plugin

准备好rpm安装包
/opt/presto-server-rpm-0.224.rpm
[root@DM02 ~]# presto-admin server install /opt/presto-server-rpm-0.224.rpm
Using rpm_specifier as a local path
Fetching local presto rpm at path: /opt/presto-server-rpm-0.224.rpm
Found existing rpm at: /opt/presto-server-rpm-0.224.rpm
Deploying rpm on 172.25.5.40...
Deploying rpm on 172.25.5.39...
Deploying rpm on 172.25.5.38...
Package deployed successfully on: 172.25.5.40
Package deployed successfully on: 172.25.5.38
Package deployed successfully on: 172.25.5.39
Package installed successfully on: 172.25.5.40
Deploying configuration on: 172.25.5.40
Package installed successfully on: 172.25.5.39
Deploying configuration on: 172.25.5.39
Deploying tpch.properties catalog configurations on: 172.25.5.40
Deploying tpch.properties catalog configurations on: 172.25.5.39
Package installed successfully on: 172.25.5.38
Deploying configuration on: 172.25.5.38
Deploying tpch.properties catalog configurations on: 172.25.5.38

[root@DM02 presto]# presto-admin server start
[172.25.5.40] out: Starting presto
[172.25.5.40] out: Started as 1702
[172.25.5.40] out:
Waiting to make sure we can connect to the Presto server on 172.25.5.40, please wait. This check will time out after 2 minutes if the server does not respond.
[172.25.5.38] out: Starting presto
[172.25.5.39] out: Starting presto
[172.25.5.39] out: Started as 27667
[172.25.5.39] out:
Waiting to make sure we can connect to the Presto server on 172.25.5.39, please wait. This check will time out after 2 minutes if the server does not respond.
[172.25.5.38] out: Started as 29199
[172.25.5.38] out:
Waiting to make sure we can connect to the Presto server on 172.25.5.38, please wait. This check will time out after 2 minutes if the server does not respond.
Server started successfully on: 172.25.5.38
Server started successfully on: 172.25.5.40
Server started successfully on: 172.25.5.39


[root@DM02 presto]# presto-admin server status
Server Status:
172.25.5.38(IP: 172.25.5.38, Roles: coordinator, worker): Running
Node URI(http): http://172.25.5.38:8080
Presto Version: 0.224-b3a49f7
Node status: active
Catalogs: system, tpch
Server Status:
172.25.5.39(IP: 172.25.5.39, Roles: worker): Running
Node URI(http): http://172.25.5.39:8080
Presto Version: 0.224-b3a49f7
Node status: active
Catalogs: system, tpch
Server Status:
172.25.5.40(IP: 172.25.5.40, Roles: worker): Running
Node URI(http): http://172.25.5.40:8080
Presto Version: 0.224-b3a49f7
Node status: active
Catalogs: system, tpch

有报错看日志:
This could mean that the server failed to start or that there was no coordinator or worker up.
Please check /var/log/presto/server.log and /var/log/presto/launcher.log

http://172.25.5.38:8080/ui/

原文地址:https://www.cnblogs.com/songyuejie/p/13710872.html