在火山引擎 E-MapReduce(EMR) 中,安装完 Impala 和 Kudu 之后,需要在 Impala 设置 Kudu Master 的地址。
注意
如果 Impala 和 Kudu 同时安装,那么,kudu_master_hosts 会自动设置,无需再手动设置
登录 EMR 控制台。
在左侧导航栏中,选择并进入 集群列表 > 集群名称 > 服务列表 > Impala 界面。
点击进入 服务参数 页签,搜索参数:kudu_master_hosts。
设置参数的值为 Kudu Master 的地址,如下图所示:
Kudu Master 地址获取,详见:Kudu 基础操作。
参数配置完成后,点击右上角 保存 按钮,然后 服务操作中 重启 Impalad 服务。
Impala-shell 是一个可以直接连接 impala 的命令行工具。火山引擎 EMR 在安装 impala 时,已默认将 impala-shell 设置到系统环境变量,您可以 ssh 到 master-1 节点,使用 impala-shell。登录方式详见:SSH连接主节点。
如果集群已经开启了 LDAP 认证,可以参看文档:impala-shell 来使用 impala-shell。
例如:
root@emr-4nw5w1f78f3l0xxx-master-1:~# impala-shell /opt/emr/2.0.0/impala-3.4.1/shell/impala-shell-3.4.1-RELEASE/ext-py2/bitarray-0.9.0-py2.7-linux-x86_64.egg/bitarray/_bitarray.py:3: UserWarning: Module backports was already imported from None, but /usr/lib/python2.7/dist-packages is being added to sys.path Starting Impala Shell without Kerberos authentication Opened TCP connection to emr-4nw5w1f78f3l0xxx-master-1:21000 Connected to emr-4nw5w1f78f3l0xxx-master-1:21000 Server version: impalad version 3.4.1-RELEASE RELEASE (build eb1ed66fa435a722fa8c6a7c58ff53edc10c572e) *********************************************************************************** Welcome to the Impala shell. (Impala Shell v3.4.1-RELEASE (eb1ed66) built on Fri Sep 16 15:03:38 CST 2022) Want to know what version of Impala you're connected to? Run the VERSION command to find out! *********************************************************************************** [emr-4nw5w1f78f3l0xxx-master-1:21000] default>
CREATE TABLE my_first_table ( id BIGINT, name STRING, PRIMARY KEY(id) ) PARTITION BY HASH PARTITIONS 16 STORED AS KUDU TBLPROPERTIES ('kudu.num_tablet_replicas' = '1');
例如:
[emr-4nw5w1f78f3l0xxx-master-1:21000] default> CREATE TABLE my_first_table > ( > id BIGINT, > name STRING, > PRIMARY KEY(id) > ) > PARTITION BY HASH PARTITIONS 16 > STORED AS KUDU > TBLPROPERTIES ('kudu.num_tablet_replicas' = '1'); Query: CREATE TABLE my_first_table ( id BIGINT, name STRING, PRIMARY KEY(id) ) PARTITION BY HASH PARTITIONS 16 STORED AS KUDU TBLPROPERTIES ('kudu.num_tablet_replicas' = '1') +-------------------------+ | summary | +-------------------------+ | Table has been created. | +-------------------------+ Fetched 1 row(s) in 7.00s
[emr-4nw5w1f78f3l0xxx-core-1:21000] default> insert into my_first_table values(1, 'Tom'); Query: insert into my_first_table values(1, 'Tom') Query submitted at: 2022-09-28 11:48:46 (Coordinator: http://emr-4nw5w1f78f3l0xxx-core-1:25000) Query progress can be monitored at: http://emr-4nw5w1f78f3l0xxx-core-1:25000/query_plan?query_id=f74dc7a765df77a3:15c929b200000000 Modified 1 row(s), 0 row error(s) in 5.76s
[emr-4nw5w1f78f3l0xxx-master-1:21000] default> select * from my_first_table; Query: select * from my_first_table Query submitted at: 2022-10-17 18:49:49 (Coordinator: http://emr-4nw5w1f78f3l0xxx-core-1:25000) Query progress can be monitored at: http://emr-4nw5w1f78f3l0xxx-core-1:25000/query_plan?query_id=7b44cc81e2374b76:0066f4bf00000000 +----+------+ | id | name | +----+------+ | 1 | Tom | +----+------+ Fetched 1 row(s) in 1.28s