elasticsearch|elasticsearch for hadoop

CREATE EXTERNAL TABLE `elasticsearch.t_elastic_xxxx`( `wayuid` string COMMENT 'from deserializer', `rpt_date` string COMMENT 'from deserializer', `start_org_code` string COMMENT 'from deserializer', `start_org_name` string COMMENT 'from deserializer', `customer_code` string COMMENT 'from deserializer', `customer_name` string COMMENT 'from deserializer', `waybill_no` string COMMENT 'from deserializer', `order_channel_type` string COMMENT 'from deserializer', `loading_time` string COMMENT 'from deserializer', `seller_id` string COMMENT 'from deserializer', `seller_name` string COMMENT 'from deserializer') ROW FORMAT SERDE 'org.elasticsearch.hadoop.hive.EsSerDe' STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler' WITH SERDEPROPERTIES ( 'serialization.format'='1') LOCATION 'hdfs://xxxxxx/t_fengchao_test' TBLPROPERTIES ( 'COLUMN_STATS_ACCURATE'='false', 'es.batch.size.bytes'='100000000', 'es.batch.size.entries'='100000', 'es.batch.write.refresh'='false', 'es.index.auto.create'='false', 'es.mapping.id'='wayuid', 'es.net.http.auth.pass'='changeme', 'es.net.http.auth.user'='elastic', 'es.nodes'='xxx.xxx.xxx.xxx', 'es.nodes.wan.only'='true', 'es.port'='9200', 'es.resource'='order_no_sign-2020.11.01-1/t_order_no_sign', 'es.write.operation'='upsert')

如果ES使用x-pack后,需要增加
'es.net.http.auth.pass'='changeme',
'es.net.http.auth.user'='elastic',
参数进行权限验证。
【elasticsearch|elasticsearch for hadoop】如果设置为upset,请注意同一批次内_id一定不要有重复数据。

    推荐阅读