ES-密码设置及JAVA应用

一、开启密码验证

（一）开启密码验证

　　1、安装x-pack

　　在ES6.3之前添加用户密码验证需要安装x-pack插件，在6.3之后被去掉，可以直接设置。

　　在es的安装目录中，执行：

./bin/elasticsearch-plugin install x-pack

　　2、设置密码

　　在安装x-pack后，es的bin目录中会产生一个x-pack目录，进入该目录，进行密码设置

./setup-passwords interactive

　　该命令会为es、logstash、kibana同时设置密码，需要手动输入。

　　　　　　

 　　3、设置es的配置文件

http.cors.enabled: true
http.cors.allow-origin: '*'
http.cors.allow-headers: Authorization,X-Requested-With,Content-Length,Content-Type

　　4、测试

　　不使用用户名密码访问：curl 172.20.10.6:9200，返回信息状态为401，无权限操作。

　　使用用户名密码访问：curl 172.20.10.6:9200 -u elastic:espass，访问状态为成功

　　5、修改用户密码：curl -H "Content-Type:application/json" -XPOST -u elastic 'http://127.0.0.1:9200/_xpack/security/user/elastic/_password' -d '{ "password" : "123456" }'

　　6、添加用户

　　添加用户接口为：POST/_xpack/security/user/,下面以添加一个test用户并添加至admin角色为例:

　　　　

 　　7、重置超级管理员密码

　　先停止es，然后使用下述命令创建一个本地认证的超级用户：

bin/x-pack/users useradd my_admin -p my_password -r superuser

　　然后通过api设置

curl -u my_admin -XPUT 'http://localhost:9200/_xpack/security/user/elastic/_password?pretty' -H 'Content-Type: application/json' -d'
{
  "password" : "new_password"
}

　　验证：curl -u elastic:123456 'http://172.20.10.6:9200/_xpack/security/_authenticate?pretty'

（二）ES-HEAD和Kibana使用密码访问

　　上面对ES做了密码设置后，访问ES-HEAD时，就需要加上用户名密码进行访问：http://172.20.10.6:9100/?auth_user=elastic&auth_password=123456

　　对于kibana的设置，需要修改kibana的配置文件，添加访问es的用户名密码

elasticsearch.username: "kibana"
elasticsearch.password: "123456"

二、JAVA API

　　ES的JAVA API很多，这里介绍一个Java High Level REST Client，每个版本变化较大，这里以6.3.2来演示。

（一）客户端配置

　　pom文件：

<dependency>
            <groupId>org.elasticsearch</groupId>
            <artifactId>elasticsearch</artifactId>
            <version>${elasticsearch.version}</version>
        </dependency>
        <dependency>
            <groupId>org.elasticsearch.client</groupId>
            <artifactId>transport</artifactId>
            <version>${elasticsearch.version}</version>
        </dependency>
        <dependency>
            <groupId>org.elasticsearch.client</groupId>
            <artifactId>elasticsearch-rest-client</artifactId>
            <version>${elasticsearch.version}</version>
        </dependency>
        <dependency>
            <groupId>org.elasticsearch.plugin</groupId>
            <artifactId>transport-netty4-client</artifactId>
            <version>${elasticsearch.version}</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.data</groupId>
            <artifactId>spring-data-elasticsearch</artifactId>
            <version>${spring.data.elasticsearch.version}</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-elasticsearch</artifactId>
        </dependency>

        <!--  es升级需要依赖的 -->
        <dependency>
            <groupId>org.elasticsearch.client</groupId>
            <artifactId>elasticsearch-rest-high-level-client</artifactId>
            <version>${elasticsearch.version}</version>
        </dependency>

　　配置文件

elasticsearch:
  clientIps: 172.20.10.6 # ES网关地址，实际地址在ES实例概览中查看，注意至少配置两个网关地址
  httpPort: 9200  # ES网关端口
  username: elastic
  password: 123456

　　配置客户端：下面提供了使用用户名密码和不使用用户名密码两个Bean，可以任选其一

@Configuration
public class ElasticSearchClientConfig {
    @Value("${elasticsearch.clientIps}")
    private String clientIps;

    @Value("${elasticsearch.httpPort}")
    private int httpPort;

    @Value("${elasticsearch.username}")
    private String username;

    @Value("${elasticsearch.password}")
    private String password;

    private HttpHost[] getHttpHosts(String clientIps, int esHttpPort) {
        String[] clientIpList = clientIps.split(",");
        HttpHost[] httpHosts = new HttpHost[clientIpList.length];
        for (int i = 0; i < clientIpList.length; i++) {
            httpHosts[i] = new HttpHost(clientIpList[i], esHttpPort, "http");
        }
        return httpHosts;
    }

    /**
     * 创建带HTTP Basic Auth认证rest客户端
     */
    @Bean
    public RestHighLevelClient restHighLevelClient() {
        CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
        credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(username, password));
        return new RestHighLevelClient(RestClient.builder(getHttpHosts(clientIps, httpPort)).setHttpClientConfigCallback((HttpAsyncClientBuilder httpAsyncClientBuilder) -> httpAsyncClientBuilder.setDefaultCredentialsProvider(credentialsProvider)));
    }

    //不带用户名密码验证
    //@Bean
    public RestHighLevelClient restClient() {
        return new RestHighLevelClient(RestClient.builder(getHttpHosts(clientIps, httpPort)));
    }
}

（二）索引操作

　　添加索引

@Test
    public void creatIndex() throws Exception{

        Map<String, Object> properties = new HashMap<>();
        Map<String, Object> orderIdIndex = new HashMap<>();
        orderIdIndex.put("type", "long");
        orderIdIndex.put("store", true);
        orderIdIndex.put("index", true);
        properties.put("orderId", orderIdIndex);

        Map<String, Object> orderNameIndex = new HashMap<>();
        orderNameIndex.put("type", "text");
        orderNameIndex.put("store", true);
        orderNameIndex.put("index", true);
        properties.put("orderName", orderNameIndex);

        Map<String, Object> content = new HashMap<>();
        content.put("type", "text");
        content.put("store", true);
        content.put("index", true);
        properties.put("content", content);

        Map<String, Object> mapping = new HashMap<>();
        mapping.put("properties", properties);

        CreateIndexRequest request = new CreateIndexRequest(INDEX);
        request.settings(Settings.builder().put("index.number_of_shards", 3).put("index.number_of_replicas", 2).build());
        request.mapping(TYPE, new Gson().toJson(mapping), XContentType.JSON);

        //request.mapping(mapping);
        CreateIndexResponse response = restHighLevelClient.indices().create(request);
        log.info("response====================={}", new Gson().toJson(response));
    }

　　删除索引

    @Test
    public void deleteIndex() throws Exception{
        DeleteIndexRequest request = new DeleteIndexRequest(INDEX);
        DeleteIndexResponse response = restHighLevelClient.indices().delete(request);
        log.info("response====================={}", new Gson().toJson(response));
    }

（三）数据操作

　　1、添加数据，添加索引可以使用Json格式、Map格式、XContentBuilder格式等，具体如下所示：

@Test
    public void saveDataByJson() throws Exception{
        OrderInfo orderInfo = OrderInfo.builder().orderId(12345).orderName("测试12345的订单").content("的撒出手大方【平时都i法第四u发").build();
        IndexRequest request = new IndexRequest(INDEX, TYPE, String.valueOf(orderInfo.getOrderId()));
        request.source(new Gson().toJson(orderInfo), XContentType.JSON);
        this.save(request);
    }

    @Test
    public void saveDataBMap() throws Exception{
        Map<String, Object> jsonMap = new HashMap<>();
        jsonMap.put("orderId", 11111);
        jsonMap.put("orderName", "1111的订单");
        jsonMap.put("content", "破欸俗人伟大撒打发的");
        IndexRequest request = new IndexRequest(INDEX,TYPE, jsonMap.get("orderId").toString()).source(jsonMap);
        this.save(request);
    }

    @Test
    public void saveDataByContentBuild() throws Exception{
        XContentBuilder builder = XContentFactory.jsonBuilder();
        builder.startObject();
        builder.field("orderId",2222);
        builder.field("orderName", "2222的订单");
        builder.field("content", "送iu恶意然后我i和拉票；萨拉宽度");
        builder.endObject();
        IndexRequest request = new IndexRequest(INDEX, TYPE, "2222").source(builder);
        this.save(request);
    }

    @Test
    public void saveDataByObject() throws Exception{
        XContentBuilder builder = XContentFactory.jsonBuilder();
        builder.startObject();
        builder.field("orderId",2222);
        builder.field("orderName", "2222的订单");
        builder.field("content", "送iu恶意然后我i和拉票；萨拉宽度");
        builder.endObject();
        IndexRequest request = new IndexRequest(INDEX, TYPE, "3333").source(
                "orderId",3333,"orderName", "3333的订单","content", "laisuyeiluqwgkjhgkjsgd"
        );
        this.save(request);
    }
    private boolean save(IndexRequest request) throws Exception{
        IndexResponse response = restHighLevelClient.index(request);
        log.info("response====================={}", new Gson().toJson(response));
        if (response.status().getStatus() == 200) {
            return true;
        }
        return false;
    }

　　2、删除数据

    @Test
    public void deleteData() throws Exception{
        DeleteRequest request = new DeleteRequest(INDEX,TYPE,"2222");
        DeleteResponse response = restHighLevelClient.delete(request);
        RestStatus restStatus = response.status();
    }

　　3、获取数据

    @Test
    public void getIncludeData() throws Exception{
        GetRequest request = new GetRequest(INDEX, TYPE, "11111");
        String[] includes = new String[]{"content", "修改"};
        String[] excludes = Strings.EMPTY_ARRAY;
        FetchSourceContext fetchSourceContext = new FetchSourceContext(true, includes, excludes);
        request.fetchSourceContext(fetchSourceContext);
        this.getData(request);
    }

    private void getData(GetRequest request) throws Exception{
        GetResponse response = restHighLevelClient.get(request);
        log.info("response====================={}", new Gson().toJson(response));
        Map<String, Object> source = response.getSource();
        log.info("source====================={}", new Gson().toJson(source));
    }

　　4、更新数据

    @Test
    public void updateByMapping() throws Exception{
        Map<String, Object> jsonMap = new HashMap<>();
        jsonMap.put("orderId", 11111);
        jsonMap.put("orderName", "修改========1111的订单");
        jsonMap.put("content", "修改==============破欸俗人伟大撒打发的");
        UpdateRequest request = new UpdateRequest(INDEX,TYPE, jsonMap.get("orderId").toString()).doc(jsonMap);
        this.update(request);
    }


    private boolean update(UpdateRequest request) throws Exception{
        UpdateResponse response = restHighLevelClient.update(request);
        RestStatus restStatus = response.status();
        log.info("response====================={}", new Gson().toJson(response));
        if (response.status().getStatus() == 200) {
            return true;
        }
        return false;
    }

　　5、批量添加

    @Test
    public void batch() throws Exception{
        BulkRequest request = new BulkRequest();
        request.add(new IndexRequest(INDEX, TYPE, "5678").source(
                new Gson().toJson(OrderInfo.builder().orderId(5678).orderName("5678的订单").content("我饿请问一日").build()),
                XContentType.JSON));
        request.add(new IndexRequest(INDEX, TYPE, "56781").source(
                new Gson().toJson(OrderInfo.builder().orderId(56781).orderName("56781的订单").content("我饿请问一日").build()),
                XContentType.JSON));
        request.add(new IndexRequest(INDEX, TYPE, "56782").source(
                new Gson().toJson(OrderInfo.builder().orderId(56782).orderName("56782的订单").content("我饿请问一日").build()),
                XContentType.JSON));
        request.add(new IndexRequest(INDEX, TYPE, "56783").source(
                new Gson().toJson(OrderInfo.builder().orderId(56783).orderName("56783的订单").content("我饿请问一日").build()),
                XContentType.JSON));

        BulkResponse response = restHighLevelClient.bulk(request);
        log.info("response====================={}", new Gson().toJson(response));
    }

　　6、数据查询

@Test
    public void searchAll() throws Exception{
        SearchRequest request = new SearchRequest(INDEX);
        request.types(TYPE);
        //SearchRequest request = new SearchRequest();
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
        searchSourceBuilder.query(QueryBuilders.matchAllQuery());
        request.source(searchSourceBuilder);
        this.search(request);
    }

    @Test
    public void searchPage() throws Exception{
        SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
        sourceBuilder.query(QueryBuilders.termQuery("orderId", "5678"));
        //sourceBuilder.from(2);
        //sourceBuilder.size(5);
        SearchRequest request = new SearchRequest(INDEX);
        request.source(sourceBuilder);
        this.search(request);
    }


    @Test
    public void matchQuery() throws Exception{
        /*QueryBuilder matchQueryBuilder = QueryBuilders.matchQuery("orderId", "5678")
                .fuzziness(Fuzziness.AUTO)
                .prefixLength(3)
                .maxExpansions(10);*/
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
        //searchSourceBuilder.query(matchQueryBuilder);
        //排序
        //searchSourceBuilder.sort(new ScoreSortBuilder().order(SortOrder.DESC));
        searchSourceBuilder.sort(new FieldSortBuilder("orderId").order(SortOrder.ASC));

        //过滤字段
        String[] includeFields = new String[] {"orderId", "orderName", "content"};
        String[] excludeFields = new String[] {"_type"};
        searchSourceBuilder.fetchSource(includeFields, excludeFields);

        //添加高亮
        highlightBuilder = new HighlightBuilder();
        HighlightBuilder.Field highligthName = new HighlightBuilder.Field("orderName"); //设置高亮字段
        highligthName.highlighterType("unified");
        highlightBuilder.field(highligthName);

        HighlightBuilder.Field highligthContent = new HighlightBuilder.Field("content"); //设置高亮字段
        highligthName.highlighterType("unified");
        highlightBuilder.field(highligthContent);
        SearchRequest request = new SearchRequest(INDEX);
        request.types(TYPE);
        request.source(searchSourceBuilder);
        this.search(request);
    }



    private void search(SearchRequest request) throws Exception{
        SearchResponse response = restHighLevelClient.search(request);
        log.info("response====================={}", new Gson().toJson(response));
        //关于请求执行本身的有用信息，如HTTP状态代码、执行时间或请求是否提前终止或超时：
        RestStatus status = response.status();
        TimeValue took = response.getTook();
        Boolean terminatedEarly = response.isTerminatedEarly();
        boolean timedOut = response.isTimedOut();
        log.info("status:{}=======took:{}====terminatedEarly:{}=========timedOut:{}",status,took,terminatedEarly,timedOut);
        //有关碎片级执行的信息，提供了有关受搜索影响的碎片总数以及成功碎片和不成功碎片的统计信息
        int totalShards = response.getTotalShards();
        int successfulShards = response.getSuccessfulShards();
        int failedShards = response.getFailedShards();
        log.info("totalShards:{}=======successfulShards:{}====failedShards:{}",totalShards,successfulShards,failedShards);
        for (ShardSearchFailure failure : response.getShardFailures()) {
            log.info("failure==============={}", new Gson().toJson(failure));
        }
        //将返回的数据返回到SearchHits对象中
        SearchHits hits = response.getHits();
        //SearchHits包含了总点击次数、最大得分等信息：
        long totalHits = hits.getTotalHits();
        float maxScore = hits.getMaxScore();
        log.info("totalHits:{}=======maxScore:{}",totalHits,maxScore);
        //对SearchHits进行遍历得到单个的SearchHit对象，SearchHit包含索引、类型、数据id和数据的得分情况
        SearchHit[] searchHits = hits.getHits();
        for (SearchHit hit : searchHits) {
            String index = hit.getIndex();
            String type = hit.getType();
            String id = hit.getId();
            float score = hit.getScore();
            log.info("index:{}=======type:{}=======id:{}=======score:{}",index,type,id,score);
            //可以将数据JSON字符串或键/值对映射的形式返回。
            String sourceAsString = hit.getSourceAsString();
            Map<String, Object> sourceAsMap = hit.getSourceAsMap();
            String documentTitle = (String) sourceAsMap.get("title");
            List<Object> users = (List<Object>) sourceAsMap.get("name");
            Map<String, Object> innerObject = (Map<String, Object>) sourceAsMap.get("innerObject");
            log.info("sourceAsString:{}=======documentTitle:{}=======users:{}",sourceAsString,documentTitle,new Gson().toJson(users));
            //设置高亮
            Map<String, HighlightField> highlightFields = hit.getHighlightFields();
            //获取title高亮显示
            HighlightField highlight = highlightFields.get("title");
            if(highlight != null){
                //获取高亮显示的字段
                Text[] fragments = highlight.fragments();
                String fragmentString = fragments[0].string();
                log.info("fragments:{}=======fragmentString:{}",fragments,fragmentString);
            }
        }
    }