Elastic安装配置说明

發表於 | 分類於 | | 閱讀次數:

Elastic下载安装

下载Elastic

1
2
3
[root@elonsu cloud]# wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.6.0.tar.gz
[root@elonsu cloud]# tar -zxvf elasticsearch-5.6.0.tar.gz 
[root@elonsu cloud]# mv elasticsearch-5.6.0 elastic

创建用户

Elastic默认不支持root用户启动, 这里创建用户dennisit

1
2
3
4
5
6
7
8
9
10
11
12
[root@elonsu cloud]# pwd
/export/cloud
[root@elonsu ~]# adduser dennisit
[root@elonsu ~]# passwd dennisit
Changing password for user dennisit.
New password: 
Retype new password: 
passwd: all authentication tokens updated successfully.
[root@elonsu cloud]# chown dennisit:dennisit elastic -R
[root@elonsu cloud]# su dennisit
[dennisit@elonsu cloud]$ cd elastic
[dennisit@elonsu elastic]$ bin/elasticsearch &

Elastic分词插件

中文分词插件

1
[dennisit@elonsu elastic]$ bin/elasticsearch-plugin install https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v6.0.0/elasticsearch-analysis-ik-6.0.0.zip

拼音分词插件

1
[dennisit@elonsu elastic]$ bin/elasticsearch-plugin install https://github.com/medcl/elasticsearch-analysis-pinyin/releases/download/v6.0.0/elasticsearch-analysis-pinyin-6.0.0.zip

简繁转换插件

1
[dennisit@elonsu elastic]$ bin/elasticsearch-plugin install https://github.com/medcl/elasticsearch-analysis-stconvert/releases/download/v6.0.0/elasticsearch-analysis-stconvert-6.0.0.zip

安装插件查看

1
2
3
4
[dennisit@elonsu elastic]$ bin/elasticsearch-plugin list
analysis-ik
analysis-pinyin
analysis-stconvert

Elastic插件-Head插件

插件地址: https://github.com/mobz/elasticsearch-head/
推荐安装: Chrome插件方式(简单省事)

Elastic插件-IK分词器

插件主页

https://github.com/medcl/elasticsearch-analysis-ik

安装IK分词器

1
2
3
4
5
6
7
[dennisit@elonsu cloud]$ cd elastic
[dennisit@elonsu elastic]$ bin/elasticsearch-plugin install https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v5.6.0/elasticsearch-analysis-ik-5.6.0.zip
-> Downloading https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v5.6.0/elasticsearch-analysis-ik-5.6.0.zip
[=================================================] 100%   
-> Installed analysis-ik
[dennisit@elonsu elastic]$ bin/elasticsearch &
[1] 33893

查看安装插件

1
2
3
4
[elon@icloud-store elasticsearch-6.0.0]$ bin/elasticsearch-plugin list
analysis-ik
analysis-pinyin
analysis-stconvert

验证IK分词器

创建index
1
2
3
[dennisit@elonsu elastic]$ curl -XPUT http://localhost:9200/index
[2017-09-26T14:08:12,782][INFO ][o.w.a.d.Monitor          ] try load config from /export/cloud/elastic/config/analysis-ik/IKAnalyzer.cfg.xml
[2017-09-26T14:08:13,243][INFO ][o.e.c.m.MetaDataCreateIndexService] [OWIX3y4] [index] creating index, cause [api], templates [], shards [5]/[1], mappings []
创建mapping
1
2
3
4
5
6
7
8
9
10
11
12
13
[dennisit@elonsu elastic]$ curl -XPOST http://localhost:9200/index/fulltext/_mapping -d'
> {
>         "properties": {
>             "content": {
>                 "type": "text",
>                 "analyzer": "ik_max_word",
>                 "search_analyzer": "ik_max_word"
>             }
>         }
>     
> }'
[2017-09-26T14:08:58,805][INFO ][o.e.c.m.MetaDataMappingService] [OWIX3y4] [index/ZNJu9Nz_R-qJKTDUnoIlxQ] create_mapping [fulltext]
{"acknowledged":true}
索引Document
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
[dennisit@elonsu elastic]$ curl -XPOST http://localhost:9200/index/fulltext/3?pretty -d  '{"content":"中韩渔警冲突调查：韩警平均每天扣1艘中国渔船"}'
{
  "_index" : "index",
  "_type" : "fulltext",
  "_id" : "3",
  "_version" : 1,
  "result" : "created",
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "created" : false
}

[dennisit@elonsu elastic]$ curl -XPOST http://localhost:9200/index/fulltext/4?pretty -d  '{"content":"中国驻洛杉矶领事馆遭亚裔男子枪击 嫌犯已自首"}'
{
  "_index" : "index",
  "_type" : "fulltext",
  "_id" : "4",
  "_version" : 1,
  "result" : "created",
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "created" : true
}

[dennisit@elonsu elastic]$ curl -XPOST http://localhost:9200/index/fulltext/5?pretty -d  '{"content":"10月1日是中国国庆节"}'
{
  "_index" : "index",
  "_type" : "fulltext",
  "_id" : "5",
  "_version" : 1,
  "result" : "created",
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "created" : true
}
查询Document
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
[dennisit@elonsu elastic]$ curl -XPOST http://localhost:9200/index/fulltext/_search?pretty  -d'
 {
     "query" : { "match" : { "content" : "中国" }},
     "highlight" : {
         "pre_tags" : ["<tag1>", "<tag2>"],
         "post_tags" : ["</tag1>", "</tag2>"],
         "fields" : {
             "content" : {}
         }
     }
 }
'
{
  "took" : 6,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 3,
    "max_score" : 0.27233246,
    "hits" : [
      {
        "_index" : "index",
        "_type" : "fulltext",
        "_id" : "5",
        "_score" : 0.27233246,
        "_source" : {
          "content" : "10月1日是中国国庆节"
        },
        "highlight" : {
          "content" : [
            "10月1日是<tag1>中国</tag1>国庆节"
          ]
        }
      },
      {
        "_index" : "index",
        "_type" : "fulltext",
        "_id" : "4",
        "_score" : 0.27179778,
        "_source" : {
          "content" : "中国驻洛杉矶领事馆遭亚裔男子枪击 嫌犯已自首"
        },
        "highlight" : {
          "content" : [
            "<tag1>中国</tag1>驻洛杉矶领事馆遭亚裔男子枪击 嫌犯已自首"
          ]
        }
      },
      {
        "_index" : "index",
        "_type" : "fulltext",
        "_id" : "3",
        "_score" : 0.27179778,
        "_source" : {
          "content" : "中韩渔警冲突调查：韩警平均每天扣1艘中国渔船"
        },
        "highlight" : {
          "content" : [
            "中韩渔警冲突调查：韩警平均每天扣1艘<tag1>中国</tag1>渔船"
          ]
        }
      }
    ]
  }
}
[dennisit@elonsu elastic]$
删除Document
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
[dennisit@elonsu elastic]$ curl -XDELETE  http://localhost:9200/index/fulltext/3?pretty
{
  "found" : true,
  "_index" : "index",
  "_type" : "fulltext",
  "_id" : "3",
  "_version" : 2,
  "result" : "deleted",
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  }
}
[dennisit@elonsu elastic]$ curl -XDELETE  http://localhost:9200/index/fulltext/5?pretty
{
  "found" : true,
  "_index" : "index",
  "_type" : "fulltext",
  "_id" : "5",
  "_version" : 2,
  "result" : "deleted",
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  }
}
[dennisit@elonsu elastic]$ curl -XDELETE  http://localhost:9200/index/fulltext/9?pretty
{
  "found" : false,
  "_index" : "index",
  "_type" : "fulltext",
  "_id" : "9",
  "_version" : 1,
  "result" : "not_found",
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  }
}
[dennisit@elonsu elastic]$ curl -XPOST http://localhost:9200/index/fulltext/_search?pretty  -d'
 {
     "query" : { "match" : { "content" : "中国" }},
     "highlight" : {
         "pre_tags" : ["<tag1>", "<tag2>"],
         "post_tags" : ["</tag1>", "</tag2>"],
         "fields" : {
             "content" : {}
         }
     }
 }
'
{
  "took" : 7,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 1,
    "max_score" : 0.27179778,
    "hits" : [
      {
        "_index" : "index",
        "_type" : "fulltext",
        "_id" : "4",
        "_score" : 0.27179778,
        "_source" : {
          "content" : "中国驻洛杉矶领事馆遭亚裔男子枪击 嫌犯已自首"
        },
        "highlight" : {
          "content" : [
            "<tag1>中国</tag1>驻洛杉矶领事馆遭亚裔男子枪击 嫌犯已自首"
          ]
        }
      }
    ]
  }
}
修改索引查询最大窗口
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
[root@localhost ~ ]$ curl http://localhost:9200/idx_index/_settings/?pretty
{
  "idx_index" : {
    "settings" : {
      "index" : {
        "creation_date" : "1529829607133",
        "number_of_shards" : "5",
        "number_of_replicas" : "1",
        "uuid" : "XtX1yMORQWqH2jihDqLomw",
        "version" : {
          "created" : "6020499"
        },
        "provided_name" : "idx_index"
      }
    }
  }
}
[root@localhost ~ ]$ curl -H "Content-Type: application/json" -XPUT http://localhost:9200/idx_index/_settings\?pretty -d '{ "index" : { "max_result_window" : 100000000}}'
{
  "acknowledged" : true
}
[root@localhost ~ ]$  curl http://localhost:9200/idx_index/_settings/\?pretty
{
  "idx_index" : {
    "settings" : {
      "index" : {
        "number_of_shards" : "5",
        "provided_name" : "idx_index",
        "max_result_window" : "100000000",
        "creation_date" : "1529829607133",
        "number_of_replicas" : "1",
        "uuid" : "XtX1yMORQWqH2jihDqLomw",
        "version" : {
          "created" : "6020499"
        }
      }
    }
  }
}

安装Nginx

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
[root@elonsu cloud]# wget http://nginx.org/download/nginx-1.12.1.tar.gz
[root@elonsu cloud]# tar -zxvf nginx-1.12.1.tar.gz 
[root@elonsu cloud]# cd nginx-1.12.1
[root@elonsu nginx-1.12.1]# ./configure \
  --prefix=/usr/local/nginx  \
  --sbin-path=/usr/local/nginx/sbin/nginx \
  --conf-path=/usr/local/nginx/conf/nginx.conf \
  --http-log-path=/usr/local/nginx/logs/access.log \
  --error-log-path=/usr/local/nginx/logs/error.log \
  --pid-path=/usr/local/nginx/logs/nginx.pid \
  --lock-path=/usr/local/nginx/lock/nginx.lock \
  --http-client-body-temp-path=/usr/local/nginx/client_body_temp \
  --http-proxy-temp-path=/usr/local/nginx/proxy_temp \
  --http-fastcgi-temp-path=/usr/local/nginx/fastcgi-temp \
  --http-uwsgi-temp-path=/usr/local/nginx/uwsgi-temp \
  --http-scgi-temp-path=/usr/local/nginx/scgi-temp \
  --user=root \
  --group=root \
  --with-http_ssl_module \
  --with-http_flv_module \
  --with-http_mp4_module \
  --with-http_gzip_static_module \
  --with-http_stub_status_module
[root@elonsu nginx-1.12.1]# make 
[root@elonsu nginx-1.12.1]# make install

配置nginx

上面我们安装了nginx在目录/usr/local/nginx/,接下来配置upstream

Nginx主配置文件修改

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
[root@elonsu conf]# pwd
/usr/local/nginx/conf
[root@elonsu conf]# cat nginx.conf

#user  nobody;
worker_processes  1;

#error_log  logs/error.log;
#error_log  logs/error.log  notice;
#error_log  logs/error.log  info;

#pid        logs/nginx.pid;

events {
    worker_connections  1024;
}

http {
    include       mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  logs/access.log  main;  

    sendfile        on;
    
    keepalive_timeout  65;

    gzip  on;
    
    server_names_hash_bucket_size 64;
    server_names_hash_max_size 512;
    
    include domains/*;    

    server {
        listen       80;
        server_name  localhost;
        
        location / {
            root   html;
            index  index.html index.htm;
        }
        
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }
       
    }
}

该配置文件我们做了最简单的配置, 在nginx默认的配置上做了如下修改:

  • 打开log_format
  • 增加server_names_hash相关配置
  • 使用include domains/*; 加载我们自定义upstream配置

自定义Upstream配置

这里我们对elastic进行代理

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
[root@elonsu domains]# pwd
/usr/local/nginx/conf/domains
[root@elonsu domains]# cat elastic.icloud.com.conf 
upstream domain.elastic.local {
                server 127.0.0.1:9200  weight=10 max_fails=2 fail_timeout=300s;
}
server {
                listen                   80;
                server_name              elastic.icloud.com;
                root                     /export/cloud/elastic/;
                access_log               /export/cloud/elastic/logs/elastic.icloud.com_access.log main;
                error_log                /export/cloud/elastic/logs/elastic.icloud.com_error.log warn;
                error_page              403 404 /40x.html;

                location / {
                        index index.html index.htm;
                        proxy_next_upstream     http_500 http_502 http_503 http_504 error timeout invalid_header;
                        proxy_set_header        Host  $host;
                        proxy_set_header        X-Real-IP        $remote_addr;
                        proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
                        proxy_pass              http://domain.elastic.local;
                        expires                 0d;
                }
}

配置完启动nginx

Nginx的启停脚本

启动Nginx并访问

1
2
3
4
5
6
7
8
9
10
11
12
13
14
[root@elonsu domains]# service nginx restart
Stopping Nginx:                                            [  OK  ]
Starting Nginx:                                            [  OK  ]
[root@elonsu domains]# curl -XGET  http://127.0.0.1:9200/index/fulltext/4\?pretty
{
  "_index" : "index",
  "_type" : "fulltext",
  "_id" : "4",
  "_version" : 1,
  "found" : true,
  "_source" : {
    "content" : "中国驻洛杉矶领事馆遭亚裔男子枪击 嫌犯已自首"
  }
}

本机使用外网域名访问

上面服务器上Elastic的反向代理已经配置好, 本机使用域名访问(因为域名非公网域名,所以本机需要配置host)

1
2
3
4
5
6
7
8
9
10
11
[root@localhost ~ ]$ curl  http://elastic.icloud.com/index/fulltext/4\?pretty
{
  "_index" : "index",
  "_type" : "fulltext",
  "_id" : "4",
  "_version" : 1,
  "found" : true,
  "_source" : {
    "content" : "中国驻洛杉矶领事馆遭亚裔男子枪击 嫌犯已自首"
  }
}