使用ES聚合查询Kong日志
目录
ES版本
- ES 7.10
配置Kong日志写入ES
|
1 |
curl -X POST localhost:8001/plugins/ --data "name=http-log" --data "config.http_endpoint=http://localhost:9200/foundry-bot/_doc/" --data "config.method=POST" --data "config.timeout=1000" --data "config.keepalive=1000" --data "config.flush_timeout=2" --data "config.retry_count=15" |
指定字段更改mapping
|
1 2 3 4 5 6 7 8 9 |
POST /test/_mapping { "properties": { "route.paths": { "type": "text", "fielddata": true } } } |
检索并聚合日志
- 根据指定字段，检索日志
- 聚合查询
- 基于时间维度，聚合 路由，统计状态码的数量
- 基于时间维度，聚合路由，统计Pxx
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 |
POST /foundry-bot/_search { "size": 0, "query": { "bool": { "should": [ {"term": {"request.headers.project": "sfsdf"}}, {"term": {"request.querystring._project": "sfsdf"}} ], "filter": [ {"range": {"timestamp": {"gte": "now-1h"}}} ], "minimum_should_match" : 1 } }, "aggs": { "aggs_all_time": { "date_histogram": { "field": "timestamp", "fixed_interval": "5m", "time_zone": "+08:00", "format": "yyyy-MM-dd H:m:ss" } }, "aggs_by_route_and_status": { "date_histogram": { "field": "timestamp", "fixed_interval": "5m", "time_zone": "+08:00", "format": "yyyy-MM-dd H:m:ss" }, "aggs": { "aggs_route": { "terms": { "field": "route.paths.keyword", "size": 100}, "aggs": { "aggs_status": { "terms": { "field": "response.status", "size": 100} } } } } }, "aggs_by_route_and_latencies": { "date_histogram": { "field": "timestamp", "fixed_interval": "5m", "time_zone": "+08:00", "format": "yyyy-MM-dd H:m:ss" }, "aggs": { "aggs_route": { "terms": { "field": "route.paths.keyword", "size": 100}, "aggs": { "pxx": { "percentiles": { "field": "latencies.request", "percents": [ 95, 99, 99.9 ]} } } } } } } } |