原创

elasticsearch使用Filter过滤查询操作(使用marvel插件)

1.filter查询语句

1)通过filter可以获取想要的数据

2)通过filter查询有缓存,大大提高查询效率,所以推荐使用filter查询


2.cache缓存


3.实例数据,准备数据:

POST /store/products/_bulk
{"index":{"_id":1}}
{"price":10,"productID":"SD1002136"}
{"index":{"_id":2}}
{"price":20,"productID":"SD2678421"}
{"index":{"_id":3}}
{"price":30,"productID":"SD8897573"}
{"index":{"_id":4}}
{"price":40,"productID":"SD4535233"}

4.查看测试数据

GET /store/products/_mget
{
  "ids":[1,2,3,4]
}

5.filter查询

最简单的filter查询

先看看sql语句: select document from products where price = 20

filter的语句

GET /store/products/_search
{
  "query": {
    "filtered": {
      "query": {
        "match_all": {}
      },
      "filter": {
        "term": {
          "price": "20"
        }
      }
    }
  }
}

也可以指定多个值

GET /store/products/_search
{
  "query": {
    "filtered": {
      "query": {
        "match_all": {}
      },
      "filter": {
        "terms": {
          "price": [
            "10",
            "20"
          ]
        }
      }
    }
  }
}

再看另一个  select * from products where productId = "SD4535233"

GET /store/products/_search
{
  "query": {
    "filtered": {
      "query": {
        "match_all": {}
      },
      "filter": {
        "term": {
          "productID": "sd1002136"
        }
      }
    }
  }
}

注意:这里的大写查不到

对字段进行分析:

查看字段的分析结果,可以看到分析结果是小写的,所以上面的大写查找不到

GET /_analyze?text=SD4535233

结果如下:

{
   "tokens": [
      {
         "token": "sd4535233",
         "start_offset": 0,
         "end_offset": 9,
         "type": "<ALPHANUM>",
         "position": 1
      }
   ]
}

如果想要大写也可以搜索到,那么需要对该字段的mapping设置为not_analyzed,不进行分析器分析

DELETE /store

PUT /store
{
  "mappings": {
    "products":{
      "properties": {
        "productID":{
          "type":"string",
          "index":"not_analyzed"
        }
      }
    }
  }
}

6.bool过滤查询,可以做组合过滤查询

SELECT product FROM products where (price = 20 OR productID  = 'SD1002136' ) AND (price != 30)

查询价格等于20的或者productID 为SD1002136的商品,排除价格30元的

类似的,elasticsearch也有and,or,not这样的组合条件查询方式

格式如下:

{
    "bool":{
        "must":[],
        "should":[],
        "must_not":[]

    }
}

must : 条件必须满足,相当于and 

should:条件可以满足也可以不满足,相当于or

must_not:条件不需要满足,相当于not

示例:

GET /store/products/_search
{
  "query":{
    "filtered": {
      "query": {
        "match_all": {}
      },
      "filter": {
        "bool": {
          "should": [
            {"term": {
              "price": "20"
            }},
            {"term": {
              "productID": "SD2678421"
            }}
          ],
          "must_not": [
            {"term": {
              "price": "30"
            }}
          ]
        }
      }
    }
  }
}

7.嵌套查询

1)示例sql:select * from products where productID = "SD1002136" OR (productID = "SD4535233" AND price = 30)

示例查询:

GET /store/products/_search
{
  "query": {
    "filtered": {
      "query": {
        "match_all": {}
      },
      "filter": {
        "bool": {
          "should": [
            {"term": {
              "productID": "sd1002136"
            }},
            {"bool": {
              "must": [
                {
                  "term": {
                    "productID": "sd4535233"
                  }
                },
                {
                  "term": {
                    "price": "30"
                  }
                }
              ]
            }}
          ]
        }
      }
    }
  }

2) 另外一种:and , or , not查询

没有bool,直接使用and ,or ,not

示例,and的:

GET /store/products/_search
{
  "query": {
    "filtered": {
      "query": {
        "match_all": {}
      },
      "filter": {
        "and":[
          {
            "term":{
              "price" : 10
            }
          },
          {
            "term":{
              "productID":"sd1002136"
            }
          }
        ]
      }
    }
  }
}

示例,or的:

GET /store/products/_search
{
  "query": {
    "filtered": {
      "query": {
        "match_all": {}
      },
      "filter": {
        "or":[
          {
            "term":{
              "price" : 10
            }
          },
          {
            "term":{
              "productID":"sd4535233"
            }
          }
        ]
      }
    }
  }
}

not示例:

GET /store/products/_search
{
  "query": {
    "filtered": {
      "query": {
        "match_all": {}
      },
      "filter": {
        "not":{
          "term":{
            "price" : 10
          }
        }
      }
    }
  }
}

 8.range范围查询

示例sql:   select * from products where price between 20 and 40

gt > 大于

lt < 小于

gte >= 大于等于

lte <= 小于等于

示例:

GET /store/products/_search
{
  "query": {
    "filtered": {
      "query": {"match_all": {}},
      "filter": {
        "range": {
          "price": {
            "gte": 10,
            "lte": 20
          }
        }
      }
    }
  }
}

9.过滤空和非空

1)建立测试数据

POST /test_index/test/_bulk
{"index":{"_id":1}}
{"tags":["search"]}
{"index":{"_id":2}}
{"tags":["search","open_source"]}
{"index":{"_id":3}}
{"other_field":"some data"}
{"index":{"_id":4}}
{"tags":null}
{"index":{"_id":5}}
{"tags":["search",null]}

2)处理null空值的方法

select tags from test where tags is not null

select tags from test where tags is null

示例:

is not null

GET /test_index/test/_search
{
  "query": {
    "filtered": {
      "query": {"match_all": {}},
      "filter": {
        "exists": {
          "field": "tags"
        }
      }
    }
  }
}

is null

GET /test_index/test/_search
{
  "query": {
    "filtered": {
      "query": {"match_all": {}},
      "filter": {
        "missing": {
          "field": "tags"
        }
      }
    }
  }
}



10.cache缓存

elasticsearch在执行带有filter查询时,会打开索引的每个segment文件(lucene底层文件),然后去判断里面的文档是否符合filter要求。

注意:旧的segment文件不会变,新来的数据会产生新的segment。

匹配的结果会用一个大型的BigSet数组来存储,这个数组的值只有0和1

匹配:1

不匹配:0

BigSet值是存在内存里的,而不是硬盘里,所以速度快!

开启方式:在filter查询语句后面加"_cache":true

注意:

Script filters,Geo-filters,Date ranges这样的过滤方式开启cache毫无意义

exists,missing,range,term和terms查询时默认开启cache的

示例:

GET /store/products/_search
{
  "query":{
    "filtered": {
      "query": {
        "match_all": {}
      },
      "filter": {
        "bool": {
          "should": [
            {"term": {
              "price": "20"
            }},
            {"term": {
              "productID": "SD2678421"
            }}
          ],"_cache":true,
          "must_not": [
            {"term": {
              "price": "30"
            }}
          ]
        }
      }
    }
  }
}


正文到此结束
本文目录