Optimizing use the QueryCache

[kkewwei]

Description

In my use-case, I discover the utilization percentage of QueryCache(with a capacity of 3GB and only 50MB used) is extremely low. Most of the queries are as follows:

POST index1/_search
{
   "size": 300,
   "query": {
      "bool": {
         "filter": [
            {
               "terms": {
                  "user_type": [0, 1, 3, 5, 4, 6]
               }
            }
         ],
         "should": [
            {
               "match": {
                  "name_ik": {
                     "query": "ab cd ed gh",
                     "operator": "OR",
                     "analyzer": "ik_max_word",
                  }
               }
            }
         ]
      }
   }
}

• should-match condition will match over 500 documents, and the query value keeps changing, due to the should clause, it won't be cached by QueryCache.
• filter-terms will match over 100,000,000 documents, the user_type has several fixed values, it will also not be cached because of the skipCacheFactor(100,000,000 / skipCacheFactor > 500).
  

  lucene/lucene/core/src/java/org/apache/lucene/search/LRUQueryCache.java

  Line 773 in 067b472

  if (cost / skipCacheFactor > leadCost) {

There seem to be several points for optimization:

• When the utilization percentage of QueryCache is not full utilized, can we loosen the restrictions to cache more queries that don't meet the current conditions?
• User could be allowed to dynamically modify the skip_factor of QueryCache. Alternatively, this parameter could be deprecated, If a query meets minFrequencyToCache, meaning it is frequent, it should be placed into QueryCache.

Furthermore, if user were able to dynamically adjust maxSize, maxRamBytesUsed in QueryCache, considering that the overhead of such adjustments is rather small.

[kkewwei]

@jpountz, please help confirm when you are free, really appreciate feedback from you.

[msokolov]

I believe query cache is highly customizable: QueryCache is just an interface; you can implement whatever you want. And the main impl LRUQueryCache has many extension points, esp. including QueryCachingPolicy. Have you tried to implement what you want using these tools?

As a side note, it's generally not effective or appreciated to call out individuals in public here or on the mailing list unless there is some ongoing dialog. Not sure if Adrien is already engaged with you on this?

[kkewwei]

Thank you very much for your reminder. I apologize for my offense and I will not do it again next time.

It seems that the QueryCachingPolicy fails to fulfill my all requirements:

• It supports method: shouldCache, which can relax the requirements when there is a large amount of cache space available.
• And I also hope that the cache size can be adjusted dynamically, considering that the overhead of such adjustments is rather small for QueryCache.
• If a query meets minFrequencyToCache, meaning it is high frequent, it should be placed into QueryCache. As importing skipCacheFactor to avoid slow query down too much, we just increase the minFrequencyToCache to solve the problem.

[kkewwei]

In our product, we change skip_factor from 10->10,000, as a result, p99 of search time is reduced: 300ms->140ms, meanwhile, the usage of querycache is expanded: 200mb->2gb(total 3g)

The p99 of search time:

We experimented with changing the value of skip_factor in three clusters where the usage of querycache is relatively low, remarkably, query performance has been improved by more than 50%. Based on these outcomes, skip_factor seems worthwhile to be changed/dynamic.

[sgup432]

@kkewwei Great to see the improvement by changing the skip factor.

I see there have been many discussions around finding the right value for skip_factor (here and here), mostly the concern being that caching large query clauses(as per their cost and eventually size) may not make much sense, considering they will make the overall query slow due to cache overhead(locks, eviction).

As per your experiment, I see query cache was heavily underutilized and increasing skip factor caused more queries to be cached. However, the cache didn't reach the limit(3gb) and I assume there were no evictions. I wonder what happens when you start observing more evictions? I suspect the overall gains you got by increasing skip_factor would eventually fade away due to the overhead(caused by evictions/locks). Have you started noticing any evictions from the cache yet?

But I certainly believe we should have the ability to set skip_factor dynamically within cache by having a simple setter method? Looks like a harmless change to me.

it will also not be cached because of the skipCacheFactor(100,000,000 / skipCacheFactor > 500).

I believe you meant 500 / skipCacheFactor > 100,000,000 and not the other way around.