Aggregations

An aggregation provides a way to summarize your data through metrics, statistics, or other forms of analysis. Aggregations enable you to answer questions such as:

• What is the average number of connections of this endpoint?

• Which employees generate the greatest number of login fails?

• What qualifies as a large data transfer on my network?

• How many network services have presence in the customer network?

The Search API categorizes aggregations into two main types:

• Metric Aggregations: Calculate numerical metrics like totals, averages, or other values from specific fields.

• Bucket Aggregations: Group documents into buckets (or bins) based on criteria like field values, ranges, or other conditions.

How to run an aggregation

Aggregations can be executed as part of a search by using the aggs parameter in the search API. For example, the following search performs a terms aggregation on the field field:

GET /datalakeapi/customerlog-socname-customername-cef-*
{
  "aggs": {
    "your_aggregation_name": {
      "terms": {
        "field": "deviceAction"
      }
    }
  }
}

The results of aggregations are included in the aggregations object within the response:

GET /datalakeapi/customerlog-socname-customername-cef-*
{
  "aggregations": {
    "your_aggregation_name": {                           
      "doc_count_error_upper_bound": 0,
      "sum_other_doc_count": 0,
      "buckets": []
    }
  }
}

Set the aggregation scope

Use the query parameter to restrict the documents processed by an aggregation:

GET /datalakeapi/customerlog-socname-customername-cef-*
{
  "query": {
    "range": {
      "@timestamp": {
        "gte": "now-1d/d",
        "lt": "now/d"
      }
    }
  },
  "aggs": {
    "your_aggregation_name": {
      "terms": {
        "field": "deviceAction"
      }
    }
  }
}

Return only aggregation results

By default, searches with aggregations return both search documents and aggregation results. To retrieve only the aggregation results, set size to 0:

GET /datalakeapi/customerlog-socname-customername-cef-*
{
  "size": 0,
  "aggs": {
    "your_aggregation_name": {
      "terms": {
        "field": "deviceAction"
      }
    }
  }
}

Run multiple aggregations

It's possible to include multiple aggregations within a single request:

GET /datalakeapi/customerlog-socname-customername-cef-*
{
  "aggs": {
    "your_aggregation_name": {
      "terms": {
        "field": "deviceAction"
      }
    },
    "your_second_aggregation_name ": {
      "terms ": {
        "field": "sourceAddress"
      }
    }
  }
}

Run sub-aggregations

Bucket aggregations can include either bucket or metric sub-aggregations. For instance, a terms aggregation with an terms sub-aggregation computes the aggregation value for each group of documents in the buckets. Sub-aggregations can be nested without any restriction on depth or levels:

GET /datalakeapi/customerlog-socname-customername-cef-*
{
  "aggs": {
    "your_aggregation_name": {
      "terms": {
        "field": "deviceAction"
      },
      "aggs": {
        "your_sub_aggregation_name": {
          "terms": {
            "field": "destinationAddress"
          }
        }
      }
    }
  }
}

Add custom metadata

The meta object allows you to attach custom metadata to an aggregation:

GET /datalakeapi/customerlog-socname-customername-cef-*
{
  "aggs": {
    "your_aggregation_name": {
      "terms": {
        "field": "deviceAction"
      },
      "meta": {
        "my-metadata-field": "foo"
      }
    }
  }
}