2.1.5. Sigflow engine

2.1.5.1. Introduction

2.1.5.1.1. For what types of threats is this engine designed?

The Sigflow engine analyzes the whole network traffic and generate metadata and files and can, according to the rules, generate alerts.

Embedded detections are the following:

Detections of uncommon behaviour or unrecommended actions perform on the corporate network
Detections of suspicious/malicious actions perform on the corporate network such as malicious download, beaconing or file extraction
Detections based on automatic generated rules based on indices of compromissions (IOCs) originated from CTI

2.1.5.1.2. How does this particular engine detect threats?

The Sigflow engine uses rules for threat detection.
These rules should describe the characteristics of the attacks and also be optimized to reduce false positives.
The GCenter allows to compile several sources and therefore several sets of rules.

2.1.5.1.3. How does Sigflow engine work in the GCenter?

Sigflow must be configured and this configuration is done by the GCap profile.

The`Ignored profile contains:

Advanced configuration of the GCap parameters (including choice of protocols to monitor...)
A structured list of rules

Note

If the protocol monitored by rules is not selected then Sigflow will report these rules as incorrect and ignore them.

The GCap profile is created and updated on the GCenter.
Then this file is sent to the GCap.
The GCap retrieves packets from the TAP and reconstructs the files and creates the corresponding metadata.
Then, based on the defined rules, Sigflow applies the rules.
If a match is found between the rule and network traffic, it creates an alert indicating the reason for the detection.

2.1.5.1.3.1. For more information

Rules are organized: see Organizing the rules.
The rules manager manages the rules and sources: see Sigflow engine signature sources.
The set of rules to be applied to the GCap is defined in a ruleset: see Rulesets.
Sigflow is configured by the GCAP profile:  see GCAP Profiles.

2.1.5.2. Detailed operation of the rules

2.1.5.2.1. Organizing the rules

The rules sent to the Sigflow engine are organized as follows:

A list of sources providing signature set files grouped in categories
A list of signatures capable of adapting to the needs of the environment to be monitored
A list of Ruleset enabling signatures to be linked to a GCap

Note

Managing the sources, categories, rules and rulesets is done in the rules manager built into the GCenter.

The following paragraphs describe the steps required to provide these rules in Ruleset form to the Sigflow module of the GCap through the GCenter:

Management of available rule sources: see Sigflow engine signature sources paragraph

Sources are used to report repositories where signatures are made available.

Once downloaded and unzipped, the rules must be added to the GCenter interface.

These sources update automatically in the case of public source/HTTP if the GCenter is connected to the internet, otherwise, a manual update can be made at this interface in order to have the latest signatures of available.
The creation of rulesets from the sources: see the Rulesets paragraph

Once the rules have been added, it is possible to directly assign this source to different rulesets
The generation of rulesets: refer to Generating rulesets
Applying rulesets to the GCap: refer to Detection Rulesets
Advanced configuration of the GCap parameters: refer to GCAP Profiles

2.1.5.2.2. Sigflow engine signature sources

The sources include Suricata type detection signatures.

There are native sources: CTI like community CTI...

It is also possible to add public sources to the rules manager.
Each source issues a file of rule sets, grouped into categories.
It is possible to manage categories and rules.
The rules manager enables to:

Define and manage the sources of signatures for the detection engine
Manage the rule set files made available by the sources
Manage the categories and rules of these files
View the rules contained in the available sources

The signatures are updated using the `Threat-DB update` fonction.

The user (operator) can directly assign one or more sources to a Ruleset.

The source management manager is described in the `Rulesets` screen.

For implementation, see Managing the Sigflow engine rule sources.

2.1.5.2.3. Rulesets

Ruleset is the security and detection policy to be applied to the GCap.
As noted above, a ruleset is therefore composed of sources that include the various detection signatures.
The creation of the ruleset is mandatory for the GCap probe to be able to analyze the network traffic and raise alerts.
Modifications can be made to signatures or categories in order to adapt a rule to the specificities of the information system or to a particular need.
The source management manager is described in the `Rulesets` screen.

For implementation, see Managing a Sigflow engine ruleset.

2.1.5.2.3.1. Optimization of rulesets

Optimization of a ruleset consists of adapting it to the type of traffic analyzed (the different collection points).
The ruleset can be edited at any time by adding, modifying or deleting rules, categories or sources.
A modified ruleset must be applied to the GCap in order for the changes to take effect.

For implementation, see Managing a Sigflow engine ruleset.

2.1.5.2.3.2. Changing signatures

Signatures and categories can be enabled, disabled, or modified for a given Ruleset.

It is possible to:

Activate signatures and categories
Deactivate signatures and categories
Modify the functioning of a signature in order to adapt it to the supervised information system by setting up a threshold or suppression of alert for a specific network.

To modify a signature in the following way:

Create a Suppress Rule on a rule: removes the raising of an alert according to a source or destination IP
Create a Threshold Rule: limit the number of alerts to be displayed based on a source or destination IP

Note

It is not possible to directly interact with the rule content from the rule manager.

For its implementation, see the Modifying Sigflow engine rules procedure.

2.1.5.2.3.2.1. Definition of signatures

Signatures in the sources can include references to blogs, CVEs, websites, and the like that can be accessed from the rules manager.

To better understand how a signature works, here is an example of a signature:

A signature is always made up of:

An action
A protocol
Network parameters such as IP and source and destination port
A message

Example of a signature:

The following protocols may be subject to a rule:

This signature is composed of:

The first part "action": corresponds to the action to be performed in case of detection.

For example: "alert, pass, drop..."
The 2nd part "the header": it is this part that allows to define the meaning of the alert as well as the networks and protocols.

For example: "tcp any -> any "

This part is composed of:

"Protocol" part: indicates the monitored protocol.

For example: "tcp, udp, icmp"

"Source and destination" section: specifies the source of traffic and the destination of traffic respectively.

It is possible to assign IP addresses (IPv4 and IPv6 are supported) and IP ranges with operations if needed.

It is also possible to use variables such as $HOME_NET and $EXTERNAL_NET that are managed in the `Net variables` section of the `Config Gcaps profiles` menu and Net variables

These variables are used to increase the accuracy of the alerts provided by the signatures.

The following syntax can be used to specify the addresses:

The part "Ports (source and destination)" indicates the ports : the first is the source port, the second is the destination port (see the direction of the directional arrow).

Traffic between and out of ports. Different protocols have different port numbers.

For example, the default port for HTTP is 80 while 443 is usually the port for HTTPS.

Note however that the port does not dictate which protocol is used in the communication. Rather, it determines which application receives the data.

The list below gives examples of port specifications.

the "Direction" part: indicates the direction of the flow.

The last part corresponds to the options applied to the rule.

These are surrounded by parentheses and separated by semicolons.

Some options have parameters (such as msg), which are specified by the option keyword, followed by a colon, followed by the parameters.

Others have no parameters, they are simply the keyword (like nocase).

For its implementation, see the procedure Modifying Sigflow engine rules.

2.1.5.2.3.3. Removing or limiting the number of alerts

The number of alerts sent can be limited automatically or manually:

see Manual creation of the deletion or limitation of the number of alerts
see Automatic creation of limits on the number of alerts

Deletion is used when alerts need to be deleted, that is, do not generate alerts for that particular signature.

This can only be done manually: see Manual creation of the deletion or limitation of the number of alerts.

2.1.5.2.3.3.1. Manual creation of the deletion or limitation of the number of alerts

The number of alerts for a particular signature can be controlled by deletion or limitation.
The limitation is usually used when the number of alerts must be kept to a minimum.
For example, if the need is to see only a maximum of one alert per minute for this signature.
Deletion is used when alerts need to be deleted, that is, do not generate alerts for that particular signature.
A rule can therefore contain the keyword threshold with the following syntax:

Syntaxe :

threshold: type <threshold|limit|both>, track <by_src|by_dst|by_rule|by_both>, count <N>, seconds <T>

The various fields are:

`type` field: defines the limitation type. There are three possibilities:

`limit`: limit alerts to no more than “N” times

`threshold`: minimum threshold for a rule before it generates an alert

`both`: limitation and threshold are applied

`track` field: defines the scope. There are several possibilities:

`by_src`: the limitation is applied to alerts with the same source IP address

`by_dst`: the limitation is applied to alerts with the same destination IP address

`by_rule`: the limitation is applied to all alerts triggered by the rule

`by_both`: the limitation is applied to alerts with the same IP pair source address + destination

`count` field: defines the quantity (limit or threshold) according to the type
`seconds` field: defines the reference period

Example :

`threshold: type threshold, track by_src, count 10, seconds 60;

This rule only generates an alert after 10 raised alerts with the same source IP address within a period of one minute.

The graphical interface for manually creating this limit or threshold rule is described in `Rulesets` screen.
The implementation of a limitation or deletion of alerts is done by modifying rules.
To do this, refer to Modifying Sigflow engine rules.

2.1.5.2.3.3.2. Automatic creation of limits on the number of alerts

Principle:

The number of alerts for one or more GCaps can be controlled by limitation
The GCenter counts alerts during an observation period and compares them to a programmable trigger point
The GCenter determines the rules that generated this threshold exceeding and based on the selected criterion (filter type), GCenter automatically creates alarm limits
These limits grouped by the GCap are communicated to the Sigflow engine present in each GCap to implement these limitations

There are several types of filtration:

`by_src`: the limitation is applied to alerts with the same source IP address

`by_dst`: the limitation is applied to alerts with the same destination IP address

`by_rule`: the limitation is applied to all alerts triggered by the rule

`by_both`: the limitation is applied to alerts with the same IP pair source address + destination

For more information, refer to the description given in `Auto-threshold` screen.

For implementation, refer to Creation of automatic alarm limits.

2.1.5.2.3.4. Generating rulesets

Once the ruleset is created and its configuration saved, the ruleset is generated and applied on the GCap.

For implementation, see the Modifying Sigflow engine rules.

2.1.5.2.4. GCAP Profiles

From this configuration interface, users will be able to apply specific policy rules. They can customize the settings from the following categories:

Detection Rulesets
Base variables
Net variables
Flow timeouts
Files rules management
Packet filtering

In order for the GCap to raise alerts, the user must first apply a ruleset to the GCap.
The GCap profile manager and its configurations are described in the `GCaps profiles` screen.
For implementation, see Configuring GCaps using the `GCaps Profiles`.

2.1.5.2.4.1. Detection Rulesets

The `Detection Rulesets` section enables the management of the network interfaces and defines the ruleset(s) used to inspect the captured traffic.

Note

It is necessary to define rules in a ruleset before applying it to GCaps.
Failure to do so will result in no rules being applied to the GCap.
Only the ruleset previously saved in the Sigflow rules manager can be selected from this menu.

The `Detection Rulesets` section enables three configuration options:

The Single-tenant
The Multi-tenant by interface
The Multi-tenant by VLAN

Note

These configuration options are exclusive.

This means that it will not be possible to apply a single tenant and multi-tenant configuration at the same time.

2.1.5.2.4.1.1. Single-tenant

In Single-tenant mode, all traffic seen by the GCap is analyzed by the same ruleset.
The detection ruleset manager is described in the `Detection Rulesets` section of the `GCaps profiles` screen.
For implementation, see the procedure Configuring the detection rulesets via the GCap profile.

2.1.5.2.4.1.2. Multi-tenant by interface

In multi-tenant mode by interface, the user can apply a different ruleset depending on the capture interface.
Each tenant is a set of interfaces: for example, one can create the tenant 1 = {mon0} and the tenant 2 = {mon1, mon2}.
An interface can only be owned by one tenant.
If an interface is not assigned to any tenant, it will be in the "default tenant".
The default tenant scans all traffic that is not already scanned by a specific tenant.

The detection ruleset manager is described in the `Detection Rulesets` section of the `GCaps profiles` screen.

For implementation, see Configuring the detection rulesets via the GCap profile.

2.1.5.2.4.1.3. Multi-tenant by VLAN

The Multi-tenant by VLAN mode enables:

The assigning one ruleset per VLAN
The assigning a ruleset for the default VLAN for those VLANs not created via the interface

The Multi-tenant by VLAN enables a configuration to be applied for each VLAN previously created in the interface and to have distinct monitoring on different networks.
Thus, it is possible to apply a ruleset independently for each VLAN.
A VLAN named "default" is created as standard in the interface. It enables a ruleset to be applied for all VLANs not explicitly specified in the interface.

The detection ruleset manager is described in the `Detection Rulesets` section of the `GCaps profiles` screen.

For implementation, see Configuring the detection rulesets via the GCap profile.

2.1.5.2.4.2. Base variables

The Base variables section enables the probe's capture parameters to be adjusted using the advanced Sigflow functions that can be configured from GCenter.
Changes to this configuration have an impact on the alerts sent from the GCap probe to the GCenter.
Enabling certain options will enable the sending of alerts, anomalies, metadata, file information, and protocol-specific records.
Alerts are records of events triggered by the matching of a rule with network traffic.
An alert will be created with associated metadata, such as the application layer record (HTTP, DNS, etc).
The graphical interface of this base variable manager is described in `Base variables` Section of the `GCaps profiles` screen.

The basic variables section is composed of:

Stream analysis and file extraction
HTTP Proxy
Payload
Community ID
Alerting and logging

2.1.5.2.4.2.1. Stream analysis and file extraction

The stream analysis and file extraction area enables you to control how the Sigflow engine handles maximum stream and file extraction sizes.

The graphical interface and the details of the parameters are described in the `Stream analysis and file extraction` zone.

2.1.5.2.4.2.2. HTTP Proxy

The HTTP Proxy area enables enhanced metadata and alerts for streams mandated with the X-Forwarded-For (XFF) http header.

The graphical interface and the details of the parameters are described in the `HTTP Proxy` Zone.

Note

XFF is a standard header enabling to identify the original IP address of a client connecting to a web server through an HTTP proxy or load balancer.

2.1.5.2.4.2.3. Payload

The Payload section enables enabling or disabling various fields present (Http body, Payload printable, Http body printable...) in the events.

The graphical interface and the details of the parameters are described in the `Payload` zone.

2.1.5.2.4.2.4. Community ID

The Community ID section enables or disables this feature in events.

This enables identifying the network streams being analyzed.

Note

This field is present in solutions other than Gatewatcher. It can therefore enable correlating the different tools of the same information system.

The graphical interface and the details of the parameters are described in the `Community ID` zone.

2.1.5.2.4.2.5. Alerting and logging

This section enables configuring the alerting and logging of the protocols used by the GCap.

Note

If the GCap is one version ahead of the GCenter, it is possible that some protocols are not yet implemented in the latter.

This is discussed in more detail in the GCAP-documentation in the section Sigflow detection engine > Rebuilding files.

Terminology for alerting and logging:

alerting consists in activating the Sigflow signature detection for a given protocol. Indeed, if Sigflow is activated for a protocol then the stream that is identified by a signature will raise an alert on the GCenter side.
logging consists in enabling the generation of metadata for a given protocol. Indeed, if the latter is enabled for a protocol then each observed session will generate metadata for that protocol on the GCenter side.

Managing the configuration of the protocols is done:

By using default profiles such as Essential, Balanced, Deep, Paranoid and Performance.
By modifying the content of the profile uploaded to the GCap

The choice of the default profile and the loading on the GCap is done in the graphical interface detailed in the `GCaps Pairing` screen.
Changing the configuration of the profile loaded into the GCap is done in the graphical interface detailed in the `Alerting and logging` zone.
The default configuration of protocols varies depending on the GCap profile used: the list is shown in the Default settings for existing profiles available.
The list of protocols that can be configured with the alerting option and with the logging option is provided in the Default settings for existing profiles available paragraph.
For implementation, see the Configuring the Base variables parameters via the GCap profile.

2.1.5.2.4.3. Net variables

The Net variables area of the GUI enables defining the network variables used in the Sigflow rules.
The list of variables and their default values are given in the `Net variables` section of the `Config Gcaps profiles` menu.
This graphical interface is described in the `Net variables` section of the `Config Gcaps profiles` menu.
For implementation, see the procedure  Configuring the network variables parameters via the GCap profile.

2.1.5.2.4.4. Flow timeouts

The Flow timeouts section enables configuring the time in seconds that Sigflow keeps a flow in memory depending on its status.
The list of variables and their default values are given in the `Flow timeouts` section of the `GCaps profiles` screen.
This graphical interface is described in the `Flow timeouts` section of the `GCaps profiles` screen.

2.1.5.2.4.5. Files rules management

The Files rules management section enables choosing the file types that the probe will retrieve for a given protocol.
File extraction works in parallel with the Sigflow signatures defined for these same protocols.
Files are reconstructed and then saved to disk with metadata that includes information such as:

Time stamp
Source/destination IP address
Protocol
Source/destination port
Size
Md5sum, etc

Managing the configuration of the file extraction rules is done:

By using default profiles such as Essential, Balanced, Deep, Paranoid and Performance: in order from most to least complete
By modifying the content of the profile uploaded to the GCap

The choice of the default profile and the loading on the GCap are done in the graphical interface detailed in the `GCaps Pairing` screen.
Changing the configuration of the profile loaded into the GCap is done in the graphical interface detailed in the `File rules` section of the `GCaps profiles` screen.
The list of protocols as well as the list of file types are given in the `GCaps profiles` screen.

For implementation, see the Configuring the file reconstruction rules via the GCap profile.

2.1.5.2.4.6. Packet filtering

The `Packet filters` section enables specific traffic to be ignored directly at the GCap network card level.
This feature enables the GCap to avoid overloading the GCap with "unnecessary" traffic such as encrypted streams, backup streams, etc., or traffic that may cause the cpus to overload such as Elephant Flow, Miles Flow, etc.
The selection of traffic to be ignored is based on vlan, network prefix, protocol, and network ports.

This graphical interface is described in the `Packet filters` section of the `Gcaps profiles` screen.

For implementation, see the Configuring filters on targeted parts of the analyzed traffic via the GCap profile.

2.1.5.3. Events generated

The events generated by Sigflow are:

Events of type "alert" when the engine has found a match with the signature defined in the alert rules.

Events are displayed in the main GCenter interface as well as in Kibana.
Events of type "fileinfo" when the Sigflow engine has reconstructed the network flow files`.

These fileinfo events are only visible in the Kibana interface.
Events of type "meta-data" when the Sigflow engine has recognized the network flow.

These metadata events are only visible in the Kibana interface.

These events (or logs) are structured in the same way and this structure is presented in the Data structure of the Events of type "alert" paragraph.

2.1.5.3.1. Events of type "alert"

Events of type "alert" are generated when the engine has found a match with the signature defined in the alert rules.

These are displayed:

In the main interface named WebUI on the GCenter in the `Alerts` screen

The main interface named WebUI is described in Overview of the WEB UI.

It is possible to view the found signature and its definition.
- To view the alerts, select the `Sigflow` engine filter.
  
  See the presentation of the WebUI `Alerts` screen.
- Click on the selected alert.
  
  The `Alert details` window is displayed.
  
  The detailed information of this alert is displayed: see Example of a Sigflow alert in the WebUI.
  
  The displayed counters are given in the Engine log data structure appendix.
In the interface named Kibana UI
- In the main interface WebUI, to view the alerts, select the `Sigflow` engine filter.
  
  The screen displays only the list of Sigflow alerts.
- After selecting the alert, click on the `Open flow details` command of the `Actions` menu.
  
  See the presentation of the WebUI `Alerts` screen.
  
  Kibana is opened on the `Sigflow` category of the `Alerts ` section.
  
  The interface displayed is the interface named Kibana UI (described in Overview of the Kibana GUI).
- Click on the `Messages` tab (1).
  
  The Kibana database is filtered with the filters:
  - network.flow_id: unique ID of the flow selected
  - event.module: sigflow_alert
  This page displays all alerts detected by Sigflow in this network flow (whatever the IP addresses or the direction).
- Click on the toggle icon on the left of the alert (2).
  
  The `Expanded document` window (3) is displayed.
  
  The detailed information of this alert can be viewed in table or json format (see the Events of type "alert").
  
  The displayed counters are given in the Engine log data structure appendix.
  
  Note
  
  The Kibana UI can also be accessed without any filters via the `Hunting` icon on the left side menu bar in the WebUI.

2.1.5.3.1.1. Example of a Sigflow alert in the WebUI

The presentation of the `Alert details` window is given in the `Alert details` window.

The counters are given in the Example of an alert exported.

2.1.5.3.1.2. List of protocols in the Events of type "alert"

The list of protocols that can generate alerts is indicated by the parameter Default settings for existing profiles available (app_proto field)`.

If a protocol changes along the way (for example if SMTP is upgraded to TLS via STARTTLS) or if the protocols used are not the same in both directions of the flow, the following fields may appear:

app_proto_tc (to client)
app_proto_ts (to server)
app_proto_orig

Note

The protocol actually recognized by Sigflow is defined in the GCap profile in the `Alerting and logging` tab in the `Alerting` column.

2.1.5.3.1.3. Data structure of the Events of type "alert"

The logs are composed of different parts:

The header part
The source part defined by "_source"
The field part defined by "_fields"

This information is displayed in the `Expanded document` screen of Kibana.

2.1.5.3.1.3.1. The header part of the Events of type "alert"

The header section contains:

"_index": "engines_alerts-2024.11.26-000022",
"_id": "yZzDZ5MBe7GX5B2fDKfY",
"_version": 1,
"_score": 0,

The detailed information is given in the table (Counters of the header part of logs).

2.1.5.3.1.3.2. The source part of the Events of type "alert"

The source part defined by "_source" contains:

Note

The data displayed on the WebUI (alerts details window) is a part of the data displayed on the `Extended document` screen on the Kibana interface.
All data can be exported to a SIEM via syslog (an example of an exported alert is shown).
The detailed information is given in the table (Counters of the header part of logs).

The example given here is a Kibana example.

 "http": {
   "response": {
     "status": 200,
     "bytes": 1197,
     "mime_type": "application/octet-stream"
   },
   "http_response_body": "TVqQAAMAAAAA//8AALgAAAOAADwELAQYAAAAAAAAAA",
   "http_response_body_printable": "MZ.....!..L.!This program cannot be run in DOS mode.\r\r\n$..............."
   "hostname": "",
   "version": "HTTP/1.1",
   "request": {
     "method": "GET"
   }
 },
 "source": {
   "ip": "",
   "port": 41710,
   "mac": ""
 },
 "flow": {
   "pkts_toserver": 4,
   "bytes_toserver": 338,
   "pkts_toclient": 12,
   "bytes_toclient": 15280,
   "start": "2024-11-26T09:16:56.277148+0000"
 },
 "observer": {
   "gcap": {
     "ingress": {
       "interface": {
         "name": "monvirt"
       }
     },
     "hostname": "gcap.gatewatcher.com",
     "version": "2.5.x"
   },
   "product": "gcenter",
   "hostname": "gcenter.domain.local",
   "version": "2.5.3.103",
   "log_format_version": "1.0.0",
   "uuid": "fc1e66e3-a397-5eb4-9277-754be778f317",
   "vendor": "gatewatcher"
 },
 "network": {
   "tx_id": 0,
   "protocol": "http",
   "timestamp": "2024-11-26T09:16:56.278736+0000",
   "transport": "tcp",
   "flow_id": 95140270586524
 },
 "ecs": {
   "version": "8.6.0"
 },
 "metadata": {
   "flowbits": [
     "min.gethttp",
     "exe.no.referer"
   ]
 },
 "destination": {
   "ip": "x.y.z.a",
   "port": 16122,
   "mac": "10:e2:ba:a6:a4:70"
 },
 "sigflow": {
   "action": "allowed",
   "payload_printable": "GET /emd.exe HTTP/1.1\r\nHost: opred.net\r\nConnection: Keep-Alive\r\n\r\n",
   "stream": 1,
   "packet": "kOK6pqSQkOK6pqSRmgk2FpoJNha",
   "packet_info": {
     "linktype": 1
   },
   "signature_id": 2019714,
   "metadata": {
     "updated_at": [
       "2024_04_22"
     ],
     "performance_impact": [
       "Significant"
     ],
     "created_at": [
       "2014_11_15"
     ]
   },
   "gid": 1,
   "signature": "ET CURRENT_EVENTS Terse alphanumeric executable downloader high likelihood of being hostile",
   "payload": "R0VUIC9lbWQuZXhlIEhUVFAvMS4mUNCg0K",
   "category": "Potentially Bad Traffic",
   "rev": 11
 },
 "url": {
   "domain": "opred.net",
   "path": "/emd.exe"
 },
 "@timestamp": "2024-11-26T09:16:56.278Z",
 "event": {
   "created": "2024-11-26T09:16:56.278736+0000",
   "dataset": "alert",
   "module": "sigflow_alert",
   "kind": "alert",
   "category": [
     "network",
     "intrusion_detection"
   ],
   "severity": 2,
   "id": "ce95f31d-6bb1-45f9-9f06-1661f5431329"
 },
 "@version": "1"
}

2.1.5.3.1.3.3. List of counters of the Events of type "alert"

Note

The alert counters are visible:

In the `Alert details` screen of the WebUI
In the `Expanded document` screen of Kibana
In the export to the SIEM

The detailed information is given in the table (Counters of the source part of logs).

2.1.5.3.2. Events of type "fileinfo"

Events of type "fileinfo" are generated when the Sigflow engine has reconstructed the network flow files.

These fileinfo events are visible in the Kibana interface.

Click on the `Hunting` icon of the `WebUI`.

The interface displayed is the interface named Kibana UI (described in Overview of the Kibana GUI).

Click on the `Network Metadata` section (2) then the `File Transactions` category (1).
Click on the `Messages` tab (3).

This page displays all events of fileinfo type corresponding to the reconstructed files in this network flow.
- Click on the toggle icon on the left of the alert (1).
The detailed information (2) of this event can be viewed in table or json format.

Note

The Kibana UI can also be accessed without any filters via the `Hunting` icon on the left side menu bar in the WebUI.

2.1.5.3.2.1. Example of the events of type "fileinfo"

The counters are given in the Example of an alert exported.

2.1.5.3.2.2. Data structure of the events of type "fileinfo"

The logs are composed of different parts:

The header part
The source part defined by "_source"
The field part defined by "_fields"

This information is displayed in the `Expanded document` screen of Kibana.

2.1.5.3.2.2.1. The header part of the events of type "fileinfo"

The header section contains:

"_index": "engines_metadata-2025.02.03-000012",
"_id": "c1ivzJQBlS-iq-hJEPiS",
"_version": 1,
"_score": 0,

The detailed information is given in the table (Counters of the header part of logs).

2.1.5.3.2.2.2. The source part of the events of type "fileinfo"

The source part defined by "_source" contains:

Note

All data can be exported to a SIEM via syslog (an example of an event of type "fileinfo" is shown).

The detailed information is given in the table (Counters of the header part of logs).

The example given here is a Kibana example.

"user_agent": {
  "original": "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729)"
},
"destination": {
  "port": 19609,
  "ip": "x.y.z.a"
},
"source": {
  "port": 8080,
  "ip": "a.b.c.d"
},
"event": {
  "dataset": "network_metadata",
  "module": "sigflow_file",
  "category": [
    "network",
    "file"
  ],
  "created": "2025-02-03T16:39:29.577001+0000",
  "id": "cbfb9498-72f5-43ab-a53a-aad5bb7b4ad2",
  "kind": "event"
},
"@version": "1",
"file": {
  "tx_id": 1,
  "magic": "Macromedia Flash data (compressed), version 14",
  "name": "/6SuCHKKkf8Sf1aFXJPqD0R6r3oEDCrbwHFm23EU-Af2zwWdHgpn6mEGu5XlxFust",
  "sid": [
    1100020
  ],
  "state": "CLOSED",
  "file_id": 1153,
  "hash": {
    "sha256": "350836364013549b6a76aab79d57d109df6acc143759e24a952d3ff5d6a76ec4",
    "md5": "67ca9a31f220bc7b68f203c07ad668b9"
  },
  "size": 77068,
  "gaps": false,
  "stored": true
},
"http": {
  "http_refer": "http://tsevid-synonymi.justdanceatsea.com:8080/ndf4xx22ci.php",
  "response": {
    "mime_type": "application/x-shockwave-flash",
    "status": 200,
    "bytes": 77068
  },
  "hostname": "tsevid-synonymi.justdanceatsea.com",
  "http_port": 8080,
  "version": "HTTP/1.1",
  "request": {
    "method": "GET"
  }
},
"url": {
  "domain": "tsevid-synonymi.justdanceatsea.com",
  "path": "/6SuCHKKkf8Sf1aFXJPqD0R6r3oEDCrbwHFm23EU-Af2zwWdHgpn6mEGu5XlxFust"
},
"observer": {
  "product": "gcenter",
  "uuid": "78f4fed1-c9ad-52b9-b509-6b87767f501f",
  "hostname": "gcenter.domain.local",
  "gcap": {
    "ingress": {
      "interface": {
        "name": "monvirt"
      }
    },
    "hostname": "gcap.gatewatcher.fr",
    "version": "2.5.x"
  },
  "version": "2.5.x",
  "vendor": "gatewatcher",
  "log_format_version": "1.0.0"
},
"@timestamp": "2025-02-03T16:39:29.577Z",
"ecs": {
  "version": "8.6.0"
},
"network": {
  "protocol": "http",
  "transport": "tcp",
  "timestamp": "2025-02-03T16:39:29.577001+0000",
  "flow_id": 1156097574154966
}

2.1.5.3.2.2.3. List of counters of the events of type "fileinfo"

Note

The alert counters are visible:

In the `Expanded document` screen of Kibana
In the export to the SIEM

The detailed information is given in the table (Counters of the source part of logs).

2.1.5.3.3. Events of type "meta-data"

Events of type "metadata" are generated when the Sigflow engine has recognized the network flow protocols.

These metadata events are visible in the Kibana interface.

Click on the `Hunting` icon of the `WebUI`.

The interface displayed is the interface named Kibana UI (described in Overview of the Kibana GUI).

Click on the `Network Metadata` section then the protocol category (HTTP for example).
Click on the `Messages` tab.

This page displays all events of metadata type (metadata) corresponding to the detected protocol in this network flow.
Select the arrow to the left of the event.

The detailed information of this event can be viewed in table or json format.

2.1.5.3.3.1. Example of the events of type "meta-data"

The counters are given in the Example of an alert exported.

2.1.5.3.3.2. Data structure of the events of type "meta-data"

The logs are composed of different parts:

The header part
The source part defined by "_source"
The field part defined by "_fields"

This information is displayed in the `Expanded document` screen of Kibana.

2.1.5.3.3.2.1. The header part of the events of type "meta-data"

The header section contains:

"_index": "engines_metadata-2025.02.03-000012",
"_id": "A1i0zJQBlS-iq-hJVPqO",
"_version": 1,
"_score": 0,

The detailed information is given in the table (Counters of the header part of logs).

2.1.5.3.3.2.2. The source part of the events of type "meta-data"

The source part defined by "_source" contains:

Note

All data can be exported to a SIEM via syslog (an example of an event of type "meta-data" is shown).

The detailed information is given in the table (Counters of the header part of logs).

The example given here is a Kibana example.

 "user_agent": {
  "original": "\"Mozilla/5.2 (Windows NT 6.2; rv:50.2) Gecko/20200103 Firefox/50.2\""
},
"destination": {
  "port": 16122,
  "ip": "x.y.z.a"
},
"source": {
  "port": 62219,
  "ip": "a.b.c.d"
},
"event": {
  "dataset": "network_metadata",
  "module": "sigflow_http",
  "category": [
    "network"
  ],
  "created": "2025-02-03T16:45:17.241433+0000",
  "id": "609a229b-d352-4b70-a0db-7915ac2aaeec",
  "kind": "event"
},
"@version": "1",
"ether": {},
"http": {
  "connection": "Keep-Alive",
  "response": {
    "mime_type": "text/plain",
    "status": 200,
    "bytes": 186503
  },
  "server": "Microsoft-IIS/5.0",
  "date": "Thu, 01 Jun 2017 20:55:46 GMT",
  "version": "HTTP/1.1",
  "request": {
    "mime_type": "text/plain",
    "bytes": 251904,
    "method": "GET"
  },
  "last_modified": "Thu, 01 Jun 2017 20:25:44 GMT",
  "hostname": "fabrique.com",
  "accept": "*/*",
  "accept_encoding": "gzip, deflate",
  "accept_language": "en-US"
},
"url": {
  "domain": "fabrique.com",
  "path": "/7rvmnb"
},
"observer": {
  "product": "gcenter",
  "uuid": "78f4fed1-c9ad-52b9-b509-6b87767f501f",
  "hostname": "gcenter.domain.local",
  "gcap": {
    "ingress": {
      "interface": {
        "name": "monvirt"
      }
    },
    "hostname": "gcap.gatewatcher.fr",
    "version": "2.5.x"
  },
  "version": "2.5.x",
  "vendor": "gatewatcher",
  "log_format_version": "1.0.0"
},
"@timestamp": "2025-02-03T16:45:17.241Z",
"ecs": {
  "version": "8.6.0"
},
"network": {
  "tx_id": 0,
  "protocol": "http",
  "transport": "tcp",
  "community_id": "1:cxHEBTJgZCFfWpryH1AkluiYEGk=",
  "timestamp": "2025-02-03T16:45:17.241433+0000",
  "flow_id": 575568319996003
}

2.1.5.3.3.2.3. List of counters of the events of type "meta-data"

Note

The alert counters are visible:

In the `Expanded document` screen of Kibana
In the export to the SIEM

The detailed information is given in the table (Counters of the source part of logs).

2.1.5.4. Management of the engine

2.1.5.4.1. View the status of Sigflow

The current engine status is displayed in the `Health checks` screen.

2.1.5.4.2. Engine update

There are updates (Updates) for the Sigflow engine.
These updates can be done manually or scheduled via Threat-DB update module.
See Dedicated modules for managing updates: Threat DB update and Software update and in particular the part Update presentation.

2.1.5.4.3. Rules update

The rule sources are defined in the GCaps profiles via the rulesets.
These sources have to be manually or automatically updated according to the configuration.
For further information, refer to Organizing the rules.
To manage the rules, see the Managing the Sigflow engine rule sources.
To update the rules, see the Updating rules sources.

2.1.5.4.4. Sigflow configuration

The engine is configured by profiles.
For more information, see the GCAP Profiles paragraph.
For implementation, see the Configuring GCaps using the `GCaps Profiles`.

2.1.5.5. Alert Analysis

The alerts are displayed on a specific screen described in the WebUI `Alerts` screen.
The general procedure for analyzing alerts is described in the Using of NDR dashboards.
The specific procedure for analyzing Sigflow alerts is described in the file Analyzing the Sigflow alerts.