Kestrel Session

A Kestrel session provides an isolated stateful runtime space for a huntflow.

A huntflow is the source code or script of a cyber threat hunt, which can be developed offline in a text editor or interactively as the hunt goes. A Kestrel session provides the runtime space for a huntflow that allows execution and inspection of hunt statements in the huntflow. The Session class in this module supports both non-interactive and interactive execution of huntflows as well as comprehensive APIs besides execution.

Examples

A non-interactive execution of a huntflow:

from kestrel.session import Session
with Session() as session:
    open(huntflow_file) as hff:
        huntflow = hff.read()
    session.execute(huntflow)

An interactive composition and execution of a huntflow:

from kestrel.session import Session
with Session() as session:
    try:
        hunt_statement = input(">>> ")
    except EOFError:
        print()
        break
    else:
        output = session.execute(hunt_statement)
        print(output)

Export Kestrel variable to Python:

from kestrel.session import Session
huntflow = """newvar = GET process
                       FROM stixshifter://workstationX
                       WHERE [process:name = 'cmd.exe']"""
with Session() as session:
    session.execute(huntflow)
    cmds = session.get_variable("newvar")
for process in cmds:
    print(process["name"])
class kestrel.session.Session(session_id=None, runtime_dir=None, store_path=None, debug_mode=False)[source]

Bases: AbstractContextManager

Kestrel Session class

A session object needs to be instantiated to create a Kestrel runtime space. This is the foundation of multi-user dynamic composition and execution of huntflows. A Kestrel session has two important properties:

  • Stateful: a session keeps track of states/effects of statements that have been previously executed in this session, e.g., the values of previous established Kestrel variables. A session can invoke more than one execute(), and each execute() can process a block of Kestrel code, i.e., multiple Kestrel statements.

  • Isolated: each session is established in an isolated space (memory and file system):

    • Memory isolation is accomplished by OS process and memory space management automatically – different Kestrel session instances will not overlap in memory.

    • File system isolation is accomplished with the setup and management of a temporary runtime directory for each session.

Parameters
  • runtime_dir (str) – to be used for runtime_directory.

  • store_path (str) – the file path or URL to initialize store.

  • debug_mode (bool) – to be assign to debug_mode.

session_id

The Kestrel session ID, which will be created as a random UUID if not given in the constructor.

Type

str

runtime_directory

The runtime directory stores session related data in the file system such as local cache of queried results, session log, and may be the internal store. The session will use a temporary directory derived from session_id if the path is not specified in constructor parameters.

Type

str

store

The internal store used by the session to normalize queried results, implement cache, and realize the low level code generation. The store from the firepit package provides an operation abstraction over the raw internal database: either a local store, e.g., SQLite, or a remote one, e.g., PostgreSQL. If not specified from the constructor parameter, the session will use the default SQLite store in the runtime_directory.

Type

firepit.SqlStorage

debug_mode

The debug flag set by the session constructor. If True, a fixed debug link /tmp/kestrel of runtime_directory will be created, and runtime_directory will not be removed by the session when terminating.

Type

bool

runtime_directory_is_owned_by_upper_layer

The flag to specify who owns and manages runtime_directory. False by default, where the Kestrel session will manage session file system isolation – create and destory runtime_directory. If True, the runtime directory is created, passed in to the session constructor, and will be destroyed by the calling site.

Type

bool

symtable

The continuously updated symbol table of the running session, which is a dictionary mapping from Kestrel variable names str to their associated Kestrel internal data structure VarStruct.

Type

dict

data_source_manager

The data source manager handles queries to all data source interfaces such as local file stix bundle and stix-shifter. It also stores previous queried data sources for the session, which is used for a syntax sugar when there is no data source in a Kestrel GET statement – the last data source is implicitly used.

Type

kestrel.datasource.DataSourceManager

analytics_manager

The analytics manager handles all analytics related operations such as executing an analytics or getting the list of analytics for code auto-completion.

Type

kestrel.analytics.AnalyticsManager

execute(codeblock)[source]

Execute a Kestrel code block.

A Kestrel statement or multiple consecutive statements constitute a code block, which can be executed by this method. New Kestrel variables can be created in a code block such as newvar = GET .... Two types of Kestrel variables can be legally referred in a Kestrel statement in the code block:

  • A Kestrel variable created in the same code block prior to the reference.

  • A Kestrel variable created in code blocks previously executed by the session. The session maintains the symtable to keep the state of all previously executed Kestrel statements and their established Kestrel variables.

Parameters

codeblock (str) – the code block to be executed.

Returns

A list of outputs that each of them is the output for each statement in the inputted code block.

parse(codeblock)[source]

Parse a Kestrel code block.

Parse one or multiple consecutive Kestrel statements (a Kestrel code block) into the abstract syntax tree. This could be useful for frontends that need to parse a statement without executing it in order to render some type of interface.

Parameters

codeblock (str) – the code block to be parsed.

Returns

A list of dictionaries that each of them is an abstract syntax tree for one Kestrel statement in the inputted code block.

get_variable_names()[source]

Get the list of Kestrel variable names created in this session.

get_variable(var_name)[source]

Get the data of Kestrel variable var_name, which is list of homogeneous entities (STIX SCOs).

create_variable(var_name, objects, object_type=None)[source]

Create a new Kestrel variable var_name with data in objects.

This is the API equivalent to Kestrel command NEW, while allowing more flexible objects types (Python objects) than the objects serialized into text/JSON in the command NEW.

Parameters
  • var_name (str) – The Kestrel variable to be created.

  • objects (list) – List of Python objects, currently support either a list of str or a list of dict.

  • object_type (str) – The Kestrel entity type for the created Kestrel variable. It overrides the type field in objects. If there is no type field in objects, e.g., objects is a list of str, this parameter is required.

do_complete(code, cursor_pos)[source]

Kestrel code auto-completion.

This function gives a list of suggestions on the inputted partial Kestrel code to complete it. The current version sets the context for completion on word level – it will reason around the last word in the input Kestrel code to provide suggestions. Data sources and analytics names can also be completed since the entire URI are single words (no space in data source or analytic name string). This feature can be used to list all available data sources or analytics, e.g., giving the last partial word stixshifter://.

Currently this method computes code completion based on:

  • Kestrel keywords

  • Kestrel variables

  • data source names

  • analytics names

Parameters
  • code (str) – Kestrel code.

  • cursor_pos (int) – the position to start completion (index in code).

Returns

A list of suggested strings to complete the code.

close()[source]

Explicitly close the session.

This may be executed by a context manager or when the program exits.