Kestrel Session
A Kestrel session provides an isolated stateful runtime space for a huntflow.
A huntflow is the source code or script of a cyber threat hunt, which can be
developed offline in a text editor or interactively as the hunt goes. A Kestrel
session provides the runtime space for a huntflow that allows execution and
inspection of hunt statements in the huntflow. The Session
class in this
module supports both non-interactive and interactive execution of huntflows as
well as comprehensive APIs besides execution.
Examples
A non-interactive execution of a huntflow:
from kestrel.session import Session
with Session() as session:
open(huntflow_file) as hff:
huntflow = hff.read()
session.execute(huntflow)
An interactive composition and execution of a huntflow:
from kestrel.session import Session
with Session() as session:
try:
hunt_statement = input(">>> ")
except EOFError:
print()
break
else:
output = session.execute(hunt_statement)
print(output)
Export Kestrel variable to Python:
from kestrel.session import Session
huntflow = """newvar = GET process
FROM stixshifter://workstationX
WHERE [process:name = 'cmd.exe']"""
with Session() as session:
session.execute(huntflow)
cmds = session.get_variable("newvar")
for process in cmds:
print(process["name"])
- class kestrel.session.Session(session_id=None, runtime_dir=None, store_path=None, debug_mode=False)[source]
Bases:
AbstractContextManager
Kestrel Session class
A session object needs to be instantiated to create a Kestrel runtime space. This is the foundation of multi-user dynamic composition and execution of huntflows. A Kestrel session has two important properties:
Stateful: a session keeps track of states/effects of statements that have been previously executed in this session, e.g., the values of previous established Kestrel variables. A session can invoke more than one
execute()
, and eachexecute()
can process a block of Kestrel code, i.e., multiple Kestrel statements.Isolated: each session is established in an isolated space (memory and file system):
Memory isolation is accomplished by OS process and memory space management automatically – different Kestrel session instances will not overlap in memory.
File system isolation is accomplished with the setup and management of a temporary runtime directory for each session.
- Parameters
runtime_dir (str) – to be used for
runtime_directory
.store_path (str) – the file path or URL to initialize
store
.debug_mode (bool) – to be assign to
debug_mode
.
- session_id
The Kestrel session ID, which will be created as a random UUID if not given in the constructor.
- Type
str
- runtime_directory
The runtime directory stores session related data in the file system such as local cache of queried results, session log, and may be the internal store. The session will use a temporary directory derived from
session_id
if the path is not specified in constructor parameters.- Type
str
- store
The internal store used by the session to normalize queried results, implement cache, and realize the low level code generation. The store from the
firepit
package provides an operation abstraction over the raw internal database: either a local store, e.g., SQLite, or a remote one, e.g., PostgreSQL. If not specified from the constructor parameter, the session will use the default SQLite store in theruntime_directory
.- Type
firepit.SqlStorage
- debug_mode
The debug flag set by the session constructor. If True, a fixed debug link
/tmp/kestrel
ofruntime_directory
will be created, andruntime_directory
will not be removed by the session when terminating.- Type
bool
- runtime_directory_is_owned_by_upper_layer
The flag to specify who owns and manages
runtime_directory
. False by default, where the Kestrel session will manage session file system isolation – create and destoryruntime_directory
. If True, the runtime directory is created, passed in to the session constructor, and will be destroyed by the calling site.- Type
bool
- symtable
The continuously updated symbol table of the running session, which is a dictionary mapping from Kestrel variable names
str
to their associated Kestrel internal data structureVarStruct
.- Type
dict
- data_source_manager
The data source manager handles queries to all data source interfaces such as local file stix bundle and stix-shifter. It also stores previous queried data sources for the session, which is used for a syntax sugar when there is no data source in a Kestrel
GET
statement – the last data source is implicitly used.- Type
kestrel.datasource.DataSourceManager
- analytics_manager
The analytics manager handles all analytics related operations such as executing an analytics or getting the list of analytics for code auto-completion.
- Type
kestrel.analytics.AnalyticsManager
- execute(codeblock)[source]
Execute a Kestrel code block.
A Kestrel statement or multiple consecutive statements constitute a code block, which can be executed by this method. New Kestrel variables can be created in a code block such as
newvar = GET ...
. Two types of Kestrel variables can be legally referred in a Kestrel statement in the code block:A Kestrel variable created in the same code block prior to the reference.
A Kestrel variable created in code blocks previously executed by the session. The session maintains the
symtable
to keep the state of all previously executed Kestrel statements and their established Kestrel variables.
- Parameters
codeblock (str) – the code block to be executed.
- Returns
A list of outputs that each of them is the output for each statement in the inputted code block.
- parse(codeblock)[source]
Parse a Kestrel code block.
Parse one or multiple consecutive Kestrel statements (a Kestrel code block) into the abstract syntax tree. This could be useful for frontends that need to parse a statement without executing it in order to render some type of interface.
- Parameters
codeblock (str) – the code block to be parsed.
- Returns
A list of dictionaries that each of them is an abstract syntax tree for one Kestrel statement in the inputted code block.
- get_variable(var_name, deref=True)[source]
Get the data of Kestrel variable
var_name
, which is list of homogeneous entities (STIX SCOs).
- create_variable(var_name, objects, object_type=None)[source]
Create a new Kestrel variable
var_name
with data inobjects
.This is the API equivalent to Kestrel command
NEW
, while allowing more flexible objects types (Python objects) than the objects serialized into text/JSON in the commandNEW
.- Parameters
var_name (str) – The Kestrel variable to be created.
objects (list) – List of Python objects, currently support either a list of
str
or a list ofdict
.object_type (str) – The Kestrel entity type for the created Kestrel variable. It overrides the
type
field inobjects
. If there is notype
field inobjects
, e.g.,objects
is a list ofstr
, this parameter is required.