Install Runtime

Kestrel runs in a Python environment on Linux, macOS, or Windows. On Windows, please use Python inside Windows Subsystem for Linux (WSL).

General Requirements

Python 3 is required.

OS-specific Requirements

If you are using following Linux distributions or newer, the requirement is already met:

  • Alpine 3.6

  • Archlinux

  • Debian 10

  • Fedora 33

  • Gentoo

  • openSUSE 15.2

  • Ubuntu 20.04

  • RedHat 8

Otherwise, check the SQLite version in a terminal with command sqlite3 --version and upgrade sqlite3 >= 3.24 as needed, which is required by firepit, a Kestrel dependency, with default config.

Full installation of Xcode is required, especially for Mac with Apple silicon (M1/M2/…).

The basic xcode-select --install may not install Python header files, or set incorrect architecture argument for dependent package compilation, so the full installation of Xcode is required.

Nothing needed.

Choose Where to Install

It is a good practice to install Kestrel in a Python virtual environment so there will be no dependency conflict with Python packages in the system, plus all dependencies will be the latest.

To setup and activate a Python virtual environment named huntingspace:

$ python3 -m venv huntingspace
$ . huntingspace/bin/activate
$ pip install --upgrade pip setuptools wheel

If you don’t like Python virtual environment or think it is too complicated, you can directly install Kestrel under a user.

There is nothing you need to do in this step besides opening a terminal under that user, or login to the remote host under that user.

The downside is all Python packages under that user are in the same namespace. If Kestrel requires a specific version of a library package, and another application requires a different version of the same library package, that will cause a conflict (pip in the next step will give a warning if happens).

It is not recommended to install Kestrel as system packages since the configurations of Kestrel is under the user who runs it. However, it is possible to install Kestrel as system package, just open a terminal and swtich to root as follows:

$ sudo -i

Kestrel Runtime Installation

Kestrel runtime has two major versions:

  • Kestrel 1: the classic interpreter runtime that uses STIX patterns and firepit (flattened STIX data) as its internal pattern and data representation.

    To install Kestrel 1, execute the commands in the terminal you opened in the last step. If you use Python virtual environment, the virtual environment should be activated.

    $ pip install kestrel-jupyter
    $ kestrel_jupyter_setup
    
  • Kestrel 2: the new just-in-time (JIT) compiler runtime that implements Kestrel intermediate representation (IR). Kestrel 2 debuts at Black Hat USA 2024.

    • Execution: per output commands such as DISP, Kestrel 2 identifies its minimal dependent IR graph, further segments the subgraph regarding different datasources/interfaces, then compiles and executes each subgraph on each corresponding Kestrel interface.

    • Lazy evaluation: execution is only triggered by output commands such as DISP. This makes it possible to take into account all dependent commands or dependent IR graph to optimize the evaluation. Instead of result retrieval for each Kestrel command by the Kestrel 1 interpreter, Kestrel 2 compiles IR subgraphs (multiple Kestrel commands that can be executed on the same interface/datasource) into deeply nested query on each interface.

    • Generic syntax support: besides STIX, users can now use entities and attributes in OCSF and OpenTelemetry in the Kestrel language. The syntax is normalized to OCSF in Kestrel IR, and data between different Kestrel interfaces are normalized into OCSF.

    Kestrel 2 is currently in beta (for experimental use). To install Kestrel 2, execute the commands. If you use Python virtual environment, the virtual environment should be activated.

    $ pip install kestrel-jupyter==2.0.0b2
    $ kestrel_jupyter_setup
    
    $ git clone git://github.com/opencybersecurityalliance/kestrel-lang
    $ cd kestrel-lang
    $ make install
    

Kestrel Front-Ends

Kestrel runtime currently supports three front-ends (Kestrel in a Nutshell). Use the following command to invoke any of them:

This is the most popular front-end for Kestrel and it provides an interactive way to develop Hunt Flow and Huntbook. Start the Jupyter Notebook and dive into Kestrel + Jupyter:

$ jupyter nbclassic

The kestrel command is designed for batch execution and hunting automation. Use it right away in a terminal:

$ kestrel myfirsthuntflow.hf

Check out the Hello World Hunt for more information.

You can use/call Kestrel from any Python program.

What’s to Do Next