Edit

Controller Setup

Introduction

The Rookout ETL Controller is a component that can be installed within your network.
The ETL Controller is only meant for advanced deployments meeting complex network and security requirements.
Using the Rookout ETL Controller you may perform all related data processing inside your own network.

Run as Container

The easiest way to deploy the ETL Controller is as a Docker container available here.
The ETL Controller can be configured to your needs using environment variables.

1. Docker

To run as a Docker container, simply execute:

docker run -p 7488:7488 -e "ROOKOUT_TOKEN=[Your Rookout Token]" rookout/controller

2. Kubernetes and Helm

If you are running on Kubernetes, we recommend using our provided helm chart available here.

Simply execute:

helm repo add rookout https://helm-charts.rookout.com
helm repo update
helm install --name my-release rookout/controller --set controller.token=[Your Rookout Token]

Run as Daemon

The Rookout ETL Controller is also available as a Linux Daemon.
To install it using a setup script, run the following commands:

export ROOKOUT_TOKEN=[Your Rookout Token]
curl -fs https://get.rookout.com | bash

Alternatively, you can download the script to disk and execute it with command line arguments (see more options below):

curl -fs https://get.rookout.com > setup.sh
bash setup.sh --token=[Your Rookout Token]

Linux Daemon Configuration

  • The environment variables for the Linux daemon are accessible by editing the /etc/default/rookout-controller file.

  • By default, the ETL Controller listens only on localhost when running in daemon mode.
    This can easily be changed by adding the following line to the configuration file:

export ROOKOUT_LISTEN_ALL=TRUE

You can also do this when installing the ETL controller:

setup.sh --token=[Your Rookout Token] --listen-all

Linux Daemon OS Support

The Linux daemon is supported in the following operating systems:

Operating SystemVersion
DebianGNU/Linux 9 (strech)
Ubuntu14.04 LTS, 16.04 LTS
CentOS6,7
Red Hat EnterpriseLinux 6, 7

Linux Daemon Update

The linux deamon can be updated to the latest version by rerunning the setup script:

If you would like to keep the pre-existing configuration rather than overwriting it with the flags specified now, use --keep-old-config

curl -fs https://get.rookout.com > setup.sh
bash setup.sh --keep-old-config

If you don't specify --keep-old-config, the new version will be installed using the new settings.

Linux Daemon Restart

When modifying the configuration it is important to restart the ETL Controller using one of the following options:

/etc/init.d/rookout-controller restart
systemctl restart rookout-controller

Linux Daemon Uninstall

To uninstall the linux daemon run the following command:

curl -fs https://get.rookout.com/remove_controller.sh | sudo bash

Data Processing

Disable Sending Data

The ETL Controller connects to the Rookout Service to receive commands and report telemetry information.
Data collected from within the application may be sent to Rookout for interactive debugging sessions.

You may configure a local policy preventing the ETL Controller from sending application data to Rookout by adding the following line to the configuration file at /etc/default/rookout-controller .

export ROOKOUT_SEND_DATA=FALSE

Local Breakpoint Targets

As the ETL Controller runs within your network, it allows you to direct the collected data into data sinks.
For example, debug messages can be sent to Elasticsearch or Splunk clusters.

Data Redaction

All data received by the ETL agent undergoes a data redaction process based on the configuration set by the user.

Proxy Support

The Rookout ETL Controller has HTTPS proxy support for advanced network configurations.
As most production systems do have automated means for proxy detection, you should configure it statically.

This can be done when installing the ETL Controller:

setup.sh --token=[Your Rookout Token] --https-proxy=[Your Proxy Server]

Or by adding the ROOKOUT_PROXY configuration to the ETL Controller configuration file at /etc/default/rookout-controller :

export ROOKOUT_PROXY=[Your Proxy Server]

Security options

The controller uses the system certificate store to verify secure connections to servers. If you prefer to skip certificate verification, you can set the ROOKOUT_SKIP_SSL_VERIFY environment variable.

export ROOKOUT_SKIP_SSL_VERIFY=1

Note: This might compromise the security of your system.

Performance Considerations

A single ETL Controller can handle thousands of concurrent applications connected to it, but the default configuration is optimized for about 100 concurrent applications.

The default limits include:

  1. The ETL Controller is restricted to use a single CPU core - to change the limit set ROOKOUT_CONTROLLER_MAX_CPU to the desired number of cores.
  2. The ETL Controller is restricted to 512MB of RAM - to change the limit set ROOKOUT_CONTROLLER_MAX_MEMORY to the desired memory in megabytes. Should the ETL Controller exceed that, it will exit with message similar to:
Memory limit reached (520 Mb) The limit is (512 Mb) - exiting

You can also adjust the limit when installing the ETL controller:

setup.sh --token=[Your Rookout Token] --max-mem=1024

Health Check

If you would like to perform a health check on the Rookout ETL Controller, you can access http://<ROOKOUT_CONTROLLER_HOST>:<ROOKOUT_CONTROLLER_PORT>/healthz. For example, a default configuration on your localhost might make the health check URL available at http://localhost:7488/healthz. A healthy ETL Controller will return an HTTP 200 OK response.

License

The Rookout ETL Controller usage license may be found here: Rookout ETL Controller License

← CollaborationsScripting Reference →