Cheat sheet

Use these commands when you need a bit of boilerplate to get up and running.

Docker commands

Container start

docker run --env-file license.env \
           -v $(pwd)/your-config.json:/home/config.json \
           shadowtraffic/shadowtraffic:latest \
           --config /home/config.json \
           --watch --sample 10 --stdout

• license.env contains your license environment variables and values.
• your-config.json is your ShadowTraffic config file.

Docker Compose example

shadowtraffic:
  image: shadowtraffic/shadowtraffic:latest
  volumes:
    - .:/workspace
  entrypoint: ["java", "-jar", "/home/shadowtraffic.jar"]
  command: ["--config", "/workspace/your-config.json"]
  env_file:
    - shadowtraffic/license.env

---

The CLI

CLI switches

  -c, --config <file>                      Path to configuration file.
      --config-base64 <text>               Instead of a file, use this Base64 encoded configuration data.
  -s, --stdout                             Ignore original output targets and forward all generated data to standard out.
      --no-pretty                          Do not use a pretty printer with --stdout when generating data.
      --sample <n>                         Generates the specified number of events and then stops immediately.
  -w, --watch                              Keep running and restart generation on config file changes.
  -q, --quiet                              Do not print any status text other than generated data to the command line.
      --seed <n>                           Initializes random generators with this seed, enabling repeatable runs.
      --with-studio                        Starts ShadowTraffic Studio on port 8080 to visually interact with your generated data in the browser. Must run with --watch and --sample.
      --action <action>                    What ShadowTraffic should do (run, bootstrap). Defaults to run.
      --bootstrap-from-json-schema <file>  Uses the provided JSON Schema file to approximate the ShadowTraffic configuration.
      --bootstrap-from-avro-schema <file>  Uses the provided Avro Schema file to approximate the ShadowTraffic configuration.
      --bootstrap-to <to>                  The target connection type to bootstrap the ShadowTraffic configuration file to.
  -h, --help                               Display this information.

---

Metrics

Prometheus metrics

ShadowTraffic exposes metrics through Prometheus. Inside the container, curl http://localhost:9400 to see the metrics, and forward port 9400 to your host if you want to ingest or graph the metrics into your own observability platform.

All metrics use two labels - connection name, and output target. For example, if you're connection is to Kafka and it's named "kafka", and the topic you're writing to is named "clicks", you'll see a label ("kafka", "clicks").

Metric list

name	kind	description
generator_events_sent_total	counter	The total number of events successfully written to the target system. Use this counter with Prometheus rate to calculate events/second.
generator_iteration_duration_seconds	histogram	Measures the duration it takes to generate a single event, excluding the time to write it to your target system. Use this to see how fast or slow the data creation part of your ShadowTraffic configuration is.

All standard JVM metrics are also sent through Prometheus's JvmMetrics reporter.

Bootstrap

Bootstrapping

If you already have a formal schema that your data needs to adhere to, you can use ShadowTraffic's bootstrap functionality to save you some work. By specifying the schema and a connection type, ShadowTraffic will convert your schema into a configuration file, stubbing out values where appropriate. This can save you a ton of typing if your schema is large.

Currently, JSON Schema and Avro are supported as an input source. See below.

JSON schema

As an example, imagine you have a simple JSON schema file named mySchema.json:

{
    "type": "object",
    "properties": {
        "id": { "type": "integer" },
        "name": { "type": "string" },
        "age": { "type": "integer" },
        "isActive": { "type": "boolean" },
    },
    "required": [ "id", "name" ]
}

You can use bootstrap to turn this schema into a configuration file with stubs for each concrete value. In the following command, mySchema.json is mounted into the container so that ShadowTraffic can see it.

docker run --env-file license.env -v $(pwd)/mySchema.json:/home/mySchema.json shadowtraffic/shadowtraffic:latest --action bootstrap --bootstrap-from-json-schema /home/mySchema.json --bootstrap-to kafka

The switch --action bootstrap tells ShadowTraffic to try and write a configuration file. Alongside are the --bootstrap-from-json-schema and --bootstrap-to switches, the latter of which indicates which connection type the data should be generated to.

After running this command, ShadowTraffic should output roughly the following stub with instructions on how to run it. 🎉

{
  "generators" : [
    {
      "topic" : "sandbox",
      "value" : {
        "id" : {
          "_gen" : "uniformDistribution",
          "bounds" : [
            1,
            50
          ],
          "decimals" : 0
        },
        "name" : {
          "_gen" : "string",
          "expr" : "#{Name.fullName}"
        },
        "age" : {
          "_gen" : "uniformDistribution",
          "bounds" : [
            1,
            50
          ],
          "decimals" : 0
        },
        "isActive" : {
          "_gen" : "boolean"
        }
      }
    }
  ],
  "connections" : {
    "kafka" : {
      "kind" : "kafka",
      "producerConfigs" : {
        "bootstrap.servers" : "localhost9092",
        "key.serializer" : "io.shadowtraffic.kafka.serdes.JsonSerializer",
        "value.serializer" : "io.shadowtraffic.kafka.serdes.JsonSerializer"
      }
    }
  }
}

Avro

Just like bootstrapping from a JSON schema, you can bootstrap from an Avro schema, too. Use the following command, with the steps the same as JSON scheama:

docker run --env-file license.env -v $(pwd)/mySchema.avsc:/home/mySchema.avsc shadowtraffic/shadowtraffic:latest --action bootstrap --bootstrap-from-avro-schema /home/mySchema.avsc --bootstrap-to kafka