Skip to main content

learn

changelog

You can subscribe to this changelog through the RSS feed.

What's new​

2.0.3​

Thu Jul 23 14:52:29 PDT 2026

2.0.2​

Thu Jul 23 10:03:49 PDT 2026

  • Fixes MAP and certain timestamps insertions for the Databricks connection.
  • Fixes regression in heading function.

2.0.1​

Wed Jul 22 09:01:42 PDT 2026

  • Fixes STRUCT and ARRAY insertions for the Databricks connection.

2.0.0​

Mon Jul 20 13:47:04 PDT 2026

  • Improves performance of nearly all workloads by a factor of 3, and for some adversarial cases > 10.
  • Breaking change: the filesystem connection now exclusively uses a temporary staging directory to improve performance. If you run ShadowTraffic in --read-only mode, provide a writable location using one of the fallback approaches.

1.19.8​

Mon Jul 20 07:57:09 PDT 2026

1.19.7​

Thu Jul 9 08:23:33 PDT 2026

1.19.6​

Thu Jun 25 08:42:17 PDT 2026

1.19.5​

Thu Jun 18 11:06:10 PDT 2026

  • Fixes spurious linting errors when using env functions.

1.19.4​

Thu Jun 18 09:06:35 PDT 2026

  • Adds proper exit codes for all terminal states. Exit code 0 indicates success, and 1 any kind of failure.

1.19.3​

Thu Jun 18 07:52:54 PDT 2026

1.19.2​

Mon Jun 15 09:09:06 PDT 2026

1.19.1​

Wed Jun 10 14:24:04 PDT 2026

1.19.0​

Tue Jun 9 10:37:32 PDT 2026

1.18.10​

Mon Jun 8 08:55:37 PDT 2026

  • Improves Avro schema errors that arise when writing incorrectly typed Parquet or Iceberg data.
  • Adds logging when Iceberg files dynamically split because of rotating partition values.
  • Echoes Iceberg table schema and properties on startup.

1.18.9​

Fri Jun 5 17:28:43 PDT 2026

  • Avoids failure to write to object storage when ShadowTraffic is run in a read-only container.

1.18.8​

Fri Jun 5 09:15:21 PDT 2026

  • Fixes date/int type conversion for Iceberg partitioned tables.

1.18.7​

Fri Jun 5 07:52:33 PDT 2026

1.18.6​

Thu Jun 4 11:25:08 PDT 2026

  • Improves memory utilization/performance for Iceberg, JSON, JSONL, and log serialization formats.

1.18.5​

Fri May 29 09:53:09 PDT 2026

  • Fixes Redis's unlink startup policy for non-templated patterns in multi node clusters.

1.18.4​

Thu May 21 08:30:22 PDT 2026

1.18.3​

Tue May 19 15:38:41 PDT 2026

  • Improves memory utilization and performance with Parquet serialization.
  • Switches S3 to upload file parts in parallel.

1.18.2​

Wed May 13 09:10:40 PDT 2026

  • Lint mode now prints out validation errors if the configuration can't be recognized.

1.18.1​

Thu May 7 08:05:28 PDT 2026

1.18.0​

Wed May 6 09:48:36 PDT 2026

  • Adds Google Vertex as a model provider for the ai function.

1.17.11​

Wed Apr 29 12:40:54 PDT 2026

  • Improves Iceberg writing performance.

1.17.10​

Mon Apr 27 21:16:53 PDT 2026

  • Fixes bug that causes Iceberg files to split too quickly when using forks.

1.17.9​

Mon Apr 27 09:10:21 PDT 2026

  • Makes ShadowTraffic's Iceberg writer resilient to AWS Glue throttling.

1.17.8​

Mon Apr 27 06:57:44 PDT 2026

  • Enhances the Iceberg writer to break apart batched rows that cross partition boundaries.

1.17.7​

Tue Apr 21 15:58:24 PDT 2026

  • Fixes file size chunking when writing files to Iceberg.
  • Fixes pathalogical case that may deplete cache of the string function.

1.17.6​

Mon Apr 20 13:11:55 PDT 2026

  • Fixes issues where Iceberg S3 files are written with a superfluous leading slash.

1.17.5​

Fri Apr 17 13:38:17 PDT 2026

  • Enables Iceberg formatted tables to work with AWS Glue.

1.17.4​

Fri Apr 17 08:46:38 PDT 2026

1.17.3​

Fri Apr 17 08:17:25 PDT 2026

  • Fixes Redis's unlink startup policy for multi node clusters.

1.17.2​

Wed Apr 15 10:02:23 PDT 2026

  • Adds new startupPolicy parameter for the Redis connection to clear the key space before running.

1.17.1​

Tue Apr 14 08:48:01 PDT 2026

  • Adds OS wallclock time to debugging mode.

1.17.0​

Wed Apr 8 13:09:08 PDT 2026

1.16.2​

Thu Apr 2 15:54:19 PDT 2026

1.16.1​

Fri Mar 27 08:03:10 PDT 2026

1.16.0​

Wed Mar 25 15:16:01 PDT 2026

  • Adds new Redis connection.

1.15.7​

Fri Mar 20 14:51:04 PDT 2026

  • Fixes linter false positive on stateMachine's exposeVars parameter.

1.15.6​

Wed Mar 11 09:18:29 PDT 2026

1.15.5​

Thu Mar 5 10:51:50 PST 2026

1.15.4​

Mon Mar 2 15:50:00 PST 2026

  • Improves error messages from illegal Avro schema characters.

1.15.3​

Wed Feb 25 11:53:22 PST 2026

  • Adds new sqlHintDdl configuration to specify SQL DDL statements directly.

1.15.2​

Mon Feb 23 12:53:40 PST 2026

  • Fixes false positive from linter on avroHint

1.15.1​

Fri Feb 20 09:26:12 PST 2026

1.15.0​

Thu Feb 19 08:10:18 PST 2026

  • Adds new profile function for generating realistic data about people.

1.14.6​

Wed Feb 18 08:46:13 PST 2026

1.14.5​

Thu Feb 12 10:23:00 PST 2026

1.14.4​

Tue Feb 10 11:02:15 PST 2026

  • Upates Docker base image to Wolfi-base, bringing its OS CVE count to near zero.

1.14.3​

Mon Feb 9 13:34:40 PST 2026

  • Upgrades Docker base image to use a hardened version of Debian Bookworm, further reducing CVE surface area.

1.14.2​

Tue Jan 27 13:25:07 PST 2026

  • Fixes Kafka tombstone validation when using Schema Registry serializers. ShadowTraffic previously inappropriately applied schema checks on null value payloads.

1.14.1​

Thu Jan 22 08:44:58 PST 2026

  • Postgres, MotherDuck, MySQL, SQL Server, and Oracle connections now log their DDL statements when they create tables for you.
  • Fixes spurious linter warning about customFunction argument maps.

1.14.0​

Tue Jan 20 10:19:14 PST 2026

  • Switches Docker base image from Oracle Linux 9 to Debian Bookworm Slim. Debian's base image has a smaller footprint and reduced CVE surface. Unnecessary OS packages have been removed to further harden the image while retaining debugging utilities (ping, curl, top, ps, netstat).
  • Fixes spurious linter warning about geolocation configuration.

1.13.7​

Fri Jan 16 08:43:42 PST 2026

1.13.6​

Thu Jan 15 10:39:15 PST 2026

  • Fixes bug in repeatedly that prevented stateful functions from advancing in certain cases.

1.13.5​

Thu Jan 15 09:33:43 PST 2026

1.13.4​

Fri Jan 9 14:34:33 PST 2026

  • Fixes issue where Kafka Data Quality rules may not fire for Avro serialized data.

1.13.3​

Fri Jan 9 08:44:38 PST 2026

1.13.2​

Wed Jan 7 13:01:17 PST 2026

  • Fixes histogram in lookup calls with multiple depending generators.

1.13.1​

Tue Jan 6 10:25:45 PST 2026

1.13.0​

Mon Jan 5 12:02:05 PST 2026

1.12.7​

Mon Dec 29 14:34:22 PST 2025

1.12.6​

Mon Dec 22 10:58:08 PST 2025

  • Kafka topics can now be created with custom configuration.

1.12.5​

Fri Dec 19 09:56:55 PST 2025

1.12.4​

Wed Dec 17 11:57:02 PST 2025

1.12.3​

Mon Dec 15 13:59:44 PST 2025

1.12.2​

Thu Dec 11 15:33:33 PST 2025

  • Echos the version of ShadowTraffic you're running on startup.
  • Fixes spurious linter warnings on the easing and easingChain functions.

1.12.1​

Wed Dec 10 13:31:31 PST 2025

  • Fixes bug that prevented manual table policies from working with backticks in MySQL.

1.12.0​

Mon Dec 8 14:14:26 PST 2025

  • Fixes a deadlock bug that prevents shutdown when using the MotherDuck connection with a > 1 pool size.
  • Adds logging to show MotherDuck connection acquisition timing.

1.11.14​

Fri Dec 5 08:29:37 PST 2025

  • Fixes OS CVEs in the ShadowTraffic Oracle Linux base image.

1.11.13​

Wed Dec 3 14:50:26 PST 2025

1.11.12​

Tue Dec 2 10:06:28 PST 2025

  • Fixes repeatedly to correctly track stateful, nested function calls.

1.11.11​

Mon Nov 17 15:19:44 PST 2025

  • Fixes linter bug misapplied to reader generators.

1.11.10​

Mon Nov 17 10:45:58 PST 2025

  • Fixes spurious linter warnings on startup for some functions.

1.11.9​

Fri Nov 14 18:10:46 PST 2025

  • Fixes a regression in the MotherDuck connection, introduced in 1.11.5, that can prevent ShadowTraffic from writing greater than 100K rows to MotherDuck.

1.11.8​

Thu Nov 13 10:05:33 PST 2025

  • Automatically includes lint output when generating data, too.

1.11.7​

Wed Nov 12 15:05:45 PST 2025

1.11.6​

Wed Nov 5 13:45:28 PST 2025

  • S3, Google Cloud Storage, Azure Blob Storage, and file system connectors can now specify customSuffix to override their file output extensions.

1.11.5​

Wed Nov 5 10:49:29 PST 2025

1.11.4​

Tue Nov 4 13:08:26 PST 2025

  • Fixes bug where MotherDuck tables were always created in the default schema. Tables can now be fully qualified with db.schema.table syntax.

1.11.3​

Mon Nov 3 09:51:08 PST 2025

  • Fixes reader generator metrics on the MotherDuck connection.

1.11.2​

Mon Oct 27 18:55:55 CDT 2025

  • stateMachine can now take functional arguments for both its initial and transitions parameter, making it more customizable.

1.11.1​

Fri Oct 24 13:12:02 PDT 2025

  • Fixes spurious stacktrace during shutdown of a long running MotherDuck connection.

1.11.0​

Thu Oct 23 18:54:19 PDT 2025

1.10.1​

Tue Oct 21 19:18:14 PDT 2025

  • Dramatically improves MotherDuck insertion performance.
  • Fixes Postgres table creation bug introduced in previous release.

1.10.0​

Mon Oct 20 15:26:16 PDT 2025

1.9.3​

Thu Oct 16 13:39:15 PDT 2025

  • Fixes a bug that caused maxEvents to prematurely terminate a generator under certain conditions, like forks with high number of contending keys.

1.9.2​

Thu Oct 16 10:25:52 PDT 2025

1.9.1​

Wed Oct 15 12:08:08 PDT 2025

  • Introduces batching to the KurrentDB connection.

1.9.0​

Tue Oct 14 15:03:55 PDT 2025

  • Upgrades the dependencies underpinning Kafka to Confluent 8.0.
  • Upgrades the EventStore connection to the latest, newly branded KurrentDB.

1.8.9​

Wed Oct 8 11:56:03 PDT 2025

  • Fixes bug where time wasn't respected as a global configuration parameter.

1.8.8​

Mon Oct 6 19:31:13 PDT 2025

  • Fixes bug where Avro schemas would be incorrectly constructed on dynamically nested arrays.

1.8.7​

Fri Oct 3 09:51:23 PDT 2025

  • Extends serialize to handle SQL Server native datetime columns.

1.8.6​

Fri Oct 3 09:16:45 PDT 2025

  • Removes hanging debug statement from previous release.

1.8.5​

Fri Oct 3 08:58:27 PDT 2025

  • The metrics port can now be set with --metrics-port, and set to 0 to randomize its value.

1.8.4​

Wed Oct 1 13:21:30 PDT 2025

  • Fixes ShadowTraffic Studio's display when multiple generators share the same output target.
  • Fixes SQL Server connection always writing to the default dbo schema.

1.8.3​

Mon Sep 29 10:22:12 PDT 2025

1.8.2​

Thu Sep 25 14:12:03 PDT 2025

  • Permits dynamic values in easing's duration and events parameters.

1.8.1​

Wed Sep 24 13:44:26 PDT 2025

  • Performs a deep merge when loadJsonFile is used with an array of files, making it easier to specify nested overrides.

1.8.0​

Mon Sep 22 09:52:42 PDT 2025

1.7.1​

Mon Sep 15 11:13:32 PDT 2025

  • easing and easingChain can now iterate along a number of events, not just a timed duration.

1.7.0​

Tue Sep 9 09:10:11 PDT 2025

  • Adds new duration function for expressing wallclock values.
  • Fixes multiBlob Parquet serialization with the file system connection.
  • Adds launch-time validation to catch illegal lookup function calls to future schedule stages that haven't yet run.

1.6.3​

Mon Sep 8 10:26:38 PDT 2025

  • Fixes Parquet serialization when multiBlob is used with S3 and other object stores. It previously applied an incorrect schema per blob.

1.6.2​

Fri Sep 5 07:44:22 PDT 2025

1.6.1​

Thu Sep 4 08:05:56 PDT 2025

  • Fixes bug in math that would throw a stacktrace when expr is missing instead of a helpful error message.

1.6.0​

Wed Sep 3 12:11:32 PDT 2025

  • Adds new AI function that can generate free text from AWS Bedrock and Ollama.

1.5.8​

Tue Sep 2 09:00:14 PDT 2025

1.5.7​

Thu Aug 21 10:53:55 PDT 2025

  • easing can now take dynamic to and from parameters.

1.5.6​

Mon Aug 18 15:12:51 PDT 2025

  • Adds new sample function modifier.

1.5.5​

Thu Aug 14 14:05:32 PDT 2025

1.5.4​

Thu Aug 14 08:26:52 PDT 2025

  • Fixes a bug in Kafka schema inference that creates single-type unions.

1.5.3​

Thu Aug 14 07:57:45 PDT 2025

  • Fixes an issue where bytes would emit too many bytes when serialized with Avro through Kafka.
  • Fixes poor Avro validation error messages when serializing incorrect deeply nested, unionized data.
  • Mitigates CVE-2024-57699.

1.5.2​

Wed Aug 13 11:25:42 PDT 2025

  • Fixes a bug where Avro serialization errors would falsely claim valid data is the problem. In particular, it would incorrectly target records with an implicit parent namespace.

1.5.1​

Tue Aug 12 12:06:41 PDT 2025

  • Fixes a bug where scheduled generators couldn't lookup each other if they shared the same name.

1.5.0​

Tue Aug 12 10:49:08 PDT 2025

1.4.3​

Fri Aug 8 07:37:24 PDT 2025

  • Allows throughput to take dynamic values. The docs previously erroneously specified it could do this.

1.4.2​

Wed Aug 6 14:03:03 PDT 2025

  • Fixes authentication forwarding for auto-schema downloading with Kafka connections.

1.4.1​

Wed Aug 6 13:37:52 PDT 2025

  • Fixes a spurious authentication issue with Kafka Schema Registry when using SR-enabled serializers.

1.4.0​

Wed Aug 6 10:54:16 PDT 2025

  • Fixes a performance regression when using lookup with histogram. Large lookup targets > 10K elements would dramatically reduce output speed. This is now fixed.

1.3.0​

Mon Aug 4 12:09:11 PDT 2025

1.2.0​

Thu Jul 31 09:48:08 PDT 2025

  • Adds support for Kafka/Confluent client-side field level encryption (AWS, Azure, GCP, Vault, and local). Follow Confluent's docs to enable it.

1.1.11​

Mon Jul 28 11:49:18 PDT 2025

1.1.10​

Wed Jul 23 15:27:52 PDT 2025

  • intervals no longer requires a fallback value or index.

1.1.9​

Wed Jul 23 10:10:21 PDT 2025

  • Adds new function cycle to emit looping sequences of elements.

1.1.8​

Mon Jul 14 10:58:08 PDT 2025

  • Loosens the schema for waypoints so that coordinates can be an array of function calls.

1.1.7​

Fri Jul 11 08:09:41 PDT 2025

1.1.6​

Wed Jul 9 10:34:30 PDT 2025

1.1.5​

Wed Jul 2 11:56:11 PDT 2025

  • Permits startingFrom in sequentialString to be a function call.
  • Fixes a bug related to customFunction that would incorrectly advance other fields state.

1.1.4​

Tue Jul 1 12:03:42 PDT 2025

  • Fixes a bug in ShadowTraffic Studio that crashes the UI when state machine delays are used.

1.1.3​

Tue Jul 1 11:20:59 PDT 2025

1.1.2​

Fri Jun 27 11:55:57 PDT 2025

1.1.1​

Mon Jun 23 08:55:28 PDT 2025

  • Improves error messages from AWS KMS while creating leases.

1.1.0​

Wed Jun 18 10:44:20 PDT 2025

  • Adds new MySQL connection.

1.0.10​

Fri Jun 13 08:42:19 PDT 2025

1.0.9​

Tue Jun 10 09:35:00 PDT 2025

  • Adds new avroHint function modifier to override automatic schema inference guesses.

1.0.8​

Wed Jun 4 15:25:44 PDT 2025

  • Fixes bug where named lookup calls against the same collection would collide results.

1.0.7​

Wed Jun 4 09:02:21 PDT 2025

  • Improves error output when ShadowTraffic is fed an invalid license.

1.0.6​

Tue Jun 3 11:33:45 PDT 2025

  • Fixes Avro schema inference bug on array types.

1.0.5​

Mon Jun 2 11:34:14 PDT 2025

1.0.4​

Wed May 28 10:53:32 PDT 2025

  • Defaults ShadowTraffic to use 65% of available container memory, increased from 25%.
  • Adds concurrency and buffer depth parameters for S3.
  • Fixes an issue with the S3 connection where large objects could deplete memory.

1.0.3​

Thu May 15 14:36:00 PDT 2025

  • Fixes Unicode rendering issues related to GraalVM that would emit ? characters in some log lines.

1.0.2​

Thu May 15 11:14:49 PDT 2025

  • Narrows the scope of state that customFunction calls carry to minimize type conversion errors.

1.0.1​

Wed May 14 09:57:35 PDT 2025

  • Fixes customFunction composite types caused by some JavaScript return values.

1.0.0​

Tue May 13 14:22:41 PDT 2025

  • Introduces customFunction that lets you extend ShadowTraffic with Python, Ruby, and JavaScript.
  • Changes the default JVM to use GraalVM 23.0.2, mainly in support of custom functions.

0.19.4​

Mon Apr 28 10:51:34 PDT 2025

  • Deprecates toInstant and instead adds a new serialize target for Postgres timestamps. This makes it easier decouple type-system specifics from the data you're generating.

0.19.3​

Mon Apr 28 09:20:53 PDT 2025

  • Expands waypoints so that its coordinates parameter can be resolved through lookups.

0.19.2​

Fri Apr 25 12:36:01 PDT 2025

  • Adds new function modifier serialize to control last mile byte arrangement.

0.19.1​

Mon Apr 21 09:31:42 PDT 2025

  • Fixes a bug where stateful functions would fail if nested under map calls.

0.19.0​

Tue Apr 15 15:40:33 PDT 2025

  • Adds new leasing capability for Enterprise users. Leasing allows central teams to issue scoped, short-term access to subteams.

0.18.9​

Mon Apr 14 12:27:27 PDT 2025

  • Fixes further edge cases continuing through Kafka data quality rules errors.

0.18.8​

Wed Apr 9 13:28:28 PDT 2025

  • Dramatically improves the helpfulness of error messages related to Kafka Avro serialization.

0.18.7​

Mon Apr 7 08:44:20 PDT 2025

  • Fixes issues related to Kafka data quality rules suppressing errors.
  • Relaxes schema of lookup to no longer require path.

0.18.6​

Fri Apr 4 14:10:15 PDT 2025

  • Adds new parameter to continue generating data to Kafka when data quality rules are violated.

0.18.5​

Wed Apr 2 14:47:15 PDT 2025

  • Fixes --bootstrap-from-avro-schema to correctly handle union types by specifying type hints in generated output.

0.18.4​

Wed Apr 2 13:06:08 PDT 2025

  • Expands elide's whenAbsent check to trigger on false too, not just null.
  • --bootstrap will now bundle the original schema as a hint with Kafka targeted generators.
  • Swaps the order the decimals and clamp modifiers run. decimals now runs afterward since clamp may introduce more floating point places.

0.18.3​

Tue Apr 1 11:24:56 PDT 2025

  • Fixes a bug where reusing a generator across stages could throw an exception.

0.18.2​

Mon Mar 31 10:49:40 PDT 2025

  • Upgrades the set of available Kafka serializers and permits use of kafka-schema-rules.

0.18.1​

Mon Mar 31 08:15:38 PDT 2025

  • Fixes a bug where schedule may throw an exception when used with --stdout.
  • Fixes --bootstrap-* commands for Kafka to now output the right corresponding serializers for the requested format.

0.18.0​

Tue Mar 25 10:05:42 PDT 2025

  • Breaking change: all events are now instantaneously available through the lookup function. The previous behavior, which prevented events from being looked up until they were acknowledged by the target backend systems, consistently caused confusion. This change now unifies how data is generated with and without the --stdout flag. See below to restore the old behavior.
  • Adds new lookupScope configuration. Set this value to committed to restore the old behavior.
  • Drops the --allow-unsafe-lookups CLI flag. Use lookupScope instead.

0.17.16​

Mon Mar 24 13:23:10 PDT 2025

0.17.15​

Tue Mar 18 12:42:38 PDT 2025

0.17.14​

Mon Mar 17 09:43:26 PDT 2025

0.17.13​

Thu Mar 13 15:25:01 PDT 2025

  • Fixes geolocation support, which recently broke through the underlying Docker image dependencies.

0.17.12​

Tue Mar 11 12:01:43 PDT 2025

  • Improves the performance of the Azure Blob Storage connector for large events.

0.17.11​

Thu Mar 6 13:05:51 PST 2025

  • Fixes toInstant's schema not requiring the ms parameter.

0.17.10​

Wed Mar 5 11:28:08 PST 2025

  • Fixes a bug where UUIDs weren't casting correctly to Postgres.
  • Fixes Postgres dropAndCreate so that drops cascading, avoiding throwing errors on dependent object chains.

0.17.9​

Wed Mar 5 09:25:47 PST 2025

  • Fixes bug where top-level math variables that refer to other variables could evaluate in the wrong order.

0.17.8​

Tue Mar 4 11:24:53 PST 2025

0.17.7​

Tue Mar 4 09:45:32 PST 2025

  • Adds new log format for file system, S3, GCS, and Azure Blob Storage. The same parameter works across all 4 connectors.

0.17.6​

Mon Mar 3 10:46:58 PST 2025

0.17.5​

Mon Mar 3 09:38:14 PST 2025

0.17.4​

Fri Feb 28 11:59:53 PST 2025

  • Configuration files can now contain JavaScript style comments (both // single line and slash asterisks multiline). If use comments, you may want to suffix your configuration files with *.js so your editor tolerates them.

0.17.3​

Thu Feb 27 10:12:45 PST 2025

0.17.2​

Wed Feb 26 11:40:24 PST 2025

  • Adds the ability to log HTTP exchanges in the webhook connector.
  • Fixes a bug that caused output to interleave when using stages with Kafka. Previously, the underlying Kafka producer did not flush its buffer between stages, which could cause out of order output on the topic's partitions.

0.17.1​

Tue Feb 25 13:54:59 PST 2025

  • --config can now take a URL to download from a network/the internet.

0.17.0​

Fri Feb 21 16:03:45 PST 2025

0.16.5​

Thu Feb 20 10:27:41 PST 2025

  • File suffixes generated by the file connection will now always increase in strict order.

0.16.4​

Wed Feb 19 08:55:37 PST 2025

0.16.3​

Tue Feb 18 15:43:20 PST 2025

  • Adds auth mechanisms to connect to Kafka clusters hosted on Amazon MSK.

0.16.2​

Fri Feb 14 12:37:34 PST 2025

  • Fixes static variable validation check missed parent-defined repeatedly local variables.

0.16.1​

Fri Feb 14 11:11:26 PST 2025

  • Fixes loadJsonFile not printing enough error information during some misconfigured multi-file loads.

0.16.0​

Thu Feb 13 15:30:16 PST 2025

  • Breaking change: the fileSystem has been updated to require a directory at the generator-level instead of a connection level. This makes it identical in behavior to the S3, Azure, and GCS connectors. Apologies for the breakage.
  • Adds the ability to merge multiple JSON files together during preprocessing.
  • Adds the ability to write multiple files for both file system and Azure connections.
  • Adds new serialization option explodeJsonlArrays to break arrays onto multiple lines.
  • Adds new optional localVars parameter to repeatedly to specify local, randomized values.
  • S3, Azure, GCS, and file system connections can now turn off batching and instantenously write files on each iteration.

0.15.8​

Thu Feb 13 12:44:20 PST 2025

  • Fixes additional SSL connections issues with Kafka and Java 23.

0.15.7​

Mon Feb 10 08:55:58 PST 2025

  • Adds new --allow-unsafe-lookups CLI flag. This flag allows lookup to access uncommitted data, which is required for generating identical data streams to asynchronously written connections. In other words, this flag allows you to trade safety for determinism.

0.15.6​

Wed Feb 5 15:00:43 PST 2025

  • Adds explicit support for S3 path-style access, which is useful for S3-compatible services like MinIO.
  • Fixes Kafka SSL connectivity, which regressed in 0.15.0 after upgrading the backing Docker image to Java 23.

0.15.5​

Wed Feb 5 10:41:16 PST 2025

  • Fixes a bug where nested stateMachine calls wouldn't terminate the generator when they had no further transitions.

0.15.4​

Wed Feb 5 10:10:11 PST 2025

0.15.3​

Tue Feb 4 15:10:18 PST 2025

  • Fixes a regression introduced in 0.15.0 where S3 connections would crash in certain environments.

0.15.2​

Tue Feb 4 13:41:11 PST 2025

  • Fixes a bug where ShadowTraffic would crash if a variable referred to another variable that called lookup.

0.15.1​

Mon Feb 3 19:24:32 PST 2025

0.15.0​

Mon Feb 3 15:53:17 PST 2025

0.14.6​

Mon Feb 3 11:29:03 PST 2025

0.14.5​

Fri Jan 31 14:51:47 PST 2025

  • Adds new strategy parameter to lookup to bias how events are chosen: first, last, or randomized.

0.14.4​

Tue Jan 28 11:29:39 PST 2025

  • Fixes bug where file connections that use subdir could roll the same file suffix multiple times.

0.14.3​

Tue Jan 28 08:30:54 PST 2025

  • Increases the speed with which new forks are launched, which is helpful if you're starting a large number of small forks.

0.14.2​

Mon Jan 27 13:16:38 PST 2025

  • Changes the semantics of fork keys so that if key is ever null, an error is raised. Previously, ShadowTraffic would silently stop forking, which was confusing and unhelpful.
  • Fixes a bug where the use of oneTimeKeys would not always allow ShadowTraffic to exit if it generated all expected data.
  • Fixes bootstrap generated output that targets Kafka connections. Previously the default Kafka connection URL was malformed.

0.14.1​

Tue Jan 21 10:17:45 PST 2025

0.14.0​

Fri Jan 17 16:32:54 PST 2025

0.13.11​

Wed Jan 15 12:23:32 PST 2025

  • Fixes lack of schema checks on map function.
  • Fixes situation where using iterateVars on repeatedly would break with nested variables.

0.13.10​

Tue Jan 14 14:32:59 PST 2025

  • Improves the error message raised when you using Avro with missing fields in a record.

0.13.9​

Mon Jan 13 12:33:26 PST 2025

  • Adds new --report-benchmark command line flag to print performance metrics after data generation completes.

0.13.8​

Thu Jan 2 08:48:02 PST 2025

  • Improves the performance of weightedOneOf.

0.13.7​

Fri Dec 13 11:33:25 PST 2024

  • Fixes bug where oneOf and weightedOneOf may throw exceptions when used with subnested, stateful functions.

0.13.6​

Fri Dec 13 10:35:33 PST 2024

  • Adds new loopAfter parameter to sequentialInteger.

0.13.5​

Fri Dec 13 07:57:48 PST 2024

  • Improves performance related to throttling generators.
  • Removes stray debug output.

0.13.4​

Thu Dec 12 14:18:44 PST 2024

  • Fixes a bug where ShadowTraffic might hang when using more outgoing connections than number of cores on the machine.

0.13.3​

Wed Dec 11 11:07:18 PST 2024

0.13.2​

Mon Dec 9 11:31:56 PST 2024

  • Fixes issue with EventStore client where certain exceptions were suppressed.

0.13.1​

Fri Dec 6 12:22:24 PST 2024

  • Fixes ShadowTraffic from hanging on termination. This bug was introduced in 0.12.9

0.13.0​

Thu Dec 5 09:46:18 PST 2024

0.12.15​

Wed Dec 4 12:29:29 PST 2024

  • Adds support for method, headers, and query parameters in the webhook connection.

0.12.14​

Tue Dec 3 11:26:42 PST 2024

0.12.13​

Mon Dec 2 14:05:21 PST 2024

  • Fixes fork keys functions that use internal state from throwing errors.

0.12.12​

Mon Dec 2 09:52:19 PST 2024

  • Adds optional keyPrefix for lookups against S3 and Azure Blob Storage to narrow searched output.
  • Fixes sequentialString by requiring the presence of an expr parameter.

0.12.11​

Mon Dec 2 08:10:24 PST 2024

  • Fixes --bootstrap so only zero-argument Faker expressions are generated. This means the backing configuration file will always work with no further modifications.

0.12.10​

Wed Nov 27 09:04:46 PST 2024

  • Fixes an additional memory leak related to serializing Parquet data.

0.12.9​

Tue Nov 26 22:57:05 PST 2024

  • Fixes a memory leak affecting long-lived generators that write to S3, Google Cloud Storage, Azure Blob Storage, Postgres, and file systems.

0.12.8​

Mon Nov 18 11:04:15 PST 2024

  • Bootstrap now uses prediction to guess string expressions rather than defaulting to a single stub expression.

0.12.7​

Mon Nov 18 08:57:47 PST 2024

  • Fixes --bootstrap-to bug for fileSystem, proton, and timeplus arguments.

0.12.6​

Fri Nov 15 14:44:23 PST 2024

  • Adds new --bootstrap command line switch to automatically generate ShadowTraffic files from JSON Schema files.

0.12.5​

Thu Nov 14 12:42:33 PST 2024

0.12.4​

Thu Nov 14 11:17:21 PST 2024

  • Fixes ShadowTraffic Studio from rendering bad output if loaded mid-run.

0.12.3​

Wed Nov 13 13:14:39 PST 2024

  • Fixes leaked library debug message on startup.
  • Improves baseline performance by about 50%.

0.12.2​

Tue Nov 12 13:41:59 PST 2024

  • Allows string's locale to be overriden, defaulting to United States/English.

0.12.1​

Fri Nov 8 11:10:55 PST 2024

0.12.0​

Thu Nov 7 11:34:54 PST 2024

  • Improves baseline performance of ShadowTraffic by about 25%.
  • Replaces string templating library with Datafaker, resulting in much higher performance for string operations.
  • Switches S3 connection to use concurrent uploads. Keys will begin uploading in serial order, but may complete out of order.

0.11.5​

Tue Nov 5 13:46:04 PST 2024

  • Improves the performance of the string function by eliding unused code.

0.11.4​

Tue Nov 5 08:07:19 PST 2024

  • Fixes cardinality from working with certain stateful functions.

0.11.3​

Tue Oct 29 08:05:56 PDT 2024

  • Allows configuring log level connections to Kafka.

0.11.2​

Thu Oct 24 09:19:41 PDT 2024

  • Fixes ShadowTraffic Studio failing to render data on state machines with overriden throttles.

0.11.1​

Wed Oct 23 09:42:25 PDT 2024

  • Fixes the EventStore client from suppressing connection errors if ShadowTraffic exits too quickly.

0.11.0​

Tue Oct 22 14:08:40 PDT 2024

  • Deprecates kafkaKeyAvroSchemaHint, kafkaValueAvroSchemaHint, kafkaKeyProtobufSchemaHint, and kafkaValueProtobufSchemaHint in favor of two simpler configurations: avroSchemaHint and protobufSchemaHint.

0.10.11​

Tue Oct 22 08:26:00 PDT 2024

  • Fixes bug where multiple sequentialInteger calls in the same level of a map would erase each other's state.

0.10.10​

Mon Oct 21 15:57:59 PDT 2024

0.10.9​

Mon Oct 21 14:44:50 PDT 2024

  • Fixes timeMultiplier throwing spurious errors.
  • Fixes bug causing competiting state in vars and stateMachine not to advance correctly.

0.10.8​

Wed Oct 16 10:40:34 PDT 2024

  • S3, Google Cloud Storage, Azure Blob Storage, and filesystem connections now support batching on number of serialized bytes with batchBytes.
  • Adds uniform support for json, pretty json, jsonl, and parquet as serialization options for S3, Google Cloud Storage, Azure Blob Storage, and filesystem connections.
  • Ditto for compression: all of these connections now support gzip.

0.10.7​

Tue Oct 15 08:43:30 PDT 2024

  • Fixes regression on updates and deletes to Postgres connection.

0.10.6​

Mon Oct 14 13:46:46 PDT 2024

  • Adds Parquet serialization support for Google Cloud Storage.

0.10.5​

Mon Oct 14 10:46:09 PDT 2024

  • Fixes uniformDistribution throwing error on decimal bounds under 1.

0.10.4​

Fri Oct 11 13:37:01 PDT 2024

  • Fixes EventStore connection eventId fields not handling UUIDs correctly.

0.10.3​

Fri Oct 11 11:34:00 PDT 2024

  • Removes addresses from geolocation that don't specify a city.

0.10.2​

Fri Oct 11 08:02:36 PDT 2024

  • Fixes error output from the webhook connector when connections timeout.

0.10.1​

Thu Oct 10 11:56:57 PDT 2024

  • Adds new keepAlive parameter so that forks can continue to respawn even after reaching maxForks.

0.10.0​

Wed Oct 9 12:59:56 PDT 2024

  • Adds support for SQL Server.
  • Allows top-level stateMachine functions to override throttleMs local configuration.

0.9.10​

Tue Oct 8 15:15:51 PDT 2024

  • Fixes JSON formatting in ShadowTraffic Studio for booleans and UUIDs.

0.9.9​

Tue Oct 8 10:48:41 PDT 2024

  • Adds new optional names parameter to the math function to isolate local variable usage.
  • Fixes error message from the math function when the expression is malformed.

0.9.8​

Mon Oct 7 15:49:50 PDT 2024

  • Increases the default settings of the webhook connection to support higher concurrency.

0.9.7​

Mon Oct 7 15:23:37 PDT 2024

  • Exposes maxRequests parameter in the webhook connection to control request concurrency.

0.9.6​

Mon Oct 7 10:59:18 PDT 2024

  • Fixes CVE-2024-47561 pertaining to Avro.
  • Upgrades Confluent Kafka serializers to 7.7.1

0.9.5​

Thu Oct 3 15:07:47 PDT 2024

  • Adds support for new string function transformation removeSubstring.
  • Fixes uncaught error when uniformDistributionBoundaries aren't legal. A better error is now printed.

0.9.4​

Thu Oct 3 10:38:01 PDT 2024

  • Adds guards to prevent the webhook connection from exhausting memory on too many outbound requests.

0.9.3​

Wed Oct 2 14:16:49 PDT 2024

  • Dramatically improves the performance of the webhook connection.

0.9.2​

Wed Oct 2 11:40:46 PDT 2024

  • Fixes how UUIDs are printed in ShadowTraffic Studio.
  • Fixes timeline stretching beyond viewport in ShadowTraffic Studio.

0.9.1​

Tue Oct 1 14:59:57 PDT 2024

  • Adds new ulid function.

0.9.0​

Tue Oct 1 09:12:41 PDT 2024

0.8.12​

Thu Sep 26 08:18:48 PDT 2024

  • Fixes bug preventing UUIDs from being written as a first-class type to Postgres with pgHint.

0.8.11​

Wed Sep 25 14:30:06 PDT 2024

  • Fixes vertical scrolling in ShadowTraffic Studio with a large number of generators.

0.8.10​

Tue Sep 24 15:43:14 PDT 2024

  • Extends maxBytes to work for Kafka and webhook connections.

0.8.9​

Tue Sep 24 14:46:10 PDT 2024

  • Adds new maxBytes configuration to limit the number of data written to an output connection. Note this currently only works with S3, Azure Blob Storage, and file system connections.

0.8.8​

Tue Sep 24 11:58:45 PDT 2024

  • Adds new throughput configuration to control how many events per second are generated.

0.8.7​

Mon Sep 23 12:57:04 PDT 2024

  • Fixes broken images in ShadowTraffic Studio when used with Chrome.

0.8.6​

Mon Sep 23 11:05:45 PDT 2024

  • Improves the error message when Protobuf schema hints aren't set correctly.

0.8.5​

Mon Sep 23 10:53:46 PDT 2024

  • Adds new Kafka type hints for Protobuf serialization: kafkaKeyProtobufHint and kafkaValueProtobufHint.

0.8.4​

Fri Sep 20 09:23:13 PDT 2024

  • Adds new advanceBy parameter to sequentialInteger and allows both the starting and step values to be functions.

0.8.3​

Wed Sep 11 12:37:03 PDT 2024

  • Adds new cardinality function modifier to limit the number of uniquely generated values.

0.8.2​

Tue Sep 10 15:25:55 PDT 2024

  • Fixes visibility of lookup calls from forked generators in ShadowTraffic Studio.

0.8.1​

Mon Sep 9 14:42:29 PDT 2024

  • Drops dataShape parameter from webhooks, allowing arbitrary shapes of data to be generated.
  • Drops top-level Decodable support, which was misdesigned as a piggyback onto webhooks.

0.8.0​

Mon Sep 9 09:33:35 PDT 2024

  • Improves schema checks on fork configuration.
  • Raises an error when the reserved var mapItem is used outside of a mapEach call.
  • Removes deprecated geospatialPath and geospatialPathScheme functions. Use waypoints instead.

0.7.23​

Thu Sep 5 13:05:24 PDT 2024

  • Fixes forkKey variable references in varsOnce blocks. These previously returned null always.

0.7.22​

Wed Sep 4 14:34:31 PDT 2024

  • Fixes lookup calls in varOnce blocks. These previously returned null.

0.7.21​

Tue Sep 3 15:00:02 PDT 2024

  • Permits null transitions in state machines to terminate the generator early.

0.7.20​

Tue Sep 3 11:28:02 PDT 2024

  • Fixes bug that corrupted state when using multiple, inline state machines.

0.7.19​

Tue Sep 3 09:20:33 PDT 2024

  • Permits dynamic key prefixes for Azure Blob Storage.
  • Improves pretty printing on deeply nested data when using --stdout.

0.7.18​

Fri Aug 30 13:17:20 PDT 2024

  • Loosens the JSON schema for Kafka connections, permitting an open-ended range of authentication methods and parameters.

0.7.17​

Fri Aug 30 12:45:36 PDT 2024

  • Prints out the offending line and column number when the configuration file fails JSON parsing.

0.7.16​

Thu Aug 29 10:04:15 PDT 2024

  • Removes pluckEach in favor of a more general solution. See next lines.
  • Adds new function mapEach that supports general transformations of arrays.
  • Adds new reserved variable mapItem.

0.7.15​

Wed Aug 28 15:29:14 PDT 2024

  • Allows fork keys to be traced as origins in ShadowTraffic Studio.

0.7.14​

Wed Aug 28 14:32:20 PDT 2024

  • Allows add, subtract, multiply, divide, min, and max to take functions as arguments instead on only arrays.

0.7.13​

Wed Aug 28 14:22:27 PDT 2024

  • Adds new pluckEach transformation function.

0.7.12​

Fri Aug 23 08:47:43 PDT 2024

  • Adds new clamp function modifier.

0.7.11​

Wed Aug 21 14:35:29 PDT 2024

  • Fixes ShadowTraffic Studio running with development-time resources instead of a production build.

0.7.10​

Tue Aug 20 15:40:06 PDT 2024

  • Adds new oneTimeKeys parameter to fork, making it possible for terminates forks to never restart.
  • Loosens stateMachine schema and allows sequential state transitions to run 0 times. This is useful if you want to sometimes run a state and sometimes skip it.

0.7.9​

Mon Aug 19 13:42:37 PDT 2024

  • Adds new optional compression parameter to the S3 connection.

0.7.8​

Fri Aug 16 10:30:36 PDT 2024

  • math now produces a better error when you reference a variable that evaluates to null.

0.7.7​

Thu Aug 15 11:12:58 PDT 2024

  • Changes file suffixes for the S3 and Azure Blob Storage connections to use ULIDs instead of nanosecond timestamps. This ensures multiple writers won't collide and write to the same file, while still making the file names sortable.

0.7.6​

Tue Aug 13 09:13:59 PDT 2024

  • Adds new whenPresent and whenAbsent parameters for the elide function modifier.
  • Fixes schema check for null rates

0.7.5​

Mon Aug 12 15:58:03 PDT 2024

  • Fixes elide used in the position of an array index. This would previously raise an exception.

0.7.4​

Mon Aug 12 13:27:43 PDT 2024

  • Fixes constant's ability to handle functions.

0.7.3​

Wed Aug 7 10:47:04 PDT 2024

  • Adds new infix math function to make it easier to evaluate long, nested math expressions.

0.7.2​

Thu Aug 1 11:35:01 PDT 2024

  • Fixes cases where the Kafka io.shadowtraffic.kafka.serdes.JsonSerializer may incorrectly serialize decimals values as literal ratios and produce incorrect JSON.

0.7.1​

Wed Jul 31 08:30:44 PDT 2024

  • Uncaps the throttle of the free trial for the first 5,000,000 generated events. This helps people assess ShadowTraffic's true performance firsthand before buying.

0.7.0​

Tue Jul 30 13:47:33 PDT 2024

0.6.12​

Thu Jul 18 13:45:33 PDT 2024

  • Adds new pow function.

0.6.11​

Thu Jul 18 13:45:33 PDT 2024

  • Adds new optional names parameter to the string function, letting you scope identifiers that may be used within expr.
  • Raises a better error when iterateVars names a non-existent variable.

0.6.10​

Tue Jul 16 14:40:54 PDT 2024

  • Fixes iterateVars parameter of repeatedly to refresh variables in dependency order.
  • Fixes nested function call errors that could overlap each other and hide the real error message.

0.6.9​

Tue Jul 16 11:52:08 PDT 2024

  • Fixes references to generators in schedule stages that terminate and are reused through a lookup. This previously caused a stacktrace, but is now fixed.

0.6.8​

Tue Jul 16 11:11:29 PDT 2024

  • Adds new iterateVars parameter to repeatedly so that element variable references can vary between elements.

0.6.7​

Tue Jul 16 10:06:29 PDT 2024

  • Allows geolocation to generate multiple formats per invocation.

0.6.6​

Mon Jul 15 14:26:36 PDT 2024

  • Adds jsonl as an output format for the file system connector.

0.6.5​

Fri Jul 12 10:17:51 PDT 2024

  • Fixes drilling into object formatting with path for the geolocation function.

0.6.4​

Mon Jul 8 10:50:45 PDT 2024

  • Fixes cosmetic dependency warning when using the Azure Blob Storage connection. The dependencies were always present, but not checkable through the approach Azure uses. This has been fixed.

0.6.3​

Mon Jul 1 15:31:09 PDT 2024

  • Adds new transformation parameters to the output of the string function.

0.6.2​

Mon Jul 1 09:23:43 PDT 2024

  • Fixes bug where multiple generators on the filesystem connector would start their file suffixes at different numbers.

0.6.1​

Fri Jun 28 14:40:29 PDT 2024

  • Adds Azure Blob storage as a new connection type.
  • Improves the error that gets emitted when ShadowTraffic tries to automatically generate a schema for a function whose type it can't infer.
  • Improves the error that gets emitted when Avro serialization fails because of either missing fields or mistyped unions.

0.6.0​

Tue Jun 25 13:44:05 PDT 2024

  • Adds new feature, schedule, that allows orchestrating when generators run. See the overview, and reference sections for stages and loop.

0.5.11​

Mon Jun 24 12:24:09 PDT 2024

  • Adds support for headers in Kafka records.
  • Fixes bug where forks with state initialization wouldn't have access to variables.
  • Prints better error message when waypoints receives coordinates that don't match expectations.
  • Removes the number function. This was added early on with a different API in mind. Use uniformDistribution or normalDistribution to generate floating point numbers instead.

0.5.10​

Fri Jun 21 11:14:27 PDT 2024

  • Adds new fileSystem connection.

0.5.9​

Thu Jun 20 09:36:11 PDT 2024

  • Adds new maxMs generator configuration to cap how long a generator may run for.
  • Adds new default parameter to the env function for fallback values.

0.5.8​

Wed Jun 19 10:25:27 PDT 2024

  • Auto-creating Kafka topics will now use the broker default partition count and replication factor, instead of forcing 1 for both respectively.

0.5.7​

Thu Jun 13 11:54:35 PDT 2024

  • Adds new time global configuration to override the starting wallclock time.
  • Adds new timeMultiplier global configuration to change the rate at which time advances.

0.5.6​

Wed Jun 12 11:19:16 PDT 2024

  • loadJsonFile and env now work recursively, meaning you can chain loaded files and nest environment variables within.
  • loadJsonFile's data parameter is now evaluated for other preprocessors, like env, before it executes.

0.5.5​

Tue Jun 11 13:27:16 PDT 2024

  • Prevents raw stack trace from leaking out when state machines transition to a non-existent state. This now shows a helpful error message.

0.5.4​

Tue Jun 11 12:35:21 PDT 2024

  • Adds support for eventId in Event Store connections.
  • Allows eventType to be the result of a function call in Event Store connections.
  • Allows eventType to be overriden within a state machine in Event Store connections.
  • Adds new optional streamSuffix to qualify Event Store stream names.

0.5.3​

Tue Jun 11 10:35:13 PDT 2024

  • Adds new throttleMs generator configuration, which supercedes throttle ms. Same configuration, improved name.
  • Adds new maxHistoryEvents generator configuration, which supercedes history events max. Same configuration, improved name.
  • Removes the integer function, which was confusing. Superceded by the decimals function modifier, which when set to 0, produces integers.
  • Removes the degenerateDistribution function, which was a redundant, more narrow version of the constant function.

0.5.2​

Tue Jun 11 09:44:06 PDT 2024

  • Fixes automatic Avro schema creation for Kafka topics with a - in their name. Dashes are incompatible with Avro, so ShadowTraffic now replaces them with _.

0.5.1​

Mon Jun 10 15:11:15 PDT 2024

  • Fixes --watch mode hanging when certain exceptions were thrown.

0.5.0​

Mon Jun 10 13:35:14 PDT 2024

  • Allows transitions into stateMachine functions to be specified sequentially, allowing imperative walks through a set of states.

0.4.24​

Fri Jun 7 14:19:50 PDT 2024

  • Fixes oneOf and weightedOneOf not passing up initialization from stateful functions.
  • Loosens schema to allow waypoints to take dynamic coordinates.

0.4.23​

Thu Jun 6 11:28:36 PDT 2024

  • Allows metadata to optionally be supplied for Event Store generators.

0.4.22​

Wed Jun 5 11:38:19 PDT 2024

  • Fixes bug in waypoints where passing speed as a function threw an error.

0.4.21​

Wed Jun 5 10:37:01 PDT 2024

  • Removes stray print output.

0.4.20​

Wed Jun 5 09:38:23 PDT 2024

  • Adds new optional data parameter to loadJsonFunction to perform simple templating substitutions.

0.4.19​

Tue Jun 4 15:30:58 PDT 2024

  • Allows trichotomy bounds parameters to be function calls.

0.4.18​

Tue Jun 4 14:25:12 PDT 2024

  • Adds new trichotomy math function.

0.4.17​

Tue Jun 4 10:00:22 PDT 2024

  • --seed now randomizes for each run when running with --watch instead of locking for the entire session.

0.4.16​

Fri May 31 10:19:53 PDT 2024

  • Adds new loadJsonFile function to load configuration fragments from other files.
  • Adds new cast function modifier to cast values to/from strings.
  • Fixes static check for required properties on digitString and characterString functions.

0.4.15​

Thu May 30 10:27:24 PDT 2024

  • Fixes bug where function calls may not get invoked through multiple vars / varsOnce call chains.

0.4.14​

Wed May 29 09:17:47 PDT 2024

  • Fixes automatic schema creation for Postgres binary types.
  • Allows bytes function to generate zero length arrays.
  • Fixes uniformDistribution and normalDistribution parameters for very large numbers.
  • Fixes regression in someKeys not evaluating nested values.
  • Allows characterString and digitString to generate varying sized text.

0.4.13​

Tue May 28 13:31:43 PDT 2024

  • Dramatically increases the performance of conditional generators like oneOf and weightedOneOf.

0.4.12​

Fri May 24 14:20:59 PDT 2024

  • Adds automatic schema generation for the waypoints function.
  • Prevents throttle from applying on terminal states of a state machine. Now, when the state machine is finished, the generator exits immediately.

0.4.11​

Thu May 23 09:42:14 PDT 2024

  • Adds new mask parameter to the env function, which suppresses logging of sensitive environment variables.

0.4.10​

Thu May 23 09:10:35 PDT 2024

  • Fixes bug where --watch wouldn't reset --seed to produce the same data each run.
  • Fixes bug where using forked generators and --seed might not produce the same data each run.

0.4.9​

Wed May 22 15:27:25 PDT 2024

  • Fixes bug where functions in globalConfigs weren't referencable.

0.4.8​

Wed May 22 14:57:24 PDT 2024

  • Adds a new tablePolicy option to Postgres, create, and makes it the default. Prior to this release, ShadowTraffic would default to the dropAndCreate mode to truncate tables, which was overly destructive.

0.4.7​

Wed May 22 11:03:39 PDT 2024

  • Permits intervals function to be used in any spot outside of data, including throttle.

0.4.6​

Mon May 20 11:35:44 PDT 2024

  • Fork keys powered by a lookup call now spawn forks in the order the original data was created. This behavior is far more intuitive as a default, especially for use cases that make a fork for each element in another data set.

0.4.5​

Mon May 20 10:31:31 PDT 2024

  • Further improves the efficiency that new forks are spawned, minimizing the work to find the next logical fork to launch.

0.4.4​

Fri May 17 14:31:48 PDT 2024

  • Improves the efficiency that new forks are spawned. Prior to this release, you'd see slight delays of forks producing output with the --stdout flag. These delays are no longer present.

0.4.3​

Fri May 17 08:10:04 PDT 2024

  • Adds new maxEvents generator configuration, which supercedes events exactly. Same configuration, improved name.

0.4.2​

Thu May 16 11:18:26 PDT 2024

  • Permit variables in all spots of waypoints.

0.4.1​

Thu May 16 10:45:54 PDT 2024

  • Looses the schema for waypoints to permit coordinates from function calls.

0.4.0​

Thu May 16 09:04:31 PDT 2024

  • Changes the require path syntax for lookup calls against Postgres. Previously, path automatically prepended "row", which was confusing and shortsighted. Now, it no longer prepends, giving you access to the entire geneerated event. Apologies for the breakage.
  • Simplifies the waypoints function. Instead of having this function feed into geoPosition, it can now be used alone.
  • Removes the geoPosition function, according to the above.
  • Adds new path function modifier so that it can be applied to any function.
  • Adds new decimals function modifier so that it can be applied to any function.

0.3.23​

Wed May 15 09:49:13 PDT 2024

  • Fixes bug where lookup functions inside of a variable against a fork would throw an exception.

0.3.22​

Tue May 14 10:48:14 PDT 2024

  • Adds new env function to load environment variables.

0.3.21​

Mon May 13 09:28:13 PDT 2024

  • Kafka topic creation now behaves idempotently to prevent races between competing ShadowTraffic containers.

0.3.20​

Tue May 7 08:25:25 PDT 2024

  • Adds new bytes function for controlling payload size.

0.3.19​

Fri May 3 09:06:45 PDT 2024

  • Fixes an issue where forks with state machines that terminate wouldn't release their fork key.

0.3.18​

Mon Apr 29 09:55:32 PDT 2024

  • Adds a new optional loop parameter to the waypoints function, instructing it to travel an infinite loop around the defined coordinates.

0.3.17​

Fri Apr 26 14:48:52 PDT 2024

  • Fixes bug where null rate could throw an error in nested generators.

0.3.16​

Fri Apr 26 14:03:32 PDT 2024

  • geoPosition and geoWander now include heading in the generated data to describe what direction the object is moving.

0.3.15​

Fri Apr 26 13:04:04 PDT 2024

  • Fixes determinism when using seeded generation with someKeys.
  • Reduces the overload of calling generators that perform a random-nth operation.

0.3.14​

Thu Apr 25 11:42:46 PDT 2024

  • Top-level state machines can now merge their previously generated events, which is useful for modeling how an object changes over time.

0.3.13​

Thu Apr 18 11:40:38 PDT 2024

  • Fixes bug in waypoints where multiple segments could become disjointed.

0.3.12​

Wed Apr 17 11:41:42 PDT 2024

  • Removes CLI option --show-progress. Instead, use Prometheus metrics exposed on port 9400 to monitor generated events/s.
  • Sets smaller Prometheus histogram buckets to discern how long each generator iteration takes.

0.3.11​

Wed Apr 17 09:50:30 PDT 2024

  • Adds new experimental geoWander, geoPosition, and waypoints generators with the intent to deprecate the older geospatial functions.
  • Adds new selectKeys option to gran a subset of keys from an object.

0.3.10​

Tue Apr 16 14:18:13 PDT 2024

  • Fixes double serialization of records with Kafka JSON Schema serializer.

0.3.9​

Mon Apr 15 14:44:53 PDT 2024

  • Adds endpoint option for connectionConfigs to S3 connection to make it possible to connect to MinIO and similiar S3-compatible systems.

0.3.8​

Mon Apr 15 09:31:39 PDT 2024

  • Adds localConfigs options kafkaKeyAvroSchemaHint and kafkaValueAvroSchemaHint so you can manually supply Kafka Avro schemas. This is useful if your schema is hard to infer, or the inference doesn't guess it correctly.

0.3.7​

Mon Apr 15 08:04:58 PDT 2024

  • Dramatically improves performance for deeply nested generators.
  • Fixes fractional throttle values.

0.3.6​

Thu Apr 11 14:06:53 PDT 2024

  • Critical: Fixes a memory leak where fork and lookup interact across generators.
  • Optimizes memory footprint by no longer storing history for generators that don't service lookups.

0.3.5​

Wed Apr 10 13:56:12 PDT 2024

  • Introduces Prometheus metrics to observe how ShadowTraffic is performing.

0.3.4​

Fri Apr 5 13:47:22 PDT 2024

  • Fixes heading calculation formula.

0.3.3​

Thu Apr 4 13:44:30 PDT 2024

  • Adds new keyNames option to rename keys from any generators that return objects.
  • Adds new heading function to calculate direction in degrees.

0.3.2​

Thu Apr 4 10:31:25 PDT 2024

  • Fixes step size bug in geospatialPathScheme function.
  • Fixes radians bug in geospatialPathScheme function.

0.3.1​

Wed Apr 3 14:01:10 PDT 2024

  • Relaxes restriction on all path arguements requiring at least one element. It can now take zero elements to access the object directly.

0.3.0​

Wed Apr 3 13:48:13 PDT 2024

  • Adds new experimental connection type to EventStore.

0.2.7​

Wed Apr 3 11:06:03 PDT 2024

  • Adds new pgHint generator option for explicitly defining how Postgres columns get created.

0.2.6​

Tue Apr 2 15:24:36 PDT 2024

  • Fixes bug where Postgres table isn't auto created correctly with UUIDs.

0.2.5​

Thu Mar 28 08:43:05 PDT 2024

  • Introduces new geolocation generator.
  • Fixes race condition where dynamic JSON Schema classes didn't always load on time.
  • Fixes bug where local generator overrides for Kafka producer configuration didn't work.
  • Fixes bug where Postgres tables weren't always correctly created with certain column types.
  • Adds a utility JsonDeserializer class.

0.2.4​

Thu Mar 21 12:21:34 PDT 2024

  • Remove debug statement.

0.2.3​

Thu Mar 21 11:14:48 PDT 2024

  • Fixes bug where an automatically generated Avro schema would be incorrect for nested records.

0.2.2​

Wed Mar 20 15:12:03 PDT 2024

  • Relaxes schema of uniformDistribution to be a double, not an integer.

0.2.1​

Wed Mar 20 11:49:52 PDT 2024

  • Removes extraneous information from being written to S3 bucket.

0.2.0​

Wed Mar 20 11:30:15 PDT 2024

  • Added new connection type - s3 - for Amazon S3 endpoints.

0.1.36​

Mon Mar 18 09:40:38 PDT 2024

  • Fixed bug where inferring an Avro schema on a forked lookup would throw an error.
  • Made the uuid generator compatible with all string serializers.

0.1.35​

Tue Mar 12 09:38:43 PDT 2024

  • Added new mode for Kafka topicPolicy - create, and switched it to be the default to prevent surprising behavior.

0.1.34​

Mon Mar 11 11:32:44 PDT 2024

  • Added new topicPolicy to Kafka connection map, with the default to auto-create non-existent topics (and drop/recreate them otherwise). This makes it easier to iterate on data generators without having to flip back and forth between ShadowTraffic and Kafka.
  • Updated default Postgres behavior for tablePolicy from manual to dropAndCreate, for the same reason as Kafka.

0.1.33​

Mon Mar 11 09:22:35 PDT 2024

  • Fixed bug in Avro serialization where oneOf / weightedOneOf could cause problems with type unions.

0.1.32​

Fri Mar 8 10:00:49 PST 2024

  • Fixed bug where forking could incorrectly raise an uncaught exception.

0.1.31​

Mon Mar 4 10:37:34 PST 2024

  • Fixed erroneous exception thrown when looking up data across two forked generators.

0.1.30​

Mon Mar 4 09:48:00 PST 2024

  • Fixed a case where runtime exceptions print a poor error message.

0.1.29​

Fri Feb 23 08:58:10 PST 2024

  • Fixed a critical bug with the Postgres connection type that prevented lookup generators from working correctly.

0.1.28​

Thu Feb 22 10:56:22 PST 2024

  • Loosened configuration validation for Kafka serializers to permit custom serializers. See the Kafka reference docs for instructions.
  • Removed Kafka XML serializer - instead, use a custom serializer.

0.1.27​

Tue Feb 20 13:23:32 PST 2024

  • Fixed generated Avro types for some dynamically-typed generators like map, merge, etc.

0.1.26​

Tue Feb 20 13:11:06 PST 2024

  • Added support for Confluent Avro Schema Registry integration. ShadowTraffic now infers the Avro schema of your data, then registers or updates the remote schema.

0.1.25​

Tue Feb 20 11:52:40 PST 2024

  • Added optional unit to geospatial functions. Default is radians, but may use degrees.

0.1.24​

Tue Feb 20 10:31:01 PST 2024

  • Loosens schema for throttle ms and stagger ms, permitting not just integers, but now doubles. This is useful to apply throttling, but still allow > 10K events/second to be generated.

0.1.23​

  • Adds new decimals option to uniformDistribution and normalDistribution generators to limit the number of decimal places.

0.1.22​

  • Adds new constant generator to return known values, which is useful for adding null and elide parameters.

0.1.21​

  • Adds new elide generator option to periodically elide a key from a map.

0.1.20​

  • Adds new map function to generate key/value pairs.

0.1.19​

  • Fixes Confluent JSON Schema serializer integration.

0.1.18​

  • Fixes edge cases communicating with the TimePlus API.

0.1.17​

  • Fixes Proton and TimePlus dependency chain.

0.1.16​

  • Fixes Proton and TimePlus lookup generator calls.

0.1.15​

  • Added new experimental Proton and TimePlus connections.

0.1.14​

  • Added new tablePolicy for Postgres connections. Set to value dropAndCreate to automatically create target tables if they don't exist. Default is manual, which leaves it to you to create the tables.

0.1.13​

  • Added new digitString and characterString functions to do fast random alphanumeric generation.

0.1.12​

  • Added new op attribute to Postgres generators, allowing updates and deletes, not just inserts.

0.1.11​

  • Fixed bug in how CLI errors are printed.

0.1.10​

  • Added new command line switch --config-base64 to accept base64 encoded literal representations of configuration files, which is useful for programmatically working with ShadowTraffic.

0.1.9​

  • Added new io.shadowtraffic.kafka.serdes.XmlSerializer Kafka serde.

0.1.8​

  • Fixed bug where non-string variables couldn't be referenced in string exprs.

0.1.7​

  • string calls can now reference variables. Variables will attempt to be resolved before any Java Faker invocations.

0.1.6​

  • Added --seed flag to enable repeatable runs. The only things not repeatable are calls to the current wallclock time with now and Java Faker's Internet.uuid generator, which can't be overriden. For repeatable UUIDs, use the uuid generator.

0.1.5​

  • Improved error messages when on bad Java Faker expressions. Now links to the docs with the list of valid exprs.

0.1.4​

  • Fixed bad build - changes from 0.1.3 appear here, not previous version.

0.1.3​

  • Improved runtime error messages to show all raw and evaluated local values.

0.1.2​

  • Fixed bug allowing multiple forks to concurrently run under the same forkKey.

0.1.1​

  • Fixed bug that sometimes transitioned state machines to a null state.

0.1.0​

  • Removed for in stateMachine, superseded by fork (see below).
  • Removed stringSeries call, superseded by sequentialString.
  • Added sequentialString call to mirror behavior of sequentialInteger.
  • Added motion calls: geospatialPathScheme and geospatialPath.
  • Added new feature fork to dynamically create new generators.
  • Added new feature varsOnce to lock variable definitions for the lifetime of a generator.

0.0.41​

  • Added support for Kafka sasl mechanism SCRAM-SHA-512.

0.0.40​

  • Added support for Kafka sasl mechanism SCRAM-SHA-256.

0.0.39​

  • Dramatically improved performance to all backends by a factor of 20-30x.
  • Added new delay configuration option.

0.0.38​

  • Fixed a bug causing an evaluation error using oneOf when choices is supplied a generator.

0.0.37​

  • Added new repeat local and global configuration options to generate an event multiple times.

0.0.36​

  • Improved error message when supplying an invalid license expiration date.

0.0.35​

  • Added static error handling when trying to reference an undefined var.

0.0.34​

  • Introduced discard local and global configuration option.
  • Fixed eager evaluation of oneOf and weightedOneOf parameters.