Skip to main content

learn

changelog

You can subscribe to this changelog through the RSS feed.

What's new​

0.6.12​

Thu Jul 18 13:45:33 PDT 2024

  • Adds new pow function.

0.6.11​

Thu Jul 18 13:45:33 PDT 2024

  • Adds new optional names parameter to the string function, letting you scope identifiers that may be used within expr.
  • Raises a better error when iterateVars names a non-existent variable.

0.6.10​

Tue Jul 16 14:40:54 PDT 2024

  • Fixes iterateVars parameter of repeatedly to refresh variables in dependency order.
  • Fixes nested function call errors that could overlap each other and hide the real error message.

0.6.9​

Tue Jul 16 11:52:08 PDT 2024

  • Fixes references to generators in schedule stages that terminate and are reused through a lookup. This previously caused a stacktrace, but is now fixed.

0.6.8​

Tue Jul 16 11:11:29 PDT 2024

  • Adds new iterateVars parameter to repeatedly so that element variable references can vary between elements.

0.6.7​

Tue Jul 16 10:06:29 PDT 2024

  • Allows geolocation to generate multiple formats per invocation.

0.6.6​

Mon Jul 15 14:26:36 PDT 2024

  • Adds jsonl as an output format for the file system connector.

0.6.5​

Fri Jul 12 10:17:51 PDT 2024

  • Fixes drilling into object formatting with path for the geolocation function.

0.6.4​

Mon Jul 8 10:50:45 PDT 2024

  • Fixes cosmetic dependency warning when using the Azure Blob Storage connection. The dependencies were always present, but not checkable through the approach Azure uses. This has been fixed.

0.6.3​

Mon Jul 1 15:31:09 PDT 2024

  • Adds new transformation parameters to the output of the string function.

0.6.2​

Mon Jul 1 09:23:43 PDT 2024

  • Fixes bug where multiple generators on the filesystem connector would start their file suffixes at different numbers.

0.6.1​

Fri Jun 28 14:40:29 PDT 2024

  • Adds Azure Blob storage as a new connection type.
  • Improves the error that gets emitted when ShadowTraffic tries to automatically generate a schema for a function whose type it can't infer.
  • Improves the error that gets emitted when Avro serialization fails because of either missing fields or mistyped unions.

0.6.0​

Tue Jun 25 13:44:05 PDT 2024

  • Adds new feature, schedule, that allows orchestrating when generators run. See the overview, and reference sections for stages and loop.

0.5.11​

Mon Jun 24 12:24:09 PDT 2024

  • Adds support for headers in Kafka records.
  • Fixes bug where forks with state initialization wouldn't have access to variables.
  • Prints better error message when waypoints receives coordinates that don't match expectations.
  • Removes the number function. This was added early on with a different API in mind. Use uniformDistribution or normalDistribution to generate floating point numbers instead.

0.5.10​

Fri Jun 21 11:14:27 PDT 2024

  • Adds new fileSystem connection.

0.5.9​

Thu Jun 20 09:36:11 PDT 2024

  • Adds new maxMs generator configuration to cap how long a generator may run for.
  • Adds new default parameter to the env function for fallback values.

0.5.8​

Wed Jun 19 10:25:27 PDT 2024

  • Auto-creating Kafka topics will now use the broker default partition count and replication factor, instead of forcing 1 for both respectively.

0.5.7​

Thu Jun 13 11:54:35 PDT 2024

  • Adds new time global configuration to override the starting wallclock time.
  • Adds new timeMultiplier global configuration to change the rate at which time advances.

0.5.6​

Wed Jun 12 11:19:16 PDT 2024

  • loadJsonFile and env now work recursively, meaning you can chain loaded files and nest environment variables within.
  • loadJsonFile's data parameter is now evaluated for other preprocessors, like env, before it executes.

0.5.5​

Tue Jun 11 13:27:16 PDT 2024

  • Prevents raw stack trace from leaking out when state machines transition to a non-existent state. This now shows a helpful error message.

0.5.4​

Tue Jun 11 12:35:21 PDT 2024

  • Adds support for eventId in Event Store connections.
  • Allows eventType to be the result of a function call in Event Store connections.
  • Allows eventType to be overriden within a state machine in Event Store connections.
  • Adds new optional streamSuffix to qualify Event Store stream names.

0.5.3​

Tue Jun 11 10:35:13 PDT 2024

  • Adds new throttleMs generator configuration, which supercedes throttle ms. Same configuration, improved name.
  • Adds new maxHistoryEvents generator configuration, which supercedes history events max. Same configuration, improved name.
  • Removes the integer function, which was confusing. Superceded by the decimals function modifier, which when set to 0, produces integers.
  • Removes the degenerateDistribution function, which was a redundant, more narrow version of the constant function.

0.5.2​

Tue Jun 11 09:44:06 PDT 2024

  • Fixes automatic Avro schema creation for Kafka topics with a - in their name. Dashes are incompatible with Avro, so ShadowTraffic now replaces them with _.

0.5.1​

Mon Jun 10 15:11:15 PDT 2024

  • Fixes --watch mode hanging when certain exceptions were thrown.

0.5.0​

Mon Jun 10 13:35:14 PDT 2024

  • Allows transitions into stateMachine functions to be specified sequentially, allowing imperative walks through a set of states.

0.4.24​

Fri Jun 7 14:19:50 PDT 2024

  • Fixes oneOf and weightedOneOf not passing up initialization from stateful functions.
  • Loosens schema to allow waypoints to take dynamic coordinates.

0.4.23​

Thu Jun 6 11:28:36 PDT 2024

  • Allows metadata to optionally be supplied for Event Store generators.

0.4.22​

Wed Jun 5 11:38:19 PDT 2024

  • Fixes bug in waypoints where passing speed as a function threw an error.

0.4.21​

Wed Jun 5 10:37:01 PDT 2024

  • Removes stray print output.

0.4.20​

Wed Jun 5 09:38:23 PDT 2024

  • Adds new optional data parameter to loadJsonFunction to perform simple templating substitutions.

0.4.19​

Tue Jun 4 15:30:58 PDT 2024

  • Allows trichotomy bounds parameters to be function calls.

0.4.18​

Tue Jun 4 14:25:12 PDT 2024

  • Adds new trichotomy math function.

0.4.17​

Tue Jun 4 10:00:22 PDT 2024

  • --seed now randomizes for each run when running with --watch instead of locking for the entire session.

0.4.16​

Fri May 31 10:19:53 PDT 2024

  • Adds new loadJsonFile function to load configuration fragments from other files.
  • Adds new cast function modifier to cast values to/from strings.
  • Fixes static check for required properties on digitString and characterString functions.

0.4.15​

Thu May 30 10:27:24 PDT 2024

  • Fixes bug where function calls may not get invoked through multiple vars / varsOnce call chains.

0.4.14​

Wed May 29 09:17:47 PDT 2024

  • Fixes automatic schema creation for Postgres binary types.
  • Allows bytes function to generate zero length arrays.
  • Fixes uniformDistribution and normalDistribution parameters for very large numbers.
  • Fixes regression in someKeys not evaluating nested values.
  • Allows characterString and digitString to generate varying sized text.

0.4.13​

Tue May 28 13:31:43 PDT 2024

  • Dramatically increases the performance of conditional generators like oneOf and weightedOneOf.

0.4.12​

Fri May 24 14:20:59 PDT 2024

  • Adds automatic schema generation for the waypoints function.
  • Prevents throttle from applying on terminal states of a state machine. Now, when the state machine is finished, the generator exits immediately.

0.4.11​

Thu May 23 09:42:14 PDT 2024

  • Adds new mask parameter to the env function, which suppresses logging of sensitive environment variables.

0.4.10​

Thu May 23 09:10:35 PDT 2024

  • Fixes bug where --watch wouldn't reset --seed to produce the same data each run.
  • Fixes bug where using forked generators and --seed might not produce the same data each run.

0.4.9​

Wed May 22 15:27:25 PDT 2024

  • Fixes bug where functions in globalConfigs weren't referencable.

0.4.8​

Wed May 22 14:57:24 PDT 2024

  • Adds a new tablePolicy option to Postgres, create, and makes it the default. Prior to this release, ShadowTraffic would default to the dropAndCreate mode to truncate tables, which was overly destructive.

0.4.7​

Wed May 22 11:03:39 PDT 2024

  • Permits intervals function to be used in any spot outside of data, including throttle.

0.4.6​

Mon May 20 11:35:44 PDT 2024

  • Fork keys powered by a lookup call now spawn forks in the order the original data was created. This behavior is far more intuitive as a default, especially for use cases that make a fork for each element in another data set.

0.4.5​

Mon May 20 10:31:31 PDT 2024

  • Further improves the efficiency that new forks are spawned, minimizing the work to find the next logical fork to launch.

0.4.4​

Fri May 17 14:31:48 PDT 2024

  • Improves the efficiency that new forks are spawned. Prior to this release, you'd see slight delays of forks producing output with the --stdout flag. These delays are no longer present.

0.4.3​

Fri May 17 08:10:04 PDT 2024

  • Adds new maxEvents generator configuration, which supercedes events exactly. Same configuration, improved name.

0.4.2​

Thu May 16 11:18:26 PDT 2024

  • Permit variables in all spots of waypoints.

0.4.1​

Thu May 16 10:45:54 PDT 2024

  • Looses the schema for waypoints to permit coordinates from function calls.

0.4.0​

Thu May 16 09:04:31 PDT 2024

  • Changes the require path syntax for lookup calls against Postgres. Previously, path automatically prepended "row", which was confusing and shortsighted. Now, it no longer prepends, giving you access to the entire geneerated event. Apologies for the breakage.
  • Simplifies the waypoints function. Instead of having this function feed into geoPosition, it can now be used alone.
  • Removes the geoPosition function, according to the above.
  • Adds new path function modifier so that it can be applied to any function.
  • Adds new decimals function modifier so that it can be applied to any function.

0.3.23​

Wed May 15 09:49:13 PDT 2024

  • Fixes bug where lookup functions inside of a variable against a fork would throw an exception.

0.3.22​

Tue May 14 10:48:14 PDT 2024

  • Adds new env function to load environment variables.

0.3.21​

Mon May 13 09:28:13 PDT 2024

  • Kafka topic creation now behaves idempotently to prevent races between competing ShadowTraffic containers.

0.3.20​

Tue May 7 08:25:25 PDT 2024

  • Adds new bytes function for controlling payload size.

0.3.19​

Fri May 3 09:06:45 PDT 2024

  • Fixes an issue where forks with state machines that terminate wouldn't release their fork key.

0.3.18​

Mon Apr 29 09:55:32 PDT 2024

  • Adds a new optional loop parameter to the waypoints function, instructing it to travel an infinite loop around the defined coordinates.

0.3.17​

Fri Apr 26 14:48:52 PDT 2024

  • Fixes bug where null rate could throw an error in nested generators.

0.3.16​

Fri Apr 26 14:03:32 PDT 2024

  • geoPosition and geoWander now include heading in the generated data to describe what direction the object is moving.

0.3.15​

Fri Apr 26 13:04:04 PDT 2024

  • Fixes determinism when using seeded generation with someKeys.
  • Reduces the overload of calling generators that perform a random-nth operation.

0.3.14​

Thu Apr 25 11:42:46 PDT 2024

  • Top-level state machines can now merge their previously generated events, which is useful for modeling how an object changes over time.

0.3.13​

Thu Apr 18 11:40:38 PDT 2024

  • Fixes bug in waypoints where multiple segments could become disjointed.

0.3.12​

Wed Apr 17 11:41:42 PDT 2024

  • Removes CLI option --show-progress. Instead, use Prometheus metrics exposed on port 9400 to monitor generated events/s.
  • Sets smaller Prometheus histogram buckets to discern how long each generator iteration takes.

0.3.11​

Wed Apr 17 09:50:30 PDT 2024

  • Adds new experimental geoWander, geoPosition, and waypoints generators with the intent to deprecate the older geospatial functions.
  • Adds new selectKeys option to gran a subset of keys from an object.

0.3.10​

Tue Apr 16 14:18:13 PDT 2024

  • Fixes double serialization of records with Kafka JSON Schema serializer.

0.3.9​

Mon Apr 15 14:44:53 PDT 2024

  • Adds endpoint option for connectionConfigs to S3 connection to make it possible to connect to MinIO and similiar S3-compatible systems.

0.3.8​

Mon Apr 15 09:31:39 PDT 2024

  • Adds localConfigs options kafkaKeyAvroSchemaHint and kafkaValueAvroSchemaHint so you can manually supply Kafka Avro schemas. This is useful if your schema is hard to infer, or the inference doesn't guess it correctly.

0.3.7​

Mon Apr 15 08:04:58 PDT 2024

  • Dramatically improves performance for deeply nested generators.
  • Fixes fractional throttle values.

0.3.6​

Thu Apr 11 14:06:53 PDT 2024

  • Critical: Fixes a memory leak where fork and lookup interact across generators.
  • Optimizes memory footprint by no longer storing history for generators that don't service lookups.

0.3.5​

Wed Apr 10 13:56:12 PDT 2024

  • Introduces Prometheus metrics to observe how ShadowTraffic is performing.

0.3.4​

Fri Apr 5 13:47:22 PDT 2024

  • Fixes heading calculation formula.

0.3.3​

Thu Apr 4 13:44:30 PDT 2024

  • Adds new keyNames option to rename keys from any generators that return objects.
  • Adds new heading function to calculate direction in degrees.

0.3.2​

Thu Apr 4 10:31:25 PDT 2024

  • Fixes step size bug in geospatialPathScheme function.
  • Fixes radians bug in geospatialPathScheme function.

0.3.1​

Wed Apr 3 14:01:10 PDT 2024

  • Relaxes restriction on all path arguements requiring at least one element. It can now take zero elements to access the object directly.

0.3.0​

Wed Apr 3 13:48:13 PDT 2024

  • Adds new experimental connection type to EventStore.

0.2.7​

Wed Apr 3 11:06:03 PDT 2024

  • Adds new pgHint generator option for explicitly defining how Postgres columns get created.

0.2.6​

Tue Apr 2 15:24:36 PDT 2024

  • Fixes bug where Postgres table isn't auto created correctly with UUIDs.

0.2.5​

Thu Mar 28 08:43:05 PDT 2024

  • Introduces new geolocation generator.
  • Fixes race condition where dynamic JSON Schema classes didn't always load on time.
  • Fixes bug where local generator overrides for Kafka producer configuration didn't work.
  • Fixes bug where Postgres tables weren't always correctly created with certain column types.
  • Adds a utility JsonDeserializer class.

0.2.4​

Thu Mar 21 12:21:34 PDT 2024

  • Remove debug statement.

0.2.3​

Thu Mar 21 11:14:48 PDT 2024

  • Fixes bug where an automatically generated Avro schema would be incorrect for nested records.

0.2.2​

Wed Mar 20 15:12:03 PDT 2024

  • Relaxes schema of uniformDistribution to be a double, not an integer.

0.2.1​

Wed Mar 20 11:49:52 PDT 2024

  • Removes extraneous information from being written to S3 bucket.

0.2.0​

Wed Mar 20 11:30:15 PDT 2024

  • Added new connection type - s3 - for Amazon S3 endpoints.

0.1.36​

Mon Mar 18 09:40:38 PDT 2024

  • Fixed bug where inferring an Avro schema on a forked lookup would throw an error.
  • Made the uuid generator compatible with all string serializers.

0.1.35​

Tue Mar 12 09:38:43 PDT 2024

  • Added new mode for Kafka topicPolicy - create, and switched it to be the default to prevent surprising behavior.

0.1.34​

Mon Mar 11 11:32:44 PDT 2024

  • Added new topicPolicy to Kafka connection map, with the default to auto-create non-existent topics (and drop/recreate them otherwise). This makes it easier to iterate on data generators without having to flip back and forth between ShadowTraffic and Kafka.
  • Updated default Postgres behavior for tablePolicy from manual to dropAndCreate, for the same reason as Kafka.

0.1.33​

Mon Mar 11 09:22:35 PDT 2024

  • Fixed bug in Avro serialization where oneOf / weightedOneOf could cause problems with type unions.

0.1.32​

Fri Mar 8 10:00:49 PST 2024

  • Fixed bug where forking could incorrectly raise an uncaught exception.

0.1.31​

Mon Mar 4 10:37:34 PST 2024

  • Fixed erroneous exception thrown when looking up data across two forked generators.

0.1.30​

Mon Mar 4 09:48:00 PST 2024

  • Fixed a case where runtime exceptions print a poor error message.

0.1.29​

Fri Feb 23 08:58:10 PST 2024

  • Fixed a critical bug with the Postgres connection type that prevented lookup generators from working correctly.

0.1.28​

Thu Feb 22 10:56:22 PST 2024

  • Loosened configuration validation for Kafka serializers to permit custom serializers. See the Kafka reference docs for instructions.
  • Removed Kafka XML serializer - instead, use a custom serializer.

0.1.27​

Tue Feb 20 13:23:32 PST 2024

  • Fixed generated Avro types for some dynamically-typed generators like map, merge, etc.

0.1.26​

Tue Feb 20 13:11:06 PST 2024

  • Added support for Confluent Avro Schema Registry integration. ShadowTraffic now infers the Avro schema of your data, then registers or updates the remote schema.

0.1.25​

Tue Feb 20 11:52:40 PST 2024

  • Added optional unit to geospatial functions. Default is radians, but may use degrees.

0.1.24​

Tue Feb 20 10:31:01 PST 2024

  • Loosens schema for throttle ms and stagger ms, permitting not just integers, but now doubles. This is useful to apply throttling, but still allow > 10K events/second to be generated.

0.1.23​

  • Adds new decimals option to uniformDistribution and normalDistribution generators to limit the number of decimal places.

0.1.22​

  • Adds new constant generator to return known values, which is useful for adding null and elide parameters.

0.1.21​

  • Adds new elide generator option to periodically elide a key from a map.

0.1.20​

  • Adds new map function to generate key/value pairs.

0.1.19​

  • Fixes Confluent JSON Schema serializer integration.

0.1.18​

  • Fixes edge cases communicating with the TimePlus API.

0.1.17​

  • Fixes Proton and TimePlus dependency chain.

0.1.16​

  • Fixes Proton and TimePlus lookup generator calls.

0.1.15​

  • Added new experimental Proton and TimePlus connections.

0.1.14​

  • Added new tablePolicy for Postgres connections. Set to value dropAndCreate to automatically create target tables if they don't exist. Default is manual, which leaves it to you to create the tables.

0.1.13​

  • Added new digitString and characterString functions to do fast random alphanumeric generation.

0.1.12​

  • Added new op attribute to Postgres generators, allowing updates and deletes, not just inserts.

0.1.11​

  • Fixed bug in how CLI errors are printed.

0.1.10​

  • Added new command line switch --config-base64 to accept base64 encoded literal representations of configuration files, which is useful for programmatically working with ShadowTraffic.

0.1.9​

  • Added new io.shadowtraffic.kafka.serdes.XmlSerializer Kafka serde.

0.1.8​

  • Fixed bug where non-string variables couldn't be referenced in string exprs.

0.1.7​

  • string calls can now reference variables. Variables will attempt to be resolved before any Java Faker invocations.

0.1.6​

  • Added --seed flag to enable repeatable runs. The only things not repeatable are calls to the current wallclock time with now and Java Faker's Internet.uuid generator, which can't be overriden. For repeatable UUIDs, use the uuid generator.

0.1.5​

  • Improved error messages when on bad Java Faker expressions. Now links to the docs with the list of valid exprs.

0.1.4​

  • Fixed bad build - changes from 0.1.3 appear here, not previous version.

0.1.3​

  • Improved runtime error messages to show all raw and evaluated local values.

0.1.2​

  • Fixed bug allowing multiple forks to concurrently run under the same forkKey.

0.1.1​

  • Fixed bug that sometimes transitioned state machines to a null state.

0.1.0​

  • Removed for in stateMachine, superseded by fork (see below).
  • Removed stringSeries call, superseded by sequentialString.
  • Added sequentialString call to mirror behavior of sequentialInteger.
  • Added motion calls: geospatialPathScheme and geospatialPath.
  • Added new feature fork to dynamically create new generators.
  • Added new feature varsOnce to lock variable definitions for the lifetime of a generator.

0.0.41​

  • Added support for Kafka sasl mechanism SCRAM-SHA-512.

0.0.40​

  • Added support for Kafka sasl mechanism SCRAM-SHA-256.

0.0.39​

  • Dramatically improved performance to all backends by a factor of 20-30x.
  • Added new delay configuration option.

0.0.38​

  • Fixed a bug causing an evaluation error using oneOf when choices is supplied a generator.

0.0.37​

  • Added new repeat local and global configuration options to generate an event multiple times.

0.0.36​

  • Improved error message when supplying an invalid license expiration date.

0.0.35​

  • Added static error handling when trying to reference an undefined var.

0.0.34​

  • Introduced discard local and global configuration option.
  • Fixed eager evaluation of oneOf and weightedOneOf parameters.