Skip to main content

Generator configuration

protobufSchemaHint

Commentary​

added in 0.11.0

local configuration

Explicitly defines the Protobuf schema and message for generated data. Takes a map of generator keys to schema and message objects.

Note that when writing data to Kafka, this configuration takes precedence over the deprecated kafkaKeyProtobufHint and kafkaValueProtobufHint configs.

Examples​

Providing the Protobuf schema​

Set the keys in the map to the fields in the generator. In this example, the schema is defined for the data field for events output to Google Cloud Storage.

{
    "generators": [
        {
            "topic": "customers",
            "value": {
                "id": {
                    "_gen": "uuid"
                }
            },
            "localConfigs": {
                "protobufSchemaHint": {
                    "value": {
                        "schemaFile": "/path/to/protobuf-schema.proto",
                        "message": "UserIdentifier"
                    }
                }
            }
        }
    ],
    "connections": {
        "kafka": {
            "kind": "kafka",
            "producerConfigs": {
                "bootstrap.servers": "localhost:9092",
                "schema.registry.url": "http://localhost:8081",
                "key.serializer": "io.shadowtraffic.kafka.serdes.JsonSerializer",
                "value.serializer": "io.confluent.kafka.serializers.protobuf.KafkaProtobufSerializer"
            }
        }
    }
}

Specification​

JSON schema​

{
    "type": "object",
    "patternProperties": {
        "^.*$": {
            "type": "object",
            "properties": {
                "schemaFile": {
                    "type": "string"
                },
                "message": {
                    "type": "string"
                }
            },
            "required": [
                "schemaFile",
                "message"
            ]
        }
    }
}