Strimzi HTTP Bridge for Apache Kafka

1. Overview

The Strimzi HTTP Bridge for Apache Kafka provides a REST API for integrating HTTP based client applications with a Kafka cluster. You can use the API to create and manage consumers and send and receive records over HTTP rather than the native Kafka protocol.

1.1. Version information

Version : 0.1.0

1.2. Contact information

Contact Email : strimzi@redhat.com

1.3. License information

License : Apache 2.0
License URL : http://www.apache.org/licenses/LICENSE-2.0.html
Terms of service : http://swagger.io/terms/

1.4. URI scheme

Host : bridge.swagger.io
BasePath : /
Schemes : HTTP

1.5. Tags

  • Consumers : Consumer operations to create consumers in your Kafka cluster and perform common actions, such as subscribing to topics, retrieving processed records, and committing offsets.

  • Producer : Producer operations to send records to a specified topic or topic partition.

  • Seek : Seek operations that enable a consumer to begin receiving messages from a given offset position.

  • Topics : Topic operations to send messages to a specified topic or topic partition. You can include message keys in requests.

1.6. Consumes

  • application/json

1.7. Produces

  • application/json

2. Definitions

2.1. AssignedTopicPartitions

Type : < string, < integer (int32) > array > map

2.2. Consumer

Name Description Schema

auto.offset.reset
optional

Resets the offset position for the consumer. If set to earliest, messages are read from the first offset. If set to latest, messages are read from the latest offset.

string

consumer.request.timeout.ms
optional

Sets the maximum amount of time, in milliseconds, for the consumer to wait for messages for a request. If the timeout period is reached without a response, an error is returned.

integer

enable.auto.commit
optional

If set to true, message offsets are committed automatically for the consumer. If set to false, message offsets must be committed manually.

boolean

fetch.min.bytes
optional

Sets the minimum ammount of data, in bytes, for the consumer to receive. The broker waits until the data to send exceeds this amount.

integer

format
optional

The allowable message format for the consumer, which can be binary (default) or json. The messages are converted into a JSON format.

string

name
optional

The unique name for the consumer instance. The name is unique within the scope of the consumer group. The name is used in URLs.

string

2.3. ConsumerRecord

Name Schema

key
optional

string

offset
optional

integer (int64)

partition
optional

integer (int32)

topic
optional

string

value
optional

string

2.4. ConsumerRecordList

Type : < ConsumerRecord > array

2.5. CreatedConsumer

Name Description Schema

base_uri
optional

Base URI used to construct URIs for subsequent requests against this consumer instance.

string

instance_id
optional

Unique ID for the consumer instance in the group.

string

2.6. Error

Name Schema

error_code
optional

integer (int32)

message
optional

string

2.7. OffsetCommitSeek

Name Schema

offset
required

integer (int64)

partition
required

integer (int32)

topic
required

string

2.8. OffsetCommitSeekList

Name Schema

offsets
optional

< OffsetCommitSeek > array

2.9. OffsetRecordSent

Name Schema

offset
optional

integer (int64)

partition
optional

integer (int32)

2.10. OffsetRecordSentList

Name Schema

offsets
optional

< OffsetRecordSent > array

2.11. Partition

Name Schema

partition
optional

integer (int32)

topic
optional

string

2.12. Partitions

Name Schema

partitions
optional

< Partition > array

2.13. ProducerRecord

Name Schema

partition
optional

integer (int32)

2.14. ProducerRecordList

Name Schema

records
optional

< ProducerRecord > array

2.15. ProducerRecordToPartition

Type : object

2.16. ProducerRecordToPartitionList

Name Schema

records
optional

< ProducerRecordToPartition > array

2.17. SubscribedTopicList

Name Schema

partitions
optional

< AssignedTopicPartitions > array

topics
optional

Topics

2.18. Topics

Name Description Schema

topic_pattern
optional

A regex topic pattern for matching multiple topics

string

topics
optional

< string > array

3. Paths

3.1. POST /consumers/{groupid}

3.1.1. Description

Creates a consumer instance in the given consumer group. You can optionally specify a consumer name and supported configuration options. It returns a base URI which must be used to construct URLs for subsequent requests against this consumer instance.

3.1.2. Parameters

Type Name Description Schema

Path

groupid
required

ID of the consumer group in which to create the consumer.

string

Body

body
required

Name and configuration of the consumer. The name is unique within the scope of the consumer group. If a name is not specified, a randomly generated name is assigned. All parameters are optional. The supported configuration options are shown in the following example.

Consumer

3.1.3. Responses

HTTP Code Description Schema

200

Consumer created successfully.

CreatedConsumer

409

A consumer instance with the specified name already exists in the Kafka Bridge.

Error

422

One or more consumer configuration options have invalid values.

Error

3.1.4. Consumes

  • application/vnd.kafka.v2+json

3.1.5. Produces

  • application/vnd.kafka.v2+json

3.1.6. Tags

  • Consumers

3.1.7. Example HTTP request

Request body
{
  "name" : "consumer1",
  "format" : "binary",
  "auto.offset.reset" : "earliest",
  "enable.auto.commit" : false,
  "fetch.min.bytes" : 512,
  "consumer.request.timeout.ms" : 30000
}

3.1.8. Example HTTP response

Response 200
{
  "application/vnd.kafka.v2+json" : {
    "instance_id" : "consumer1",
    "base_uri" : "http://localhost:8080/consumers/my-group/instances/consumer1"
  }
}
Response 409
{
  "application/vnd.kafka.v2+json" : {
    "error_code" : 409,
    "message" : "A consumer instance with the specified name already exists in the Kafka Bridge."
  }
}
Response 422
{
  "application/vnd.kafka.v2+json" : {
    "error_code" : 422,
    "message" : "One or more consumer configuration options have invalid values."
  }
}

3.2. DELETE /consumers/{groupid}/instances/{name}

3.2.1. Description

Deletes a specified consumer instance. The request for this operation MUST use the base URL (including the host and port) returned in the response from the POST request to /consumers/{groupid} that was used to create this consumer.

3.2.2. Parameters

Type Name Description Schema

Path

groupid
required

ID of the consumer group to which the consumer belongs.

string

Path

name
required

Name of the consumer that you want to delete.

string

3.2.3. Responses

HTTP Code Description Schema

204

Consumer removed successfully.

No Content

404

The specified consumer instance was not found.

Error

3.2.4. Consumes

  • application/vnd.kafka.v2+json

3.2.5. Produces

  • application/vnd.kafka.v2+json

3.2.6. Tags

  • Consumers

3.2.7. Example HTTP response

Response 404
{
  "application/vnd.kafka.v2+json" : {
    "error_code" : 404,
    "message" : "The specified consumer instance was not found."
  }
}

3.3. POST /consumers/{groupid}/instances/{name}/assignments

3.3.1. Description

Assigns one or more topic partitions to a consumer.

3.3.2. Parameters

Type Name Description Schema

Path

groupid
required

ID of the consumer group to which the consumer belongs.

string

Path

name
required

Name of the consumer to which you want to assign topic partitions.

string

Body

body
required

List of topic partitions to assign to the consumer.

Partitions

3.3.3. Responses

HTTP Code Description Schema

204

Partitions assigned successfully.

No Content

404

The specified consumer instance was not found.

Error

409

Subscriptions to topics, partitions, and patterns are mutually exclusive.

Error

3.3.4. Consumes

  • application/vnd.kafka.v2+json

3.3.5. Produces

  • application/vnd.kafka.v2+json

3.3.6. Tags

  • Consumers

3.3.7. Example HTTP request

Request body
{
  "partitions" : [ {
    "topic" : "topic",
    "partition" : 0
  }, {
    "topic" : "topic",
    "partition" : 1
  } ]
}

3.3.8. Example HTTP response

Response 404
{
  "application/vnd.kafka.v2+json" : {
    "error_code" : 404,
    "message" : "The specified consumer instance was not found."
  }
}
Response 409
{
  "application/vnd.kafka.v2+json" : {
    "error_code" : 409,
    "message" : "Subscriptions to topics, partitions, and patterns are mutually exclusive."
  }
}

3.4. POST /consumers/{groupid}/instances/{name}/offsets

3.4.1. Description

Commits a list of consumer offsets. To commit offsets for all records fetched by the consumer, leave the request body empty.

3.4.2. Parameters

Type Name Description Schema

Path

groupid
required

ID of the consumer group to which the consumer belongs.

string

Path

name
required

Name of the consumer.

string

Body

body
optional

List of consumer offsets to commit to the consumer offsets commit log. You can specify one or more topic partitions to commit offsets for.

OffsetCommitSeekList

3.4.3. Responses

HTTP Code Description Schema

204

Commit made successfully.

No Content

404

The specified consumer instance was not found.

Error

3.4.4. Consumes

  • application/vnd.kafka.v2+json

3.4.5. Produces

  • application/vnd.kafka.v2+json

3.4.6. Tags

  • Consumers

3.4.7. Example HTTP request

Request body
{
  "offsets" : [ {
    "topic" : "topic",
    "partition" : 0,
    "offset" : 15
  }, {
    "topic" : "topic",
    "partition" : 1,
    "offset" : 42
  } ]
}

3.4.8. Example HTTP response

Response 404
{
  "application/vnd.kafka.v2+json" : {
    "error_code" : 404,
    "message" : "The specified consumer instance was not found."
  }
}

3.5. POST /consumers/{groupid}/instances/{name}/positions

3.5.1. Description

Configures a subscribed consumer to fetch offsets from a particular offset the next time it fetches a set of records from a given topic partition. This overrides the default fetch behavior for consumers. You can specify one or more topic partitions.

3.5.2. Parameters

Type Name Description Schema

Path

groupid
required

ID of the consumer group to which the consumer belongs.

string

Path

name
required

Name of the subscribed consumer.

string

Body

body
required

List of partition offsets from which the subscribed consumer will next fetch records.

OffsetCommitSeekList

3.5.3. Responses

HTTP Code Description Schema

204

Seek performed successfully.

No Content

404

The specified consumer instance was not found, or the specified consumer instance did not have one of the specified partitions assigned.

Error

3.5.4. Consumes

  • application/vnd.kafka.v2+json

3.5.5. Produces

  • application/vnd.kafka.v2+json

3.5.6. Tags

  • Consumers

  • Seek

3.5.7. Example HTTP request

Request body
{
  "offsets" : [ {
    "topic" : "topic",
    "partition" : 0,
    "offset" : 15
  }, {
    "topic" : "topic",
    "partition" : 1,
    "offset" : 42
  } ]
}

3.5.8. Example HTTP response

Response 404
{
  "application/vnd.kafka.v2+json" : {
    "error_code" : 404,
    "message" : "The specified consumer instance was not found."
  }
}

3.6. POST /consumers/{groupid}/instances/{name}/positions/beginning

3.6.1. Description

Configures a subscribed consumer to seek (and subsequently read from) the first offset in one or more given topic partitions.

3.6.2. Parameters

Type Name Description Schema

Path

groupid
required

ID of the consumer group to which the subscribed consumer belongs.

string

Path

name
required

Name of the subscribed consumer.

string

Body

body
required

List of topic partitions to which the consumer is subscribed. The consumer will seek the first offset in the specified partitions.

Partitions

3.6.3. Responses

HTTP Code Description Schema

204

Seek to the beginning performed successfully.

No Content

404

The specified consumer instance was not found, or the specified consumer instance did not have one of the specified partitions assigned.

Error

3.6.4. Consumes

  • application/vnd.kafka.v2+json

3.6.5. Produces

  • application/vnd.kafka.v2+json

3.6.6. Tags

  • Consumers

  • Seek

3.6.7. Example HTTP request

Request body
{
  "partitions" : [ {
    "topic" : "topic",
    "partition" : 0
  }, {
    "topic" : "topic",
    "partition" : 1
  } ]
}

3.6.8. Example HTTP response

Response 404
{
  "application/vnd.kafka.v2+json" : {
    "error_code" : 404,
    "message" : "The specified consumer instance was not found."
  }
}

3.7. POST /consumers/{groupid}/instances/{name}/positions/end

3.7.1. Description

Configures a subscribed consumer to seek (and subsequently read from) the offset at the end of one or more of the given topic partitions.

3.7.2. Parameters

Type Name Description Schema

Path

groupid
required

ID of the consumer group to which the subscribed consumer belongs.

string

Path

name
required

Name of the subscribed consumer.

string

Body

body
optional

List of topic partitions to which the consumer is subscribed. The consumer will seek the last offset in the specified partitions.

Partitions

3.7.3. Responses

HTTP Code Description Schema

204

Seek to the end performed successfully.

No Content

404

The specified consumer instance was not found, or the specified consumer instance did not have one of the specified partitions assigned.

Error

3.7.4. Consumes

  • application/vnd.kafka.v2+json

3.7.5. Produces

  • application/vnd.kafka.v2+json

3.7.6. Tags

  • Consumers

  • Seek

3.7.7. Example HTTP request

Request body
{
  "partitions" : [ {
    "topic" : "topic",
    "partition" : 0
  }, {
    "topic" : "topic",
    "partition" : 1
  } ]
}

3.7.8. Example HTTP response

Response 404
{
  "application/vnd.kafka.v2+json" : {
    "error_code" : 404,
    "message" : "The specified consumer instance was not found."
  }
}

3.8. GET /consumers/{groupid}/instances/{name}/records

3.8.1. Description

Retrieves records for a subscribed consumer, including message values, topics, and partitions. The request for this operation MUST use the base URL (including the host and port) returned in the response from the POST request to /consumers/{groupid} that was used to create this consumer.

3.8.2. Parameters

Type Name Description Schema

Path

groupid
required

ID of the consumer group to which the subscribed consumer belongs.

string

Path

name
required

Name of the subscribed consumer for which you want to retrieve records.

string

Query

max_bytes
optional

The maximum size, in bytes, of unencoded keys and values that can be included in the response. Otherwise, an error response with code 422 is returned.

integer

Query

timeout
optional

The maximum amount of time, in milliseconds, that the HTTP Bridge spends retrieving records before timing out the request.

integer

3.8.3. Responses

HTTP Code Description Schema

200

Poll request executed successfully.

ConsumerRecordList

404

The specified consumer instance was not found.

Error

406

The format used in the consumer creation request does not match the embedded format in the Accept header of this request or the bridge got a message from the topic which is not JSON encoded.

Error

422

Response exceeds the maximum number of bytes the consumer can receive

Error

3.8.4. Produces

  • application/vnd.kafka.json.v2+json

  • application/vnd.kafka.binary.v2+json

  • application/vnd.kafka.v2+json

3.8.5. Tags

  • Consumers

3.8.6. Example HTTP response

Response 200
{
  "application/vnd.kafka.json.v2+json" : [ {
    "topic" : "topic",
    "key" : "key1",
    "value" : {
      "foo" : "bar"
    },
    "partition" : 0,
    "offset" : 2
  }, {
    "topic" : "topic",
    "key" : "key2",
    "value" : [ "foo2", "bar2" ],
    "partition" : 1,
    "offset" : 3
  } ],
  "application/vnd.kafka.binary.v2+json" : "[\n  {\n    \"topic\": \"test\",\n    \"key\": \"a2V5\",\n    \"value\": \"Y29uZmx1ZW50\",\n    \"partition\": 1,\n    \"offset\": 100,\n  },\n  {\n    \"topic\": \"test\",\n    \"key\": \"a2V5\",\n    \"value\": \"a2Fma2E=\",\n    \"partition\": 2,\n    \"offset\": 101,\n  }\n]"
}
Response 404
{
  "application/vnd.kafka.v2+json" : {
    "error_code" : 404,
    "message" : "The specified consumer instance was not found."
  }
}
Response 406
{
  "application/vnd.kafka.v2+json" : {
    "error_code" : 406,
    "message" : "The `format` used in the consumer creation request does not match the embedded format in the Accept header of this request."
  }
}
Response 422
{
  "application/vnd.kafka.v2+json" : {
    "error_code" : 422,
    "message" : "Response exceeds the maximum number of bytes the consumer can receive"
  }
}

3.9. POST /consumers/{groupid}/instances/{name}/subscription

3.9.1. Description

Subscribes a consumer to one or more topics. You can describe the topics to which the consumer will subscribe in a list (of Topics type) or as a topic_pattern field. Each call replaces the subscriptions for the subscriber.

3.9.2. Parameters

Type Name Description Schema

Path

groupid
required

ID of the consumer group to which the subscribed consumer belongs.

string

Path

name
required

Name of the consumer that you want to unsubscribe from topics.

string

Body

body
required

List of topics to which the consumer will subscribe.

Topics

3.9.3. Responses

HTTP Code Description Schema

204

Consumer subscribed successfully.

No Content

404

The specified consumer instance was not found.

Error

409

Subscriptions to topics, partitions, and patterns are mutually exclusive.

Error

422

A list (of Topics type) or a topic_pattern must be specified.

Error

3.9.4. Consumes

  • application/vnd.kafka.v2+json

3.9.5. Produces

  • application/vnd.kafka.v2+json

3.9.6. Tags

  • Consumers

3.9.7. Example HTTP request

Request body
{
  "topics" : [ "topic1", "topic2" ]
}

3.9.8. Example HTTP response

Response 404
{
  "application/vnd.kafka.v2+json" : {
    "error_code" : 404,
    "message" : "The specified consumer instance was not found."
  }
}
Response 409
{
  "application/vnd.kafka.v2+json" : {
    "error_code" : 409,
    "message" : "Subscriptions to topics, partitions, and patterns are mutually exclusive."
  }
}
Response 422
{
  "application/vnd.kafka.v2+json" : {
    "error_code" : 422,
    "message" : "A list (of Topics type) or a topic_pattern must be specified."
  }
}

3.10. GET /consumers/{groupid}/instances/{name}/subscription

3.10.1. Description

Retrieves a list of the topics to which the consumer is subscribed.

3.10.2. Parameters

Type Name Description Schema

Path

groupid
required

ID of the consumer group to which the subscribed consumer belongs.

string

Path

name
required

Name of the consumer that you want to unsubscribe from topics.

string

3.10.3. Responses

HTTP Code Description Schema

200

List of subscribed topics and partitions.

SubscribedTopicList

404

The specified consumer instance was not found.

Error

3.10.4. Produces

  • application/vnd.kafka.v2+json

3.10.5. Tags

  • Consumers

3.10.6. Example HTTP response

Response 200
{
  "topics" : [ "my-topic1", "my-topic2" ],
  "partitions" : [ {
    "my-topic1" : [ 1, 2, 3 ]
  }, {
    "my-topic2" : [ 1 ]
  } ]
}
Response 404
{
  "application/vnd.kafka.v2+json" : {
    "error_code" : 404,
    "message" : "The specified consumer instance was not found."
  }
}

3.11. DELETE /consumers/{groupid}/instances/{name}/subscription

3.11.1. Description

Unsubscribes a consumer from all topics.

3.11.2. Parameters

Type Name Description Schema

Path

groupid
required

ID of the consumer group to which the subscribed consumer belongs.

string

Path

name
required

Name of the consumer that you want to unsubscribe from topics.

string

3.11.3. Responses

HTTP Code Description Schema

204

Consumer unsubscribed successfully.

No Content

404

The specified consumer instance was not found.

Error

3.11.4. Tags

  • Consumers

3.11.5. Example HTTP response

Response 404
{
  "error_code" : 404,
  "message" : "The specified consumer instance was not found."
}

3.12. GET /healthy

3.12.1. Description

Check if the bridge is running. This does not necessarily imply that it is ready to accept requests.

3.12.2. Responses

HTTP Code Description Schema

200

The bridge is healthy

No Content

3.13. GET /openapi

3.13.1. Description

Retrieves the OpenAPI v2 specification in JSON format.

3.13.2. Responses

HTTP Code Description Schema

200

OpenAPI v2 specification in JSON format retrieved successfully.

string

3.13.3. Produces

  • application/json

3.14. GET /ready

3.14.1. Description

Check if the bridge is ready and can accept requests.

3.14.2. Responses

HTTP Code Description Schema

200

The bridge is ready

No Content

3.15. POST /topics/{topicname}

3.15.1. Description

Sends one or more records to a given topic, optionally specifying a partition, key, or both.

3.15.2. Parameters

Type Name Description Schema

Path

topicname
required

Name of the topic to which you want to send records.

string

Body

body
required

ProducerRecordList

3.15.3. Responses

HTTP Code Description Schema

200

Records sent successfully.

OffsetRecordSentList

404

The specified topic was not found.

Error

422

The record list is not valid.

Error

3.15.4. Consumes

  • application/vnd.kafka.json.v2+json

  • application/vnd.kafka.binary.v2+json

3.15.5. Produces

  • application/vnd.kafka.v2+json

3.15.6. Tags

  • Producer

  • Topics

3.15.7. Example HTTP request

Request body
{
  "records" : [ {
    "key" : "key1",
    "value" : "value1"
  }, {
    "value" : "value2",
    "partition" : 1
  }, {
    "value" : "value3"
  } ]
}

3.15.8. Example HTTP response

Response 200
{
  "application/vnd.kafka.v2+json" : {
    "offsets" : [ {
      "partition" : 2,
      "offset" : 0
    }, {
      "partition" : 1,
      "offset" : 1
    }, {
      "partition" : 2,
      "offset" : 2
    } ]
  }
}
Response 404
{
  "application/vnd.kafka.v2+json" : {
    "error_code" : 404,
    "message" : "The specified topic was not found."
  }
}
Response 422
{
  "application/vnd.kafka.v2+json" : {
    "error_code" : 422,
    "message" : "The record list contains invalid records."
  }
}

3.16. POST /topics/{topicname}/partitions/{partitionid}

3.16.1. Description

Sends one or more records to a given topic partition, optionally specifying a key.

3.16.2. Parameters

Type Name Description Schema

Path

partitionid
required

ID of the partition to which you want to send records.

integer

Path

topicname
required

Name of the topic containing the partition to which you want to send records.

string

Body

body
required

List of records to send to a given topic partition, including a value (required) and a key (optional).

ProducerRecordToPartitionList

3.16.3. Responses

HTTP Code Description Schema

200

Records sent successfully.

OffsetRecordSentList

404

The specified topic partition was not found.

Error

422

The record is not valid.

Error

3.16.4. Consumes

  • application/vnd.kafka.json.v2+json

  • application/vnd.kafka.binary.v2+json

3.16.5. Produces

  • application/vnd.kafka.v2+json

3.16.6. Tags

  • Producer

  • Topics

3.16.7. Example HTTP request

Request body
{
  "records" : [ {
    "key" : "key1",
    "value" : "value1"
  }, {
    "value" : "value2"
  } ]
}

3.16.8. Example HTTP response

Response 200
{
  "application/vnd.kafka.v2+json" : {
    "offsets" : [ {
      "partition" : 2,
      "offset" : 0
    }, {
      "partition" : 1,
      "offset" : 1
    }, {
      "partition" : 2,
      "offset" : 2
    } ]
  }
}
Response 404
{
  "application/vnd.kafka.v2+json" : {
    "error_code" : 404,
    "message" : "The specified topic partition was not found."
  }
}
Response 422
{
  "application/vnd.kafka.v2+json" : {
    "error_code" : 422,
    "message" : "The record is not valid."
  }
}

Revised on 2019-12-02 20:54:38 +0100