RabbitMQ Dokumentation
Die Microservices des AS4-Systems kommunizieren größtenteils asynchron über AMQP. Ein Marktkommunikation-Backend ist ebenfalls per AMQP anzuschließen. Diese Art der Kommunikation entkoppelt die Services. Sie können leicht skaliert werden und der Ausfall einzelner Services wirkt sich nicht auf andere Services aus. AMQP erfordert einen Message Broker zum Übertragen der Nachrichten.
Als AMQP Message Broker kann RabbitMQ verwendet werden. Mehr dazu ist in der offiziellen Dokumentation nachzulesen.
Die folgende Konfiguration ist ein Beispiel aus unserem Testsystem. Für ein produktives System wird stattdessen ein RabbitMQ Cluster empfohlen.
Docker-Compose
In einer Docker-compose Umgebung kann die definitions.json als volume eingebunden werden.
rabbitmq3:
image: rabbitmq:3.8-management
container_name: rabbitmq3
hostname: rabbitmq3
restart: always
ports:
- "5672:5672"
- "15672:15672"
environment:
- TZ=${TIME_ZONE}
volumes:
- rabbitmq3:/var/lib/rabbitmq/mnesia/
- ./conf/rabbitmq/enabled_plugins:/etc/rabbitmq/enabled_plugins
- ./conf/rabbitmq/definitions.json:/etc/rabbitmq/definitions.json:ro
Schema Definition
Viele Exchanges und Queues werden von den Microservices automatisch angelegt.
Für bestimmte Queues ist es wichtig, dass sie vor dem Start der Microservices angelegt werden. Dazu kann eine definitions.json verwendet werden. User und Passwort werden auch in dieser json-Datei hinterlegt.
{
"users": [
{
"name": "guest",
"password": "guest",
"hashing_algorithm": "rabbit_password_hashing_sha256",
"tags": "administrator",
"limits": {}
}
],
"vhosts": [{
"name": "/"
}
],
"permissions": [{
"user": "guest",
"vhost": "/",
"configure": ".*",
"write": ".*",
"read": ".*"
}
],
"topic_permissions": [],
"parameters": [],
"global_parameters": [{
"name": "internal_cluster_id",
"value": "rabbitmq-cluster-id-ZN4mSvA1esBHKCUX2VrpLw"
}
],
"policies": [],
"queues": [{
"name": "as4.inbound.delivery",
"vhost": "/",
"durable": true,
"auto_delete": false,
"arguments": {
"x-dead-letter-exchange": "DLX",
"x-dead-letter-routing-key": "as4.inbound.delivery",
"x-queue-type": "quorum"
}
}, {
"name": "as4.inbound.pathswitch",
"vhost": "/",
"durable": true,
"auto_delete": false,
"arguments": {
"x-dead-letter-exchange": "DLX",
"x-dead-letter-routing-key": "as4.inbound.pathswitch",
"x-queue-type": "quorum"
}
},{
"name": "as4.receipt.outbound.delivery",
"vhost": "/",
"durable": true,
"auto_delete": false,
"arguments": {
"x-dead-letter-exchange": "DLX",
"x-dead-letter-routing-key": "as4.receipt.outbound.delivery",
"x-queue-type": "quorum"
}
},{
"name": "as4.outbound.pathswitch",
"vhost": "/",
"durable": true,
"auto_delete": false,
"arguments": {
"x-dead-letter-exchange": "DLX",
"x-dead-letter-routing-key": "as4.outbound.pathswitch",
"x-queue-type": "quorum"
}
},{
"name": "b2b.json.default",
"vhost": "/",
"durable": true,
"auto_delete": false,
"arguments": {
"x-dead-letter-exchange": "DLX",
"x-dead-letter-routing-key": "b2b.json.default",
"x-queue-type": "quorum"
}
},{
"name": "as4.send.receipt.default",
"vhost": "/",
"durable": true,
"auto_delete": false,
"arguments": {
"x-dead-letter-exchange": "DLX",
"x-dead-letter-routing-key": "as4.send.receipt.default",
"x-queue-type": "quorum"
}
},{
"name": "as4.relation.create.default",
"vhost": "/",
"durable": true,
"auto_delete": false,
"arguments": {
"x-dead-letter-exchange": "DLX",
"x-dead-letter-routing-key": "as4.relation.create.default",
"x-queue-type": "quorum"
}
}
],
"exchanges": [{
"name": "as4.inbound",
"vhost": "/",
"type": "topic",
"durable": true,
"auto_delete": false,
"internal": false,
"arguments": {}
}, {
"name": "as4.receipt.outbound",
"vhost": "/",
"type": "topic",
"durable": true,
"auto_delete": false,
"internal": false,
"arguments": {}
}, {
"name": "b2b.json",
"vhost": "/",
"type": "topic",
"durable": true,
"auto_delete": false,
"internal": false,
"arguments": {}
}, {
"name": "as4.send.receipt",
"vhost": "/",
"type": "direct",
"durable": true,
"auto_delete": false,
"internal": false,
"arguments": {}
}, {
"name": "as4.relation.create",
"vhost": "/",
"type": "topic",
"durable": true,
"auto_delete": false,
"internal": false,
"arguments": {}
}
],
"bindings": [{
"source": "as4.inbound",
"vhost": "/",
"destination": "as4.inbound.delivery",
"destination_type": "queue",
"routing_key": "https://www.bdew.de/as4/communication/services/MP",
"arguments": {}
},{
"source": "as4.inbound",
"vhost": "/",
"destination": "as4.inbound.pathswitch",
"destination_type": "queue",
"routing_key": "https://www.bdew.de/as4/communication/services/pathSwitch",
"arguments": {}
},{
"source": "as4.receipt.outbound",
"vhost": "/",
"destination": "as4.receipt.outbound.delivery",
"destination_type": "queue",
"routing_key": "https://www.bdew.de/as4/communication/services/MP",
"arguments": {}
},{
"source": "as4.receipt.outbound",
"vhost": "/",
"destination": "as4.outbound.pathswitch",
"destination_type": "queue",
"routing_key": "https://www.bdew.de/as4/communication/services/pathSwitch",
"arguments": {}
},{
"source": "b2b.json",
"vhost": "/",
"destination": "b2b.json.default",
"destination_type": "queue",
"routing_key": "#",
"arguments": {}
},{
"source": "as4.send.receipt",
"vhost": "/",
"destination": "as4.send.receipt.default",
"destination_type": "queue",
"routing_key": "default",
"arguments": {}
},{
"source": "as4.relation.create",
"vhost": "/",
"destination": "as4.relation.create.default",
"destination_type": "queue",
"routing_key": "default",
"arguments": {}
}
]
}
Verwenden Sie für Multi Tenancy das folgende Schema:
{
"users": [
{
"name": "guest",
"password": "upQ4VVrEGhxSWHoIKsue",
"hashing_algorithm": "rabbit_password_hashing_sha256",
"tags": "administrator",
"limits": {}
}
],
"vhosts": [{
"name": "/"
}
],
"permissions": [{
"user": "guest",
"vhost": "/",
"configure": ".*",
"write": ".*",
"read": ".*"
}
],
"topic_permissions": [],
"parameters": [],
"global_parameters": [{
"name": "internal_cluster_id",
"value": "rabbitmq-cluster-id-ZN4mSvA1esBHKCUX2VrpLw"
}
],
"policies": [],
"queues": [{
"name": "as4.inbound.delivery",
"vhost": "/",
"durable": true,
"auto_delete": false,
"arguments": {
"x-dead-letter-exchange": "DLX",
"x-dead-letter-routing-key": "as4.inbound.delivery",
"x-queue-type": "quorum"
}
}, {
"name": "as4.inbound.pathswitch",
"vhost": "/",
"durable": true,
"auto_delete": false,
"arguments": {
"x-dead-letter-exchange": "DLX",
"x-dead-letter-routing-key": "as4.inbound.pathswitch",
"x-queue-type": "quorum"
}
},{
"name": "as4.receipt.outbound.delivery",
"vhost": "/",
"durable": true,
"auto_delete": false,
"arguments": {
"x-dead-letter-exchange": "DLX",
"x-dead-letter-routing-key": "as4.receipt.outbound.delivery",
"x-queue-type": "quorum"
}
},{
"name": "as4.receipt.outbound.pathswitch",
"vhost": "/",
"durable": true,
"auto_delete": false,
"arguments": {
"x-dead-letter-exchange": "DLX",
"x-dead-letter-routing-key": "as4.outbound.pathswitch",
"x-queue-type": "quorum"
}
},{
"name": "b2b.json.default",
"vhost": "/",
"durable": true,
"auto_delete": false,
"arguments": {
"x-dead-letter-exchange": "DLX",
"x-dead-letter-routing-key": "b2b.json.default",
"x-queue-type": "quorum"
}
},{
"name": "as4.send.receipt.default",
"vhost": "/",
"durable": true,
"auto_delete": false,
"arguments": {
"x-dead-letter-exchange": "DLX",
"x-dead-letter-routing-key": "as4.send.receipt.default",
"x-queue-type": "quorum"
}
},{
"name": "as4.outbound.9903111045603",
"vhost": "/",
"durable": true,
"auto_delete": false,
"arguments": {
"x-dead-letter-exchange": "DLX",
"x-dead-letter-routing-key": "as4.outbound.9903111045603",
"x-queue-type": "quorum"
}
},{
"name": "as4.outbound.9900794076504",
"vhost": "/",
"durable": true,
"auto_delete": false,
"arguments": {
"x-dead-letter-exchange": "DLX",
"x-dead-letter-routing-key": "as4.outbound.9900794076504",
"x-queue-type": "quorum"
}
},{
"name": "as4.send.receipt.9903111045603",
"vhost": "/",
"durable": true,
"auto_delete": false,
"arguments": {
"x-dead-letter-exchange": "DLX",
"x-dead-letter-routing-key": "as4.send.receipt.9903111045603",
"x-queue-type": "quorum"
}
},{
"name": "as4.send.receipt.9900794076504",
"vhost": "/",
"durable": true,
"auto_delete": false,
"arguments": {
"x-dead-letter-exchange": "DLX",
"x-dead-letter-routing-key": "as4.send.receipt.9900794076504",
"x-queue-type": "quorum"
}
},{
"name": "as4.relation.create.default",
"vhost": "/",
"durable": true,
"auto_delete": false,
"arguments": {
"x-dead-letter-exchange": "DLX",
"x-dead-letter-routing-key": "as4.relation.create.default",
"x-queue-type": "quorum"
}
}
],
"exchanges": [{
"name": "as4.inbound",
"vhost": "/",
"type": "topic",
"durable": true,
"auto_delete": false,
"internal": false,
"arguments": {}
}, {
"name": "as4.receipt.outbound",
"vhost": "/",
"type": "topic",
"durable": true,
"auto_delete": false,
"internal": false,
"arguments": {}
}, {
"name": "b2b.json",
"vhost": "/",
"type": "topic",
"durable": true,
"auto_delete": false,
"internal": false,
"arguments": {}
}, {
"name": "as4.send.receipt",
"vhost": "/",
"type": "direct",
"durable": true,
"auto_delete": false,
"internal": false,
"arguments": {}
}, {
"name": "as4.relation.create",
"vhost": "/",
"type": "topic",
"durable": true,
"auto_delete": false,
"internal": false,
"arguments": {}
}
],
"bindings": [{
"source": "as4.inbound",
"vhost": "/",
"destination": "as4.inbound.delivery",
"destination_type": "queue",
"routing_key": "https://www.bdew.de/as4/communication/services/MP",
"arguments": {}
},{
"source": "as4.inbound",
"vhost": "/",
"destination": "as4.inbound.pathswitch",
"destination_type": "queue",
"routing_key": "https://www.bdew.de/as4/communication/services/pathSwitch",
"arguments": {}
},{
"source": "as4.receipt.outbound",
"vhost": "/",
"destination": "as4.receipt.outbound.delivery",
"destination_type": "queue",
"routing_key": "https://www.bdew.de/as4/communication/services/MP",
"arguments": {}
},{
"source": "as4.receipt.outbound",
"vhost": "/",
"destination": "as4.receipt.outbound.pathswitch",
"destination_type": "queue",
"routing_key": "https://www.bdew.de/as4/communication/services/pathSwitch",
"arguments": {}
},{
"source": "b2b.json",
"vhost": "/",
"destination": "b2b.json.default",
"destination_type": "queue",
"routing_key": "#",
"arguments": {}
},{
"source": "as4.send.receipt",
"vhost": "/",
"destination": "as4.send.receipt.default",
"destination_type": "queue",
"routing_key": "default",
"arguments": {}
},{
"source": "as4.outbound",
"vhost": "/",
"destination": "as4.outbound.9903111045603",
"destination_type": "queue",
"routing_key": "9903111045603",
"arguments": {}
},{
"source": "as4.outbound",
"vhost": "/",
"destination": "as4.outbound.9900794076504",
"destination_type": "queue",
"routing_key": "9900794076504",
"arguments": {}
},{
"source": "as4.send.receipt",
"vhost": "/",
"destination": "as4.send.receipt.9903111045603",
"destination_type": "queue",
"routing_key": "9903111045603",
"arguments": {}
},{
"source": "as4.send.receipt",
"vhost": "/",
"destination": "as4.send.receipt.9900794076504",
"destination_type": "queue",
"routing_key": "9900794076504",
"arguments": {}
},{
"source": "as4.relation.create",
"vhost": "/",
"destination": "as4.relation.create.default",
"destination_type": "queue",
"routing_key": "default",
"arguments": {}
}
]
}
Verwenden Sie für Multi Tenancy und Redundanz das folgende Schema:
{
"users": [
{
"name": "guest",
"password": "upQ4VVrEGhxSWHoIKsue",
"hashing_algorithm": "rabbit_password_hashing_sha256",
"tags": "administrator",
"limits": {}
}
],
"vhosts": [{
"name": "/"
}
],
"permissions": [{
"user": "guest",
"vhost": "/",
"configure": ".*",
"write": ".*",
"read": ".*"
}
],
"topic_permissions": [],
"parameters": [],
"global_parameters": [{
"name": "internal_cluster_id",
"value": "rabbitmq-cluster-id-ZN4mSvA1esBHKCUX2VrpLw"
}
],
"policies": [],
"queues": [{
"name": "as4.inbound.delivery",
"vhost": "/",
"durable": true,
"auto_delete": false,
"arguments": {
"x-dead-letter-exchange": "DLX",
"x-dead-letter-routing-key": "as4.inbound.delivery",
"x-queue-type": "quorum"
}
}, {
"name": "as4.inbound.pathswitch",
"vhost": "/",
"durable": true,
"auto_delete": false,
"arguments": {
"x-dead-letter-exchange": "DLX",
"x-dead-letter-routing-key": "as4.inbound.pathswitch",
"x-queue-type": "quorum"
}
},{
"name": "as4.receipt.outbound.delivery",
"vhost": "/",
"durable": true,
"auto_delete": false,
"arguments": {
"x-dead-letter-exchange": "DLX",
"x-dead-letter-routing-key": "as4.receipt.outbound.delivery",
"x-queue-type": "quorum"
}
},{
"name": "as4.receipt.outbound.pathswitch",
"vhost": "/",
"durable": true,
"auto_delete": false,
"arguments": {
"x-dead-letter-exchange": "DLX",
"x-dead-letter-routing-key": "as4.outbound.pathswitch",
"x-queue-type": "quorum"
}
},{
"name": "b2b.json.default",
"vhost": "/",
"durable": true,
"auto_delete": false,
"arguments": {
"x-dead-letter-exchange": "DLX",
"x-dead-letter-routing-key": "b2b.json.default",
"x-queue-type": "quorum"
}
},{
"name": "as4.outbound.9903111045603",
"vhost": "/",
"durable": true,
"auto_delete": false,
"arguments": {
"x-dead-letter-exchange": "DLX",
"x-dead-letter-routing-key": "as4.outbound.9903111045603",
"x-queue-type": "quorum"
}
},{
"name": "as4.outbound.9900794076504",
"vhost": "/",
"durable": true,
"auto_delete": false,
"arguments": {
"x-dead-letter-exchange": "DLX",
"x-dead-letter-routing-key": "as4.outbound.9900794076504",
"x-queue-type": "quorum"
}
},{
"name": "as4.send.receipt.9903111045603_1",
"vhost": "/",
"durable": true,
"auto_delete": false,
"arguments": {
"x-dead-letter-exchange": "DLX",
"x-dead-letter-routing-key": "as4.send.receipt.9903111045603_1",
"x-queue-type": "quorum"
}
},{
"name": "as4.send.receipt.9903111045603_2",
"vhost": "/",
"durable": true,
"auto_delete": false,
"arguments": {
"x-dead-letter-exchange": "DLX",
"x-dead-letter-routing-key": "as4.send.receipt.9903111045603_2",
"x-queue-type": "quorum"
}
},{
"name": "as4.send.receipt.9900794076504",
"vhost": "/",
"durable": true,
"auto_delete": false,
"arguments": {
"x-dead-letter-exchange": "DLX",
"x-dead-letter-routing-key": "as4.send.receipt.9900794076504",
"x-queue-type": "quorum"
}
},{
"name": "as4.relation.create.default",
"vhost": "/",
"durable": true,
"auto_delete": false,
"arguments": {
"x-dead-letter-exchange": "DLX",
"x-dead-letter-routing-key": "as4.relation.create.default",
"x-queue-type": "quorum"
}
}
],
"exchanges": [{
"name": "as4.inbound",
"vhost": "/",
"type": "topic",
"durable": true,
"auto_delete": false,
"internal": false,
"arguments": {}
}, {
"name": "as4.receipt.outbound",
"vhost": "/",
"type": "topic",
"durable": true,
"auto_delete": false,
"internal": false,
"arguments": {}
}, {
"name": "b2b.json",
"vhost": "/",
"type": "topic",
"durable": true,
"auto_delete": false,
"internal": false,
"arguments": {}
}, {
"name": "as4.send.receipt",
"vhost": "/",
"type": "topic",
"durable": true,
"auto_delete": false,
"internal": false,
"arguments": {}
}, {
"name": "as4.relation.create",
"vhost": "/",
"type": "topic",
"durable": true,
"auto_delete": false,
"internal": false,
"arguments": {}
}
],
"bindings": [{
"source": "as4.inbound",
"vhost": "/",
"destination": "as4.inbound.delivery",
"destination_type": "queue",
"routing_key": "https://www.bdew.de/as4/communication/services/MP",
"arguments": {}
},{
"source": "as4.inbound",
"vhost": "/",
"destination": "as4.inbound.pathswitch",
"destination_type": "queue",
"routing_key": "https://www.bdew.de/as4/communication/services/pathSwitch",
"arguments": {}
},{
"source": "as4.receipt.outbound",
"vhost": "/",
"destination": "as4.receipt.outbound.delivery",
"destination_type": "queue",
"routing_key": "https://www.bdew.de/as4/communication/services/MP",
"arguments": {}
},{
"source": "as4.receipt.outbound",
"vhost": "/",
"destination": "as4.receipt.outbound.pathswitch",
"destination_type": "queue",
"routing_key": "https://www.bdew.de/as4/communication/services/pathSwitch",
"arguments": {}
},{
"source": "b2b.json",
"vhost": "/",
"destination": "b2b.json.default",
"destination_type": "queue",
"routing_key": "#",
"arguments": {}
},{
"source": "as4.outbound",
"vhost": "/",
"destination": "as4.outbound.9903111045603",
"destination_type": "queue",
"routing_key": "9903111045603",
"arguments": {}
},{
"source": "as4.outbound",
"vhost": "/",
"destination": "as4.outbound.9900794076504",
"destination_type": "queue",
"routing_key": "9900794076504",
"arguments": {}
},{
"source": "as4.send.receipt",
"vhost": "/",
"destination": "as4.send.receipt.9903111045603_1",
"destination_type": "queue",
"routing_key": "9903111045603",
"arguments": {}
},{
"source": "as4.send.receipt",
"vhost": "/",
"destination": "as4.send.receipt.9903111045603_2",
"destination_type": "queue",
"routing_key": "9903111045603",
"arguments": {}
},{
"source": "as4.send.receipt",
"vhost": "/",
"destination": "as4.send.receipt.9900794076504",
"destination_type": "queue",
"routing_key": "9900794076504",
"arguments": {}
},{
"source": "as4.relation.create",
"vhost": "/",
"destination": "as4.relation.create.default",
"destination_type": "queue",
"routing_key": "default",
"arguments": {}
}
]
}
Wenn Sie das Schema nicht nutzen sondern die Queues selbst konfigurieren möchten, gehen Sie wie folgt vor:
Exchange as4.receipt.outbound
Die Exchange as4.receipt.outbound (Type = topic) routet Marktübertragungsnachrichten in Richtung des Backends, während Pathswitch Nachrichten in Richtung des AS4-Address-Service geroutet werden. Entsprechend müssen folgende Bindings eingerichtet werden:
| to queue | routing key |
|---|---|
| as4.receipt.outbound.pathswitch | https://www.bdew.de/as4/communication/services/pathSwitch |
| as4.receipt.outbound.delivery | https://www.bdew.de/as4/communication/services/MP |
Legen Sie die zugehörigen Queues an, falls sie noch nicht existieren (Type = Quorum). Ergänzen Sie auch die notwendigen Argumente:
- “x-dead-letter-exchange”: “DLX”,
- “x-dead-letter-routing-key”: (siehe Schema Definition)
Exchange as4.inbound
Die Exchange as4.inbound (Type = topic) routet Marktübertragungsnachrichten in Richtung des Backends, während Pathswitch Nachrichten in Richtung des AS4-Address-Service geroutet werden. Entsprechend müssen folgende Bindings eingerichtet werden:
| to queue | routing key |
|---|---|
| as4.inbound.pathswitch | https://www.bdew.de/as4/communication/services/pathSwitch |
| as4.inbound.delivery | https://www.bdew.de/as4/communication/services/MP |
Legen Sie die zugehörigen Queues an, falls sie noch nicht existieren (Type = Quorum). Ergänzen Sie auch die notwendigen Argumente:
- “x-dead-letter-exchange”: “DLX”,
- “x-dead-letter-routing-key”: (siehe Schema Definition)
Shovel Plugin
In RabbitMQ, the Shovel Plugin provides a robust and automated way to move messages between queues, including transferring messages from a Dead Letter Queue (DLQ) back to its originating queue. By configuring a Shovel, you can define source and destination queues, ensuring messages are moved efficiently and reliably, without manual intervention.
Dynamic Configuration (via Management UI)
- Go to the RabbitMQ Management UI (http://
:15672). - Navigate to the Admin tab.
- Under the Shovels section, click “Add a new shovel.”
- Configure the shovel:
- Name: Give the shovel a name (e.g., dlq_to_main).
- Source URI: amqp:// (or the URI of the RabbitMQ broker containing the DLQ).
- Source Queue: The name of the DLQ (e.g., dlq).
- Destination URI: amqp:// (or the URI of the RabbitMQ broker containing the main queue).
- Destination Queue: The name of the main queue (e.g., main_queue).
- Set Ack Mode to on-confirm or on-publish for reliability.
- Click Add Shovel
Here we can define the prefetch count which regulates the amount of messages moved at once. It acts as a flow control mechanism to balance the load between the source and destination queues, preventing the Shovel from overwhelming the destination or consuming too much memory by fetching too many messages.
configuration for shovel in definitions.json looks like below:
"parameters": [
{
"value": {
"ack-mode": "on-confirm",
"dest-add-forward-headers": false,
"dest-protocol": "amqp091",
"dest-queue": "as4.outbound.request.default",
"dest-uri": "amqp://",
"reconnect-delay": 5,
"src-delete-after": "never",
"src-protocol": "amqp091",
"src-queue": "as4.outbound.request.default.dlq",
"src-uri": "amqp://"
},
"vhost": "/",
"component": "shovel",
"name": "dlq_to_base_as4_outbound_request"
}
]
And the mounted enabled_plugins must contain rabbitmq_shovel_management like below: [rabbitmq_management,rabbitmq_prometheus,rabbitmq_shovel,rabbitmq_shovel_management]
View Me Edit Me