Pipeline Engine offline or temporarily unavailable

[titto_antony]

Hi Team,

I'm working with the Datahub 2.3 Trail instance. I was trying to execute the "message broker in SAP Data Hub, trial edition 2.3" tutorial in SAP Datahub. But frequently i'm getting this Pipeline Engine offline or temporarily unavailable error message when i'm trying to execute the graph.

This error was even affecting execution of Enrich device data graph created earlier which was working fine previously. Is it a known issue? I was also getting below error in the Kafka producer and consumer transform:

Error messgage default.Kafka Producer: kafka: client has run out of available brokers to talk to (Is your cluster reachable?)

Please help to resolve the issue

[titto_antony]

Any suggestions to resolve the issue in Kafka Connection

[regysm21]

Hello,

I know it has been a while now since u have asked this question however I just wanted to ask if you could get this question resolved and how?

Thanks,

Regys

[titto_antony]

Thanks for your reply Thorsten Schneider.

For issue 2, I have setup the Kafka in the same network/subnetwork and also i tried to implement the steps mentioned by Douglas Maltby in the below question raised earlier on the same issue . Please let us know what all to be checked in GCP to get he Kafka communicate correctly

https://answers.sap.com/questions/678816/tutorial-issue-use-a-message-broker-in-sap-data-hu.html

[schneidertho]

Hi Titto,

for issue 1 (offline...): do you have a screenshot?

for issue 2 (Kafka): I think, this very much depends on your overall setup on GCP. Data Hub and Kafka need to be able to communicate. If that is possible or not, depends on the network setup (in particular: networks, subnets, firewall rules).
From my experience, the easiest to achieve this is to ensure that your Data Hub and your Kafka run in the same network/subnet and all internal traffic inside this network is allowed. Can you check that you use the same network/subnet and can you check the firewall rules. I am thiking how to improve the description in the tutorial.

Cheers

Thorsten