Unable to see running ESP project from remote ESP Studio

[Former Member]

Dear SAP Event Stream Processor Experts,

I have been sifting through the community and can't seem to find a case that would help me with my issue. I hope you can point me to the solution.

We are just starting out with development and so far have one ESP 5.1 SP12 installation running on SUSE Linux in the cloud. The cluster database and node1 is running. From ESP Studio on a Windows terminal server in the same private network, I can connect and run a first test project to manually write data from an input stream to disk on the SUSE Server.

Now I am trying to do the same from my Laptop, which obviously is not within the private network of the SUSE Server. I am able to connect to the cluster via its' public IP on port 19011 and I can see the contained workspaces and my test project displayed.

However, I don't see the input stream, eventhough the project is running. When I stop the project on the SUSE Server, and then try starting it from ESP Studio on my Laptop, this actually does start the project (I can see it is running again from Studio on the terminal server). But still, I am unable to see the contained stream on my Laptop. Right-clicking the project also continues to give me the option to start the project, not to stop it (eventhough it is running). If I try starting the running project from my Laptop, an error message tells me, that the project is running. [FAILURE:Application current status is not stopped][CODE:710077]

If I am trying to connect via private IP or public IP from the terminal server within the private network, both is working fine. I am noting this, because connecting with public IP from the terminal would be analogous to connecting with public IP from my laptop. That it is working with public IP from within the network makes me guess, that the problem is boiling down to how I set up the IPs and DNS names in the configuration files.

My configuration looks like this:

start_db.sh on SUSE:
ESP_CLUSTER_DB_NAME=esp_cluster_esp1_esp.public.com

cluster.cfg on SUSE has this line: jdbc-url=jdbc:sybase:Tds:<public-IP>:19111

cluster.xml on SUSE has this line: 
<Macro name="STREAMING_HOSTNAME" type="value">esp.private.com</Macro>

/etc/hosts on SUSE maps 
esp.public.com --> public IP 
esp.private.com --> private IP

hosts on Windows Terminal Server maps
esp.private.com --> private IP

hosts on Windows Laptop maps
esp.private.com --> public IP

If I try using esp.public.com as STREAMING_HOSTNAME, the cluster node can not even be started (please see attached error log from node1log.txt). This is why I ended up using above configuration with esp.private.com, which is working fine within the private network and also allows me to connect to the cluster database from remote ESP Studio.

So my question is: why can I not see the running project and the contained stream, after connecting from my laptop? Do additional ports need to be opened on the cloud network for external access or how do I get the configuration right for my purpose?

At this point, I also want to note, that the end goal of this exercise is to be able to connect a publisher written in Java to the stream. That publisher is running on a sensor device, which is supposed to be hooked to the internet from anywhere to stream the data to ESP on the SUSE. I was unable to connect that publisher with following exception:

Exception in thread "main" java.io.IOException: Failed call to:https://esp.private.com:48931/RPC2 (Failed to read server's response: Connection timed out: connect)at com.sybase.esp.sdk.internal.ProjectGateway.isBigEndian(ProjectGateway.java:1036)   at com.sybase.esp.sdk.impl.ProjectImpl.doUpdatePropertiesFromProject(ProjectImpl.java:1656)   at com.sybase.esp.sdk.impl._5_1_SP10.ProjectImpl.doUpdatePropertiesFromProject(ProjectImpl.java:39)   at com.sybase.esp.sdk.impl.ProjectImpl.doConnect(ProjectImpl.java:1421)   at com.sybase.esp.sdk.impl.ProjectImpl.connect(ProjectImpl.java:172)  at sensordata.program.ConnectWithESP(program.java:103)   at sensordata.program.main(program.java:76)

It seems to me, the device also is able to connect to the server, but then unable to connect the publisher to the stream. So, in order to troubleshoot this, my thinking is, that I should first be able to connect ESP Studio from a remote location. So in the end, my problem is not really about connecting ESP Studio for development purposes - I could develop from Studio on the terminal server - it really is about connecting a Java publisher.

You may want to point me to this solution, which apparently is not helping:
http://help.sap.com/saphelp_esp_51sp12_tb/

I would be glad if you had any suggestions for solving this issue!

Thank you very much!

Daniel

[RobertWaywell · Accepted solution]

Since you are working with a cloud server instance, you will want to explicitly configure the ports being used by your ESP project. So far it sounds like you had your network admin open 1 of the 3 required ports, and since the default behaviour is for the ports to be dynamically allocated each time you start the project, you will find that the ports change each time you start/restart the project. By configuring the project to always use the same 3 ports, you will then be able to explicitly open just the 3 ports required for the project. Take a look at this blog post for instructions on how to specify the ports:

Specifying the Command Port to be used by a Smart Data Streaming Project

While that post was specifically written for HANA Smart Data Streaming, the project configuration steps and dialogues are the same for ESP projects.

[Former Member · Accepted solution]

Setting the three project ports in the ccr file and then opening the firewall for them as described is the exact solution! Thanks! 🙂

[Former Member]

Thank you for your very quick answer! 🙂

I tried configuring the network copnnections in ESP Studio as proposed, but it did not help. Also, I made sure there is no proxy involved in the connection.

Still, I am a step ahead now. The issue here really is that the port, which is used for connecting via https was blocked and needed to be opened - i. e. I checked which port the connection currently uses (since this seems to be randomly assigned in a range of something like port 40000 to 50000) and asked my admin to open that specific port.

The result is, that I can now connect and disconnect the project from Studio on my laptop. I can see the contained stream and view it within Stream View.

However, this leads me to the next issue and leaves me with an open question about port configuration.

First, the next issue is that now, if I try to feed data via Manual Input within Studio into the stream, I get "Connection timed out: connect".

If I now try connecting with my Java sensor device, I also get "Connection timed out: connect", but with a little more information around the exception:

Exception in thread "main" java.net.ConnectException: Connection timed out: connect   
at sun.nio.ch.Net.connect0(Native Method)   
at sun.nio.ch.Net.connect(Unknown Source)   
at sun.nio.ch.Net.connect(Unknown Source)   
at sun.nio.ch.SocketChannelImpl.connect(Unknown Source)   
at com.sybase.esp.sdk.internal.channel.SocketInformationChannel.open(SocketInformationChannel.java:30)   
at com.sybase.esp.sdk.internal.channel.SslSocketInformationChannel.open(SslSocketInformationChannel.java:29)   
at com.sybase.esp.sdk.internal.DataAccessSsl.open(DataAccessSsl.java:50)   
at com.sybase.esp.sdk.impl.PublisherImpl.doConnect(PublisherImpl.java:281)   
at com.sybase.esp.sdk.impl.PublisherImpl.connect(PublisherImpl.java:49)   
at sensordata.program.ConnectWithESP(program.java:106)   
at sensordata.program.main(program.java:76)

While I don't seem to quickly find an answer to this issue in the community, and it may be related to my overall configuration, do you have a solution for this?

Second, the open question about port configuration is, how to configure the pool of ports, which can or should be used for the https connection. Since ports seem to be randomly assigned - which port range has to be openly available for standard ESP installations? Is it possible to nail this down to fewer or even a specific single port, for each device that is supposed to feed into ESP? We would prefer to keep our system as sealed as possible and have some control over necessary ports. I would be thankful if you were able to point me towards the relevant documentation regarding this.

Many greetings!

Daniel

[JWootton]

This is most likely because you are connecting via a proxy. While I suppose it could be caused by other things, I have seen this exact problem many times and it was always a proxy setting. Try this:

1. In ESP Studio, go to the menu Window >> Preferences

2. In the navigation bar, select "General" and then "Network Connections"

3. Sect "Active Provider" to "Manual"

4. Configure your HTTPS schema, adding the host name for your proxy (if you don't know, try "proxy") along with the port number (most likely 8080).