Hi there,
I hope this is not a duplicate. We currently want to develop an integration in the CPI which searches in different directories on an On-Premise FTP server for matching files (like (/xml-files/foobar.xml and /pdf-files/foobar.pdf), zips those files and uploads the zip file to an public SFTP server. As a reverse proxy we already use the Cloud Connector, which I enhanced for the hostname of our On-Premise FTP server.
Especially regarding the "searching for matching files in different directories" on the FTP server, I can only think of a Groovy script which establishes the connection to the On-Premise FTP server via the Cloud Connector (I guess you can ignore that the target is an FTP server since I cannot even reach the FTP server from the Groovy Script, so it is more a general question about how to reach On-Premise systems via the Cloud Connector from a Groovy Script in the CPI-Cloud Foundry Environment).
I found the following information regarding using a TCP connection from the BTP (Cloud Foundry) via the Cloud Connector which acts as a SOCKS5-Proxy Server in this case: Using the TCP Protocol for Cloud Applications. I used the coding (which basically manually handles the custom SOCKS5 authentication via a JWT-Token (see: Authenticate against Cloud Connector/SOCKS5-Proxy) and the SOCKS5 Connect command to create a TCP socket.
The authentication against the Cloud Connector/SOCKS5 Proxy is working fine, but the SOCKS5 Connect command is failing when I run the integration flow. The response from the Cloud Connector/SOCKS5-Proxy is "SOCKS5 command failed with status: FORBIDDEN"

According to Using the TCP Protocol for Cloud Applications (bottom) this indicates that the Cloud Connector/SOCKS5 Proxy does not contain a ruleset/system mapping for the (virtual) host.
The system mapping in the Cloud Connector is definitely set up (the ports beginning at 30000 are the data ports of the FTP server) ...

... and via the Connectivity Test from within the CPI I can definitely reach the On-Premise FTP server and even see the content of the root directory, so the communication port and the data ports of the FTP server should all be set correctly in the Cloud Connector system mappings.

The only way to force the Connectivity Test to produce the same error message (SOCKS5 command failed with status: FORBIDDEN) that I get from the Integration Flow/Groovy Script within the the Integration Flow is to enter a (virtual) hostname with Proxy Type=On-Premise that definitely does not exist in the Cloud Connector system mapping like "does.not.exist.in.cloud.connector.com". When testing this non-existent host name via the CPI Connectivitiy Test I can also see a corresponding entry in the Audit log of the Cloud Connector


Note: The timestamps are 4 mins apart since the system running the Cloud Connector has configured a slightly different/wrong time
However. The connection attempts via the Integration Flow/Groovy Script within the the Integration Flow are not visible in the Audit log of the cloud connector. I therefore somehow have the feeling that from within the CPI Integration Flow I am not really "hitting" the correct Cloud Connector (which has my system mapping for the host name of the FTP server).
The proxy host and proxy port that I retrieve via the environment variables in the Groovy Script via the coding provided by the SAP Help under Using the TCP Protocol for Cloud Applications
private static final String SOCKS5_PROXY_HOST_PROPERTY = "onpremise_proxy_host";
private static final String SOCKS5_PROXY_PORT_PROPERTY = "onpremise_socks5_proxy_port";
...
protected InetSocketAddress getProxyAddress() {
try {
JSONObject credentials = extractEnvironmentCredentials();
String proxyHost = credentials.getString(SOCKS5_PROXY_HOST_PROPERTY);
int proxyPort = Integer.parseInt(credentials.getString(SOCKS5_PROXY_PORT_PROPERTY));
...
}
...
}
are => connectivityproxy.internal.cf.eu10.hana.ondemand.com:20004 (since I am on a Cloud Foundry environment in EU10/Frankfurt)
I also checked the individual bytes from the SOCKS5 connect command via the monitor log and they look quite alright when I compare them against e.g. the wiki article about the SOCKS protocol:
SOCKS5 connect command payload 05 // 0x05 => Socks version 5 01 // 0x01 establish a TCP/IP stream connection (0x02 and 0x03 not supported by SAP) 00 // reserved, always 0x00 03 // addr 0x03 => domain name 16 // addr/domain name length 0x16 => 22 ........................................6465 // hex to ascii => ....................de (obfucsated, since I don't want to share the host name) 0015 // port 0x0015 => 21
Info: Cloud Connector version is 2.12.1.1, so not the most current version, but as far as I researched maybe 18 months old.
So my question is: Does anybody know if the proxy host and proxy port to "hit" the correct Cloud Connector (which is connected to the BTP (Cloud Foundry) subaccount) has to be different /than the ones retrieved via the environment variables when the connection should be established via the CPI/Integration Suite instance?
I know that even when the connection could be establish that there are additional steps such that the FTP protocol will work in this scenario (especially since SAP uses the custom SOCKS5 authentication method (0x80) with the JWT-Token which is not existing in the SOCKS-Standard, but I think I would have a solution for that and anyways this would be step 2 once the connection attempts via the Groovy Script over the Cloud Connector would work.