retrieve data from website

[former_member184293]

hi all,

I have a requirement to retrieve data from the website http://www.bnm.gov.my/?tpl=exchangerates -- Ringgit Foreign Exchange Conversion.

we need to convert the whole data if possible only selling row data to xml. and do some modifications and send it to ECC system.

Can you please suggest us a way to get the data from website and convert it to xml using SAP PI.

we are using SAP PI 7.4.

please let me know if i am not clear.

Thanks

kavya.

[iaki_vila]

HI Kavya,

If the response is a html as Eng pointed out, why don't do a proxy to proxy scenario with a SOAP lookup. You can start the process in your ECC endpoint and at mapping level do a soap lookup (http://scn.sap.com/community/pi-and-soa-middleware/blog/2013/08/07/how-to--soap-lookup), you will need a java development anyway. Your endpoint process will be obviously your same ECC.

Regards.

[engswee]

Hi Kavya

If you so happen to be on the latest SP09 of PI 7.4, then you can consider using the new REST adapter instead of the SOAP Axis adapter.

I checked the BNM site, and the content can be retrieved with just a HTTP GET operation (which is also possible with the HTTP_AAE adapter.)

However, I've had a look at the response data from the site and the content type returned is HTML. This will be the main challenge as you will need to parse the HTML response to get the exchange rate details from the HTML table, and then convert it to an XML output for further processing.

You will need to use a Java mapping or custom adapter module to achieve this parsing. I've not done this before, but you can try searching online for some Java library for HTML parser that might help you achieve it.

Rgds

Eng Swee

[markangelo_dihiansan]

Hi,

You could use jsoup to parse the html

Regards,

Mark

[former_member184293]

thank you all,

if i am using HTTP_AAE and GET Operation, what is the value i need to provide in

Request Details : - Main Payload Parameter name?

where should i have to give the URL details?

In Michal blog, the URL which he is providing  is only a XML message, but in my case it is a HTML website.

it would be great help if you provide me any blog with similar requirement.

Thanks

kavya.

[engswee]

Unfortunately, it looks like there isn't much content on SCN on the new HTTP_AAE adapter

I haven't tried it myself since I have a REST adapter on my system which provides more robust HTTP functionality.

You can refer to the following for the configuration.

Configuring the Java HTTP Adapter on the Receiver Channel - Advanced Adapter Engine - SAP Library

For the URL details, you have to split it into the host, port and path.

As for the Main Payload Parameter Name, the documentation is quite vague about how to use it for a GET operation. Maybe you can just try putting MainDocument.

Rgds

Eng Swee

[former_member184293]

hi eng,

you have provided me receiver channel details. but i need to convert the html document into XML right. so i have to use the sender channel get operation.

please correct me if i am wrong.

Thanks

kavya.

[engswee]

Kavya

You won't be able use the sender channel because HTTP_AAE does not support sender polling. Even SAP's own REST adapter does not have the feature yet.

My suggestion of receiver channel is based on a proxy to HTTP_AAE synchronous approach.

If you want to design based on sender polling, you can probably check out the SOAP Axis approach that was provided by Ram above. I haven't worked with the SOAP Axis adapter so I won't be able to provide you further input on that.

Rgds

Eng Swee

[Former Member]

Hi Kavya,

Refer this blog, this exactly fits your requirement.

Regards,

Pranav