Skip to Content

Data Extraction from Social Media using SAP Data Services

Hi ,

I am going to extract the data from Social media like Twitter, Facebook, Linkedin and RSS feeds.

I got the job for extracting twitter data  in the Data Services Data quality blue prints.  But there we are using Python script for the extract the data.

We are going to use only Data integrator not Data Services.  Python script used in the user defined transform.  User defined transform  is part of Data quality.

We need to use only Data Integrator transforms and need to extract the data from Facebook, Twitter, Linkedin and RSS feeds.

I have checked in the SAP website and found one application called SAP Cloud for social engagement.  Is it possible to extract the data from Social engagement to extract  and store into the database.

Please help me in this ?

Thanks & Regards,

Venkata Ramana Paidi

Add comment
10|10000 characters needed characters exceeded

  • Get RSS Feed

1 Answer

  • Nov 04, 2015 at 10:09 AM

    Hi Venkata,

    I guess your requirement is not possible to implement.

    Only the user defined transform which comes under DQ Transform can be used to access data from Social media

    You can also use Entity Extraction Transform of Text Data Processing feature but DI Transforms will not help you

    Using SAP BO Data Services Text Data Processing for entities extraction from RSS feeds

    Regards

    Arun Sasi

    Add comment
    10|10000 characters needed characters exceeded

    • Arun Sasi Venkata Ramana Paidi

      Venkat,

      Accessing twitter feed/data from Twitter.com using user defined transform.

      1) Go to  https://apps.twitter.com/ and logon to twitter using your Twitter email id or user id

      2) Create a new application. You can use your own home page address

      https://twitter.com/your_login_name

      3) Get the token generated

      Consumer Key (API Key)

      Consumer Secret (API Secret)

      Access Token

      Access Token Secret

      4) Create a program in python and use the tokens generated in step 3 to extract the twitter tuple. Use this program in UDT in such a way that it will extract the tuple. I dont have Python knowledge so wont be able to help you with the code but you can search in google.

      Also the above steps look easy but its very tricky. So you willhave to take help from a Python developer.

      Better Python Development for BODS: How and Why | SCN

      Regards

      Arun Sasi