SAP Data Services for Social Media Load from Twitter

[Former Member]

Dear all,

We are currently trying to load social media posts from twitter into SAP hybris marketing 1605 on-premise. We are using SAP Data services 4.2 SP9 and SAP IPS 4.2 SP4. We followed the Rapid Data Load guide, and "Load social media data for SAP Hybris Marketing (DI6)".

During the execution of the job, we are currently getting the following errors:

  |Data flow DF_Map_Twitter_Data_NoCust|Loader Get_T_MaxID_STG_TW_MAXID
SQL submitted to ODBC data source <xx.xxx.xx.xxx> resulted in error <[SAP AG][LIBODBCHDB DLL][HDBODBC] General error;362
invalid schema name: <RDS_DS_STAGE>: line 1 col 21 (at pos 20)>. The SQL submitted is <CREATE COLUMN TABLE
<RDS_DS_STAGE>."STG_TW_MAXID" ( "CHANNEL" VARCHAR (4)  not null ,  "LANGUAGE" VARCHAR (4)  not null ,  "SEARCH_TERM" VARCHAR
(50)  not null ,  "MAX_ID" VARCHAR (50) ,   primary key ( "CHANNEL", "LANGUAGE", "SEARCH_TERM") ) >.
|Data flow DF_Map_Twitter_Data_NoCust|Loader Get_T_MaxID_STG_TW_MAXID
SQL submitted to ODBC data source <xx.xxx.xx.xxx> resulted in error <[SAP AG][LIBODBCHDB DLL][HDBODBC] General error;362
invalid schema name: <RDS_DS_STAGE>: line 1 col 21 (at pos 20)>. The SQL submitted is <CREATE COLUMN TABLE
<RDS_DS_STAGE>."STG_TW_MAXID" ( "CHANNEL" VARCHAR (4)  not null ,  "LANGUAGE" VARCHAR (4)  not null ,  "SEARCH_TERM" VARCHAR
(50)  not null ,  "MAX_ID" VARCHAR (50) ,   primary key ( "CHANNEL", "LANGUAGE", "SEARCH_TERM") ) >.

When I look into HANA though, the schema is present.

Also, I did make some changes to the repositories as following (to add the users due to "hana server is unknown" error).

Did anyone encounter this error before? Do I need to actually change something in the SAP Data Services job itself?

[Former Member · Accepted solution]

Fixed it as following. If you want more details please contact me.

1. Run Job_Load_Social_Media_Search_Query on SAP DS to create base table
2. Run Job_Twitter_Data_Load to retrieve posts
3. Run reports etc. in application server for assigning sentiment

I am now able to retrieve posts from Twitter.

Some remaining questions:

• Search queries are uploaded from Excel, so why is there a need to customize in the application server?
• How accurate is the language setting in the Search Query?

[Former Member]

Hi Komur,

Before run the Twitter job, Please follow the test script DI6_CEI12_BPD_EN_XX.docx to maintain the search query and run the job as sequence.

Best Regards

Hank

[Former Member]

Dear Hank,

I went through that guide, but nowhere in there does the HANA table STG_SMI_SEARCH_QUERY get generated. the actual table still seems to be missing in the HANA DB.

I do have search queries maintained in Customizing.

Do I need to run some kind of report to get these tables created after the search terms have been created?

[former_member318524]

Dear Akil,

We are implementing HM on-premisse 1702. As we couldn't use SAP Data Services to extract the information from Twitter SAP recommended us to use Smart Data Connector to simulate what SAP DS has to do. Then we had created a program in order to fill the interactions tables and we can get some information.

I am not really sure about this other tables: SOCIALDATA (it has information) but the table SOCIALUSERINFO is not receiving any data and we do not understand what else to do.

I was searching information regarding this subject but i don't really understand how we should fill this table with twitter information.

Also i was sure on activate this business functions: FND_SOMI_CM and FND_SOMI_DS but i did and even that i wasn't getting any user additional information.

Also i wanted to ask you which are the reports you mentioned in the point 3 that you had to run?

• "Run reports etc. in application server for assigning sentiment"

I would really appreciate your help regarding this point.

Thanks in advance.

Danitza Apaza

[Former Member]

Dear Danitza,

If I remember correctly this was:

• CUAN_IA_TA_EXTR_DELTA with the social/text analysis selections enabled.

[former_member318524]

Dear Akil,

Thanks your for answer. Could you provide the link of the documentation where talks about this program?

I found this in the operation guide for HM 1702, but it looks like this is a separate task to do in order to don't have duplicated information and merge the contact information.

https://help.sap.com/http.svc/rc/ae97aa0c077f47da8f942176eef57322/1702%20YMKT/en-US/loio1c0cc750bed1...

I followed this guide but i still can't fill this table: SOCIALUSERINFO .

I would appreciate if you can share this information.

Regards,

Danitza Apaza

[Former Member]

Dear Danitza,

We used the following guides;

https://rapid.sap.com/bp/#/browse/scopeitems/DI6

Also see the part "Load social media data for SAP Hybris Marketing - Configuration (DI5)" on that page.

With regards,

Akil

[former_member318524]

Dear Akil,

Sorry for the late answer but i wanted to share with you all SAP Answer in the note 215369 .

"Be aware that importing Tweets into SAP Hybris Marketing typically will not merge the information from the Tweets with information on your actual customers. Instead new contacts per Twitter user will be created. This is caused by a missing joint ID to match the contacts – Twitter just provides the Twitter user ID and name and no additional information like an email address that could be used for a reasonable contact matching. The match works if for your customers a Twitter ID is known. This for example can be given if you implemented a social logon solution via Twitter. For god results you need a significant match rate that is not easy to achieve. From customer projects I was involved in I feel that the other way around typically works better: You have your customers in SAP Hybris Marketing and transfer them as custom audience to an external platform. There you can use the contacts directly, create look alike audiences or update custom audiences event based having an audience for example with contacts having contracts, that expire soon for a long running campaign. This is directly supported for example for Facebook or can be implemented in a customer project for Twitter using externally executed campaigns.
Regarding filling the table SOCIALDATA and SOCIALUSERINFO. These tables are part of Social Intelligence. This is not used by SAP Hybris Marketing (to be precise – some parts are used by the mentioned RDS but this is irrelevant for SAP Hybris Marketing). Instead social postings are stored as Interactions (Business Object CUAN_INTERACTION, root table CUAND_CE_IA_RT) and the corresponding users as Interaction Contacts (Business Object CUAN_INTERACTION_CONTACT, root table CUAND_CE_IC_ROOT)."

Regards,

Danitza Apaza

[Former Member]

I fixed this first issue, by changing the datastore again, for some reason the DS_OWNER was <RDS_DS_STAGE>.

I removed < and >.

Now we get the following errors:

5272	1308	DBS-070404	7/21/2017 6:28:14 AM	|Data flow DF_Map_Twitter_Data_NoCust|Reader Get_T_Data-BreakKey-Mapping1
5272	1308	DBS-070404	7/21/2017 6:28:14 AM	SQL submitted to ODBC data source <servername> resulted in error <[SAP AG][LIBODBCHDB DLL][HDBODBC] Base table not found;259
5272	1308	DBS-070404	7/21/2017 6:28:14 AM	invalid table name:  Could not find table/view STG_SMI_SEARCH_QUERY in schema RDS_DS_STAGE: line 7 col 356 (at pos 972)>. The

Nowhere in the manuals is there a mention of creating tables manually, does anyone have a missing guide?