How to analyze survey data in Webi 4.2 with numerous records for each person?

[Former Member]

I need to analyze and report survey data. Each respondent provides responses to questions, such as, name, age, gender, birth date, state, etc. Collected data is available in three dimensions in Webi 4.2: Respondent (ID), Question, and Answer as shown in the screenshot below:

As you would notice in the screenshot above, there are many rows for each respondent. It makes it difficult to provide simple analysis, such as average age of respondents or respondent gender distribution or ranking respondents by state etc. In reality, there are 100+ questions and thousands of respondents and years' worth of data to analyze and Webi queries max out the record limit and fetching only partial results.

I believe one row should represent one record and each column should represent one attribute of the data. I would like to have each question transposed to as a Dimension, as shown below, so that it would be easy to analyze the data.

What would be the best practice to create Webi documents so that analysis and reporting can be done more efficiently?

[Former Member]

Is my question too challenging for the experts? I am eagerly looking forward to a permanent solution so that I can do my development more efficiently. Please take a moment and advise your best suggestion whether at database level or at universe level or at Webi level.

[ayman_salem]

Hi,

You can do that with "cross table".

First, define a new variable (AnswerK) from the dimension "Answer" as a measure

Then insert Cross Table

....

I hope this gives you an idea of how you can do more

Ayman

[Former Member]

Ayman,

Thanks for taking time in doing the exercise and replying with the helpful screenshots. Could you please tell me why we should use the variable as Measure? There are string values in the Answer, such as Name, State, Gender, DOB etc.

Thanks,

KP

[ayman_salem]

In "simple" crosstabs, it does not matter what content is measured.

However, if you want to add a mathematical calculation, eg. Example: "average age", you must create a new variable "AvgAge" and convert "AnswerK" to the number in the formula ex: Average (ToNumber ([AnswerK])) and so on.

[Former Member]

Thanks, Ayman. I have tried cross-tab but I ran into some limitations because of cross-tab structure. I don't think it's sustainable approach, but I agree that it is one of the approaches. Looking for more robust and full-proof solution.

[sateesh_kumar1]

Hi KP,

traditional approach would be creating detail variables for your Answer object.

Gender=[Answer] Where ([Question]="Gender")

=NoFilter([Answer] Where ([Question]="name"))

=Sum(ToNumber(NoFilter([Answer] Where ([Question]="sal")))) and so on..

Regards

Sateesh

[Former Member]

Sateesh,

Thanks for your reply. Yes, creating multiple detail variables for each question is an option but when there are 100+ questions, it becomes tedious process. Any other alternative possible?

[sateesh_kumar1]

KP,

I don't see any options apart from above , your requirement has to be deal with modeling on back end.

Thank you

Sateesh

[Former Member]

Sateesh,

Yes, that's exactly I was thinking. But where in backend? At universe level or at database level? May be at universe level this could be transposed as expected? I have provided SQL script and output above in response to Jothi, if you could take a look and advise accordingly.

Thank you,

KP

[Former Member]

One of the problems I ran into is "partial results" fetched for query. The limit is 500,000 and will not be changed per BO Admins.

I just don't see any option that's full-proof 😞

[jyothirmayee_s]

Hi,

What is your data source for WebI?. How do you see the data results when run on the backend?.

can you post that resultset?.

Thanks,

Jothi

[Former Member]

Jothi,

Thanks for your reply. Data source is Vertica > Universe > Webi.

Here is the SQL:

SELECT
  wt_fact_survey_respondents.survey_respondent,
  com_survey_mapping.question_label,
  fact_survey_results.field_value
FROM
  wt_fact_survey_respondents INNER JOIN fact_survey_results ON (wt_fact_survey_respondents.survey_respondent=fact_survey_results.survey_respondent)
   INNER JOIN com_survey_mapping ON (fact_survey_results.field_name=com_survey_mapping.question_code)

[Former Member]

Above is the output from the query.

[Former Member]

Jothi,

Do you have any comment? Let me know what you think per my responses to your questions.

Thanks,

KP