DS SDK : Expand hierarchy in visualization

[jmsrpp]

Hello,

I'm working my way through some D3 visualizations and have managed to get a tree diagram working thanks to this example:

http://www.d3noob.org/2014/01/tree-diagrams-in-d3js_11.html

The visualization takes a dimension object with an activated hierarchy and renders it like this:

With a crosstab component I can get the tree to expand:

Now I want to extend this to function like the Collapsible Tree from Mike Bostock:

http://bl.ocks.org/mbostock/4339083

I've incorporated the transitions from the update function, which all work well, but the problem comes with the click event, which is handled in the D3 code with this function:

// Toggle children on click.
function click(d) {
  if (d.children) {
  d._children = d.children;
  d.children = null;
  } else {
  d.children = d._children;
  d._children = null;
  }
  update(d);

This is called in the update function when the nodes are declared:

        // Declare the nodes

        var node = svg.selectAll("g.node")

        .data(nodes)

        .enter().append("g")

        .attr("class", "node")

        .attr("transform", function(d) { return "translate(" + d.y + "," + d.x + ")"; })

        .on("click", click);

I can see the event being fired, and d._children being switched with d.children, but nothing happens in my visualization. I even tried modifying the function to set the metadata property nodeState, hoping this would alter the datasource, but to no avail:

            function click(d) {

            if (d.children) {

              d._children = d.children;

              d.children = null;

              if (d.nodeState)

              {d.nodeState = "EXPANDED";}

            } else {

              d.children = d._children;

              d._children = null;

              if (d.nodeState)

              {d.nodeState = "COLLAPSED";}

            }

            update(d);

          }

In the console, I can see the property d.nodeState change from EXPANDED to COLLAPSED when I set the appropriate break point, but nothing changes in the viz

I'm guessing I need to do something differently to interact with the data source and change it's state but I couldn't find anything in the guide. Unfortunately, the Simple Crosstab sample doesn't provide the expand/collapse functionality either, though it does render the plus signs that indicate a hierarchy node is available.

Any ideas on how I can add this interactivity to my extension would be very welcome.

Thanks,

Jim

[mike_howles4 · Accepted solution]

If you can drop your project out on Github I will try to take a look.

[jmsrpp · Accepted solution]

Hey Mike,

Thanks for the offer ... you can find the project here:

jmsrpp/DesignStudio · GitHub

com.sap.sample.dendogram/res/js/d3-tree-renderer.js is the file I'm working on as mentioned above. Code is cluttered at the moment, so apologies.

You just need to activate a hierarchy (BW, HANA, etc.) and attach that data source to the sample component.

Thanks,

Jim

[jmsrpp · Accepted solution]

Quick update ... I forgot to update my node.append to nodeEnter.append as per the D3 example. I made those modifications and I can now expand/collapse for any node that has already been expanded in the crosstab (my transitions aren't correct yet but it works).

So the question remains, how can I control the state of the data source from within my viz?

I updated Git with the latest.

Edit: transitions are all working now as well. Also cleaned up the source and added CSS styles.

[mike_howles4 · Accepted solution]

Hey, James

Didn't have any time over the weekend to look at this like I wanted (burst pipes and stomach bug)

I cloned the repo this morning, created a new DS App, and pulled in a Data Source with one hierarchy and one measure, and then bound it to your SDK component.  For reasons I've not spent time yet on, I cannot get it to detect my hierarchy.  Below is what I see:

I did notice some 1.5 residual references in your project so it may perhaps be related to something that I cannot do in 1.4?  Maybe not, but just letting you know where I'm at.

[jmsrpp · Accepted solution]

Hey Mike,

I caught some of the bad fortune on Twitter ... hope you're on the rebound and the pipes are back in action.

I forgot to mention that you need to turn off totals from the datasource in order to get the component working properly. I need to fix that at some point but for now it should get you going. My underlying data source looks like this:

I'm also using 1.4 so the residual references shouldn't be a problem.

Thanks for looking,

Jim

[jmsrpp · Accepted solution]

Hey Mike,

Spoke to some folks in dev and it turns out that the expand/collapse functionality in the crosstab requires a server-side request to be made. I couldn't identify the facility for that to implement in the visualization.

It seems like there are 2 options I could use to resolve this:

1. Use a defaultValue property and script contribution from contribution.ztl with a BIAL breakout (i.e. this.getDataSource().expand/collapse) and let Design Studio handle the on click event from the D3 viz.
   
   
2. The D3 examples load the whole JSON stream, so I could also pre-expand the entire hierarchy structure in my data source, and let the viz operate on top of the whole data set. Possible performance impact but with a few thousand records probably not that bad.

I couldn't figure out how to pass d.key from my D3 code into the script contribution, so I'm focusing on option 2 at the moment. I got close but the update function only impacts children, so my parent leafs aren't re-positioning properly on click. This should be something I'm capable of resolving (I hope).

Interested in any thoughts you might have.

Thanks,

Jim

[mike_howles4 · Accepted solution]

Hey James,

Spoke to some folks in dev and it turns out that the expand/collapse functionality in the crosstab requires a server-side request to be made.

...

The D3 examples load the whole JSON stream, so I could also pre-expand the entire hierarchy structure in my data source, and let the viz operate on top of the whole data set. Possible performance impact but with a few thousand records probably not that bad.

Yes, I also ran into that when doing this exercise:

I had to pre-expand the hierarchy to the intended level and then use a Crosstab component to expand it further.  Was hoping for a BIAL angle to help but ran out of time chasing it.

My days are still short on extra time due to my home "event" as you can imagine but I definitely want to help see this one through because I saw all the effort you gave in the JavaScript and I think it'll be a great reference example for hierarchy data!  Maybe my (very crude) Packed Circles example has something in there to help.  Warning - It is not coded very well

EDIT: Direct link to source code:  DesignStudio1.3UtilityPack/packedcircles.js at master · entmike/DesignStudio1.3UtilityPack · GitHub

[manfred_schwarz · Accepted solution]

Hi James,

I took the exact same example and have it already working. My trick to expand the hierarchy is creating two extra events (onExpand and onCollapse). For this events the Design Studio Developer has to code a DS_1.expandHierarchy(Dimension, value).

I did not found any function in SDK that has access to the used datasource. With my "solution" you are able to expand and collapse multiple hierarchies at the same time.

I have to clean up a little bit and i will publish my work in Mike and Karols repository latest by the end of next week. Do you ned a sneak peak before that ?

Best regards

Manfred

[jmsrpp · Accepted solution]

Hi Manfred,

If you're willing to share earlier it would be quite helpful for me. I've been working on the 2nd option I mentioned above, but having some problems with the transitions. It all worked perfectly well with the crosstab but lacked the necessary interactivity.

Seeing the logic for those events would help me quite a bit, since I couldn't think of how to populate the value in that BIAL script with the correct value (i.e. the d.key from the tree).

Thanks,

Jim

james.rapp@sap.com

[jmsrpp · Accepted solution]

Hi Manfred,

Thanks to your excellent example I am now closer than ever!

I managed to incorporate the relevant parts from your project, but I'm failing to send the expandNode/collapseNode commands on click. I have confirmed that:

1. this.selectedNode is populated correctly (I used getSelectedNodeKey() in a text box successfully)

2. The onHCollapse/onHExpand events are firing properly (my exception has to do with the script contribution

When I click in the visualization, I get an exception:

Couldn't resolve reference to Member 'collapseHierarchyNode'.

in script

DENDOGRAM_1.collapseHierarchyNode();

My script contribution looks like this:

/* Collapse the hierarchy */

  @Visibility(private)

  void collapseHierarchyNode() {*

  this.getDataSource().collapseNode(this.getDataSource().getDimensions()[0].name, this.getSelectedNodeKey());

  *}

I did this following the example in the 1.4 SDK guide on P 33-34 where it explains using a default ScriptText in your contribution.xml like this:

<defaultValue property="onHCollapse">this.collapseHierarchyNode();</defaultValue>

This fires the script but obviously there is something wrong with the one I'm trying to use.

I created a new branch (script_contribution) on GitHub: jmsrpp/DesignStudio at script_contribution · GitHub

In the meantime, I'd like to leverage your solution with the explicit ScriptText for onHExpand and onHCollapse. I couldn't figure out the syntax I needed to use in Design Studio. For my test the BIAL would look like:

DS_1.expandNode("SBOP2_D05", "HIERARCHY_NODE/0HIER_NODE/KIDS_CARS");

But obviously I want dimension to be populated with dimHierarchy and I want the node to be populated with selectedNode. What format should I use in the script to take advantage of these variables?

Thanks again!

Jim

[manfred_schwarz · Accepted solution]

Hi Jim,

I had a look at your source code and changed some lines – now you get at least no error
message complaining about collapse or expand while using the ZTL functions.

What you still have to do is to avoid a reload and repaint of the tree after
selecting a component in the this.afterupdate function (or at least avoid a full repaint).

This solution works with Design Studio 1.4 only (because of the use of this.getDataSource() and the not visible ScriptText element!!

I made the following changes in your ...

contribution.ztl file:

/*
Returns the Dimension Key containing the hierarchy of the datasource. */      
String getHierarchyDimensionKey() {*          
     return this.dimHierarchy;  
*}

/* Expand the hierarchy */  
@Visibility(private)    
void expandHierarchyNode() {*
     this.getDataSource().expandNode(this.getHierarchyDimensionKey(),
     this.getSelectedNodeKey());

*}

 
/* Collapse the hierarchy */    
@Visibility(private)   
void collapseHierarchyNode() {*         
     this.getDataSource().collapseNode(this.getHierarchyDimensionKey(),
     this.getSelectedNodeKey());

*}

In the metadata-tree-converter.js

/**

     * Finds the dimension with hierarchy nodes

*/

    MetadataToTreeConverter.prototype.findHierarchicalDimension
= function(metadata) {

        var tempResult = "";

            if (metadata) {

                   for(var i=0;i< metadata.dimensions.length;i++){

                                          var dim =  metadata.dimensions[i];

                          // check if member of dim are hierarchy_dimension
and then return dim name

                          if (!dim.hasOwnProperty("containsMeasures")) {

                                 if (dim.members[0]) {

                                       if (dim.members[0].type == "HIERARCHY_NODE") {             
                                        tempResult = dim.key;                         
}                               
}                 
}

                 
} // all dimensions have been traversed

          
} // check if parse of strMetadata was sucessful
and this._metadata is filled coorectly

            return tempResult;

   }

And in the component.js

this.afterUpdate = function() {

        var root =
metadataToTreeConverter.convert(savedMetadata);

        var dimValue =
metadataToTreeConverter.findHierarchicalDimension(savedMetadata);

        this.dimHierarchy( dimValue );

        this.firePropertiesChanged(["dimHierarchy"]);

    treeRenderer.render(root,this.$(), that);

    };

Best regards

Manfred   

[jmsrpp · Accepted solution]

This was fantastic help Manfred ... thanks ever so much for the helping hand.

Using the contribution.ztl with default functions seems to be the best way to handle expand/collapse from within a component, at least until such time the function can be made more responsive.

I noticed a problem though that affects your extension as well. It seems that firing the event related to hierarchy expand/collapse:

this.getDataSource().expandNode(this.getHierarchyDimensionKey(), this.getSelectedNodeKey());

sometimes causes 2 updates to the component, and other times it doesn't cause any (depending on whether or not the node had been previously expanded). I added some conditional logic to work around this in the afterUpdate() function, especially since my extension requires a clicked node to be recreated after an expand/collapse event.

I updated the GitHub with the final product. Thanks again!