XML to JSON on Cloud Connector

[irijsdijk]

I have the following scenario in SCPI: I receive an XML request which I need to convert to a JSON format and then send an HTTP request to an external service. The response message should be returned to the sender.

My scenario looks like this:

My JSON message looks good in the logging step "store JSON request". When I'm using Postman to test this message, I get a good result. However, when I'm using SCPI to send the JSON message, I get "No JSON found in POST data" as response. In Postman I get this response when I'm sending an empty payload.

When I'm using a content modifier to directly set the message body I get a good response. It looks like the XML to JSON converter somehow messes up the payload:

Any ideas on how to get this scenario working?

[MortenWittrock]

Hi Iddo

Just making sure I understood the problem correctly: You log the output of the XML to JSON converter, and when you POST that JSON from Postman, the call succeeds? But if that same JSON gets POSTed by the HTTP receiver channel, it fails?

Regards,

Morten

[irijsdijk]

Exactly. And when I put the payload direcly in the message body using a content modifier, the calls succeeds as well.

[irijsdijk · Accepted solution]

You should not use the standard XML to JSON option in SCI. It contains several severe errors. Instead, use this XSLT transformation to do the conversion:

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:n0="urn:louwman:nl:WMS:DeliveryMatch:InsertShipment">


  <xsl:output indent="no" omit-xml-declaration="yes" method="text" encoding="UTF-8" media-type="text/x-json"/>
	<xsl:strip-space elements="*"/>
  <!--contant-->
  <xsl:variable name="d">0123456789</xsl:variable>


  <!-- ignore document text -->
  <xsl:template match="text()[preceding-sibling::node() or following-sibling::node()]"/>


  <!-- string -->
  <xsl:template match="text()">
    <xsl:call-template name="escape-string">
      <xsl:with-param name="s" select="."/>
    </xsl:call-template>
  </xsl:template>
  
  <!-- Main template for escaping strings; used by above template and for object-properties 
       Responsibilities: placed quotes around string, and chain up to next filter, escape-bs-string -->
  <xsl:template name="escape-string">
    <xsl:param name="s"/>
    <xsl:text>"</xsl:text>
    <xsl:call-template name="escape-bs-string">
      <xsl:with-param name="s" select="$s"/>
    </xsl:call-template>
    <xsl:text>"</xsl:text>
  </xsl:template>
  
  <!-- Escape the backslash (\) before everything else. -->
  <xsl:template name="escape-bs-string">
    <xsl:param name="s"/>
    <xsl:choose>
      <xsl:when test="contains($s,'\')">
        <xsl:call-template name="escape-quot-string">
          <xsl:with-param name="s" select="concat(substring-before($s,'\'),'\\')"/>
        </xsl:call-template>
        <xsl:call-template name="escape-bs-string">
          <xsl:with-param name="s" select="substring-after($s,'\')"/>
        </xsl:call-template>
      </xsl:when>
      <xsl:otherwise>
        <xsl:call-template name="escape-quot-string">
          <xsl:with-param name="s" select="$s"/>
        </xsl:call-template>
      </xsl:otherwise>
    </xsl:choose>
  </xsl:template>
  
  <!-- Escape the double quote ("). -->
  <xsl:template name="escape-quot-string">
    <xsl:param name="s"/>
    <xsl:choose>
      <xsl:when test="contains($s,'"')">
        <xsl:call-template name="encode-string">
          <xsl:with-param name="s" select="concat(substring-before($s,'"'),'\"')"/>
        </xsl:call-template>
        <xsl:call-template name="escape-quot-string">
          <xsl:with-param name="s" select="substring-after($s,'"')"/>
        </xsl:call-template>
      </xsl:when>
      <xsl:otherwise>
        <xsl:call-template name="encode-string">
          <xsl:with-param name="s" select="$s"/>
        </xsl:call-template>
      </xsl:otherwise>
    </xsl:choose>
  </xsl:template>
  
  <xsl:template name="encode-string">
    <xsl:param name="s"/>
    <xsl:choose>
      <!-- tab -->
      <xsl:when test="contains($s,'	')">
        <xsl:call-template name="encode-string">
          <xsl:with-param name="s" select="concat(substring-before($s,'	'),'\t',substring-after($s,'	'))"/>
        </xsl:call-template>
      </xsl:when>
      <!-- line feed -->
      <xsl:when test="contains($s,'
')">
        <xsl:call-template name="encode-string">
          <xsl:with-param name="s" select="concat(substring-before($s,'
'),'\n',substring-after($s,'
'))"/>
        </xsl:call-template>
      </xsl:when>
      <!-- carriage return -->
      <xsl:when test="contains($s,'
')">
        <xsl:call-template name="encode-string">
          <xsl:with-param name="s" select="concat(substring-before($s,'
'),'\r',substring-after($s,'
'))"/>
        </xsl:call-template>
      </xsl:when>
      <xsl:otherwise><xsl:value-of select="$s"/></xsl:otherwise>
    </xsl:choose>
  </xsl:template>


  <!-- number (no support for javascript mantissa) -->
  <xsl:template match="text()[not(string(number())='NaN' or
                       (starts-with(.,'0' ) and . != '0'))]">
    <xsl:value-of select="."/>
  </xsl:template>


  <!-- boolean, case-insensitive -->
  <xsl:template match="text()[translate(.,'TRUE','true')='true']">true</xsl:template>
  <xsl:template match="text()[translate(.,'FALSE','false')='false']">false</xsl:template>


  <!-- object -->
  <xsl:template match="*" name="base">
    <xsl:if test="not(preceding-sibling::*)">{</xsl:if>
    <xsl:call-template name="escape-string">
      <xsl:with-param name="s" select="name()"/>
    </xsl:call-template>
    <xsl:text>:</xsl:text>
    <!-- check type of node -->
    <xsl:choose>
      <!-- null nodes -->
      <xsl:when test="count(child::node())=0">null</xsl:when>
      <!-- other nodes -->
      <xsl:otherwise>
      	<xsl:apply-templates select="child::node()"/>
      </xsl:otherwise>
    </xsl:choose>
    <!-- end of type check -->
    <xsl:if test="following-sibling::*">,</xsl:if>
    <xsl:if test="not(following-sibling::*)">}</xsl:if>
  </xsl:template>


  <!-- array -->
  <xsl:template match="*[count(../*[name(../*)=name(.)])=count(../*) and count(../*)>1]">
    <xsl:if test="not(preceding-sibling::*)">[</xsl:if>
    <xsl:choose>
      <xsl:when test="not(child::node())">
        <xsl:text>null</xsl:text>
      </xsl:when>
      <xsl:otherwise>
        <xsl:apply-templates select="child::node()"/>
      </xsl:otherwise>
    </xsl:choose>
    <xsl:if test="following-sibling::*">,</xsl:if>
    <xsl:if test="not(following-sibling::*)">]</xsl:if>
  </xsl:template>
  
  <!-- convert root element to an anonymous container -->
  <xsl:template match="/">
    <xsl:apply-templates select="node()"/>
  </xsl:template>
    
</xsl:stylesheet>

[irijsdijk]

Update: this actually seems to be an issue in SCPI. We created a ticket @SAP.

[irijsdijk]

I set up a connection to a mockservice running on my own laptop. To my surprise both the good-request.txt and the bad-request.txt appear to contain the right payload. So the problem most likely is not in the SCI flow, but somehow in the receiving service.

I will pursue this option now.

[Sriprasadsbhat]

Hello Iddo,

Could you please share the input XML ,working JSON message ( which you are putting in content modifier ) and output of JSON converter.

Regards,

Sriprasad Shivaram Bhat

[irijsdijk]

Sure, here is the JSON:

{"client":{"id":"272","channel":"?","callback":"?","action":"?","method":"?","filter":"false"},"shipment":{"orderNumber":"?","reference":"?","language":"?","currency":"?","firstPickupDate":"?","carrier":"?","service":"?","inbound":"false","numPallets":"?","cod":"false","signature":"false","noNeighbor":"false","insured":"false","incoterm":"?"},"sender":{"address":{"name":"?","companyName":"?","address1":"?","address2":"?","street":"?","houseNr":"?","houseNrExt":"?","postcode":"?","city":"?","country":"?","zone":"?"},"contact":{"phoneNumber":"?","Email":"?"}},"customer":{"address":{"name":"?","companyName":"?","address1":"?","address2":"?","street":"?","houseNr":"?","houseNrExt":"?","postcode":"?","city":"?","country":"?","zone":"?"},"billing":{"name":"?","companyName":"?","address1":"?","address2":"?","street":"?","houseNr":"?","houseNrExt":"?","postcode":"?","city":"?","country":"?","zone":"?"},"contact":{"phoneNumber":"?","Email":"?"}},"packages":{"package":{"warehouse":"?","description":"?","weight":"?","length":"?","width":"?","height":"?"}},"quote":{"product":{"id":"?","packageNum":"?","warehouse":"?","description":"?","content":"?","hsCode":"?","dangerousGoods":{"technicalName":"?","mainDanger":"?","class":"?","packingGroup":"?","UN":"?","grossMass":"?","netMass":"?","massUnit":"?","LQ":"false","NOS":"false","environmentHazard":"false"},"value":"?","weight":"?","length":"?","width":"?","height":"?"}},"fragileGoods":"false","dangerousGoods":"false","priceIncl":"?","priceExcl":"?","weight":"?"}

[MortenWittrock]

Hi Iddo

Given that the converted JSON structure is correct, I'm thinking that this could be an encoding issue. Are you sending UTF-8 encoded data or something else?

Edit: You can explicitly set the JSON encoding to UTF-8 in the Converter step, by the way. Your current configuration takes the encoding from the SOAP request.

Regards,

Morten

[irijsdijk]

Set the encoding to UTF-8:

The result is still the same:

{"status":"failure","code":3,"message":"No JSON found in POST data"}