WebService .net Engine XML structure returned as string or any?

[Former Member]

Hi,

PB 12.5.2 Buil 5602, win 7 64 bit , .Net Engine, SDK 7.1, framework 4/4.5

we have a webservice returns a XML structure, like organisation has departements has persons etc.. simple XML Strcture.

First try to get the result was to put in a string, we got error "there is an error in xml document (1 265)".

Second try in webservice around the XML structure we put CDATA tag an thats works to get in a string.

Third try the return is any, tried to put in a string also in a any everytime is null.

Some ideas?? or how must be the webservice declaration to get as return an XML Structure in a string in Powerbuilder and maybe after doing some process with the data in datawindow etc...???

[Former Member · Accepted solution]

Fiddler get the XML Structure from WebService.  I tried also base64binary, when i deploy the proxy in PB the return value type is any, it's normal? Then i try to put in byte[] and get again the error "there is an error in xml document (1 306)"

[ricardojasso · Accepted solution]

Marko,

I thought that maybe the XML structure was being sent as a binary type but apparently this is not the case. This is the case for a web service I consumed in one of my projects where an XML document was being sent as a binary type. And PB transformed the base64binary type to a byte[] in the proxy and it worked. I wonder why PB returned "any" in your proxy. But this is meaningless if the web services indeed sends the structure as a string.

But if we step a little bit back, I'm confused as to why you are modifying the WSDL file to find a match to the datatype of the response message when in fact one of the WSDL main purpose is to specify the data types of the parameters being sent back and forth. This allows the programming language to generate proxys with matching datatypes that can send and receive such messages. If the original WSDL file says the XML Structure should be received as a string then there should be no need to change the datatype to other than a string.

I think the problem is how PB is trying to store the XML structure and apparently it cannot handle it properly. A nested XML document is something that is not easily handled in web services. From the book "Web Services" by Alonso, Casati, Kuno, and Machiraju: "...XML does not always gracefully support the necessary data types, such as binary data (e.g., an image) or nested XML documents".

Is there a way the web service you are trying to access could send the XML structure as a binary?  I would suggest you talk to the provider and see if they can change the XML structure datatype to something that PB can handle. Or wait until Appeon resolves this as it seems to be a PB bug. By the way, I use PB 12.5 Build 2511.

Regards,

Ricardo

[ricardojasso · Accepted solution]

...maybe the XML structure was being sent as a binary type...

...an XML document was being sent as a binary type...

...Is there a way the web service you are trying to access could send the XML structure as a binary? ...

Sorry, I meant base 64 binary in this previous comment.

[Former Member]

We've been using SOAPUI for testing different Web Services. It's easy to configure and simple to use.

It will download and parse the WSDL file from the WS, or you can supply your own. It remembers multiple queries per function, and gives you the full XML result so you can see exactly what came back.

You may also need to get out of PB completely for getting data from the WS. One of our clients needed data that has complex arguments and result types, and none of the PB SOAP interfaces could handle them. We created a DLL to do the work for us. wsdl2h and soapcpp2 utilities create C/C++ files that can connect and read data from the WS. We wrote new dll functions to use the generated code and structures to work with the WS and do the grunt work. PB calls our dll functions and gets real results back (either as string, structure, or whatever). It's not the easiest way to do it, but necessary, and extremely flexible.

[Former Member]

Of course when the provider of a webservice do the effort to do return as base64binary will help, but in the begin the return was string and PB with .net Engine made an error on XML Document, so i think a string is a string. If webservice return string then PB have to relay it and return also string and not an error whatever complexity of the XML structure. After you can parse etc... the XML structure. If this error is a bug then it have to be fixed i think.

[ricardojasso]

I agree with you. The string should be handled correctly no matter what data it may contain. It seems that the PB web service XML parser is not able to handle it correctly. This should be fixed.

Another issue I found with PB web service proxies is that if an attribute is defined as optional in the wsdl file (or related xsd file) PB still generates the request with the attribute included even if you set it to null in PowerScript. (For example: <xs:attribute name="hobby" use="optional">)

This, along with your issue, shows lack of completeness in the PB implementation of web services proxies when consuming web services.

[ricardojasso]

Marko,

In fairness to PowerBuilder I would like to correct myself. After having a talk with one of our web developers who has experience creating web services he is telling me that having a string that contains XML markup inside an XML document will most certainly disrupt the parser. So, PowerBuilder is correct in showing the error message.

One solution is to use <![CDATA[information]]> which tells the parser not to interpret the characters as markup. From your original post it seems that CDATA did work correctly or didn’t it? The only consideration to be taken when using CDATA is that the content characters cannot contain the sequence ]]> which will be interpreted as the end of the information.

Another solution is the one I mentioned which is to encode the whole content using base 64 binary. I believe this is the less error prone method although if involves more work from the client side.

There could be more solutions other than this two but either of them should work for your purposes.

Regards,

Ricardo

[Former Member]

Ricardo,

yes CDATA solution worked fine but how you mentioned with sequence ]] is not appropriated. Also 64binary worked. I think if the provider of webservice will not do effort to make a correct webservice that Powerbuilder can do structures etc.. then 64binary is the best solution. After that the client must do the parsing process.

So well done and thanks for the effort and explinations for this complexe request.

Regards,

Marko

[ricardojasso]

Marko,

You cannot include the sequence ]]> because it will terminate the CDATA section prematurely.

This is an example of an invalid xml document:

<?xml version="1.0" encoding="utf-8"?>

<content>

  <![CDATA[This string of characters can contain XML tags like

  <person>

    <firstname>John</firstname>

    <lastname>Doe</lastname>

  </persons>

  even if they are not well formed but if you insert the CDATA end character sequence ]]> the string will be truncated and maybe the application will show an error message]]>

</content>

Of course, it's going to be rare for you to use the CDATA ending character sequence... unless you are writing an XML tutorial and you store your content in XML files.

Regarding your concern that PB doesn’t generate objects or structures to handle the XML structure I think we’re mixing concepts here. As far as my knowledge goes PowerBuilder will generate user objects and structures based on the WSDL file (in itself an XML file) which describes how the XML communication request documents must be constructed by the web service client and what data types should be expected from the response documents.

This WSDL file along with other possible associated XSD files (schema files) tell the programming language what datatypes are going to be used in the communication process. So PB generates objects and structures to handle the elements and attributes of the request and response XML documents. And if the XML structure is sent as a string by the web service then PB will generate a string variable to store it, but it won’t parse it because there is no reason to do it. So you must parse it yourself for your application purposes using DOM or some other technique.

But the XML datatype specification does have an XML Type that could be used for what you want to do. Personally, I’ve never used those datatypes so I cannot help you anymore with this. And it needs to be seen if PB can handle such datatypes.

Our experience in web services is limited to one project in which we had to call a web service which in turn returned an XML document (electronic invoice) as one of the attributes using base64binary. We didn’t have to do any parsing with the XML document, just store it as a file. But the WSDL file had many complex types defined so we learned a lot about PB web service proxies.

Maybe someone else can shed some light on this.

Best,

Ricardo

[Former Member]

Yes - that's exactly correct. 

Since Halloween is approaching, I'll use that analogy. 

Suppose you walk up to 2 houses in your Halloween costume and ring the bell at each one.

House 1 has the treats set out on a table by type - one dish contains chocolate only, one dish contains treats with nuts, one has fruit, one has hard candy, one has bubble gum, and so on.  You'd be able to introspect the table, and see which treats you'd like to take with you, and which you knew to avoid (nut allergies, not partial to chocolate, whatever).

But house 2 just hands you a black plastic bag with the treats sealed inside.  You have to take whatever they give you, without any idea of what's inside. Only later can you open the bag and see what you've been given - and discard those treats you find objectionable.

Table 2 is the situation Marko has been given.  PB can only see the "black bag", and can't determine the individual fields inside it. It's up to Marko to write a function that can take that content (in whatever form it's sent), convert it to a string, and parse it into its constituent bits. 

Had the service been defined with strongly typed fields, PB would have been able to see the individual "bowls of treats", and know that field 1 contains chocolates, and field 2 contains bubble gum, and field 3 is a popcorn ball.  PB's web service project would have generated a structure class that exactly mirrored that structure.  Instead, all it can see is "Black Bag" (or "string", in Marko's case).

Hope this helps,

-Paul-

[Former Member]

Yes Ricardo,

when it's encoded as Base 64 it worked, Here we have luck because we can change the webservice as needed. But what to do when the provider don't offer base 64, so you can't use it.

Thanks for the help.

[ricardojasso]

Marko,

I think I got mixed up along this thread but now it's clear to me you are the creator of the web service and that the web service is hosted with a third party. It's good to know that changing the response type to base64binary worked.

Being the provider of the web service allows you to change the datatypes of its elements so that it can be consumed by clients without much problems. This will also be the case of some other provider that wishes to publish his own web service. He will build it in a way that people can consume it. If he wants to send a binary file or image he will most probably use base64binary as the datatype. Does this answer your question?

Regards,

Ricardo

[Former Member]

Ricardo,

as i mentioned before the first try was return type string and then i got the error that in Xml document is an error. By the way, just a think, the return is UTF8, it's may be that the problem why i got the error?

The try to return array of byte worked. The webservice return base64 encoded then i put it an array of byte then blob() and then string(,EncodingUTF8!). So i got XML structure into string.

Why when string it doesn't work? Encoding???

[ricardojasso]

Marko,

Web services communicate through XML documents. A request XML document is sent by the web service client to the web service and a response XML document is sent by the web service to the client to service its petition.

The error you are receiving in PB refers to the response XML document, not the XML structure. But most probably it is being caused by the XML structure itself which might contain special characters that interfere with the XML document.

So, a solution is to encode the XML structure using something like Base 64 encoding in which everything is converted to a small set of common characters (64 characters to be exact) that won't interfere with the XML document. Base 64 is commonly used to encode binary content like images or files to send them through HTTP but it may also work for your XML structure.

UTF-8 is used to encode Unicode characters to 8-bit characters that can be transmitted in the web, similar to Base 64 encoding. It is the most common encoding method for web pages. Why string(x, EncodingUTF8!)? Because the characters in the XML structure that where Base 64 encoded and decoded might be Unicode characters, so this syntax converts the byte array to Unicode characters properly. But this depends on the original format of the characters in the XML structure so a different UTF encoding might be needed.

From your reply I understand the web service indeed returns the XML structure Base 64 encoded and using the byte array worked. Is this correct?

Regards,

Ricardo

[Former Member]

Here is the WDSL File:

<!-- 


  This WSDL file is meant for Mirth Webservice Listeners that are based on the
  custom ws.AcceptXMLMessage class!

  AcceptXMLMessage Webservice listeners do not automatically generate a WSDL file, so you 
  will have to adapt this file to match your webservice, and make it available to your
  webservice clients.

  The only modification that has to be performed is the location URL at the end of the file
  (soap:address element)
  It must match your Webservice's host, port and service name.

  -->
<definitions
  xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd"
  xmlns:wsp="http://www.w3.org/ns/ws-policy" xmlns:wsp1_2="http://schemas.xmlsoap.org/ws/2004/09/policy"
  xmlns:wsam="http://www.w3.org/2007/05/addressing/metadata" xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"
  xmlns:tns="http://ws.connectors.connect.mirth.com/" xmlns:xs="http://www.w3.org/2001/XMLSchema"
  xmlns="http://schemas.xmlsoap.org/wsdl/" targetNamespace="http://ws.connectors.connect.mirth.com/"
  name="DefaultAcceptMessageService">
  <types>
  <xs:schema xmlns:tns="http://ws.connectors.connect.mirth.com/"
  xmlns:xs="http://www.w3.org/2001/XMLSchema" version="1.0"
  targetNamespace="http://ws.connectors.connect.mirth.com/">
  <xs:element name="acceptMessage" type="tns:acceptMessage" />
  <xs:element name="acceptMessageResponse" type="tns:acceptMessageResponse" />
  <xs:complexType name="acceptMessage">
  <xs:sequence>
  <xs:element name="arg0" type="xs:string" minOccurs="0" />
  <!--xs:element name="arg0" type="xs:anyType" minOccurs="0"/-->
  </xs:sequence>
  </xs:complexType>
  <xs:complexType name="acceptMessageResponse">
  <xs:sequence>
  <xs:element name="return" type="xs:string" minOccurs="0" />
  <!--xs:element name="return" type="xs:anyType" minOccurs="0" /-->
  </xs:sequence>
  </xs:complexType>
  </xs:schema>
  </types>
  <message name="acceptMessage">
  <part name="parameters" element="tns:acceptMessage" />
  </message>
  <message name="acceptMessageResponse">
  <part name="parameters" element="tns:acceptMessageResponse" />
  </message>
  <portType name="DefaultAcceptMessage">
  <operation name="acceptMessage">
  <input
  wsam:Action="http://ws.connectors.connect.mirth.com/DefaultAcceptMessage/acceptMessageRequest"
  message="tns:acceptMessage" />
  <output
  wsam:Action="http://ws.connectors.connect.mirth.com/DefaultAcceptMessage/acceptMessageResponse"
  message="tns:acceptMessageResponse" />
  </operation>
  </portType>
  <binding name="DefaultAcceptMessagePortBinding" type="tns:DefaultAcceptMessage">
  <soap:binding transport="http://schemas.xmlsoap.org/soap/http"
  style="document" />
  <operation name="acceptMessage">
  <soap:operation soapAction="" />
  <input>
  <soap:body use="literal" />
  </input>
  <output>
  <soap:body use="literal" />
  </output>
  </operation>
  </binding>
  <service name="DefaultAcceptMessageService">
  <port name="DefaultAcceptMessagePort" binding="tns:DefaultAcceptMessagePortBinding">
  <soap:address
  location="http://pc1:8100/services/Mirth" />
  </port>
  </service>
</definitions>

The Webserviceproxy in PB is well done. The function named acceptmessage returns string, first try. Try to call the function got error message. Second try if returns any then any is NULL.

[ricardojasso]

I'd suggest you try to catch the response message before it is deserialized by PowerBuilder and check if indeed the web service is sending the XML structure in the return argument. You can use Fiddler to acomplish this. http://www.telerik.com/fiddler

If the web service is sending the XML structure then it is a problem on the PB side.

[ricardojasso]

Have you tried changing the type to base64binary?

That is:

<xs:element name="return" type="base64binary" minOccurs="0" />

This will be translated to an array of bytes in the PB proxy:

byte return[]

Then just follow the steps I mentioned before.

[Former Member]

I would consider this a poorly designed web service...  There are basically two choices for service designers.

1) They create the service to return a strongly-typed result set, where every individual field is named and typed in the response.  The WSDL for this will show every field and its datatype, and PB can parse this into a well-defined structure object using the Web Service proxy project.

2) They create the service to return a single XML or JSON string, and that string can contain anything they want. This says "we're too lazy to tell you what's in this 'bag of tags', so figure it out yourself."

You've clearly been provided with option 2.

[ricardojasso]

1) They create the service to return a strongly-typed result set, where every individual field is named and typed in the response.  The WSDL for this will show every field and its datatype, and PB can parse this into a well-defined structure object using the Web Service proxy project.

Of course, the design of the web service can be improved but for this case the response is correctly named and typed so there should be no problem handling it.

      <xs:complexType name="acceptMessageResponse">

        <xs:sequence>

          <xs:element name="return" type="xs:string" minOccurs="0" />

        </xs:sequence>

      </xs:complexType>

[Former Member]

All of this advice to "convert it to a binary array" or "use a blob" - all of that advice is to help you create a string from something that is NOT a string.

But you DO have a valid string!  Your first error message was "there is an error in xml document...".

If you didn't have valid string data, you wouldn't have even gotten that far!

The question you should be asking is this:  "Does the contents of the string I've received represent a valid XML document?" The only way I know to answer that is to call the webservice from outside PB, using Fiddler or some other SOAP client, and paste the result into an XML validator website, like XML Validation: XML Validation.

-Paul-

[Former Member]

Ok, thanks and I understand the way to process the data after i got the XML Structure into a string.

But thats is the problem first. I have to get it first into a string.

When the webservice returned the XML Structure as a string i get this error "there is an error in xml document (1 265)" when i try to put it into a string in PB. Therefore we tried that the webservice returned type "any" and tried to put it into a string or an any variable then, this try, the variables are NULL.

The XML Structure is sometimes 4 Level sometimes 5 level per tags.

Some ideas?

[Former Member]

=>  i get this error "there is an error in xml document (1 265)" when i try to put it into a string in PB

1) Is the WS just returning a "String" data type populated with XML?

2) How are you trying to load the PB String

FWIW: XML can contain special characters (even binary data) - so I usually place the XML data stream into a BLOB.  

[ricardojasso]

Try putting the XML structure into a byte array, then convert the byte array to a blob, and then convert the blob to a string using the proper encoding.

For example,

Byte[] WSResponse.xml_structure (defined in the schema)

Blob lblb_xml_structure

lblb_xml_structure= Blob(l_WSResponse.xml_structure)

string ls_xml_structure

ls_xml_structure= String(lblb_xml_structure, EncodingUTF8!)

[Former Member]

Hi Ricardo;

FYI: The BLOB ( ) method defaults to "EncodingUTF16LE!". If the XML dta stream is indeed UTF8, then you might want to try ...

lblb_xml_structure= Blob(l_WSResponse.xml_structure, EncodingUTF8!)

  I'm not sure if that is the issue .. but, I just thought about this aspect as well.

Regards ... Chris

[ricardojasso]

Chris,

Syntax 2 of the Blob function has just one argument: the string or byte array.

Syntax 2 Convert a string or a byte array to a blob

Description

Converts a string or an array of bytes to a blob datatype.

Syntax

Blob ( array[ ] )

Argument	Description
stringorbytearray	An Any variable that holds a string or an array of bytes you want to convert to a blob datatype

Return Values

Blob. Returns the converted string or byte array in a blob.

Regards,

Ricardo

[Former Member]

<my bad> ... you are correct.

I was thinking about a String to Blob - not an array to blob.  

[ricardojasso]

Marko,

How is the XML structure property defined in the WSDL of the web service? What is its type? Are you creating a Web Service Proxy in PowerBuilder? What is its type once the proxy is generated?

Regards,

Ricardo

[Former Member]

Just another random thought ...

Can you try:

Blob        lb_data

Any          la_data

Byte         lb_array [ ]

lb_array    =  l_WSResponse.xml_structure

la_data     =  lb_array

lb_data     =  Blob (la_data)

[Former Member]

Sorry for the confusion. No the webservice is not created in PB and not by us.

The returned value of the webserivce is an XML structure like described upper and we tried to get it in an string, therefore I enumerated the tries to get this XML structure.

[Former Member]

Hi Marko;

  I that case you have two options:

1) Use a Web Service DataWindow to parse the returned XML

   - This only works if the XML represents a flat 2D result set like an SQL query would return.

2) Use PBDOM and parse the XML stream yourself via PowerScript.

  - We use this technique when parsing Correctional Services XML based data being sent over to Passport Canada.

HTH

Regards ... Chris

[Former Member]

Or - if this is a SOAP service and the "shape" of the response is flat (no nested hierarchies), you can just paint a datawindow and use the Web Service datasource.

-Paul-

[Former Member]

Hi Marko;

1) You have to create a 1st Structure for the returned XML data using appropriate "simple" data types.

2) You now need to create a 2nd Structure to house the 1st structure.

3) Then place the 1st structure into the 2nd structure as an unbounded array.

4) You would now return the 2nd Structure from your PB Web Service once populated with data.

Note: You can load your 1st structure directly from a DataWindow buffer if the DW's primary buffer matches the Structure using the DW's DOT notation.

For example:

  Structure  str_A

    Int         ID

    String  Name

  Structure  str_B

    str_A     lo_str [ ]

  // Load  lo_str [ ] in a loop or use DW syntax!

  str_B.lo_str  =  DS.Object.Data

  Return  str_B

HTH

Regards ... Chris

[former_member190719]

A little confused.  You're creating the service?  If so, return a structure or an NVO. The web service framework is responsible to converting that to XML for you, and the web service client framework is responsible for serializing that back into a structure or NVO.