What can go wrong?

To see what can go wrong with VO services, first we take a look at how we use VO services. We need to know where to find these services. Then we need to know how to query these services and how to interpret the results. Finally, we need to know how to validate the data.

Problems with VO Registry

• In theory, all VO registries should contain the same information. But in practice, they differ. For example, a search for "Chandra Einstein" at http://nvo.stsci.edu/VORegistry/QueryRegistry.aspx finds 19 entries, while the same search at http://voservices.net/registry/QueryRegistry.aspx finds only 12 entries.
• Another problem is that the information in the entries is not always verified. For example, a SSAP service could be listed as SIAP service.
• The format of the service URL field for is not standardized. Some URLs end with the '?' character, while others have additional parameters, for example:

  http://casjobs.sdss.org/vo/DR3SIAP/SIAP.asmx/getSiapInfo?&FORMAT=image/fits&BANDPASS=i&

  Note that ending '&' character, which is usually harmless. However, this URL shows that parameters are already present and that other additional parameters cannot be appended with a leading '?'. To add another format to the query, for example, image/jpeg, the client software should change the existing format parameter to

  FORMAT=image/fits,image/jpeg

  instead of simply appending an additional format parameter at the end of the URL.

Problems with services

Cone search and SIAP services use HTTP-GET as transport mechanism. Besides issues with underlying networking layers, web services based on HTTP-GET have their own set of problems. HTTP-GET does not provide any specialized support for handling web service exceptions.

• Bad URL
  If the service URL does not exist, then the web server will emit a HTTP error code 404 (File not found) message, which also can be in different formats depending on the configuration of the web server. Clients should not only expect responses in VOTable (XML) format, but also should be prepared to handle error information in HTTP or HTML.
• No response
  In the simple case when the server terminates abnormally, no response will be sent. In that case, the client will receive a empty response, when the network connection closes.
• Bad response
  When the service provider cannot process a request or an error occurs during processing, an error message will be sent to the client.
  Because HTTP-GET services are handled by the web server first, and then delegated to the cone search or SIAP module, the web server can catch an error and issue an error message before the actual web service engine has a chance to process the request. This type of error messages is usually encoded in HTTP or HTML.
  Depending on the implementation of the web service, the response can be encoded in:
  • plain text as a simple error message,
  • well-built VOTable containing the error message in a INFO tag, or
  • general XML format that is not a VOTable or is only a minimalistic VOTable containing just one INFO tag.
• non-conformant services
  Because many VO services were developed and implemented at the same time VO specifications were maturing, some services are not fully complaint with the latest standards. When a query is successul, the majority of VO services return a valid VOTable. Most of the non-conformant behaviors occur in special cases, such as, setting search radius/size to 0 deg or format to 'METADATA'. Some services perform minimal option checking and do not honor special requests, for example, always return fits files disrepect of the format option.

SOAP problems

Interoperability amongst different SOAP implementations is one of the most common problems concerning web services. The two main types of SOAP encoding, rpc/encoded and document/literal, are both widely used, with newer implementations leaning towards document/literal. Both the VO Registry as well as the SkyNode SOAP implementations follow the document/literal encoding style. Axis default encoding style is rpc/encoded and extra care must be taken to communicate with document/literal implementation.

There are two ways to use WSDL to specify the communication interface between server and client. First, a tool can generate code in a programming language, for example Java, that describe needed data types, classes and methods. Programmers can then use the generated code to build applications. Alternatively, WSDLs can be interpreted in runtime. In this case, all methods and parameters binding occur also in runtime.

In the first case, on the client side, the WSDL is not longer needed once the code is generated. However, any changes in the WSDL require new cycles of code generation and compilation. When server and client are out of sync, unpredicted errors can occur that are very difficult to detect. On the other hand, when WSDL is processed in runtime, any incompatiblity in the interface can be detected and handled.

In both case, some knowledge of the WSDL is necessary to make the communication work. However, when WSDL is processed in runtime, mapping of data types can be automatically handled. This can be done with dynamically typed languages, such as PHP or Python, but also with statically typed languages such as Java because the serialization of SOAP messages is inherent dynamic.

SOAP exception handling

Another common problem with SOAP web services is the use of SOAP exceptions to describe errors situations. SOAP exceptions must be thrown by the server and caught by the client application. The server application must catch other exceptions before throwing SOAP exceptions. Otherwise, a plain stack trace could be returned to the client.

Other inconsistencies

• Names and Ids
  Some services return VOTables with fields that have 'name' attribute, others have 'id' attribute. Some have both.
• Use of UCD
  Many services provide the minimal required usage of UCDs, mainly just for RA and DEC. These services simply leave the UCD attributes in the fields blank. At least in one case, POS_EQ_RA_MAIN and POS_EQ_DEC_MAIN are assigned to two different fields. In another case, PHOT_MAG appears multiple times.

Shui Hung Kwok
NVO Summer School 2005