Consuming a Web Service from Batch — Calling Java from COBOL (MVS)

This is a application development pattern sample demonstrating how a batch COBOL program might consume a Web Service by invoking methods on java objects. This procedure might be an alternative to the often used design pattern which is to code the batch COBOL program to call a CICS program using EXCI and the CICS program in turn invoking the web service and passing the response back to the batch program. The tooling provided by CICS makes this pattern fairly easy to develop. Instead, here we leverage the wsimport utility provided by the Java JDK to do all the java coding for us and then invoke the java classes/methods from the generated web service client by a batch COBOL program.

In the demonstration, we are using a web service provided by Aonaware that does a dictionary lookup on dict.org. The web service is described on this web page.

Procedure

Logon to the z/OS Unix Systems Services shell via telnet or TSO OMVS command.
Create a directory for this project.

$ mkdir DictService

Build the java artifacts for the desired webservice from the WSDL using the wsimport utility.

$ cd DictService
$ wsimport -s . "http://services.aonaware.com/DictService/DictService.asmx?WSDL"

This command generates java source code and compiles the source. The current working directory is used as a base for the package subdirectories. The artifacts generated from this WSDL will be in subdirectory com/aonaware/services/webservices:

$ ls com/aonaware/services/webservices/
ArrayOfDefinition.class         DictionaryList.java
ArrayOfDefinition.java          DictionaryListExtended.class
ArrayOfDictionary.class         DictionaryListExtended.java
ArrayOfDictionary.java          DictionaryListExtendedResponse.class
ArrayOfDictionaryWord.class     DictionaryListExtendedResponse.java
ArrayOfDictionaryWord.java      DictionaryListResponse.class
ArrayOfStrategy.class           DictionaryListResponse.java
ArrayOfStrategy.java            DictionaryWord.class
Define.class                    DictionaryWord.java
Define.java                     Match.class
DefineInDict.class              Match.java
DefineInDict.java               MatchInDict.class
DefineInDictResponse.class      MatchInDict.java
DefineInDictResponse.java       MatchInDictResponse.class
DefineResponse.class            MatchInDictResponse.java
DefineResponse.java             MatchResponse.class
Definition.class                MatchResponse.java
Definition.java                 ObjectFactory.class
DictService.class               ObjectFactory.java
DictService.java                ServerInfo.class
DictServiceHttpGet.class        ServerInfo.java
DictServiceHttpGet.java         ServerInfoResponse.class
DictServiceHttpPost.class       ServerInfoResponse.java
DictServiceHttpPost.java        Strategy.class
DictServiceSoap.class           Strategy.java
DictServiceSoap.java            StrategyList.class
Dictionary.class                StrategyList.java
Dictionary.java                 StrategyListResponse.class
DictionaryInfo.class            StrategyListResponse.java
DictionaryInfo.java             WordDefinition.class
DictionaryInfoResponse.class    WordDefinition.java
DictionaryInfoResponse.java     package-info.class
DictionaryList.class            package-info.java

There is more here than we need for the demonstration. The web service implements nine operations, we will only use the Define operation here.

We wrote java code to prove that the service works as expected and that we understood the classes that will be required since these will all be the same for a COBOL program. This is not necessary but is useful to illustrate how the Java and Object Oriented COBOL code correspond. There was quite a bit of trial & error and reviewing the generated java source–this is what we came up with:

import java.util.List;
import com.aonaware.services.webservices.DictService;
import com.aonaware.services.webservices.DictServiceSoap;
import com.aonaware.services.webservices.WordDefinition;
import com.aonaware.services.webservices.ArrayOfDefinition;
import com.aonaware.services.webservices.Definition;
public class DictWS {
  public void callWebService() {
    System.out.println("DictWS Entered");
    DictService aDictService
      = new DictService();
    DictServiceSoap aDictServiceSoapPort
      = aDictService.getDictServiceSoap();
    WordDefinition aWordDefinition
      = aDictServiceSoapPort.define("java");
    System.out.println(aWordDefinition.getWord());
    ArrayOfDefinition aDefinitionArray
      = aWordDefinition.getDefinitions();
    List aDefinitionList
      = aDefinitionArray.getDefinition();
    int DefinitionListSize = aDefinitionList.size();
    System.out.println(DefinitionListSize
      + " definitions returned\n");
    for (int idx = 0; idx < DefinitionListSize; idx++) {
      Definition aDefinition
        = (Definition) aDefinitionList.get(idx);
      System.out.println(aDefinition.getDictionary().getName());
      System.out.println(aDefinition.getWordDefinition());
    };
    System.out.println("Returning from DictWS");
  }
  public static void main(String[] args) {
  new DictWS().callWebService();
  }
}

At this point you might just want to run this code using java on the mainframe. And, you could. Here a some JCL that will run this as a batch job:

//*
//* Run a DictWS java program via JZOS
//* Assumes DictWS is in same directory as DictService/com/...
//* Check APP_HOME for correct path
//*
//* see SYS1.SAMPLIB(JVMJCL60) for instructions
//*
//JAVA EXEC PROC=JVMPRC60,JAVACLS='DictWS',ARGS=''
//SYSPRINT DD SYSOUT=*
//SYSOUT DD SYSOUT=*
//STDOUT DD SYSOUT=*
//STDERR DD SYSOUT=*
//STDENV DD *
# This is a shell script which configures
# any environment variables for the Java JVM.
# Variables must be exported to be seen by the launcher.
. /etc/profile
export JAVA_HOME=/usr/lpp/java/J6.0
export PATH=/bin:"${JAVA_HOME}"/bin
LIBPATH=/lib:/usr/lib:"${JAVA_HOME}"/bin
LIBPATH="$LIBPATH":"${JAVA_HOME}"/lib/s390
LIBPATH="$LIBPATH":"${JAVA_HOME}"/lib/s390/j9vm
LIBPATH="$LIBPATH":"${JAVA_HOME}"/bin/classic
export LIBPATH="$LIBPATH":
# Customize your CLASSPATH here
APP_HOME=<path-to-classes>/DictService                        # << Check This
CLASSPATH=$APP_HOME:"${JAVA_HOME}"/lib:"${JAVA_HOME}"/lib/ext
# Add Application required jars to end of CLASSPATH
for i in "${APP_HOME}"/*.jar; do
  CLASSPATH="$CLASSPATH":"$i"
done
export CLASSPATH="$CLASSPATH":
# Set JZOS specific options
# Use this variable to specify encoding for DD STDOUT and STDERR
#export JZOS_OUTPUT_ENCODING=Cp1047
# Use this variable to prevent JZOS from handling MVS operator commands
#export JZOS_ENABLE_MVS_COMMANDS=false
# Use this variable to supply additional arguments to main
#export JZOS_MAIN_ARGS=""
# Configure JVM options
IJO="-Xms16m -Xmx128m"
# Uncomment the following to aid in debugging "Class Not Found" problems
#IJO="$IJO -verbose:class"
# Uncomment the following if you want to run with Ascii file encoding..
#IJO="$IJO -Dfile.encoding=ISO8859-1"
export IBM_JAVA_OPTIONS="$IJO "

but, we have digressed…

Code the OO COBOL program. 😉

There are certain requirements to implement Object Oriented programming in COBOL and other requirements for invoking java classes. The resulting load module is a DLL which must be stored in a PDSE Library (or HFS) instead of a PDS. It also should be reentrant. The first line of the sample program specifies these overrides to the normal compiler options and the Program-ID stanza specifies that the program may be run recursively.

 cbl dll,thread,dbcs,pgmname(longmixed),lib
 Identification division.
 Program-id. "DICTWS" recursive.

The next difference from traditional batch COBOL pgramming is the Repository where the program identifies and creates aliases for external classes that will be referenced–similar to the import statements in a java program.

 Environment division.
 Configuration section.
 Repository.
 Class jobject is "java.lang.Object"
 Class jstring is "jstring"
 Class JavaException is "java.lang.Exception"
 Class JavaList is "java.util.List"
 Class DictService is "com.aonaware.services.webservices.DictService"
 Class DictServiceSoapPort is "com.aonaware.services.webservices.DictServiceSoap"
 Class WordDefinition is "com.aonaware.services.webservices.WordDefinition"
 Class ArrayListOfDefs is "com.aonaware.services.webservices.ArrayOfDefinition"
 Class Definition is "com.aonaware.services.webservices.Definition"
 Class Dictionary is "com.aonaware.services.webservices.Dictionary"
 .

Then, instead of putting variable declaration in Working-Storage, we put them in Local-storage which helps have the program be reentrant (and permits recursion). We declare references to external objects and classes with the object reference data type.

 Data Division.
 Local-storage section.
 77 anException object reference JavaException.
 77 aWord object reference jstring.
 77 aString object reference jstring.
 77 aDictService object reference DictService.
 77 aDictServiceSoapPort object reference DictServiceSoapPort.
 77 aWordDefinition object reference WordDefinition.
 77 aDefinitionArray object reference ArrayListofDefs.
 77 aDefinitionList object reference JavaList.
 77 aDefinition object reference Definition.
 77 aDefinition-Redef-JObject redefines aDefinition
    object reference jobject.
 77 aDictionary object reference Dictionary.
 77 Word-To-Define PIC X(32).
 77 DefinitionListSize PIC S9(9) COMP-5.
 77 idx PIC S9(9) binary.
 77 String-Length PIC S9(9) binary.
 77 String-Buffer PIC X(8192).

So, for example, aDefinition is an object reference to a Definition which (according to the Repository) is defined in the class com.aonaware.services.webservices.Definition. aDefinition is also redefined as aDefinition-Redef-JObject which is a jobject (java.lang.Object). Redefining object references effectively allows us to do casting and we need this to manipulate Definitions with ArrayOfDefintion methods which is a java.util.List List (and expects java.lang.Object objects).

Some additional setup enables the use of some Java-Native Interface (JNI) routines. This provides some callable services to convert data types from java to COBOL use and vice versa. For example, fetching a Java String and storing it in a PIC X() field:

Linkage section.
copy JNI.
Procedure division.
  Set address of JNIEnv to JNIEnvPtr
  Set address of JNINativeInterface to JNIEnv
  .

The JNI copybook is supplied as an HFS file, /usr/lpp/cobol/include/JNI.cpy, however the batch COBOL compiler cannot access it from there, so we had to copy it to a PDS using the TSO OCOPY command and list that PDS in the SYSLIB for the compiler job step.

You use the INVOKE COBOL verb to invoke a method of a class or object. For Java classes and object, all Invoke verb parameters are BY VALUE.

After each method invocation, we check for a java exception and print the stack trace and exit when there is one. The output would go the the JAVAERR DD statement in the job step JCL.

Following is the rest of the COBOL code with the Java version of the program shown as comments:

* public void callWebService() {

  callWebService.

* System.out.println("DictWS Entered");

  Display "DictWS Entered".

* DictService aDictService
* = new DictService();

  Invoke DictService New
    Returning aDictService
  Perform JavaExceptionCheck

* DictServiceSoap aDictServiceSoapPort
* = aDictService.getDictServiceSoap();

  Invoke aDictService "getDictServiceSoap"
    Returning aDictServiceSoapPort
  Perform JavaExceptionCheck

* Put the word to define into a jstring

  Move z"java" to Word-To-Define

  Call "NewStringPlatform"
    using by value JNIEnvPtr
                   Address of Word-To-Define
                   Address of aWord
                   0

* WordDefinition aWordDefinition
* = aDictServiceSoapPort.define("java");

  Invoke aDictServiceSoapPort "define"
    Using By Value aWord
    Returning aWordDefinition
  Perform JavaExceptionCheck
* System.out.println(aWordDefinition.getWord());

  Invoke aWordDefinition "getWord"
    Returning aString
  Perform JavaExceptionCheck

  Call "GetStringPlatformLength"
    using by value JNIEnvPtr
                   aString
                   Address of String-Length
                   0
  Call "GetStringPlatform"
    using by value JNIEnvPtr
                   aString
                   Address of String-Buffer 
                   Length of String-Buffer
                   0
  Display String-Buffer(1:String-Length)

* ArrayOfDefinition aDefinitionArray
* = aWordDefinition.getDefinitions();

  Invoke aWordDefinition "getDefinitions"
    Returning aDefinitionArray
  Perform JavaExceptionCheck

* List aDefinitionList
* = aDefinitionArray.getDefinition();

  Invoke aDefinitionArray "getDefinition"
    Returning aDefinitionList
  Perform JavaExceptionCheck

* int DefinitionListSize = aDefinitionList.size();

  Invoke aDefinitionList "size"
    Returning DefinitionListSize
  Perform JavaExceptionCheck

* System.out.println(DefinitionListSize
* + " definitions returned");

  Display DefinitionListSize ' definitions returned'
  Display ' '

* for (int idx=0; idx < DefinitionListSize; idx++) {

  Perform Varying idx from 0 by 1
    Until idx = DefinitionListSize

* Definition aDefinition
* = (Definition) aDefinitionList.get(idx);

  Invoke aDefinitionList "get"
    Using by value idx
    Returning aDefinition-Redef-JObject
  Perform JavaExceptionCheck

* System.out.println(aDefinition.getDictionary().getName());

  Invoke aDefinition "getDictionary"
    Returning aDictionary
  Perform JavaExceptionCheck

  Invoke aDictionary "getName"
    Returning aString
  Perform JavaExceptionCheck

  Call "GetStringPlatformLength"
    using by value JNIEnvPtr
                   aString
                   Address of String-Length
                   0
  Call "GetStringPlatform"
    using by value JNIEnvPtr
                   aString
                   Address of String-Buffer
                   Length of String-Buffer
                   0 
  Display String-Buffer(1:String-Length)
* System.out.println(aDefinition.getWordDefinition());

  Invoke aDefinition "getWordDefinition"
    Returning aString
  Perform JavaExceptionCheck

  Call "GetStringPlatformLength"
    using by value JNIEnvPtr
                   aString
                   Address of String-Length
                   0
  Call "GetStringPlatform"
    using by value JNIEnvPtr
                   aString
                   Address of String-Buffer
                   Length of String-Buffer
                   0
  Display String-Buffer(1:String-Length)
* }

  End-Perform

* System.out.println("Returning from DictWS");

  Display "Returning from DictWS"
* }
  Goback.
************************************************************
* Check for thrown Java exceptions
************************************************************
JavaExceptionCheck.
  Call ExceptionOccurred using by value JNIEnvPtr
    Returning anException
  If anException not = null then
    Call ExceptionClear using by value JNIEnvPtr
    Display "Caught an unexpected exception in DICTWS"
    Invoke anException "printStackTrace"
    move 16 to return-code
    Stop run
  End-if
.
End program "DICTWS".

Compile and link edit the COBOL program. Compiling the program is normal. The link edit step, however, needs to bring in the COBOL/Java support and the java JVM DLL side decks. In our case (for JAVA 6), we used:

//SYSLIN DD *
INCLUDE OBJMOD(DICTWS)
INCLUDE '/usr/lpp/java/J6.0/bin/j9vm/libjvm.x'
INCLUDE '/usr/lpp/cobol/lib/igzcjava.x'

Also, the Linkage editor DYNAM(DLL) and CASE(MIXED) parameters on the EXEC statement must be specified and the output library (SYSLMOD) must be a PDSE Library or HFS directory.

Run the batch job.

To execute the program properly, the runtime must be able to locate the java runtime and the classes built by the wsimport step. We did this with a STDENV DD statement in the job and we referred to it with the __CEE_ENVFILE parameter on the PARM ENVAR. Also, XPLINK(ON) and POSIX(ON) are required. Like this:

//GO EXEC PGM=DICTWS,COND=(4,LT,LKED),REGION=128M,
// PARM='/ENVAR("_CEE_ENVFILE=DD:STDENV"),
// XPLINK(ON),ALL31(ON),POSIX(ON)'
//STEPLIB DD DSN=*.LKED.SYSLMOD,DISP=SHR
//SYSOUT DD SYSOUT=*
//SYSPRINT DD SYSOUT=*
//CEEDUMP DD SYSOUT=*
//SYSUDUMP DD DUMMY
//STDENV DD *
PATH=/usr/lpp/java/J6.0/bin
LIBPATH=/usr/lpp/java/J6.0/bin/j9vm:/usr/lpp/java/J6.0/bin
CLASSPATH=DictService:/usr/lpp/java/J6.0/lib
//JAVAIN DD *
//JAVAOUT DD PATH='DictService/javaout',
// PATHOPTS=(OWRONLY,OCREAT,OTRUNC),
// PATHMODE=(SIRUSR,SIWUSR,SIRGRP)
//JAVAERR DD PATH='DictService/javaerr',
// PATHOPTS=(OWRONLY,OCREAT,OTRUNC),
// PATHMODE=(SIRUSR,SIWUSR,SIRGRP)

Performance

Performance of this demonstration is not great. Running this program on a model 2094 with 21621 SU/sec capacity uses about 8 CPU seconds. We suspected that most of this time is in building the JVM, however, this does not compare favorably with java on other platforms. We ran a modified version of the program that made five requests instead of one and displayed time stamps before and after each request and we saw that the first request took about 45 seconds and the four subsequent requests took about 1/2 second each (which is reasonable considering network delay and system load). The CPU time for the five request version of the program was the same as the single request version; i.e. within normal variances.

Update: Retesting the five request verion of the program on an updated CPU, model 2817, 38369 SU/sec, was much better. The job ran 6.5 seconds using 2.5 CPU seconds.

Putting invocations of java methods in a loop probably necessitates adding calls to DeleteLocalRef so that the execution does not run out of storage.

If you have any suggestions to improve the performance, please leave me a comment.

References

Aonaware DictService
JAX-WS wsimport utility.
Developing object-oriented programs in COBOL
Access JNI services from Enterprise COBOL 4.2.
Preparing and running OO applications in JCL or TSO/E

May 2015 Update

I didn’t attend SHARE in Seattle 2015, but there was an interesting session by Tom Ross of IBM (opinion based upon the powerpoint slides) called Practical Experiences about COBOL Programming. Make SOA Possible in z/OS batch COBOL  Almost makes me think the presenter reviewed this post at some point. 😃 Wish I’d been there.