QE Café

Byteman plays with Drools

2012-01-07T18:39:00.004+01:00

Due to my recent suspicion that there is a bug in Drools I needed to monitor its guts in detail. To be more specific, I needed to see what is in the working memory after execution of all rules.

Debbuger was not a good option since there were hundruds of invocations of the problematic part and I needed to get overview of quite a large amount of data. If I inly could store all the data for each invocation in a file and then grep through it.

But wait! Of course I can. Byteman will help me! The identified point in source code was execution of fireAllRules() in org.drools.common.AbstractWorkingMemory. Ideally, just before return from the method. So here is the corresponding rule:

RULE Debug fireAllRules
HELPER org.drools.planner.examples.tournaments.helper.BytemanHelper
CLASS ^org.drools.common.AbstractWorkingMemory
METHOD fireAllRules
AT EXIT
IF TRUE
DO
  printWorkingMemory($0.getKnowledgeRuntime());
ENDRULE

Just a few notes to it:

^ means to apply to child classes as well
$0 refers to this

As you can see there is a custom helper class to print the working memory content since Byteman rules are little bit limited in the supported Java syntax.

import org.drools.impl.StatefulKnowledgeSessionImpl;
import org.jboss.byteman.rule.Rule;
import org.jboss.byteman.rule.helper.Helper;

public class BytemanHelper extends Helper {
   protected BytemanHelper(Rule rule) {
      super(rule);
   }
   
   public void printWorkingMemory(Object obj) {
      StatefulKnowledgeSessionImpl session = (StatefulKnowledgeSessionImpl) obj;
      debug("vvvvvvvvvvvvvvvvvvvvvvvvvvv");
      debug(session.toString());
      for (Object o: session.getObjects()) {
         debug(o.toString());
      }
      debug("^^^^^^^^^^^^^^^^^^^^^^^^^^^");
   }
}

And now I was able to get some nice output of individual working memories! So the resulting bug is JBRULES-3337.

My Gimp Magic

2011-12-04T11:57:00.010+01:00

I'm a Linux user (distribution-agnostic) so I have to bolt my seat down in the airplane but once done, it is really comfortable. The same thing applies to Gimp. Today I spent some time with it on a fresh installation and found out all the good plugins I ever used. The important point here is that there are only three of them to do what I really need. And, of course, all of this can be done manually in a clean Gimp installation but it might take ages.

Contrast Blending aka HDR Photography

High dynamic range imaging is a set of techniques that allow displaying images with big differences between dark and light parts on a limited device. A limited device is a usual computer screen or a printed image. None of these media can fully display the range that a human eye is able to see. So we need to make a compromise in the scene.

There is a great Gimp plugin and tutorial for HDR blenging. The basic principle is to take three pictures with different exposure values (dark, normal and bright) and mix them together. A good algorithm can do that pretty easily for you. A sample picture can be seen below. In the left upper corner, there is the dark exposure. In the upper right corner, there is the normal exposure, the bright exposure is placed at lower left, and, eventually, the result image is at lower right. Note that you can see the background, while there is the candle wick still visible below the light.

Layer Effects

In Photoshop, layer effects are built-in. These effects can do some easy manipulation with the image and is one the most criticised lacks of Gimp. Fortunately, there is a plugin that can be easily installed and works pretty much the same way.

Shadows and Highlights

This plugin allows you to manipulate dark and light parts of the scene. It can automatically extract these parts into new layers and by using the layer opacity you can configure the light in shadows and darken the bright parts.

This is not such a miracle, but it is very useful in infrared (IR) photography.

IR Photography

The biggest issue with IR photography is that usual DSLR cameras have a fixed built-in IR filter before the image chip. This is to avoid unwanted light. But for IR photos we want it! The solution (except for camera disassembly or buying Sigma SD14 with removable filter) is to increase exposure period. It can take several minutes to get enough light with an IR filter on your lens.

Now, when we have a photo, how to process the red scene we obtained? There is a great Photoshop IR tutorial. But I wanted to do that in Gimp. There is a tutorial in a forum that resembles the Photoshop procedure. To summarize that:

Realize what is in individual color channels - red is brightness, green is sharpness, blue is noise.
In Gimp go to Colors, Levels and try Automatic or set a reference white point (which is something that produces most IR light - a green leaf on a sun for instance). You can also setup individual channels as you like.
In Gimp select Colors, Components, Channel mixer and switch the red and the blue channels. This is done by setting 100% for blue and 0% for red with the red input channel and vice versa for the blue input channel.
Now is the best time for the Shadows and Highlights plugin.
Finally, you can try Colors, Hue/Saturation and add a little bit of Hue (+15 - +24) and possibly add some Saturation as well.
Bonus step is to open Colors, Curves or Colors, Levels and fine tune the image colors.

Accessing ODBC from Java

2011-05-17T16:15:00.005+02:00

Connecting to an ODBC database from Java can be a real nightmare. Most solutions suggest using a JDBC-ODBC bridge. So what are real possibilities here? Native JDBC-ODBC bridge by Sun/Oracle sun.jdbc.odbc.JdbcOdbcDriver. Well, there are some limitations like the number of concurrent connections and poor performance. I was not able to find any free (ideally open source) JDBC-ODBC bridge. There are multiple vendors but only one seems to have a de facto standard - Easysoft. They provide trial version of their bridge so you can try the solution before you buy it. Here we realized some troubles in Java Virtual Machine consuming too much memory. I do not want to blame the bridge. It was on a separate machine but we are not able to debug it and see how it works. So not being opensource killed Easysoft's chance to get a new customer.

After all I decided to write a small Java client that would be able to access an ODBC database. We needed just some basic querying capabilities anyway. But this is a little tricky, there is no ODBC communication protocol specification. Eventually, I decided to use unixODBC library using JNI. And here is beginning of the story.

First I designed a client for accessing the database. I had some previous ODBC API knowledge so I designed the client in order to write minimum code in C.

A design note: I failed passing database handles (type void*) between C and Java, so I created an array to store real handles in the C library and pass just an index to the array. This requires tha Java part to remember which handles have been used. There definitely exist a more sophisticated solution but I wanted to query an ODBC database at first. Also note that all data are returned as Strings for simplicity. There is also a shor main() method to test the client.

package org.marvec.odbc;

public class OdbcClient {
   
   private static SortedSet usedHandles = new TreeSet();
   private int handle;
   
   public native void connect(String connection) throws IOException;
   public native void execute(String statement) throws IOException;
   public native int getNumCols();
   public native ColumnMetadata getColMetadata(int col);
   public native boolean fetch();
   public native String getColData(int col) throws IOException;
   public native void freeStatement();
   private native void close();
      
   public OdbcClient() throws IOException {
      int i = 0;
      while (usedHandles.contains(i) && i <= 1024) {
         i++;
      }
      if (i == 1024) {
         throw new IOException("All handles are currently in use. Try to free some handles by disconnecting from ODBC.");
      }
      usedHandles.add(i);
      handle = i;
   }

   public void disconnect() {
      close();
      usedHandles.remove(handle);
   }

   public static class ColumnMetadata {
       public String name;
       public int type;
       public long length;
       public int digits;
       this.nullable = nullable;

       public String toString() {
          return name + ": " + type + "(" + length + ")" + "[" + digits + "]" + (nullable ? "*" : "");
       }
   }

   public static void main(String... args) throws Exception {
       System.loadLibrary(args[0]);
       OdbcClient c = new OdbcClient();
       c.connect("DRIVER={PostgreSQL64};DATABASE=myodbcdb;SERVER=localhost;PORT=35432;Uid=admin;Pwd=admin;");
       c.execute("select * from persons where person_id < 1000");
       System.out.println("Columns: " + c.getNumCols());
       for (int i = 1, j = c.getNumCols(); i <= j; i++) {
           ColumnMetadata meta = c.getColMetadata(i);
           System.out.println("Column " + i + ": " + meta);
       }
       while (c.fetch()) {
           for (int i = 1, j = c.getNumCols(); i <= j; i++) {
               System.out.print(c.getColData(i) + (i == j ? "\n" : ", "));
           }
       }
       c.freeStatement();
       c.disconnect();
   }
}

After compilation, I neede javah utility to obtain a C header file (.h).

javah -jni org.marvec.odbc.OdbcClient

Now the hard C part begins. I'm not a C guru and my compiler reports many warnings about my code. However, it compiles and works. First I copied org_marvec_odbc_OdbcClient.h to org_marvec_odbc_OdbcClient.c and added the shared handle storage to the header file as well as necessary includes. As you can see, we need to remember three different handles for each connection.

#include 
#include 
...
SQLHENV env[1024];
SQLHDBC dbc[1024];
SQLHSTMT stmt[1024];

You might find me wasting some memory but you know, in Java world, there is always enough memory... I will show you only some parts of the solution, link to a complete package is at the end of this post. The most difficult function is getting column metadata that requires creating a new Java object from C.

JNIEXPORT jobject JNICALL Java_org_marvec_odbc_OdbcClient_getColMetadata(JNIEnv *jnienv, jobject obj, jint col) {
    int h = getStaticHandle(jnienv, obj);
    SQLSMALLINT nameLength, dataType, decimalDigits, nullable;
    SQLULEN colSize;
    jstring jstr;
    char name[512];
    
    SQLDescribeCol(stmt[h], (SQLUSMALLINT) col, name, sizeof(name), &nameLength, &dataType, &colSize, &decimalDigits, &nullable);

    jstr = (*jnienv)->NewStringUTF(jnienv, name);

    jclass clazz;
    jmethodID cid;
    jobject meta;

    clazz = (*jnienv)->FindClass(jnienv, "org/marvec/odbc/OdbcClient$ColumnMetadata");
    cid = (*jnienv)->GetMethodID(jnienv, clazz, "", "(Ljava/lang/String;IJIZ)V");
    meta = (*jnienv)->NewObject(jnienv, clazz, cid, jstr, dataType, colSize, decimalDigits, (jboolean) (nullable == SQL_NULLABLE));

    return meta;
}

In some problematic methods I also check for errors using another utility library I mostly copied from Jinput project (see function throwIOException() for example).

JNIEXPORT void JNICALL Java_org_marvec_odbc_OdbcClient_connect(JNIEnv *jnienv, jobject obj, jstring conn) {
    SQLRETURN ret;
    jbyte *str;
    int h = getStaticHandle(jnienv, obj);
    char error[10240];

    jclass cls = (*jnienv)->GetObjectClass(jnienv, obj);

    str = (*jnienv)->GetStringUTFChars(jnienv, conn, NULL);

    SQLAllocHandle(SQL_HANDLE_ENV, SQL_NULL_HANDLE, &(env[h]));
    SQLSetEnvAttr(env[h], SQL_ATTR_ODBC_VERSION, (void *) SQL_OV_ODBC3, 0);
    SQLAllocHandle(SQL_HANDLE_DBC, env[h], &(dbc[h]));
    ret = SQLDriverConnect(dbc[h], NULL, str, SQL_NTS, NULL, 0, NULL, SQL_DRIVER_COMPLETE);

    if (!SQL_SUCCEEDED(ret)) {
        extractError(dbc[h], SQL_HANDLE_DBC, error, sizeof(error));
        throwIOException(jnienv, "SQLDriverConnect failed with return value %d:\n%s", ret, error);
    }

    (*jnienv)->ReleaseStringUTFChars(jnienv, conn, str);
}

Now there are some dependencies in the C project. You must have installed the following libraries in your system:

unixODBC
unixODBC-dev
some ODBC driver for your target database like odbc-postgresql
libltdl
and some common C libraries that are likely to be already present on your computer: pthread, dl

Now you must configure ODBC to provide you the driver (see the first part of the connection string DRIVER=). This is done in /etc/odbcinst.ini and we can configure both 32- and 64-bit versions:

[PostgreSQL]
Description     = PostgreSQL driver for Linux & Win32
Driver          = /usr/lib/odbc/psqlodbca.so
Setup           = /usr/lib/odbc/libodbcpsqlS.so

[PostgreSQL64]
Description     = PostgreSQL driver for Linux & Win32
Driver          = /usr/lib64/odbc/psqlodbca.so
Setup           = /usr/lib64/odbc/libodbcpsqlS.so

To compile the C library I created a small Makefile:

CC=gcc
CFLAGS=-shared -fPIC -w -I$(JAVA_HOME)/include -I$(JAVA_HOME)/include/linux
PROJECT=org_marvec_odbc_OdbcClient

all: $(PROJECT).so

$(PROJECT).so: $(PROJECT).c
        $(CC) $(CFLAGS) util.c $(PROJECT).c -o $(PROJECT).so -lodbc -lpthread -lltdl -ldl
        ln -sf $(PROJECT).so libjavaODBC.so

clean:
        rm $(PROJECT).so libjavaODBC.so

Note the creation of a symbolic link. Java loads libraries on library path with standard name lib[name fo the library].so and only the library name is what you pass as an argument to System.loadLibrary().

Now to run the client just call:

java -Djava.library.path=. org.marvec.odbc.OdbcClient javaODBC

Feel free to download the complete source code at github. The code is provided as is, I do not take any responsibility for any damage it might cause to anybody or anything (including hardware, software, people, animals, etc). For instructions see JBoss Community document.

Invalid signature file digest for Manifest main attributes

2010-06-26T11:52:00.004+02:00

Because of some obvious reasons, security is a must when it comes to computers nowadays. I am not much into security but I had to deal with it anyway. Last couple of days I worked with signed JAR files, when my Java compiler suddenly said: "Invalid signature file digest for Manifest main attributes".

It was because some JAR files were not signed correctly since I needed to remove signatures and my automated tool for signatures removal was broken. Fine, but which files are in a bad condition? I was not able to make my compiler tell me. My class path was like 10kB of text and the day was almost over...

Signatures and certificates in a JAR file are in the META-INF directory. These are files with .SF, .DSA, and .RSA extensions. All one need to do to "unsign" a JAR file is to delete those files. Note that, some information are in MANIFEST.MF as well but it is not necessary to remove them until you want to sign the JAR file again.

It is pretty straight forward so what can be wrong? It was my bad, I assumed that all JAR files in my application use the same certificate file. I did not delete all .RSA files but just something like PRODUCT.RSA.

But it took me a while to realize what is wrong. And here is a command that can help you to recognize a broken JAR file:

find -name *.jar -exec echo {} \; -exec jarsigner -verify {} \;

It is definitely a pity that Java compiler (Sun JDK 1.6) does not print out the problematic file name.

"Always avoid working with tools that have a helve"

2010-04-02T18:41:00.002+02:00

My daddy told me, son, always try to avoid working with tools that have a helve. I succeeded, I would say. I do not have to work with such tools. At least not at a daily basis. I work as a software engineer. I work mainly with intellectual property. But sometimes I feel like I am working just because of those tools.

Just imagine your only job was to make a better hammer. I believed that a hammer proved to be optimal over the years. It is a mean of achieving a more complicated result. But all those software engineers working on operating systems or application middleware try to enhance tools for computer users or application developers all day long.

The software engineers use another tools to make tools for you. Those another tools like compiler must have been developed by another engineers. But this is not what I wanted to talk about...

The software engineers tend to think too much about their tools. It is good to know your tool, of course. But it should not be recognizable which tool you used. Usually, you cannot recognize a 9mm chisel trace on a sculpture. You should not design according to the tools you plan to use.

I keep asking myself how difficult is to master a programming language. Is it that easy? Is it just a syntax sugar you need to know? Let's take Java as an example. There are too many pitfalls you must know to understand complicated code clearly - thread local variables, dynamic proxy, anonymous inner classes. You can make sure that you do not know the language perfectly, just google sample questions for the Sun Certified Java Programmer exam.

I started programming in Java around 2001. I was able to develop my applications easily without studying many guides. I did not know almost nothing but the language grammar. Now, almost 10 years later, I feel like I know intuitively 95% of the language (some percents were gained very very hard).

And what now? After all, Java is just a tool. Like a software engineer's hammer. It is definitely less than 1% of my knowledge needed to do my job correctly. So big on one side and so small on the other side. What is the right side?

My biggest dilemma is whether I should ask for language specific details when having an interview with a possible new employee at my company. Or should I ask more sophisticated questions? "What are basic properties of each transaction?" or "What is an application server?". And what about assembler? Everything that works need assembler!

Fancy technologies

2010-03-18T13:30:00.007+01:00

What makes technologies look good for the first sight? Is it a great demo showing an easy-to-implement solution? What makes you to choose a given technology? A nice graphics, open source, wide community?

Well, I have major interest in Java related technologies so my opinion may be skewed by that. Recently, I felt for some technology (let's call it Technology X) and I chose it for my project. It has the following attributes that I appreciated (not in any particular order):

open source
active community
fancy demos
great ratio of source lines/work done
supporting tools
usable documentation
some books by known authors exist
fresh post "bleeding edge" technology

Especially the last attribute was a problem. I usually wait several months before upgrading my Linux distribution to hear from the community about major problems. I waited for a year after relatively wide adoption of Technology X. Obviously it was not enough and my decision proved to be a big mistake.

What might look fancy at the beginning could become a real nightmare. My problem was that Technology X was well prepared for many standard usage scenarios keeping the above list of attributes valid. When I started looking at some specific problems I got into a trouble. And what makes your project exceptional? A specific behavior that nobody else has, in my opinion.

To make the long story short, I ended up dwelling in the Technology X's source code to realize that there are well-known bugs and design flaws that were blockers to my project.

What makes me think of this story was another blog post of a new superb glorious Technology Y that can become a panacea to your projects. As a proof of all the positive features authors demonstrated a Getting started example of nothing too fancy, but enough to give you an idea of...

But realistically, it is possible that later the authors abandon the project and no other blog will be published. It is not likely to see a blog post describing what is not possible with Technology Y. So please, before you write a Getting started blog/article, always make sure that you have tried advanced features of that technology as well.

Eventually, I used Technology Z for my project. It was a complete restart of the development phase. This time I was more cautious. I realized that with the right technology I can start doing some real work early in the technology studying phase. The documentation is precise, yet relatively short. Technology Z has a great support in my favorite IDE. I have control of what and how is done - the technology is flexible in allowing me to choose an appropriate level of granularity in different parts of my project depending on my goals.

I couldn't believe it. Technology Z offers me the same functionality as Technology X. I do not have to write any significantly bigger amount of code or configuration files. I can implement new features in my project in approximately 20% of the original time needed with Technology X. Long live Technology Z.

Image Analysis

2009-11-28T23:18:00.003+01:00

Today, I needed to compute an average value of a function. The problem was that the function was a chart in a .gif image.

I run across a great application called ImageJ. It has a pluggable interface, many powerful functions, it is open-source and written in Java. The platform independent distribution has only 3MB.

I opened my image in the application, cropped the unnecessary parts, and used the magic wand to selected the area under the function line. Then I just clicked on measure and got the area of 69035 pixels (btw. this is the value of the integral). I divided the number by the width of the image and got 226. Now I subtracted 226 from the image height and looked at the y-axis for the average value - 8.7...

Disclaimer: ImageJ is being developed at the National Institutes of Health by an employee of the Federal Government in the course of his official duties. Pursuant to Title 17, Section 105 of the United States Code, this software is not subject to copyright protection and is in the public domain. So long and thanks for ImageJ!

Performance testing: Load generator

2009-09-07T15:14:00.016+02:00

Introduction

Performance testing is a wide term, so let me be more specific please: performance testing of a message oriented middleware from the client's point of view. A message oriented middleware can be an Enterprise Service Bus for instance. We tried several implementations - JBoss ESB, WSO2 ESB, and Mule ESB.

What does the "testing from the client's point of view" mean? It means that we don't care about internal processes of the ESB. We just care how long it takes to process our request. It reminds of integration testing in the classical V-Model.

Problem Definition

We were working in a team on the problem and after trying several load generators, we found out that there is none that is generic enough. We wanted to send various types of messages (e.g. JMS, SOAP, files...) in different ways (send all messages at once, find the maximum sustainable speed, run a long-term test...). Apache JMeter approaches the ideal but is still missing some features. Moreover, we experienced high results jitter over time. So we decided to write our own lightweight, easy-to-use load generator. Here are some basic requirements:

Stable results.
There must be a constant message inflow while received messages are being processed.
Messages must be generated by many concurrent clients.
Clients must run on remote machines not to influence results.
Speed should be measured on the client side for us not to affect the server.

Solution

Message Sender

First, there are different types of messages. So let's have an interface of a component that just sends a particular message.

public interface MessageSender {
  public void setProperty(String prop, String value);
  public void init(String address) throws Exception;
  public void close();
  public Serializable send(Serializable message, 
    Map properties) throws Exception;
  public MessageSender clone();
}

The component should be able to get configured with standard property/value pairs, initialize a connection to a given address, send a serializable message (e.g. a String), and close the connection. Moreover, it can clone itself for us not to have to configure each instance.

There are two important decision points. First, do you want MessageSender to be reusable - to be able to send more than one message? When you are careful enough, you should be able to achieve this. At least, we did not experience any problems with that.

Second, do you want your senders to be thread safe? This is a completely different situation, which is really hard to achieve and keep high performance. An example is sending of JMS messages. JMS session is not thread safe but it takes relatively long time to initialize. You might want to extend the interface and create a ThreadSafeMessageSender.

An example of an HTTPSender is shown below.

public class HTTPSender implements MessageSender {
  private URL url;
  private String method = "POST";

  public void setProperty(String prop, String value) {
    if ("method".equals(prop)) {
      method = value;
    }
  }

  public void init(String address) throws Exception {
    url = new URL(address);
  }

  public void close() {
  }

  public Serializable send(Serializable message, Map properties) throws Exception {
    HttpURLConnection rc = (HttpURLConnection) url.openConnection();
    
    // ... standard HttpUrlConnection usage ...

    return response;
  }

  @Override
  public MessageSender clone() {
    ...
  }
}

Message Generator

Next, there must be a message generator that uses message senders to generate messages in a given way.

public abstract class MessageGenerator {
  protected MessageSender sender;
  protected int threads = 1;

  public void setProperty(String property, String value) {
    if ("threads".equals(property)) {
      threads = Integer.valueOf(value);
    }
  }

  public void init(String address, MessageSender sender) throws Exception {
    this.sender = sender;
    this.sender.init(address);
  }

  public void close() {
    sender.close();
  }

  public abstract void generate(Serializable message, int count) throws Exception;
}

As you can see, an already configured instance of MessageSender must be passed to the MessageGenerator in the init() method. It is not a constructor because some configuration of the generator might be performed first in its descendants. This abstract generator supposes that the message sender is thread safe, because it uses just a single instance to pass all the messages through. Descendants might want to clone this instance in case it were not thread safe.

We used several new features of Java 6 to create a message generator that just sends all the messages to the server in a given number of threads. Let's first examine a sender task that is created per each thread. It uses shared AtomicInteger to keep track of actually sent messages.

public class SenderTask implements Runnable {
  private MessageSender sender; // shared sender
  private Serializable message; // message to send
  private AtomicInteger counter; // shared message counter
  private int count; //  number of messages for this thread

  public SenderTask(AtomicInteger counter, MessageSender sender, 
      String address, Serializable message, int count) 
      throws Exception {
    this.count = count;
    this.sender = sender;
    this.message = message;
    this.counter = counter;

    // expect sender's uninitialized clone
    this.sender.init(address);
  }

  @Override
  public void run() {
    try {
      for (int i = 0; i < count; i++) {
        sender.send(message, null);
        counter.incrementAndGet();
      }
    } catch (Exception e) {
      log.error(e);
    } finally {
      sender.close();
    }
  }
}

In the next listing, there is an implementation of the generate() method. No speed measurement code is included but the main goal here was to show the ExecutorService's usage.

@Override
public void generate(Serializable message, int count) 
    throws Exception {
  List<SenderTask> taskList = 
    new ArrayList<SenderTask>(threads);

  // Let tasks to initialize senders
  for (int i = 0; i < threads; i++) {
    taskList.add(new SenderTask(counter, sender.clone(), 
      address, message, perThreadMessageCount);
  }

  // Submit all tasks for completion
  ExecutorService es = Executors.newFixedThreadPool(threads);
  for (int i = 0; i < threads; i++) {
    es.submit(taskList.get(i));
  }

  // Wait for termination
  es.shutdown();
  boolean terminated = false;
  while (!terminated) {
    // output current state
    // take some sleep
    // updated 'terminated' variable
  }
}

As you can see, clones of the original sender are created for us not to have to take care of the thread safeness. Individual SenderTask instances are created in advance for all sender clones to get initialized. Then the instances are put in an ExecutorService and executed. While we are waiting for all threads to finish, we can periodically output current and overall throughput for instance.

Conclusion

This concept already demonstrated its value in real world scenarios. We were able to write various message senders and generators easily and quickly. Even though it may look simple, there are some factors you must take into consideration.

First is the warm up period of the server. You could either run the test several times and ignore a couple of first runs, or extend the generator to throw in some more messages and do not count them.

Second, always make sure that the client is able to generate messages several times faster than the server is able to process. Otherwise, you measure the client's performance, which is something you do not want probably.

WSDL 2.0: Quick reference

2009-09-07T12:03:00.013+02:00

I wrote this blog entry mainly for me - to beat WSDL structure in my mind in a clean and nice way. Hopefully, it will help some others as well.

I would like to introduce changes between WSDL 1.1 and 2.0, and to go through the individual parts of the new WSDL version. This blog entry is based on several other on-line resources as well as books.

Introduction

WSDL stands for Web Services Description Language. It is a document written in XML. The document describes a Web service, specifies the location of the service, and the operations (or methods) the service exposes.

WSDL 2.0 was first developed as WSDL 1.2 but was renamed because of its substantial differences. Some major changes are:

Adding further semantics to the description language.
Removal of message constructs. These are specified using the XML schema type system in the types element.
No support for operator overloading.
PortTypes renamed to interfaces. Support for interface inheritance is achieved by using the extends attribute in the interface element.
Ports renamed to endpoints.

Conceptual Model

The description consists of two parts. In the abstract part, WSDL describes a web service in:

Messages it sends and receives using a type system (typically W3C XML Schema).
Message exchange patterns that define the sequence and cardinality of messages.
Operations that associates message exchange patterns with one or more messages.
Interfaces group these operations in a transport and wire independent manner.

In the concrete part of the description:

Bindings specify the transport and wire format for interfaces.
A service endpoint associates network address with a binding.
A service groups the endpoints that implement a common interface.

Structure

Following is a basic WSDL structure

<definitions targetNamespace="xs:anyURI">
<documentation /> ?
[<import /> | <include /> ] *
<types /> ?
[<interface /> | <binding /> | <service /> ] *
</definitions>

Definitions

The definitions element serves as a container.

<definitions name="StockQuote"
  targetNamespace="http://example.com/stockquote/definitions"
  xmlns:tns="http://example.com/stockquote/definitions"
  xmlns:xsd1="http://example.com/stockquote/schemas"
  xmlns:soap="http://www.w3.org/2003/11/wsdl/soap12"
  xmlns="http://www.w3.org/2003/11/wsdl">

Include

The include element helps to modularize the web service descriptions. Included documents must have the same target namespace.

Import

The concept behind the import element is very similar to that of include element, except that the imported WSDL can be in different target namespaces.

<import namespace="http://example.com/stockquote/schemas"
  location="http://example.com/stockquote/stockquoteV20.xsd"/>

Types

The types element defines the data types used by the exchanged messages. WSDL uses W3C XML Schema as its preferred schema language.

The following example refers to an imported XSD.

<types>
  <schema 
    targetNamespace="http://example.com/stockquote/definitions">
    <element name="GetLastTradePriceInput" 
      type="xsd1:TradePriceRequest"/>
    <element name="GetLastTradePriceOutput" 
      type="xsd1:TradePrice"/>
  </schema>
</types>

Interface

An interface element encloses a named set of abstract operations and the abstract messages. It can extend one or more other interfaces. Interfaces are referred to by QName in other components.

<interface name="StockQuoteInterface">
  <operation name="GetLastTradePrice" pattern="http://www.w3.org/2003/11/wsdl/in-out">
    <input message="tns:GetLastTradePriceInput"/>
    <output message="tns:GetLastTradePriceOutput"/>
  </operation>
</interface>

Binding

The binding element defines the underlying transport and wire format for messages. Each binding references to an interface.

<binding name="StockQuoteSoapBinding" 
  interface="defs:StockQuoteInterface">
  <soap:binding protocol="http://www.w3.org/2003/11/wsdl/http"/>
  <operation name="GetLastTradePrice">
    <soap:operation 
      soapAction="http://example.com/GetLastTradePrice"/>
    <input>
      <soap:body/>
    </input>
    <output>
      <soap:body/>
    </output>
  </operation>
</binding>

Service

A service element describes a set of endpoints which refer to a single network address for a binding. All other protocol specific information is contained in the binding.

<service name="StockQuoteService">
  <documentation>My stock quote service</documentation>
  <endpoint name="StockQuoteEndPoint" 
    binding="tns:StockQuoteSoapBinding">
    <soap:address location="http://example.com/stockquote"/>
  </endpoint>
</service>

Conclusion

Despite of the fact that the WSDL format might seem over-engineered to you (because it probably is), you should now be aware of its content and the meaning of individual elements.

References

[1] Article on WSDL at www.xml.com
[2] W3C School WSDL Tutorial
[3] Web Service Platform Architecture on InformIT.com

EJB3: Conscious Session Bean

2009-05-13T11:19:00.006+02:00

Last week, I wanted to demonstrate @TransactionAttribute in a session bean using JBoss AS 5.1.0 GA and EJB3. Everything seemed pretty simple...

@Stateless
public class OmnipotentBean implements Omnipotent {
    @TransactionAttribute(REQUIRED)
    public String transactionRequired() throws SystemException {
        try {
            transactionNever();
            System.err.println("No exception occured - ERROR");
        } catch (EJBException e) {
            System.out.println("Expected exception - OK");
        }
    }

   @TransactionAttribute(NEVER)
    public String transactionNever() throws SystemException {
        ...
    }
}

As you can see, I wanted to show that calling a method with a transaction attribute set to NEVER from a method that already runs in a transaction throws an exception. But the exception was not thrown. How come?

Let's start from the client. What kind of object do we get from JNDI when we looked up OmnipotentBean/remote? It was neither OmnipotentBean or its interface. It was a dynamically generated class based on java.lang.reflect.Proxy, which handles all method calls and passes them to the server. You can call its method because it implements Omnipotent interface. Moreover, the call passes through a set of interceptors, which can perform additional work and setup the environment, inlcuding security and transactional interceptors.

The same applies at the server side. What is this inside transactionRequired() method? It is just a OmnipotentBean class instance, thus a simple POJO. We must lookup the EJB through JNDI to get a proxy object for transactional attributes to start working.

Another uncomfortable thing is that we cannot use dependency injection in our session bean to inject its proxy. Not having the bean initialized at the time of its initialization when dependencies are being injected, this is the cause.

So, are all the problems just JBoss AS's whims? What does the specification say? EJB3 - JSR-220 (file EJB Core, page 343):

The following subsections define the responsibilities of the container for managing the invocation of an enterprise bean business method when the method is invoked via the enterprise bean’s business interface (and/or home or component interface), or web service endpoint. The container’s responsibilities depend on the value of the transaction attribute.

Everything is all right. EJB container (i.e., the application server) must manage the invocations of bean's methods only when they are called via the business interface, which is the proxy object.

Maybe this is not suprising to you and you are right. It is very logical. Just bear in mind that a session bean must be conscious of its own business interface when invoking methods of its own.

RichFaces and Background Gradient

2009-04-18T19:22:00.004+02:00

I do not know whose idea it was. Maybe it is there just to demonstrate some advanced RichFaces' abilities but I (in all the humbleness) hate it. I am talking about the background gradient which is by default present at table headers and toolbars. The background is a dynamically generated image and its presence in a web page is caused by XML snippets like the following one (part of richfaces-ui.jar:/org/richfaces/skin.xcss).

<u:style name="background-image">
  <f:resource xmlns:f="http:/jsf.exadel.com/template" f:key="org.richfaces.renderkit.html.GradientA"/>
</u:style>

A friend of mine would call such a beatiful thing a funfair or a fly-by. Apart from the hardcoded gradient background I miss only one thing in RichFaces.

A scrolling text that blinks in Firefox and Opera.

It took me couple of hours to get over the problem, yet I'm not satisfied with either solution I found.

Solution no. 1: Override the default behavior

Your CSS style can be loaded later and override the background you are not satisfied with. This is even "oficially" recommended. The linked page speaks about rich:dataTable so I am adding my part for rich:Toolbar.

.rich-toolbar, .rich-toolbar-item {
    background-image:none;
    background-color:transparent;
}

You can of course set more convenient background color. The big issue here is that you still have the gradient style configured and loaded by your browser, and the background image generated at the server.

Solution no. 2: Update richfaces-ui.jar

You can grep through all .xcss files inside richfaces-ui.jar and remove the gradient from whatever place you want. You get tinier CSS and less work on the server side. But you have to repeate the procedure every time you want to upgrade RichFaces version. And you also must be careful about your richfaces-ui-without-gradient.jar.

I like neither of the solutions. For my project I went with no. 1 and I hope that there will be a new gradient style to configure in my skin.properties: gradientStyle=NONE.

TestNG and Group Filters

2009-04-09T13:10:00.018+02:00

Recently I solved an interesting problem with test groups in TestNG. To describe the problem, let me describe a sample test class with three methods, which are included into two groups.

public class SampleTest {
  @Test(groups = { "groupA" })
  public void testMethod1() {
    System.out.println("1 in A");
  }

  @Test(groups = { "groupA", "groupB" })
  public void testMethod2() {
    System.out.println("2 in A, B");
  }

  @Test(groups = { "groupB" })
  public void testMethod3() {
    System.out.println("3 in B");
  }
}

Usually, TestNG is called using an Ant task like the following one.

<testng excludedGroups="..." groups="..." ... />

What methods can be executed with various groups and excludedGroups setting presents this table:

Methods	groups	excludedGroups
1	groupA	groupB
1, 2	groupA	--
2, 3	groupB	--
3	groupB	groupA
1, 2, 3	--	--

This is pretty logical. But wait, there is a combination missing from that table. What about just the method 2, which is in groupA and groupB? To be honest, I did not find an easy way to achieve this. But there is a workaround. First, you must use an external configuration file for TestNG.

<testng classpathref="...">
  <xmlfileset dir="." includes="testng.xml"/>
</testng>

In this file, you can use Beanshell to enable/disable individual methods.

<suite name="TestSuite">
  <test name="SampleTest">
    <method-selectors>
      <method-selector>
        <script language="beanshell"><![CDATA[
          groups.containsKey("groupA") && groups.containsKey("groupB")
        ]]></script>
      </method-selector>
    </method-selectors>
    <classes>
      <class name="com.blogspot.qecafe.tests.SampleTest" />
    </classes>
  </test>
</suite>

The magic line in Beanshell will cause only methods in both groups to be executed.

There is yet another item missing - 1, 3 combination. You can run it in a similar way just like 2.

Temptations of Software Engineers

2009-04-07T10:06:00.008+02:00

Have you ever been tempted to use a technology just because of the technology itself? Was the technology such a fancy thing that you could not resist? Please do not do that! A result of your work would be barely usable in the real life. Do you know the superior Swiss knife with all the stuff inside? I am in doubt you could ever carve a little wooden ship for your son with it.
Let me tell you a true story of a software engineer and an application he developed.

Once upon a time, there was a guy who decided to learn Java and all its bright features, which brought the light on every programmer's possible problem. He chose a set of features that were most interesting to him including serialization, and multi threading. To be not all alone and to verify his code, he decided to take a community as a hostage. Let's inspect a couple of his mistakes.

First, serialization is good for remote method invocation, for sending objects over network, and even for storing small project configurations into a file. It should not be used as a primary data storage, since there are relational databases for such a purpose. If you wanted to be thread safe in a serialize-everything environment, you must keep all the objects in the memory and (de)serialize them only on an application start/shutdown. It is slow, complicated and inefficient way.

Second, multi threading is a powerful thing but it is like the fire - it can be very dangerous. Threads are not independent processes. You should be careful about them, especially if they are running third party code. Apart of that issue, the biggest problem was that Thread.stop() does not stop a waiting thread on all JVM implementations. It is not good idea to stick to an implementation detail. Later, Thread's methods stop(), suspend(), and resume() have become deprecated because it showed up that they are unsafe.

I could have continue here with similar cases but that should be enough to show what I meant by temptations. If the project were now buried deep in the binary scarp, everything was fine. Unfortunately, the community around it become large and the project is massively used nowadays. And the users must fight with those design issues hidden deep inside of it.

Guts of Programming Languages

2009-04-01T16:27:00.006+02:00

What makes it difficult to master a new programming language when you already know one or two? Is it a fancy syntax? Is it an API? I do not think so.

A syntactic sugar is not the real matter of the programming art. It should be only a hammer for your nails. Usually, a new syntax is not so much different. I mean how often do you meet a brand new paradigm? There is imperative, functional, logic and whatever programming, but you have probably chosen your way already.

Modern IDEs can help you with the API of the new language. Again, API should not be an obstacle. But there is still a thing that you must be aware of...

I do not know of any real programming language that isolates a programmer from its inside behavior (by a real programming language I mean a language that is commonly used for developing long running applications). You always need to be aware of a memory model, of a garbage collection mechanism, of a just in time compilation, of a target system architecture, of registers size, of a class inner representation, of a variable size in bytes...

There are too many "of". I wonder when there is a language that allows developers to concentrate completely on the main problem. A friend of mine suggested that PHP is such a language but I do not agree with that. It is a language which leaves a field of corpses behind itself and must finish processing sooner than anybody reveals this fact.

I have a favorite programming language, which isolates me at most. It is in the functional languages family, and it is called Haskell. You can even write a GUI application in Haskell, but an application server would be a death match.

So good luck in those hard times and do your best for you not to forget any single language implementation detail...