Java Synchronization and Concurrency Across Multiple JVMs, Multiple Computers

It is pretty standard fare for a Java application to coordinate multiple uses of a single resource by using the “synchronized” key word.  But there are times when a single resource must be used by multiple programs running in separate Java Virtual Machines (JVMs), perhaps even on different computers.  Synchronizing use of that resource involves a bit more planning.  Let’s look at single-JVM synchronization first so we can see where, how and why cross-JVM synchronization may be applied.

Typically, resources like counters and files that are used by multiple threads (multiple concurrent users) in a web application require synchronization.  In a Java Servlet, the doGet() and doPost() methods are multithreaded.  Each web browser addressing the web application will have a separate thread, executing the method independently and concurrently.  Whenever those methods update a resource that is declared outside of the method scope, it is a shared resource, and that update should be synchronized.

There are a couple easy ways to synchronize use of those resources.  We can make a method synchronized so that only one thread can use it at a time – all other threads will queue up until the synchronized method is available – they will take turns.  A second way to synchronize use of a resource is to update the resource within a synchronized block.  Let’s look at an example.

public class SynchServlet extends HttpServlet {

            private static PrintWriter fileOut; // Synchronize use of this!

            public void init( ServletConfig config ) throws ServletException {
                        super.init( config );
                        try {
                                    fileOut = new PrintWriter( "logfile.log" );
                        } catch( Exception x ) {
                                    throw new ServletException( x.toString() );
                        }
            }
                       
            public void destroy( ServletConfig config ) {
                        fileOut.close();
            }
           
            public void doGet( HttpServletRequest req, HttpServletResponse res )
                                    throws ServletException, IOException {
                        logWrite( "Using doGet()" );
                        //…
            }
           
            public void doPost( HttpServletRequest req, HttpServletResponse res )
                                    throws ServletException, IOException {
                        logWrite( "Using doPost()" );
                        //…
            }
           
            public static synchronized void logWrite( String logEntry ) {
                        fileOut.println( logEntry );
            }
}
 
In this first example, each time a browser calls on the doGet() and doPost() methods we will write to a log file. We need to have these threads take turns, so only one thread attempts to write to the file at a time – if more than one write occurs concurrently, a runtime exception will be generated.  The simplest way get the threads to take turns is to place the actual log file write in a synchronized method.  The example logWrite() method is modified with the “synchronized” key word so that each thread in this web application (Servlet) will wait until the logWrite() method is available, then the thread will lock the method for exclusive use until finished.  After that, the lock will be released, and the next thread will be able to use the method.

This synchronized method is static, so the lock is applied to the SynchServlet class, not to an instance of SynchServlet – that difference is irrelevant in this case, since there is usually only one instance of a Servlet.  However, in most cases, both “static” and “synchronized” modifiers should be used for a single resource that is shared within a JVM.  As a public static method, you can call it from other classes, and it will enforce the same synchronization controls, like this:

            SynchServlet.logWrite( "Some other message" );
 
Another way we can synchronize multiple threads using a shared resource is to put every use of the resource in synchronized blocks.  The following example shows how we might implement a doGet() success counter.

public class SynchServlet extends HttpServlet {

            private static int doGetSuccessCounter = 0;
            private static Object synchObject = new Object();
 
            public void doGet( HttpServletRequest req, HttpServletResponse res )
                                    throws ServletException, IOException {
                        try {     
                                    synchronized( synchObject ) {
                                                doGetSuccessCounter++;
                                    }
                                    //…                              

                                    synchronized( synchObject ) {
                                                System.out.println( "There are " +
                                                            doGetSuccessCounter + " doGet() successes!" );
                                    }
                        } catch( Exception x ) {
                                    synchronized( synchObject ) {
                                                doGetSuccessCounter--;
                                    }
                                    throw new ServletException( x.toString() );
                        }
            }
}
 
Notice in this example that we increment the doGetSuccessCounter integer in the doGet() method, and we decrement doGetSuccessCounter in the catch block – just for fun, we do not count a thread’s passage through doGet() as a “success” if an Exception is thrown.  Again we want to synchronize use of this shared resource (it is declared outside the method scope) so that we count and discount for every thread calling doGet(), without concurrent threads overwriting one another.  In addition, we synchronize around the report of the current value of doGetSuccessCounter so we are guaranteed to get the current value (after any other lock is released and we have exclusive access to the integer.)

Notice that in each case, when we set up the synchronized block, we specify that the synchronization lock be placed on an object named synchObject.  Any Java object can be used for synchronization locking, but not primitives – that is, we could lock on an Integer.class instance but not on an int primitive.  Our example instantiates an object of type Object named synchObject, and we use it for all the synchronized blocks.  Synchronized blocks must share the lock on a single object in order to coordinate their execution – so all our example synchronized blocks share the lock on synchObject.  Of all the synchronized blocks on synchObject, only one at a time may acquire the lock and execute.  The lock is automatically released at the end of the synchronized block.

You might observe that we have three separate synchronized blocks and that we could get rid of all of them if we just make the doGet() method synchronized.  I love code reduction and refactoring, but this is one case where it would be a very bad plan.  If we made the doGet() method synchronized, then only one browser could view our Servlet at a time; also, the entire doGet() would need to execute before the lock is released.  You always want to limit the amount of time and processing that is done in a synchronized block or synchronized method.  Look back at our examples, at the minimal synchronized block / method.

Also note that since synchObject is modified with the “static” keyword, all instances of the SynchServlet class will share the same lock – that is typically what you want for shared locks within a JVM.  If you also share the resource in another class (perhaps another servlet), you would lock on this same static object, like this:

            synchronized( SynchServlet.synchObject ) {
                        SynchServlet.doGetSuccessCounter++;
            }
 
Synchronized methods and synchronized blocks are foundational concurrency tools.  There are a number of other synchronization and concurrency techniques, many Thread-safe classes and the java.util.concurrent packages that may be also used.  All of them deal with synchronization within a single JVM.  Probably the clearest example of when all these techniques fall short is when you are running Java applications on separate computers and need to update a shared resource, like a file.

Typically, you might run an independent service to update the shared file.  You would provide a network port (or messaging) interface to the service so that all the applications that update the file can send messages to the service which would serve as a proxy to update the (shared) file on their behalf.  Alternatively, you could just update a database using JDBC.

However, it is possible to program synchronization into applications so that they can share and update a resource, even when running in separate JVMs.  In this case, we cannot share a lock on a Java object by using the synchronized keyword – no objects are shared between separate JVMs.  Instead we lock on a file.  We depend on the locking mechanisms inherent in the Operating System (OS) filesystem in order to synchronize our efforts.

Here is an example that shows how code running in separate JVMs can share updates to a file.  The shared file that we are updating is named “central.log”.  I’m using the escaped double backskash to indicate the file is found on a fileshare named \\server\dir.  I refer to another file named “lock.lock” on that same fileshare.  We will get an exclusive file lock on the “lock.lock” file before we do any updates to “central.log”.  Note that the method, centralLog() can be included in several applications, running in different JVMs, perhaps on different servers; and they will all be able to coordinate updates to the “central.log” file.

public class SynchServlet extends HttpServlet {
            public void init( ServletConfig config ) throws ServletException {
                        super.init( config );
                        centralLog( "Starting SynchServlet");
            }
 
            private static void centralLog( String message ) throws ServletException {
                        // Need something external to this JVM to test singularity
                        FileOutputStream fos = null;
                        FileLock fl = null;
                        PrintWriter centralOut = null;
                        try {
                            fos= new FileOutputStream( "\\\\server\\dir\\lock.lock" );
                            fl = fos.getChannel().tryLock();
                            // Null when can't get exclusive lock on file
                            while( fl == null ) {
                                      try {
                                                Thread.sleep(1000);
                                                fl = fos.getChannel().tryLock();
                                      } catch( Exception v ) {}
                            }
                            // At this point, I have exclusive lock on file!
                            centralOut = new PrintWriter( "\\\\server\\dir\\central.log" );
                            centralOut.println( message );
                        } catch( Exception x ) {
                                    throw new ServletException( x.toString() );
                        } finally {
                                    try {
                                                if( centralOut != null ) centralOut.close();
                                    } catch(Exception y) {}
                                    try {
                                                if( fl != null ) fl.release();
                                                if( fos != null ) fos.close();
                                    } catch(Exception y) {}
                        }
            }
}
 
There are three classes in the java.io package that can provide FileChannel objects: FileInputStream, FileOutputStream and RandomAccessFile.  In our example, we instantiate a FileOutputStream on the “lock.lock” file and call getChannel() to get the FileChannel object.  There are several methods to get an exclusive lock on a FileChannel.  We use the FileChannel.tryLock() method in our example.  If tryLock() is unsuccessful, it returns null, else it returns the associated FileLock object.  In our example, we call tryLock() then test the return value for null.  If it is null, we sleep for one second then try again, in the while( null ) loop.  When the FileLock is not null, we have an exclusive lock and can update shared resources.  It is very important that we release the FileLock.  For this reason, I have an independent try/catch block within the finally block that calls the release() method on the FileLock object.

I use a separate file, “lock.lock” for the FileLock, separate from the shared file that applications update, “central.log”.  We could alternatively establish the FileLock on the shared file.  I use a separate file for locking for a couple reasons: it makes the lock file job more obvious and explicit, and occasionally the object I’m sharing is not a file.  I have had to implement this style locking to accommodate a dedicated, single-user port: for example a serial port server, or in one bizarre situation, an FTP client that required a fixed port (per firewall rules), like this:

//package org.apache.commons.net.ftp;
package dac;
import org.apache.commons.net.ftp.*;
import org.apache.commons.net.ftp.parser.*;

public class FTPClient extends FTP {

            …
            private int getActivePort() {
                    if( true ) return 11111; // requires ip / port filter permission
 
Let me mention the limitation of this file locking strategy.  It does not work between JVMs running on different OS architectures.  If all your JVMs are running on Windows or all your JVMs are running on UNIX, there is no problem; however, you cannot obtain an exclusive lock on a file from a Windows computer and have that lock observed on a UNIX computer, nor vice versa.  On the bright side, you can have a JVM on a Windows computer get an exclusive lock on a file that resides on a UNIX computer, and other Windows JVMs will observe the lock.  The same is true for JVMs on UNIX obtaining exclusive locks on files that reside on Windows.

There is also a risk that while a file is locked, the JVM may unexpectedly quit without completing the finally block and releasing the lock.  And there is a risk that if the locked file is on a remote computer, there may be a network failure, or the locking computer may unexpectedly reboot; which could leave the file in a locked state.  A key to avoiding this scenario is to only keep a file locked for the minimum time required.  Lock it, do your work, unlock it ASAP.

Thus ends part one of this story, Java Synchronization Across JVMs.  In part two, I will discuss flag-passing synchronization.

The complete code follows:

package dac;

import java.io.FileOutputStream;
import java.io.IOException;
import java.io.PrintWriter;
import java.nio.channels.FileLock;
 
import javax.servlet.ServletConfig;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
 
public class SynchServlet extends HttpServlet {
            private static final long serialVersionUID = 1L;
            private static PrintWriter fileOut; // Synchronize use of this!
            private static int doGetSuccessCounter = 0;
            private static Object synchObject = new Object();
 
            @Override
            public void init(ServletConfig config) throws ServletException {
                        super.init(config);
                        try {
                                    centralLog("Starting SynchServlet");
                                    fileOut = new PrintWriter("logfile.log");
                        } catch (Exception x) {
                                    throw new ServletException(x.toString());
                        }
            }
 
            public void destroy(ServletConfig config) {
                        fileOut.close();
            }
 
            @Override
            public void doGet(HttpServletRequest req, HttpServletResponse res)
                                    throws ServletException, IOException {
                        try {
                                    synchronized (synchObject) {
                                                doGetSuccessCounter++;
                                    }
                                    logWrite("Using doPost()");
                                    // …

                                    synchronized (synchObject) {

                                                logWrite("There are " + doGetSuccessCounter
                                                                        + " doGet() successes!");
                                                System.out.println("There are " + doGetSuccessCounter
                                                                        + " doGet() successes!");
                                    }
                        } catch (Exception x) {
                                    synchronized (synchObject) {
                                                doGetSuccessCounter--;
                                    }
                                    throw new ServletException(x.toString());
                        }
            }
 
            public void altGet(HttpServletRequest req, HttpServletResponse res)
                                    throws ServletException, IOException {
                        try {
                                    synchronized (SynchServlet.synchObject) {
                                                SynchServlet.doGetSuccessCounter++;
                                    }
                                    SynchServlet.logWrite("Some other message");
                                    // …

                                    synchronized (synchObject) {

                                                logWrite("There are " + doGetSuccessCounter
                                                                        + " doGet() successes!");
                                                System.out.println("There are " + doGetSuccessCounter
                                                                        + " doGet() successes!");
                                    }
                        } catch (Exception x) {
                                    synchronized (synchObject) {
                                                doGetSuccessCounter--;
                                    }
                                    throw new ServletException(x.toString());
                        }
            }
 
            @Override
            public void doPost(HttpServletRequest req, HttpServletResponse res)
                                    throws ServletException, IOException {
                        logWrite("Using doPost()");
                        // …

            }

 
            // synchronized instance method, not class method (static)
            private static synchronized void logWrite(String logEntry) {
                        fileOut.println(logEntry);
            }
 
            private static void centralLog(String message) throws ServletException {
                        // Need something external to this JVM to test singularity
                        FileOutputStream fos = null;
                        FileLock fl = null;
                        PrintWriter centralOut = null;
                        try {
                                    fos = new FileOutputStream("\\\\server\\dir\\lock.lock");
                                    fl = fos.getChannel().tryLock();
                                    // Null when can't get exclusive lock on file
                                    while (fl == null) {
                                                try {
                                                            Thread.sleep(1000);
                                                            fl = fos.getChannel().tryLock();
                                                } catch (Exception v) {}
                                    }
                                    // At this point, I have exclusive lock on file!
                                    centralOut = new PrintWriter("\\\\server\\dir\\central.log");
                                    centralOut.println(message);
                        } catch (Exception x) {
                                    throw new ServletException(x.toString());
                        } finally {
                                    try {
                                                if (centralOut != null) centralOut.close();
                                    } catch (Exception y) {}
                                    try {
                                                if (fl != null) fl.release();
                                                if (fos != null) fos.close();
                                    } catch (Exception y) {}
                        }
            }
}