Caching in a JEE : don’t write it yourself, use LoadingCache from Google Guava libraries.

Caching data is something you use in almost every JEE project. Most of the time it’s pretty simple : put your data in a .properties file and use a PropertyManager to fetch the data.

But that’s not very flexible and manageable. Updating the values means, updating your property file, repackaging the ear file, and redeploying, and only developers can update the data.

Putting the data in JNDI entries, and using JNDI lookups may solve the problem of redeploying, but if you got a few 100 properties, it’s still not very manageable.

Most of the times, JNDI entries are entered via some application server console which, in a production environment, is not accessible for your users who need to manage this data.

So lets put the data that needs to be cached in a database, or make it accessible via a web service. That would be ideal. You can write your own application on it, and have the data managed by your users.

But that means that you have to write your own, thread safe, caching algorithms.

No big deal if the data only changes once every 10 years, but refreshing it on a time or size basis, makes the whole thing a bit more complicated. And that’s where the great LoadingCache class from the Google Guava library comes in.

What are the Guave libraries ? Well, here’s how they describe it : ‘The Guava project contains several of Google’s core libraries that we rely on in our Java-based projects: collections, caching, primitives support, concurrency libraries, common annotations, string processing, I/O, and so forth.’

Now for caching, the Guava LoadingCache class caches data in a key-object map, and lets you define a cache refreshing mechanism, all done in a thread safe manner.

So lets show a small  example and explain how it works. Suppose your cache contains a list of products that are on sale for 1 day. Depending on the no. of sold products, the price will increase during that day. This means that the cache should be updated every few seconds, to update the price, and after 1 day, the whole cache should be refreshed with new products. Suppose that price setting and product selections are in the database, updated by some back-end application, and we need the new data in our frontend application and we want to cache it.

All this can be done with this simple class :

import java.util.concurrent.TimeUnit;
import javax.ejb.EJB;
import javax.ejb.Singleton;
import be.iadvise.dao.DatabaseDAO;
import be.iadvise.entities.Product;
import com.google.common.base.Optional;
import com.google.common.cache.CacheBuilder;
import com.google.common.cache.CacheLoader;
import com.google.common.cache.LoadingCache;
import com.google.common.util.concurrent.MoreExecutors;

@Singleton
public class ProductCache {

@EJB
 DatabaseDAO databaseDAO;
 private static final Integer REFRESH_PRODUCT_AFTER_5_SECONDS = 5;
 private static final Integer EXPIRE_PRODUCT_AFTER_1_DAY = 1;
 private final LoadingCache<String, Optional<Product>> cache;

 public ProductCache() {
      cache = CacheBuilder.newBuilder()
           .expireAfterWrite(EXPIRE_PRODUCT_AFTER_1_DAY, TimeUnit.DAYS)
           .refreshAfterWrite(REFRESH_PRODUCT_AFTER_5_SECONDS, TimeUnit.SECONDS)
           .build( new CacheLoader<String, Optional<Product>>() {
                 @Override
                 public Optional<Product> load( String productId ) throws Exception {
                     return loadCache(productId);
                 }
           }
     );
 }

 public Optional<Product> getEntry( String productId ) {
      return cache.getUnchecked( productId );
 }

 private Optional<Product> loadCache(String productId) {
      Product product = databaseDAO.getProduct(productId);
      return Optional.fromNullable(product);
 }
}

Explanation

  1. In the constructor, we build the cache using the CacheLoader, defining the refresh mechanism. In our example we define 2 rules :
    - expireAfterWrite : after this period, the object will be evicted from the cache, and replaced the next time it is requested.
    - refreshAfterWrite : after this period, the object will be refreshed using the loadCache method. (with our new price)
  2. getEntry(String productId) method : will return the object with given key. So in this example, the cache is not loaded all at once, but only when the object is needed.
  3. loadCache(String productId) : will load the product and add it to the cache, or replace it if it’s already there and needs to be refreshed.

That’s all there is to it !

A few other remarks on the code

  1. There are other mechanism like expire/refresh AfterRead, which will time only from the last read, or let the cache hold only a certain no. of objects,…
  2. This code is implemented as a session bean. To make a singleton, I’m using EJB 3 annotation @Singleton. Because I only want 1 cache in my application
  3. My DAO is also injected using the @EJB annotation
  4. The LoadingCache does not want any null objects in the map (returns an error), so I’m using the Guava ‘Optional’ class here. This is basically a wrapper for my object and used to check if there is a value for my product id or not. So if someone uses a wrong productId, my cache will indicate that there is no product for this id, and I don’t have to go to the database every time it is requested.

To conclude:

Programming a caching mechanism in a JEE environment is not as trivial as it may seem. Testing it in a multithreaded environment is even more difficult. The caching classes of Guave gives you ready-to-use solution. It’s programmed, tested and used by Google, so I think we can say in all honesty : this is proven technology.

A remark on deploying on Weblogic 12c:

Weblogic also uses the Guava libraries, but an older version. This causes following error on deployment :

java.lang.NoSuchMethodError: com.google.common.util.concurrent.MoreExecutors.sameThreadExecutor()

Lcom/google/common/util/concurrent/ListeningExecutorService;

Adding the following to your weblogic-application.xml will solve the problem (force weblogic to use your deployed Guava libraries :

<wls:prefer-application-packages>
<wls:package-name>
com.google.common.*
</wls:package-name>
<wls:prefer-application-packages>

Guava libraries run under Apache license, more info/download can be found on :

https://code.google.com/p/guava-libraries/

Have fun !

Using ADF Logging in a non-ADF project

In a previous post (Starting with ADF 11G Logging), I explained how ADF logging is simple to set up, and how it will enable you to set the logging levels at runtime, without having to restart any server. When I showed this to a colleague of mine, he immedialtely popped the question : “Can’t we use this for all of our java applications, even the ones that don’t use ADF?”. Well, the answer is yes, and it turns out to be very easy. Just add the correct jar to your project and your done.

This blog will demonstrate how to get this working. I use Eclipse Juno to create a small webproject, only containing a servlet that does the logging. In fact I will use the same servlet I used in the previous post.

So I open my Eclipse , and started with a File -> New -> Dynamic Web project. Give it a name, set ‘Dynamic web module version’ to 2.5, click the  ‘Add project to an ear’ checkbox and click finish.

dyn_wb_prj

Now Eclipse has created a web and ear module for me.

Image

Now right click the web project (ADFLogging), and select New -> Servlet, give it a name, eg. TestServlet, and click finish.

Remove the generated code in the servlet, and copy the code from the servlet ‘ExecuteLogger’ from my previous post (here) and paste it in our new serlvet.

PS. : When you copied the code from my previous blog, don’t forget to set ADFLogger.createADFLogger to our current servlet class name : TestServlet.class.

We will get compile errors on HttpServletRequest,etc… and on the ADFLogger class because they are not defined in the classpath of the project. So we’ll add them in order to get our servlet compiled.  I get the 2 jar’s from a JDeveloper installation I did on my machine. We’ll only add these jar’s in order to get the servlet compiled in Eclipse. We will NOT deploy them, as they are already available on our Weblogic server.

To add the jar’s, right click on the web project, and go to Properties. In the Properties, click on ‘Java Build Path’.

buildpath

Click on ‘Add External JARs…’ , and go to the directory where you installed your JDeveloper, which in my case is : C:\Oracle\Middleware.

In that directory , get following jar’s from the sub-directory :

\oracle_common\modules\javax.servlet_1.0.0.0_2-5.jar : contains the servlet classes like HttpServletRequest/Response,etc…

\oracle_common\modules\oracle.adf.share.ca_11.1.1\adf-share-base.jar : contains the ADFLogger classes.

Now we see the the following jar’s added :

jars_added

Click OK and return to the servlet. In the servlet use CTRL-SHIFT-O to import the neccessary classes from the jar’s we just added.

Now all compile errors should be gone.

Generate the ear file as follow : File -> Export -> Ear file

Select the ear project and enter destination of the ear file

When you examine the ear, you will notice that the folder \WEB-INF\lib is empty.

As the servlet and ADFLogger jar is already available on Weblogic, there is no need to deploy it with our application.

Now deploy the ear to the Weblogic and test the servlet with following url. :

http://localhost:7101/ADFLogging/TestServlet

It will generate following output :

output

To check the logging done by this servlet :

As I used the integrated Weblogic of JDeveloper, I will look for my logs using JDeveloper, but in a production environment,

these logs can be viewed using the enterprise manager of Weblogic. For details, see my previous blog.

In the Oracle Diagnostics Logging configuration, I see my servlet after the deployment. No message level is defined, so it will take “Warning”, as this one is defined as default by the Root Logger

logger

After te execution, I see following log lines in the log analyzer.

result

So that’s it. So the bottom line is to add the ADFLogger jar to your non-ADF project, and you are ready to go !

Starting with ADF 11G Logging

In software development, logging is an indispensible part of the job. When developing  java programs, Log4j (Apache framework) is probably the most commonly used framework.

But when we are writing Java programs, using the Oracle ADF framework, there is another option : ADF Logger. This logger is integrated in the Weblogic enterprise manager, and gives you the flexibility to adjust your log-levels at runtime. This blog post briefly demonstrates how the ADF logging works, using a servlet that logs at all levels.

The first example is run on a remote Weblogic server. At the end of the post the same example is run within the integrated Weblogic server of JDeveloper.

First we start with viewing the loggers defined in Oracle Enterprise Manager (OEM) before our servlet is deployed :

em_go_to_logconfig_2

When we select ‘Log Configuration’ we get the Log Configuration screen, where some loggers are already defined and their loglevels can be configured :

em_initieel_2

We’ll deploy a servlet that uses the ADF Logger, change the loglevels at runtime, and check the logging to see what happens.

So fire up your JDeveloper and select a new Fusion Web Application (ADF) or download the JDeveloper project from  https://www.dropbox.com/s/mwkjdw8k265iadx/LoggerApp.zip

Then create a servlet like the one below, which is a simple servlet called ‘ExecuteLogger’ that logs a message on all loglevels :


package be.iadvise.loggerapp;

 import java.io.IOException;
 import java.io.PrintWriter;

 import java.util.Calendar;

 import javax.servlet.*;
 import javax.servlet.http.*;

 import oracle.adf.share.logging.ADFLogger;

 public class ExecuteLogger extends HttpServlet {
  private static final String CONTENT_TYPE = "text/html; charset=UTF-8";

 private static ADFLogger _log = ADFLogger.createADFLogger(ExecuteLogger.class);

  public void init(ServletConfig config) throws ServletException {
  super.init(config);
  }

 public void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
  response.setContentType(CONTENT_TYPE);
  PrintWriter out = response.getWriter();
  out.println("<html>");
  out.println("<head><title>Executing logging with ADFLogging and System.out.println</title></head>");

 Calendar cal = Calendar.getInstance();
  String loggingId = Long.toString(cal.getTimeInMillis());
  out.println("<h1>Executing logging with ADFLogging and System.out.println</h1>");
  out.println("<h2>Used Logging id = "+loggingId+"</h2>");
  out.println("<p>The logging id is different for each request.</br>Use the logging id to search the log messages in the logfiles or EM for this request.");
  out.println("<p>Logger Name in EM = "+_log.getName());

  out.println("<p>****************** START LOGGING ******************");

  out.println("<table >");

 // FINEST
  _log.finest("Loglevel finest : This is a logmessage with the ADFLogger. Logging id = "+loggingId);
  System.out.println("Loglevel finest : This is a logmessage with the System.out.println. Logging id = "+loggingId);
  out.println("<tr><td>FINEST</td><td>Loglevel finest : logMessage written to ADFLogger and System.out.println. Logging id = "+loggingId+"</td></tr>");

  // FINER
  _log.finer("Loglevel finer : This is a logmessage with the ADFLogger. Logging id = "+loggingId);
  System.out.println("Loglevel finer : This is a logmessage with the System.out.println. Logging id = "+loggingId);
  out.println("<tr><td>FINER</td><td>Loglevel finer : logMessage written to ADFLogger and System.out.println. Logging id = "+loggingId+"</td></tr>");

  // FINE
  _log.fine("Loglevel fine : This is a logmessage with the ADFLogger. Logging id = "+loggingId);
  System.out.println("Loglevel fine : This is a logmessage with the System.out.println. Logging id = "+loggingId);
  out.println("<tr><td>FINE</td><td>Loglevel fine : logMessage written to ADFLogger and System.out.println. Logging id = "+loggingId+"</td></tr>");

  // CONFIG
  _log.config("Loglevel config : This is a logmessage with the ADFLogger. Logging id = "+loggingId);
  System.out.println("Loglevel config : This is a logmessage with the System.out.println. Logging id = "+loggingId);
  out.println("<tr><td>CONFIG</td><td>Loglevel config : logMessage written to ADFLogger and System.out.println. Logging id = "+loggingId+"</td></tr>");

  // INFO
  _log.info("Loglevel info : This is a logmessage with the ADFLogger. Logging id = "+loggingId);
  System.out.println("Loglevel info : This is a logmessage with the System.out.println. Logging id = "+loggingId);
  out.println("<tr><td>INFO</td><td>Loglevel info : logMessage written to ADFLogger and System.out.println. Logging id = "+loggingId+"</td></tr>");

  // WARNING
  _log.warning("Loglevel warning : This is a logmessage with the ADFLogger. Logging id = "+loggingId);
  System.out.println("Loglevel warning : This is a logmessage with the System.out.println. Logging id = "+loggingId);
  out.println("<tr><td>WARNING</td><td>Loglevel warning : logMessage written to ADFLogger and System.out.println. Logging id = "+loggingId+"</td></tr>");

  // SEVERE
  _log.severe("Loglevel severe : This is a logmessage with the ADFLogger. Logging id = "+loggingId);
  System.out.println("Loglevel severe : This is a logmessage with the System.out.println. Logging id = "+loggingId);
  out.println("<tr><td>SEVERE</td><td>Loglevel severe : logMessage written to ADFLogger and System.out.println. Logging id = "+loggingId+"</td></tr>");

  out.println("</table>");

  out.println("<p>****************** END LOGGING ******************");

  out.println("<body>");
  out.println("</body></html>");
  out.close();
  }
 }
 

Then, map this servlet in your web.xml as follows :

<servlet>
 <servlet-name>ExecuteLogging</servlet-name>
 <servlet-class>be.iadvise.loggerapp.ExecuteLogger</servlet-class>
 </servlet>
 <servlet-mapping>
 <servlet-name>ExecuteLogging</servlet-name>
 <url-pattern>/executeLogging</url-pattern>
 </servlet-mapping>

The line

private static ADFLogger _log = ADFLogger.createADFLogger(ExecuteLogger.class);

will create an entry in the ADF logging panel of OEM  for the class ‘be.iadvise.loggerapp.ExecuteLogger’ during the first execution of our servlet.

So generate the ear file (in JDeveloper : Application -> Deploy -> your application) , and deploy to an ear , and deploy the ear to your remote Weblogic.

Then, execute the servlet by entering following url in a browser http://server:port/appname/executeLogger

We will receive the following output where the logging id is a unique id for every request (this will help us find the log info logged for every run) :

output_servlet_3

and in our log configuration screen, the logger for our servlet is added automatically with level WARNING :

EM_na_uitvoeren_servlet_2

When we look to the logging itself, by selecting the following :

go_to_logmessages

We see that our servlet has logged 2 lines  : level WARNING and ERROR, as the logger was default created with level “warning”.

(We look in the log files for the id generated by the servlet to get our 2 lines)

output_first_log_2

Now let’s update our log level to TRACE (finest), press apply, confirm the update, execute the servlet again , and see what happens to our logging :

trace

confirm the update :

confirm

execute the servlet again (it generates a new logging id)

new_servlet

look for the generated logging :

new_logging

As we have set the loglevel to trace (finest), which is the lowest level, we see all the generated loglines.

Remark : When we undeploy the application, the logger will remain visible in the log configuration screen until the managed server is restarted.

Using the integrated Weblogic server in JDeveloper

This screen shot shows you how get to the log configuration and logging screen in JDeveloper :

1_go_log_screen

After executing the servlet locally, the Oracle Diagnostic Logging will show you the following :

2_log_config

and the analyze log :

3_logging_first_run

then we change the log level :

4_change_log_level

and execute the servlet again and watch the output :

5_output_logging_2

That’s it !!!

The big advantage of ADFLogging is the update of the loglevel at runtime, so whenever something seems to be going wrong, just set the loglevel to e.g. TRACE and check the logfiles. When the problem is solved reset it to WARNING or ERROR. And this without having to restart your application.

Also, being able to view and search the logfiles using the EM increases the userfriendlyness of this system. You don’t have to access the files directly on the system anymore.