Functional programming a brief introduction to Scala

In this blogpost I’ll try to explain on a very basic level what functional programming (FP) is about. look at is as an introduction to the amazing world of FP. FP is already here for quite a few decades. It was mostly used in the academic world and specialized industries. Since the coming of Scala http://www.scala-lang.org, FP became more and more mainstream.

So what is functional programming?

(from Wikipedia http://en.wikipedia.org/wiki/Functional_programming) In computer sciencefunctional programming is a programming paradigm, a style of building the structure and elements of computer programs, that treats computation as the evaluation of mathematical functions and avoids changing-state and mutable data. It is a declarative programming paradigm, which means programming is done with expressions. In functional code, the output value of a function depends only on the arguments that are input to the function, so calling a function f twice with the same value for an argument x will produce the same resultf(x) each time. Eliminating side effects, i.e. changes in state that do not depend on the function inputs, can make it much easier to understand and predict the behavior of a program, which is one of the key motivations for the development of functional programming.

Installing Scala

I’m using the scala Read-Evaluate-Print-Loop (REPL) to show you the examples. It’s basically a Scala commandline interpreter. Just get the latest version of scala and install it. To get started with scala REPL, open a command prompt or terminal session; go to the directory where you installed scala and further to the bin directory. Enter the command scala or ./scala and you’re ready to go!

Read-Evaluate-Print-Loop

Scala Command line REPL

Scala

Scala is a programming language that mixes paradigms of both FP and OO. Scala uses its own compiler (scalac) to compile your code to byte code. Probably the biggest advantage of Scala is that it runs on your standard java infrastructure. It is even capable of calling your java code and visa versa although you need to take special care when integrating both languages. It’s functional As the name already states, functional programming allows the developer write your code on a functional level. In other words you don’t have to translate your functional solution to a lower, technical, level. If I want to know the outcome of: all even numbers between 1 and 10 and multiplied them by 2, you write in Scala

(1 to 10).toList.filter(_ % 2 == 0).map(_ * 2)

While in java it looks more like this

List input = {1,2,3,4,5,6,7,8,9,10}.toList();
List results = new ArrayList();
for(int val : input) {
  if(val % 2 == 0) {
    results.add(val * 2);
  }
}
return results;

Looking at the Scala code you will notice a few things.

  • The code is expressive and readable. No clutter, no unnecessary boiler plate code.
  • There is no assignment of variables (stateless)
  • Functions (_ % 2) and (_ * 2) is added as argument to a method
    • The underscore (_) represents a wildcard. In this case, each element of the list

Expressive

Scala, as all functional languages, is an expressive language. But what does it mean? First of all, it makes your code much more readable. You almost program what you say. A friend of mine, who’s working in the travel business, understood a Scala code snippet, while the java counterpart was Chinese for her. Is this important? Yes, if you know that code is read 10 times more than written.
Secondly expressions return values. Which can lead to less code.

Boilerplate code be gone!

The guys from Scala did their best to make the code as clean as possible. Every character unnecessary to run your code can be made obsolete. This includes ( ) for zero argument method calls, semi colons ‘;’, points ‘.’ etc… The scala compiler is quite intelligent and will do all the hard work for you to convert the dense code into the full-blown byte code.

Values and variables

var myName = “Ief”
myName: String = Ief

As you can see, it is not mandatory to define a type. Still Scala is strongly typed and knows that it is a String. Strongly typed means that you don’t need to define types a lot. Confused?

val myName = “Ief”

is the same as

 val myName : String = “Ief” 

Once the type of a variable or value is assigned you don’t need to repeat it again. Val represent a value, a final variable named ‘name’ of the type Sting with only an accesor method (getter).

var myName = “Ief”

Is a variable named name of the type String with an accesor method (getter) and a mutator method (setter)

Classes

A class Person with a variable name and a value surname would look like this in Scala

 class Person(var name : String, val surname : String) 

While the java counterpart would look like this

public class Person {
  private String name;
  private final String surname;

  public Person(final String surname) {
    this.surname = surname;
  }

  public String getName();
    return name;
  }

  public void setName(final String name) {
    this.name = name;
  }

  public String getSurname();
    return surname;
  }
}

You can instantiate the class using the following code

val me = new Person(“Ief”, “Cuynen”)
me: Person = Person@4d826d77

me.name
res7: String = Ief

me.name = “Jef”
me.name: String = Jef

me.surname
res8: String = Cuynen

me.surname = "Peeters"
res9: error: reassignment to val

You probably noticed that for zero argument methods you can skip the ( ) Scala is packed with syntactic sugar. A few more examples:

Regular expressions can be created just by calling the r method on a String

val regularExpression = “[0-9][a-Z]”.r

A Range of numbers from 1 to 10

1 to 10
res11: scala.collection.immutable.Range.Inclusive = Range(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)

Functions

Functions are defined using the def keyword. Every function returns the value of the last statement of the function, even without an explicit return statement. It is even possible to return a new function as return value.

def addOne(x: Int): Int = x +1
addOne: (x: Int)Int

addOne(2)
res12: Int = 3

In scala it is also possible to return multiple values in 1 go by using tuples. Tuples are a way to group different values of different types into a single container without creating an explicit class.

def printStuff(x : Int): (Int, String)  = (x, "Hello number " + x)
printStuff: (x: Int)(Int, String)

printStuff(1)
res13: (Int, String) = (1,Hello number 1)

Functions can be used as arguments for other functions. This way it is possible to add behaviour.

def isEven(x : Int): Boolean =  x % 2 == 0
def isUneven(x : Int): Boolean = x % 2 == 1

(1 to 10).filter(isEven(_))
res20: scala.collection.immutable.IndexedSeq[Int] = Vector(2, 4, 6, 8, 10)

(1 to 10).filter(isUneven(_))
res21: scala.collection.immutable.IndexedSeq[Int] = Vector(1, 3, 5, 7, 9)

Using functions as arguments greatly reduce the need for subclassing. Normally a subclass is used to ‘bind’ different behaviour to the same interface. By using functions as arguments, the behaviour is injected into interface.

Functions can be even combined into other functions which are called high order functions.

Option

Using null values is FP and Scala is a very bad practice. When you think about it, null values are pure evil in any programming for that matter. They cause unwanted side effects and you need to always check for null values to prevent NullPointerExceptions. Within Scala you can use the Option counterpart which is a decent way to solve the null evilness.

scala> def convertToInt(x: String): Option[Int] = {
     |    try {
     |        Some(Integer.parseInt(x))
     |     } catch {
     |        case e: Exception => None
     |    }
     | }
convertToInt: (x: String)Option[Int]

This function parses a String to an Int. In case the String can be successfully parsed, the Int value is returned (Some). In case an exception is thrown, None is returned

scala> convertToInt("1")
res22: Option[Int] = Some(1)
scala> convertToInt("abc")
res23: Option[Int] = None

To get the actual value, the getOrElse function can be called on the Option.

scala> val x = convertToInt("1").getOrElse(0)
x: Int = 1
scala> val x = convertToInt("abc").getOrElse(0)
x: Int = 0

Another way to get the value of the Option is to use a matcher.

scala> convertToInt("1") match {
     |     case Some(x) => println(x)
     |     case None => println("That String was not really an Int was it?")
     | }
1

scala> convertToInt("abc") match {
     |     case Some(x) => println(x)
     |     case None => println("That String was not really an Int was it?")
     | }
That String was not really an Int was it?

Pattern matching

You already saw above what you can do with a matcher. Look at it as the Java switch on steroids. It can match about any everything.

scala> def matchThis(x : Any) = x match {
     |     case 1 => println(1)
     |     case s : String => println("Match String " + s)
     |     case _ => println("whatever")
     | }
matchThis: (x: Any)Unit

scala> matchThis(1)
1

scala> matchThis("abc")
Match String abc

scala> matchThis(true)
whatever

Caveats

One of the caveats is that the java compiler does not compile scala and the scala compiler does not compile java. So 2-way dependencies probably lead to a big headache.

Java Object Cache – The Future

You will all encounter, or maybe already have encountered, at some point in time, applications that perform bad. You check the code, or better, profile it. After removing the Thread.sleep(1000) calls :-) there are still cases where the code runs too slow. The reasons can be various, from bad architecture to large object creation times and external systems with slow response times.
If the architecture is bad, you have a problem, in all other cases you can think about implementing a caching mechanism.
There are various cache libaries available or you can even write your own. It all depends on the requirements for the cache.
As an Oracle minded company, we use the Oracle caching products.

Oracle provides a caching library called the Java Object Cache (JOC). It is fairly easy to implement and supports distributed caching across multiple JVMs. JOC is a part of the OC4J and thus the Oracle Application Server, so you can use it for ‘free’ if you have licensed the app server.

Since the launch of Oracle Fusion Middleware 11G I wondered if Oracle would still supports the JOC because:
-) The current version of JOC is 10.1.2, no 11G version in sight.
-) It is embedded in OC4J but OC4J is replaced by Weblogic.
-) Oracles strategic caching product is Coherence.

So I did a little research. Indeed Coherence is the way to go. Coherence has all the features of JOC and many more. But imho Coherence true power unleashes when using it as an enterprise grid cache. And yes, there is the standard edition without the grid capabilities, but still, JOC is a better choice if you want a no nonsense, simple to use, embedded in your application, I can’t afford Coherence, cache.

Back to my initial question. Is JOC still supported? Actually it does. Webcenter 11G uses JOC in the Webcenter Spaces component. Maybe, Oracle will release an 11G version…you never know.

References
JOC home page
Coherence
JOC in Webcenter 11G
More about Oracle caching mechanisms (2005)

Java & Memory in an iAS environment

Everyone who worked with java applications probably heard of the OutOfMemory error. Like the name says, it indicates that the java application reached the limit of available memory and tries to go beyond that limit, with the OutOfMemory as a logical consequence. But why does this error occur? Your 8 CPU, 20GB server has enough resources to serve 50 of these applications simultaneously and still, the error pops-up now and then, crashing your entire application.


Before you add another set of memory modules, let’s take a quick look at the JVM internal memory management.

Automatic Memory Management
Java uses automatic instead of explicit memory management. This means that the developer is only response for the allocation of memory (Object x = new Object();), and the JVM, more specific, the garbage collector automatically cleans up unused objects.

So it is impossible to have memory leaks in my application? Well, not quite, it is not as obvious as in programming languages with explicit memory management but you still have to free your allocated resources in a proper way (ref. DB connections, Streams, …)

The Heap
The memory allocated by the JVM is called the Heap. The heap itself is divided into 3 areas also called ‘generations’.

  • young generation
  • old generation
  • permanent generation

Most objects are initially allocated in the young generation. The old generation contains objects that have survived some number of young generation collections, as well as some large objects that may be allocated directly in the old generation.
The permanent generation holds objects that the JVM finds convenient to have the garbage collector manage, such as objects describing classes and methods, as well as the classes and methods themselves.

The young generation consists of an area called Eden plus two smaller survivor spaces.
Most objects are initially allocated in Eden. (As mentioned, a few large objects may be allocated directly in the old generation.)
The survivor spaces hold objects that have survived at least one young generation collection and have thus been given additional chances to die before being considered “old enough” to be promoted to the old generation. At any given time, one of the survivor spaces holds such objects,
while the other is empty and remains unused until the next collection.

Now that we know that the young generation contains short living objects and the permanent generation contains (very)long living objects
it is easier to pinpoint the OutOfMemory errors.

iAS & memory
When OutOfMemory problems occur in your newly deployed and bugfree application :-) , you can use the Oracle Enterprise Manager (OEM) web interface
to get a first impression about the memory usage of the application.

In the system components view (The view where all the OC4J instances are listed) you can see the Memory usage in MB per OC4J.
The view also shows the total amount of memory in the server as a pie chart at the top of the screen. The pie chart is divided into 3 parts.

  • Application Server Memory Usage
  • Free Memory
  • Other Memory Usage

Be very careful while interpreting the pie chart. Unix and Linux allocate free memory in a buffer. When an application needs extra memory,
the OS can free a part of the allocated buffer and give it to the application.
Due to the allocated buffer, the pie chart shows that almost all of the system memory is allocated in the ‘Other Memory Usage’

On the other hand, there are the log files where OutOfMemory errors are logged.

  • application.log located in the %ORACLE_HOME%/j2ee/%OC4J%/application-deployments/%APPLICATION% directory.
  • Standard output log named OC4J~\{OC4J\}~\{island\}~\{jvm process\}.log (eg. OC4J~OC4J_DEMOS~default_island~1.log) located in the %ORACLE_HOME%/opmn/logs directory

Be sure to check logging just before the OutOfMemory for more details.

Solving the memory problem
The simplest way to solve OutOfMemory errors is to increase the size of the heap. To do this, you can add command line parameters to the java command.

java -Xms256m -Xmx512m

Where -Xms defines the minimum and -Xmx the maximum amount of memory allocated by the JVM.

In the application server you can alter this in the OC4J instance -> server properties -> command line options -> java options field.
Note that the properties affect all OC4J instances in the cluster for that specific OC4J.

Sometimes this does not resolve the problem. In some cases an OutOfMemory error occurs while there is still lots of memory free for the JVM.
Check standard output log files for the following message.

Permanent generation is full…
increase MaxPermSize (current capacity is set to: 67108864 bytes)

As we have seen before, the permanent generation contains long living objects. By default, the JVM allocates 64MB to the permanent generation.
For the most application this will suffice. The size permanent generation must be set apart from the heap size. To do this use the command line:

java -XX:PermSize=64m -XX:MaxPermSize=256m

Where -XX:PermSize=xxx defines the minimum and -XX:MaxPermSize=xxx the maximum amount of memory allocated for the permanent generation.

Always make sure that the -server option is added to your java options. This setting will preconfigure your JVM to run as optimal as possible in a server environment.

If these settings do not work, you should check you application for memory leaks. You can use a profile to detect these leaks… but that will be explained in a future post

References
Memory Management in the Java HotSpot Virtual Machine (pdf)
Oracle® Application Server 10g Performance Guide

Oracle Audit Vault

A while ago we took the opportunity to participate in the beta testing of a new product named Oracle Audit Vault.

The official product description sounds like this:
“Oracle Audit Vault provides a solution to help customers address the most difficult security problems remaining today, protecting against the insider threat and meeting regulatory compliance requirements. Oracle Audit Vault will help businesses secure sensitive audit data, protect corporate reputation and meet regulatory compliance guidelines. Oracle Audit Vault is a product that enables collection, monitoring, storage, verification and reporting of audit data from various systems such as databases, application servers, applications, and operating systems.”

We are going to focus on the integration with the oracle application server. Now, we are waiting for the download, which will be available mid-august.

(to be continued)