Sunday, March 6, 2011

Do we really need <> in the diamond operator?

As you may know, one of new features of upcoming Java 7 will be the diamond operator. Purpose of the diamond operator is to simplify instantiation of generic classes. For example, instead of

List<Integer> p = new ArrayList<Integer>();

with the diamond operator we can write only

List<Integer> p = new ArrayList<>();

and let compiler infer the value of type argument. Nice simplification. But do we really need to write <>? Isn't new ArrayList() enough? In this article, I will describe the arguments of the <> proponents and explain why I think that these arguments are not very strong. However, I also describe arguments why we need <>.

In Java 1.4, we had raw types only:

List p = new ArrayList();

Java 5 introduced generics:

List<Integer> p = new ArrayList<Integer>();

Many types in Java API were generified and even though we can still use generic types as raw types, there is no reason for this in Java 5 or newer. When generics were introduced, raw types were allowed for backward compatibility so that we could gradually and smoothly adopt generics. For example, code in Java 1.4 can be combined with new generic code because raw and generic types are allowed together. This is also expressed in the JLS (4.8 Raw Types):

"The use of raw types is allowed only as a concession to compatibility of legacy code. The use of raw types in code written after the introduction of genericity into the Java programming language is strongly discouraged. It is possible that future versions of the Java programming language will disallow the use of raw types."

Now let's go back to the diamond operator and ask again: "Do we really need <>?". The proponents of the <> syntax say that we need <> to preserve backward compatibility. Let's look at an example from the coin-dev conference:

class Foo<X> {
   Foo(X x) { }
   Foo<X> get(X x) { return this; }
}

class Test {
   void test() {
      Foo<?> f1 = new Foo(1).get(""); //ok - can pass String where Object is expected
      Foo<?> f2 = new Foo<>(1).get(""); //fail - cannot pass String where Integer is expected
   }
}

This shows the difference between new Foo(1) and new Foo<>(1). Clearly, these two are different and if we changed the semantics of new Foo(1), it would break backward compatibility. But wait. Backward compatibility with what? Isn't line

Foo<?> f1 = new Foo(1).get("");

a little suspicious? It uses generic type in the left part and raw type in the right part. Although it is legal, it is probably either omission or malpractice. And its legality is probably only a side effect of "a concession to compatibility of legacy code".

Let's go further and look at another example from the coin-dev conference. It shows the difference between raw type and parameterized type with the diamond:

public class X<T> {
   public X(T t) { }
   public T get() { return null; }

   public static int f(String s) { return 1; }
   public static int f(Object o) { return 2; }

   public static void main(String[] args) {
      System.out.println(f(new X<>("").get()));
      System.out.println(f(new X("").get()));
   }
}

Let's play with the code a bit. Let's assume that there was a library with the X class:

public class X {
   public X(Object o) { }
   public Object get() { return null; }
}

and some code that compiled against this library:

public class Client {
   static int f(String s) { return 1; }
   static int f(Object o) { return 2; }

   public static void main(String[] args) {
      System.out.println(f(new X("").get()));
   }
}

Then, the library was generified:

public class X<T> {
   public X(T t) { }
   public T get() { return null; }
}

and we compiled the client project against the generified version. Now, if we changed the semantics of new X("") to new X<String>("") (or new X<>("") with the diamond syntax), the code would behave differently. So, the answer to the title question is 'yes'. If we want to stay backward compatible, we need <> and we cannot put new X("") semantically equal to new X<>("").

Other questions are how long can Java evolve and remain compatible with concessions to compatibility and whether newcomers to Java will appreciate this.

Tuesday, August 3, 2010

API evolution with RefactoringNG

See my article API evolution with RefactoringNG on dzone.

Friday, June 18, 2010

RefactoringNG

RefactoringNG is a flexible and powerful Java refactoring tool, implemented as NetBeans module. Refactoring rules are described as transformations of source abstract syntax trees to destination abstract syntax trees. For example, the rule that rewrites x = x + 1 to x++ is as follows:

 // x = x + 1 -> x++
 Assignment {
    Identifier [name: "x"],
    Binary [kind: PLUS] {
       Identifier [name: "x"],
       Literal [kind: INT_LITERAL, value: 1]
    }
 } ->
 Unary [kind: POSTFIX_INCREMENT] {
    Identifier [name: "x"]
 }
The names and structure of abstract syntax trees are the same as in Sun Java compiler. The tool uses Compiler Tree API (com.sun.*), the formal language model in the JDK API (javax.language.model.*), and NetBeans infrastructure. For example, if you have ever needed to replace the constructor call with a factory method or add an argument to each method call, you may appreciate how easily such tasks can be done with RefactoringNG:

Factory method

 // new Position(<args>) -> Position.create(<args>)
 NewClass {
    null,
    List<Tree>,
    Identifier [name: "Position"],
    List<Expression> [id: args],
    null
 } ->
 MethodInvocation {
    List<Tree> { },
    MemberSelect [identifier: "create"] {
       Identifier [name: "Position"]
    },
    List<Expression> [ref: args]
 }

Another argument

 // plus (<expr1>, <expr2>) -> plus (<expr1>, <expr2>, 5)
 MethodInvocation {
    List [size: 0],
    Identifier [name: "plus"],
    List<Expression> [id: args, size: 2]
 } ->
 MethodInvocation {
    List<Tree> [size: 0],
    Identifier [name: "plus"],
    List<Expression> {
       ListItems [ref: args],
       Literal [kind: INT_LITERAL, value: 5]
    }
 }
For more information, go to http://kenai.com/projects/refactoringng. For download, go to http://kenai.com/projects/refactoringng/downloads

Tuesday, August 19, 2008

Modifier 'for'

Modifier 'for'

Method modifier 'for' enables us to declare and call "loop-like" methods.

  static for void eachEntry(int[] values, { int ==> void } block) {
    for (int p : values) {
      block.invoke(p);
    }
  }
  
  public static void main(String[] args) {
    int[] v = { 2, 3, 5, 7, 11 };
    for eachEntry(int i : v) {
      System.out.println(i);
      if (i > 3) {
        break;
      }
    }
  }
  

For example, we can declare a method that iterates through a map:

  static for <K, V> void eachMapEntry(Map<K, V> dict, 
      { K, V ==> void } block) {
    for (Map.Entry<K, V> entry : dict.entrySet()) {
      block.invoke(entry.getKey(), entry.getValue());
    }
  }
  
  public static void main(String[] args) {
    Map<String, String> dict = new HashMap<String, String>();
    ...
    for eachMapEntry(String k, String v : dict) {
      System.out.println(k + ": " + v);
    }
  }
  

Nonlocal transfer

Nonlocal transfer

In unrestricted closures, we can use the break, continue, and return statements.

  for (int i = 0; i < 5; i++) {
    System.out.println(i);
    if (i == 2) { ==> break; }.invoke();
  }
  

These statements are always lexically bound, i.e. they are bound in the context in which the closure is declared (as opposite to invoked). So, the following code will print numbers from <1,10> that are not multiples of 3.

  static void m(int i, { int => boolean } cond, { ==> void } block) {
    if (cond.invoke(i)) {
      block.invoke();
    }
  }
  
  public static void main(String[] args) {
    for (int i = 1; i < 10; i++) {
      // continue is bound to the for loop
      m(i, { int i => i % 3 == 0 }, { ==> continue; });
      System.out.println(i);
    }
  }
  

The return statement returns from the bound method.

  static int m1() {
    // return here means "return from m1"
    m2({ double d ==> if (d < 0.5) return 1; });
    return 0;
  }
  
  static void m2({ double ==> void } p) {
    // the closure invocation can cause return from m1
    p.invoke(Math.random());
  }
  

If the bound method is not active at the time when we call the closure with the return statement, we will get the UnmatchedTransfer exception. For example:

  // any type can be used as the return type of p in this
  // example because the closure always returns earlier
  // so, p can be declared also as static { ==> int } p;
  static { ==> boolean } p;
  
  static int m1() {
    p = { ==> return 1; };
    return 0;
  }
  
  public static void main(String[] args) {
    System.out.println(m1());
    System.out.println(p.invoke());
  }
  

Unrestricted closures are useful when declaring "statement-like" methods.

  static void withLock(Lock lock, { ==> void } block) {
    try {
      lock.lock();
      block.invoke();
    } finally {
      lock.unlock();
    }
  }
  
  public static void main(String[] args) {
    Lock guard = new ReentrantLock();
    double d = Math.random();
    withLock(guard) {
      if (d < 0.5) {
        return;
      }
      ...
    }
  }
  

Saturday, March 29, 2008

Method references (Implementation Issue)

Method references (Implementation Issue)

Each method reference is transformed to a closure:

  // Math#sqrt(double) will be transformed to { double d => Math.sqrt(d) }
  { double => double } sqrt = Math#sqrt(double);
  

Instance method references are transformed in a similar way.

  Circle c = new Circle();
  { => void } p = c#draw();  // the same as { => c.draw(); }
  
  { Circle => void } p = Circle#draw();  // { Circle c => c.draw(); }
  

Let us have a look at another example. We declare class Box:

  public class Box<T> {
    T v;
    public Box(T v) {
      this.v = v;
    }
    public T getValue() {
      return v;
    }
    public void print() {
      System.out.println("Box: " + v);
    }
  }
  

and refer to the print method:

  // Box#print() will be transformed to { Box<Integer> box => box.print(); }
  { Box<Integer> => void } p = Box#print();
  

Reference to a generic method will be replaced by a closure as follows:

  public class Generic {
    public static <T> Box<T> copy(Box<T> b) {
        return new Box<T>(b.getValue());
    }
  }
  
  // Generic#copy(Box<Integer>) will be transformed to 
  // { Box<Integer> box => Generic.copy(box) }
  { Box<Integer> => Box<Integer> } boxCopy = Generic#copy(Box<Integer>);
  

I do not want closures in Java because they are not simple.
This argument is wrong. We should always weigh all pros and cons. Closures are not simple. But is software development in Java simple? What should I know to be capable to develop software in Java? Knowledge of the Java programming language is clearly not enough. One should also know some Java technologies and frameworks. And are they simple? Are JPA, EJB, JAX-WS, Hibernate, or Spring (to name a few) simple? Apparently no, so it does not make sense to reject closures just for this reason.

Thursday, March 20, 2008

Method References (version 2008-03-17)

Method references (version 2008-03-17)

The compiler prototype (available at http://www.javac.info) comes with method references (also known as eta expansion). The method references are written as follows: ClassName # methodName ( listOfArgumentTypes ).

A method reference can be assigned to a function variable:

  { String => int } parseInt = Integer#parseInt(String);
  int x = parseInt.invoke("42");
  

We can use the covariant return:

  // Integer.valueOf() returns Integer
  { int => Number } p = Integer#valueOf(int);
  System.out.println(p.invoke(97));
  

And the contravariant arguments:

  class MyClass {
    static Integer print(Object o) {
      return Integer.valueOf(o.hashCode());
    }
    
    public static void main(String[] args) {
      { String => Number } pp = MyClass#print(Object);
      System.out.println(pp.invoke("hi"));
    }
  }
  

An instance method can be referenced in two ways. Either we reference a method on a given object:

  class Box {
    private int x;
    Box(int x) {
        this.x = x;
    }
    int getX() {
        return x;
    }
  }
  public class InstanceMethod {
    public static void main(String[] args) {
      Box p = new Box(10);
      { => int } getX = p#getX();
      System.out.println(getX.invoke());
    }
  }  
  

Or we reference just a method. Then the function type has an additional argument: the object on which the method is called.

  { Box => int } getX = Box#getX();  // additional argument of type Box
  Box p = new Box(10);
  System.out.println(getX.invoke(p));
  

The method is selected at runtime according to the object supplied as argument. So, in the following example, the toString method from String is called.

  { Object => String } toString = Object#toString();
  System.out.println(toString.invoke("hi"));
  

Generic methods are also supported:

  { String => Set<String> } singleton =
      Collections#<String>singleton(String);
  Set<String> set = singleton.invoke("single");
  

Monday, February 25, 2008

Control Abstraction (version 2008-02-22)

Control Abstraction (version 2008-02-22)

In this post, I will describe new features of version 2008-02-22 of the compiler prototype. It can be downloaded from http://www.javac.info.

1. java.lang.Unreachable was renamed to java.lang.Nothing.

2. Annotation @Shared was added. It may annotate a variable bound in a closure. For example:

  static void doTwice({ => void } block) {
    block.invoke();
    block.invoke();
  }
  public static void main(String[] args) {
    @Shared int x = 10;
    doTwice({ => System.out.println(x++); });
  }
  

We should always annotate bound variables with this annotation because bound variables are very different from ordinary local variables.

3. Closures can use recursion. Now we can use the function variable within a closure that is used to initialize this variable.

  { int => int } sum = { int n => n == 1 ? 1 : n + sum.invoke(n - 1) };
  

4. Unrestricted closures were partially implemented. In an urestricted closure we can use the return, break, and continue statements. Unrestricted closures use ==> instead of =>.

  { int ==> int } neg = { int x ==> -x };
  

5. A method that accepts an unrestricted closure as the last argument can be called using the control invocation syntax:

  public static void simple({ ==> void } block) {
    block.invoke();
    block.invoke();
  }
  public static void main(String[] args) {
    simple() {
      System.out.println("vrr!");
    }
  }
  

The method may have more than one parameter:

  public static void profile(String s, { ==> void } block) {
    System.out.printf(">>> %s: %s%n", s, System.nanoTime());
    block.invoke();
    System.out.printf("<<< %s: %s%n", s, System.nanoTime());
  }
  public static void main(String[] args) {
    @Shared long f = 1;
    profile("factorial") {
      for (int i = 2; i <= 59; i++) {
        f *= i;
      }
    }
  }
  

The closure used as the last argument can have parameters. They are declared before arguments of the method and are separated from them by colon (although the syntax is similar to the enhanced for loop, the colon here has another meaning):

  public static void control(int x, { int ==> void } block) {
    block.invoke(x);
  }
  public static void main(String[] args) {
    // ordinary syntax
    control(1, { int i ==> System.out.println(i); });
    // control invocation syntax
    control(int i : 2) {
      System.out.println(i);
    }
  }
  

If the closure has more parameters, they are separated by comma:

  public static void controlTwo(int x, { int, int ==> void } block) {
    block.invoke(x, x + 1);
    block.invoke(x + 2, x);
  }
  public static void main(String[] args) {
    // closure parameters are separated by comma
    controlTwo(int i, int j : 2) {
      System.out.println(i * j);
    }
  }
  

This enables us to define new "control statements" without modifying the language. For example, we can add "statement" with that closes a given closeable at the end (the code is derived from the prototype sources):

  public static <T extends Closeable> void with(T t, { T ==> void } block) {
    try {
      block.invoke(t);
    } finally {
      try {
        t.close();
      } catch (IOException e) {
        // ignore exception
      }
    }
  }
  public static void main(String[] args) {
    // class MyCloseable implements Closeable
    with (MyCloseable in : new MyCloseable()) {
      //...
    }
  }
  

Why Java cannot remain simple? Do we need new control statements?
Many people like simplicity (I am one of them). But Java must evolve if we do not want it to become outdated. In addition, the hardware is changing and these changes will probably change programming languages used in software development. For information how the hardware is changing, I recommend the article The Free Lunch Is Over: A Fundamental Turn Toward Concurrency in Software. If you are curious how future programming languages may look like, have a look at x10 being developed at IBM. I also recommend the post Is Java dying?

Monday, January 7, 2008

Usage

Usage

In this post, I express my opinion on many places. For example, when I say "it is more intuitive...", I mean "in my opinion, it is more intuitive...".

As we already mentioned, closures are transformed to anonymous inner classes at compile time. So, all we can do with closures, can be done with inner classes as well. However, the access to local variables may require a little effort. For example, the code

  int x = 4;
  { => System.out.println(++x); }.invoke();
  

can be rewritten without closures as follows:

  final int[] x = { 4 };
  new javax.lang.function.V() {
    public void invoke() {
      System.out.println(++x[0]);
    }
  }.invoke();
  

Even if closures do not increase the power of language, in many cases they are more concise and intuitive than other means. They are a natural choice when we want to parameterize an algorithm with a function. For example, sorting may be parameterized with a comparison function:

  Integer[] primes = { 19, 23, 2, 11, 17, 31, 5, 13 };
  Arrays.sort(primes, { Integer x, Integer y => y.compareTo(x) });
  

Here, it is more intuitive to use a function argument than the Comparator interface because the purpose of the argument is only to compare two values.

One should also prefer closures for predicates because predicate is a function:

  static <T> void selectAndPrint(Iterable<T> items, { T => boolean } cond) {
    for (T item: items) {
      if (cond.invoke(item)) {
        System.out.println(item);
      }
    }
  }
  
  public static void main(String[] args) {
    List<String> cities = Arrays.asList(
      "Prague", "San Francisco", "Moscow", "New York", "Paris");
    int max = ...;
    // will print cities that are not longer than max
    selectAndPrint(cities, { String s => s.length() <= max });
    char c = ...;
    // will print cities that contain character c
    selectAndPrint(cities, { String s => s.indexOf(c) >= 0 });
  }
  

Function types can also be used to describe the input of algorithm. For example, a method that returns the size of area under the function curve should have a function argument:

  static double determineFiniteIntegral(
    double from, double to, { double => double } func) {
    double area;
    //...
    return area;
  }
  

Another usage is in a method that takes a block of code as argument. For example, the execute method takes a block that should be performed in a separate thread:

  ExecutorService pool = Executors.newFixedThreadPool(5);
  pool.execute({ => System.out.println("thinking..."); });
  

Again, it is more intuitive and logical to pass a block of code than an object that implements Runnable.

Closures enable us to write methods that behave in a similar way as the foreach loop:

  static void forEach(int[] nums, { int => void } block) {
    for (int n: nums) {
      block.invoke(n);
    }
  }
  
  public static void main(String[] args) {
    int[] p = { 5, 1, 3, 4, 2 };
    int sum = 0;
    forEach(p, { int n => sum += n; }); // will sum up the numbers in array
    int min = Integer.MAX_VALUE;
    forEach(p, { int n => if (n < min) min = n; }); // will find the minimum
  }
  

For example, we can implement a method that performs an operation on each map entry:

  static <K, V> void forEachEntry(Map<K, V> map, { K, V => void } block) {
    for (Map.Entry<K, V> entry: map.entrySet()) {
      block.invoke(entry.getKey(), entry.getValue());
    }
  }
  
  public static void main(String[] args) {
    Map<Integer, String> map = new HashMap<Integer, String>();
    //...
    forEachEntry(map, { 
      Integer key, String value => 
        System.out.println(key + ": " + value); });
  }
  

We can also use closures to implement the visitor design pattern in a simple fashion:

  class Node {
    int value;
    Node left, right;
  }
  
  class Tree {
    Node root;
    //...
    void visitPreOrder({ int => void } block) {
      visitPreOrder(root, block);
    }
    void visitPreOrder(Node p, { int => void } block) {
      if (p != null) {
        block.invoke(p.value);  // first perform the operation on this node
        visitPreOrder(p.left, block);  // then on the left subtree
        visitPreOrder(p.right, block);  // and then on the right subtree
      }
    }
  }
  
  public class TreeOperations {
    public static void main(String[] args) {
      Tree tree = ...;
      List<Integer> nums = new ArrayList<Integer>();
      // will add the numbers to the list
      tree.visitPreOrder({ int v => nums.add(v); });
      //...
    }
  }
  

Completion Transparency

Completion Transparency

Completion transparency enables us to create methods that transparently return the result of a closure which is passed as argument. For example:

  static <T> T withLock(Lock lock, { => T } block) {
    lock.lock();
    try {
      return block.invoke();
    } finally {
      lock.unlock();
    }
  }
  

We can pass to the withLock method a block that returns any value. For example:

  Lock lock = new ReentrantLock();
  Integer i = withLock(lock, { => Integer.valueOf(42) });
  

Due to completion transparency, we can also use a block that does not return anything (i.e. a block of type { => void }):

  withLock(lock, { => System.out.println("mining..."); });
  

And even a block that cannot complete normally, i.e. throws exception (in this case only unchecked exceptions are allowed):

  withLock(lock, { => throw new NullPointerException(); });
  

To allow checked exceptions as well, we can use exceptions transparency:

  static <T, throws E> T withLockEx(Lock lock, { => T throws E} block) 
    throws E {
    lock.lock();
    try {
      return block.invoke();
    } finally {
      lock.unlock();
    }
  }
  
  public static void main(String[] args) {
    try {
      Lock lock = new ReentrantLock();
      withLockEx(lock, { => throw new Exception(); });
    } catch (Exception e) { e.printStackTrace(); }
  }
  

Implementation issue

Let us recall that each closure is converted to an instance of interface. For example, a closure { => System.out.println("buff"); } is converted to an instance of V:

  public interface V {
    void invoke();
  }
  

To achieve completion transparency, the compiler is allowed to use Void instead of void. For example, the code

  withLock(lock, { => System.out.println("miaow"); });
  

will be transformed approximately as follows:

  withLock(lock, new javax.lang.function.O() {
    public Void invoke() {
      System.out.println("miaow");
      return null;
    }
  }
  

In case of closures that do not complete normally, the invoke method returns java.lang.Unreachable which is a special class that is considered a subclass of all classes. This means that an Unreachable may be assigned to any reference variable. For example, the statement

  Integer i = { => throw new NullPointerException(); }.invoke();
  

will be transformed roughly as follows:

  Integer i = new javax.lang.function.O() {
    public Unreachable invoke() { throw new NullPointerException(); }
  }.invoke();
  

Friday, December 28, 2007

Closures Sources

Closures sources

Advanced Features

Advanced features

A function type can be used as type parameter:

  public static void main(String[] args) {
    List<{ int, int => int }> operations = 
      new ArrayList<{ int, int => int }>();
    operations.add({ int x, int y => x + y });
    operations.add({ int x, int y => x | y });
    int[][] param = { { 1, 2 }, { 3, 4 }, { 5, 6 } };
    for (int[] p: param) {
      for ({ int, int => int } op : operations) {
        System.out.println(op.invoke(p[0], p[1]));
      }
    }
  }
  

A closure can return another closure:

  { String => { int => String } } concat = 
    { String s => 
      { int n => String r = ""; for ( ; n > 0; n--) r += s; r } };
  

Function concat takes a single string argument and returns a function that takes a single integer argument and returns string.

  { int => String } concatABC = concat.invoke("ABC");
  String result = concatABC.invoke(3);
  

Each function that takes multiple arguments can be transformed into a function that takes a single argument. This transformation is called currying. For example, function plus can be transformed into anotherPlus:

  { int, int => int } plus = { int x, int y => x + y };
  { int => { int => int } } anotherPlus = { int x => { int y => x + y } };
  int threePlusFour = anotherPlus.invoke(3).invoke(4);
  

The reverse transformation is called uncurrying. Currying and uncurrying are used in functional programming. For more examples of functional programming in Java, see Luc Duponcheel's blog.

Exceptions

Exceptions

Closures are allowed to throw exceptions. We use the throws keyword to declare which exceptions are thrown by a closure:

  { long => void throws InterruptedException } wait = 
    { long millis => Thread.sleep(millis); };
  try {
    wait.invoke(2000);
    System.out.println("hi");
  } catch (InterruptedException e) { e.printStackTrace(); }
  

A closure may throw more than one exception:

  try {
    FileOutputStream fos = new FileOutputStream("primes");
    try {
      { int, long => 
        void throws IOException, InterruptedException } writeAndWait = 
        { int value, long millis => 
          fos.write(value); Thread.sleep(millis); };
      writeAndWait.invoke(17, 2000);
      System.out.println("first");
      writeAndWait.invoke(19, 2000);
      System.out.println("second");
    } finally { fos.close(); }
  } catch (IOException | InterruptedException e) { e.printStackTrace(); }
  

When we assign a closure to a variable of function type, the closure must not throw more than is declared in the function type.

  // constructor FileWriter throws IOException which is a subclass of 
  // Exception
  { String => void throws Exception } truncate = 
    { String name => new FileWriter(name).close(); };
  { => double throws Exception } next = 
    { => Math.random() };  // Math.random() does not throw any exception
  

Exception Transparency

If we pass a closure that throws exceptions to some method, we might want these exceptions are propagated from the method. If we used ordinary type parameter for exception, we could have passed in only a block that throws one exception. But what if we want to accept a block that throws any number of exceptions and we want exceptions are transparently propagated to the caller? We have to use exception type parameters. The exception type parameter is declared using the throws keyword and stands for any number of exceptions.

  // method performTwice throws the same exceptions as the block passed in
  static <throws E> void performTwice({ => void throws E } block) throws E {
    block.invoke();
    block.invoke();
  }
  
  static void passBlockThrowingNoExceptions() {
    performTwice({ => System.out.println("no exception..."); });
  }
  
  static void passBlockThrowingOneException() {
    try {
      // the block passed to performTwice throws one exception 
      // (InterruptedException)
      // the compiler checks that we catch this exception
      performTwice({ => 
        System.out.println("waiting..."); Thread.sleep(2000); });
    } catch (InterruptedException e) { e.printStackTrace(); }
  }
  
  static void passBlockThrowingMoreExceptions() {
    try {
      // the block passed to performTwice throws two exceptions 
      // (InterruptedException and IOException)
      // the compiler checks that we catch these exceptions
      performTwice({ => Thread.sleep(2000); new FileWriter("x").close(); });
    } catch (InterruptedException | IOException e) { e.printStackTrace(); }
  }
  

If we do not want to propagate exceptions, we can invoke a closure in the try block:

  static <throws E> void logAndPerform(
    String desc, { => void throws E } block) {
    System.out.printf("start: %s...%n", desc);
    try {
      block.invoke();
    } catch (Exception e) { e.printStackTrace(); }
    System.out.printf("end: %s%n", desc);
  }
  
  public static void main(String[] args) {
    logAndPerform("doing nothing", { => System.out.println("hi"); });
    logAndPerform("sleeping", { => Thread.sleep(2000); });
    logAndPerform("sleeping and truncating", 
      { => Thread.sleep(2000); new FileWriter("x").close(); });
  }
  

Exercises

  1. Fill in the type of p.
      static void m() throws FileNotFoundException, InterruptedException {
        //...
      }
      
      public static void main(String[] args) {
                                                     p = { => m(); };
        try {
          p.invoke();
        } catch (FileNotFoundException | InterruptedException e) { 
          e.printStackTrace();
        }
      }
      
  2. Declare method forEach that performs an operation on each item of a collection. The operation is described by a closure and may throw an exception (one or more).

Solutions

  { => void throws FileNotFoundException, InterruptedException } p = 
    { => m(); };
  
  static <T, throws E> void forEach(
    Iterable<T> items, { T => void throws E } block) throws E {
    for (Iterator<T> it = items.iterator(); it.hasNext(); ) {
      block.invoke(it.next());
    }
  }
  public static void main(String[] args) {
    List<String> nums = Arrays.asList("first", "second", "third");
    forEach(nums, { String s => System.out.println(s); });
  }
  

Closure Conversion

Closure Conversion

A closure can be assigned to a variable of compatible interface type. A compatible interface type must have a single method with a compatible return type and compatible argument types. For example:

  interface IntPlus {
    int add(int x, int y);
  }
  IntPlus plus = { int x, int y => x + y };
  

In such case, a closure is converted to an instance of anonymous class that implements the given interface:

  IntPlus plus = new IntPlus() {
    public int add(int x, int y) { return x + y; }
  };
  

Then, we call the closure by the interface method:

  int sum = plus.add(5, 6);
  

This allows us to use a closure instead of anonymous subclass of interface with a single method. For example, the closure conversion can create an instance of Runnable:

  new Thread({ => System.out.println("hi"); }).start();
  

Sometimes we can use the closure conversion even if the interface has more than one method. It is in case when all but one method are implemented in Object. The closure is then converted to a single remaining method. For example, interface Comparator contains two methods: compare() and equals(). Method equals() is in Object and so the closure is converted to method compare().

  String[] girls = { "Jane", "Eva", "Sarah", "Alice" };
  Arrays.sort(girls,
    { String s1, String s2 =>
      int r = s1.length() - s2.length(); r == 0 ? s1.compareTo(s2) : r 
    });
  

Exercises

  1. Declare interface HashCalculator.
      HashCalculator hc = 
        { String s1, String s2 => s1.hashCode() ^ s2.hashCode() };
      System.out.println(hc.hash("elephant", "giraffe"));
      
  2. Use a closure to define Comparator which sort numbers descendingly.
      Integer[] wages = { 1200, 4500, 850, 1500, 990 };
      Arrays.sort(wages,                                      );
      
  3. Sort an array of strings and count the number of comparisons.

Solutions

  interface HashCalculator {
    int hash(String s1, String s2);
  }
  
  Integer[] wages = { 1200, 4500, 850, 1500, 990 };
  Arrays.sort(wages, { Integer i, Integer j => j.compareTo(i) });
  
  String[] countries = { 
    "France", "Czech Republic", "Poland", "Hungary", "Germany" 
  };
  int c = 0;
  Arrays.sort(countries, { String s1, String s2 => c++; s1.compareTo(s2) });
  System.out.println("number of comparisons: " + c);
  

Covariant Return and Contravariant Arguments

Covariant Return and Contravariant Arguments

As we know, to a variable of function type may be assigned a closure with compatible return type and compatible argument types. "Compatible return type" means that the type is either the same as in function type or is a subclass of this type (return types are covariant).

  { => Number } p1 = 
    { => Integer.valueOf(19) };  // Integer is a subclass of Number
  { => Number } p2 = 
    { => Double.valueOf(1.25) };  // Double is a subclass of Number
  

"Compatible argument type" means that the argument type is either the same as in function type or is a superclass of this type (argument types are contravariant).

  { String => void } p = 
    // Object is a superclass of String
    { Object o => System.out.println(o); };
  

We can combine covariant return with contravariant arguments:

  { String => Number } p = { Object o => Integer.valueOf(o.hashCode()) };
  System.out.println(p.invoke("haf"));
  

These rules define hierarchy on function types. So a function type might be a subtype or supertype of another function type. For example, { => Integer } is a subtype of { => Number } and { String => void } is a supertype of { Object => void }. As for class types, to a variable of function type may be assigned an object (closure) of its type or any subtype.

What is meant by covariant and contravariant?
Let S and T be two types (classes or function types) such that S is a subtype of T. If method m of T is overridden in S, then the corresponding types from the m's signature can either preserve the relationship between S and T (i.e. the type used in S is a subtype of the corresponding type in T), reverse this relationship (i.e. the type used in S is a supertype of that in T), or neither preserve nor reverse this relationship. If they preserve the relationship of S and T, we say they are covariant, and if they reverse the relationship of S and T, we say they are contravariant.

Local Variables

Access to Local Variables

So far, closures were a concise analogy to anonymous inner classes. But they offer more. In inner classes, you can access a variable from enclosing scope only if it is final. For closures, there is no such restriction:

  public static void main(String[] args) {
    int x = 4;
    { => void } printX = 
      { => System.out.println(x); };  // x is a free variable here
    x++;
    printX.invoke();  // will print 5
  }
  

In function printX, x is a free variable because it is not declared in the printX's block of code (it is neither a parameter nor a local variable in this block). We can access and change free variables:

  int x = 100;
  { => int } nextX = { => ++x };
  System.out.println(nextX.invoke());  // will print 101
  System.out.println(nextX.invoke());  // will print 102
  x = 200;
  System.out.println(nextX.invoke());  // will print 201
  

A block of code is packed with variables of the enclosing scope, i.e. the scope where the block is declared. This is where the name "closure" comes from. For example, the following block of code will always work with variable x in the main method:

  static void doTwice({ => void } block) {
    block.invoke();  // will print 10
    int x = 20;
    block.invoke();  // will print 11
    System.out.println(x);  // will print 20
  }
  
  public static void main(String[] args) {
    int x = 10;
    // the block is "packed" with variable x
    doTwice({ => System.out.println(x++); });  
  }
  

A closure can use variables of the enclosing scope even if this scope is not active at the time of closure invocation. For example, a closure can use local variables of a method after return from this method:

  static { => int } makeSum() {
    int n = 1, s = 0;
    // the following closure uses local variables n and s
    return { => s += n; n++; s };
  }
  
  public static void main(String[] args) {
    { => int } sum = makeSum();
    System.out.println(sum.invoke());  // will print sum 0 + 1
    System.out.println(sum.invoke());  // will print sum 0 + 1 + 2
    System.out.println(sum.invoke());  // will print sum 0 + 1 + 2 + 3
  }
  

If we declare n and s local in a closure, we will get different results:

  static { => int } makeSumWrong() {
    return { => int n = 1, s = 0; s += n; n++; s };
  }
  
  public static void main(String[] args) {
    { => int } sum = makeSumWrong();
    System.out.println(sum.invoke());  // will print sum 0 + 1
    System.out.println(sum.invoke());  // will print sum 0 + 1
    System.out.println(sum.invoke());  // will print sum 0 + 1
  }
  

Variables n and s are now local in a closure, so they are initialized upon each invocation of this closure.

Reference this in a closure has the same meaning as this in enclosing scope.

  public class TestThis {
    String s = "hello";
    void test() {
      String s = "hi";
      { => System.out.println(this.s); }.invoke();  // will print "hello"
    }
    public static void main(String[] args) {
      new TestThis().test();
    }
  }
  

Implementation Issue

Local variables that are accessed from a closure are allocated on the heap, so that they are available even when we exit the block they are declared in.


Exercises

  1. Determine the output.
      class ClosureWithThis {
        int x = 1;
        void tryThis() {
          new Invoker().tryThis({ int x => 
            System.out.println(x);
            System.out.println(this.x);
          });
        }
        public static void main(String[] args) {
          new ClosureWithThis().tryThis();
        }
      }
      
      class Invoker {
        int x = 2;
        void tryThis({ int => void } p) {
            p.invoke(3);
        }
      }
      
  2. Determine the output.
      static void m({ => int } c1, { => int } c2) {
        System.out.println(c1.invoke());
        System.out.println(c2.invoke());
      }
    
      public static void main(String[] args) {
        int i = 1;
        m({ => ++i }, { => --i });
      }
      

Solutions

  1. x refers to the argument and this.x refers to field x of the ClosureWithThis instance. Thus, the first println will print 3 and the second will print 1.
  2. Closures { => ++i } and { => --i } refer to the same variable i. Thus, the first println will print 2 and the second will print 1.

Function Types

Function Types

Function types are used when we want to refer to closures. We can declare a variable of function type and assign to it a reference to a closure. A function type is written as follows: { formal parameters => return type }. Formal parameters are optional. The return type can be any type or void. For example, { int, String => void } means a function with two arguments (int and String) and return type void. Any function with a compatible list of argument types and compatible return type can be assigned to a variable of function type.

  public class DeepThought {
  
    // { => int } means a function with no arguments and return type int
    static { => int } answer = { => 42 };
 
    public static void main(String[] args) {
      int i = answer.invoke();
      System.out.println(i);
    }
 
  }
  

A function with one int argument and return type String:

  { int => String } toBinary = { int x => Integer.toBinaryString(x) };
  String binary11 = toBinary.invoke(11);  // will return 1011
  

A function with no arguments and return type void:

  { => void } sayHello = { => System.out.println("Hello"); };
  sayHello.invoke();  // will print "Hello"
  

Function types can be used as types of arguments:

  static void doTwice({ => void } block) {
    block.invoke();
    block.invoke();
  }
  
  public static void main(String[] args) {
    doTwice({ => System.out.println("deja vu"); });
  }
  

They can also serve as a return type:

  static { => boolean } makeCond() {
    return { => Math.random() < 0.8 };
  }
  
  public static void main(String[] args) {
    { => boolean } cond = makeCond();
    while (cond.invoke()) {
      System.out.println("trying...");
    }
  }
  

Exercises

  1. Write a method that invokes an action on each value of array of ints. The action is passed in a function variable.
  2. Write a method that returns a closure which checks if a value is from a given interval. Bounds of the interval are arguments of the method.
  3. Write a method that selects values from a list in accordance with a predicate passed in a function variable.

Solutions

  static void perform(int[] values, { int => void } action) {
    for (int n: values) {
      action.invoke(n);
    }
  }
  
  public static void main(String[] args) {
    int[] v = { 2, 3, 5, 7, 11 };
    perform(v, { int x => System.out.println(x); });
  }
  
  static { int => boolean } makeInterval(int a, int b) {
    return { int x => a <= x && x <= b };
  }
  
  public static void main(String[] args) {
    { int => boolean } interval = makeInterval(10, 20);
    System.out.println(interval.invoke(15));
  }
  
  static <T> List<T> select(List<T> values, { T => boolean } cond) {
    List<T> selected = new ArrayList<T>();
    for (T t: values) {
      if (cond.invoke(t)) { selected.add(t); }
    }
    return selected;
  }
  
  public static void main(String[] args) {
    List<String> cities =
      Arrays.asList("London", "New York", "Prague", "San Francisco");
    List<String> shortCities = 
      select(cities, { String s => s.length() <= 6 });
    System.out.println(shortCities);
  }
  

Closures

Closures

A closure is a form of anonymous function. It is declared using the following syntax:
{ formal parameters => statements expression }. Formal parameters, statements, and expression are optional in this clause. For example, { int x => x + 1 } is a function that takes a single int argument and returns its value incremented by 1. A closure can be invoked by the invoke method. For example, { int x => x + 1 }.invoke(10) will call the closure with argument 10.

  public class SimpleClosure {
  
    public static void main(String[] args) {
      // function with no arguments; return value is always 42
      int answer = { => 42 }.invoke();
      System.out.println(answer);
    }
  
  }
  

A closure with one argument:

  double log = { double x => Math.log(x) }.invoke(10);
  

A closure with a statement:

  // this will print "31 is odd" and return 15
  int half = { 
    int x => 
    if (x % 2 != 0) System.out.printf("%d is odd%n", x); x / 2
  }.invoke(31);
  

A closure with two arguments:

  int sum = { int x, int y => x + y }.invoke(3, 4);  // will return 7
  

A closure does not have to return any value (the function may have return type void):

  { char c => System.out.println(c); }.invoke('@');  // will print @
  

A closure that returns a string:

  String reversed = { 
    String s => 
    new StringBuilder(s).reverse().toString()
  }.invoke("abcd");  // will return "dcba"
  

A closure that returns an instance of Runnable.

  { => new Runnable() {
      public void run() {
        System.out.println("hi from Prague");
      }
    }
  }.invoke().run();  // invoke() returns an instance of Runnable and we 
                     // call run() on it immediately
  

We can declare local variables in closures:

  { int n => 
    int m = n + 1; System.out.println(m * m); 
  }.invoke(3);  // will print 16
  

Implementation issue

This paragraph describes how the prototype compiler implements closures.

For each closure, the compiler will generate an interface with single method invoke(). E.g., for closure { => 42 }, the interface will look like:

  public interface I {
    int invoke();  // no arguments and return type int
  }
  

And for closure { int x, int y => x + y }, the interface will be like:

  public interface III {
    int invoke(int x, int y);  // two int arguments and return type int
  }
  

The interface will be in the javax.lang.function package. Then the compiler will generate an anonymous subclass of this interface, create an instance of this subclass, and call the invoke method. For example, the code

  public static void main(String[] args) {
    int answer = { => 42 }.invoke();
    System.out.println(answer);
  }
  

will be tranformed approximately as follows:

  public static void main(String[] args) {
    int answer = new javax.lang.function.I() {
      public int invoke() {
        return 42;
      }
    }.invoke();
    System.out.println(answer);
  }
  

So, at runtime a closure is represented as object. In JSR, this object is called the closure object.


Exercises

  1. Write a closure that returns the lesser of two double values.
  2. Write a closure that decides if a value is even.
  3. Write a closure that returns last n characters of a string.

Solutions

  double min1 = { double x, double y => x < y ? x : y }.invoke(7.5, 8.2);
  double min2 = { 
    double x, double y => 
    double m; if (x < y) m = x; else m = y; m 
  }.invoke(10.22, 9.76);
  
  boolean even = { int x => x % 2 == 0 }.invoke(15);
  
  { String, int => String } lastNChars = 
    { String s, int n => int len = s.length(); s.substring(len - n, len) };
  

Java Closures Tutorial

Java Closures Tutorial

JSR Proposal: Closures for Java, section 2:

'This JSR provides support for operating on an arbitrary "block of Java code", or body, which is either a statement list, an expression, or a combination of both. We call the mechanism a closure expression. Wrapping statements or an expression in a closure expression does not change their meaning, but merely defers their execution. Evaluating a closure expression produces a closure object. The closure object can later be invoked, which results in execution of the body, yielding the value of the expression (if one was present) to the invoker. A closure expression can have parameters, which act as variables whose scope is the body. In this case the invoker of the closure object must provide compatible arguments, which become the values for the parameters.'

This tutorial does not describe the whole proposal but only the features implemented in the prototype compiler. All examples in this tutorial were tested against the early prototype available at http://www.javac.info. Current version: closures.tar.gz