Monday, February 25, 2008

Control Abstraction (version 2008-02-22)

Control Abstraction (version 2008-02-22)

In this post, I will describe new features of version 2008-02-22 of the compiler prototype. It can be downloaded from http://www.javac.info.

1. java.lang.Unreachable was renamed to java.lang.Nothing.

2. Annotation @Shared was added. It may annotate a variable bound in a closure. For example:

  static void doTwice({ => void } block) {
    block.invoke();
    block.invoke();
  }
  public static void main(String[] args) {
    @Shared int x = 10;
    doTwice({ => System.out.println(x++); });
  }
  

We should always annotate bound variables with this annotation because bound variables are very different from ordinary local variables.

3. Closures can use recursion. Now we can use the function variable within a closure that is used to initialize this variable.

  { int => int } sum = { int n => n == 1 ? 1 : n + sum.invoke(n - 1) };
  

4. Unrestricted closures were partially implemented. In an urestricted closure we can use the return, break, and continue statements. Unrestricted closures use ==> instead of =>.

  { int ==> int } neg = { int x ==> -x };
  

5. A method that accepts an unrestricted closure as the last argument can be called using the control invocation syntax:

  public static void simple({ ==> void } block) {
    block.invoke();
    block.invoke();
  }
  public static void main(String[] args) {
    simple() {
      System.out.println("vrr!");
    }
  }
  

The method may have more than one parameter:

  public static void profile(String s, { ==> void } block) {
    System.out.printf(">>> %s: %s%n", s, System.nanoTime());
    block.invoke();
    System.out.printf("<<< %s: %s%n", s, System.nanoTime());
  }
  public static void main(String[] args) {
    @Shared long f = 1;
    profile("factorial") {
      for (int i = 2; i <= 59; i++) {
        f *= i;
      }
    }
  }
  

The closure used as the last argument can have parameters. They are declared before arguments of the method and are separated from them by colon (although the syntax is similar to the enhanced for loop, the colon here has another meaning):

  public static void control(int x, { int ==> void } block) {
    block.invoke(x);
  }
  public static void main(String[] args) {
    // ordinary syntax
    control(1, { int i ==> System.out.println(i); });
    // control invocation syntax
    control(int i : 2) {
      System.out.println(i);
    }
  }
  

If the closure has more parameters, they are separated by comma:

  public static void controlTwo(int x, { int, int ==> void } block) {
    block.invoke(x, x + 1);
    block.invoke(x + 2, x);
  }
  public static void main(String[] args) {
    // closure parameters are separated by comma
    controlTwo(int i, int j : 2) {
      System.out.println(i * j);
    }
  }
  

This enables us to define new "control statements" without modifying the language. For example, we can add "statement" with that closes a given closeable at the end (the code is derived from the prototype sources):

  public static <T extends Closeable> void with(T t, { T ==> void } block) {
    try {
      block.invoke(t);
    } finally {
      try {
        t.close();
      } catch (IOException e) {
        // ignore exception
      }
    }
  }
  public static void main(String[] args) {
    // class MyCloseable implements Closeable
    with (MyCloseable in : new MyCloseable()) {
      //...
    }
  }
  

Why Java cannot remain simple? Do we need new control statements?
Many people like simplicity (I am one of them). But Java must evolve if we do not want it to become outdated. In addition, the hardware is changing and these changes will probably change programming languages used in software development. For information how the hardware is changing, I recommend the article The Free Lunch Is Over: A Fundamental Turn Toward Concurrency in Software. If you are curious how future programming languages may look like, have a look at x10 being developed at IBM. I also recommend the post Is Java dying?

19 comments:

Mark Mahieu said...

I believe restricted closures and function types use the => token, and unrestricted closures and function types use the ==> token.

It's worth noting that the compiler will prevent you from assigning an unrestricted closure to a variable of restricted function type, which is a good thing - for example, a method which is passed a closure and invokes it on another thread can use the restricted syntax for the closure parameter, and you then won't be able to pass it a closure which uses break, return etc.

Zdeněk Troníček said...

You are right. I have fixed it. Thanks.

Anonymous said...

deněk,

I believe your argument is flawed.

> Many people like simplicity (me too).

You can't have it both ways. BGGA closures are not simple. They add nonlocal return, continue, and break to the language. They add function types. They add capture of non-final locals. They add concurrent access to locals. They add two new tokens that are practically indistinguishable from one another but quite different in semantics (=>, ==>). You make like this, but it isn't simple. Michael Kölling likes simplicity. Here's what Michael thinks about BGGA closures.

Neal implicitly acknowledged some of this complexity when he added a second token for defining function types (==>), but I'm afraid that it only adds to the complexity.

> But Java must evolve if we do not want it to become outdated.

True, but that doesn't mean that Java's evolution should include every feature that anyone campaigns for. Each feature must justify its inclusion. That means that it must carry its weight, and it must not jeopardize the feel of the language. No less an authority than Martin Odersky has suggested that adding BGGA closures to Java may be "like strapping wings to an elephant."

> In addition, the hardware is changing and these changes will probably change programming languages used in software development.

Agreed, but you haven't argued that BGGA closures do anything to make Java better suited to this new hardware.

Josh

Zdeněk Troníček said...

To Joshua:

> BGGA closures are not simple.

Yes, unfortunately they are not simple. But this is only proposal and I think everybody involved wants them as simple as possible.

> True, but that doesn't mean that Java's evolution should include every feature that anyone campaigns for. Each feature must justify its inclusion. That means that it must carry its weight, and it must not jeopardize the feel of the language.

Right. I completely agree. I see closures as a means that enable to add new "statements". If you agree that Java must evolve and that the hardware is changing, you probably agree that Java will need new control statements, don't you?

Closures are not for these days. They are for the future.

Anonymous said...

Zdeněk,

Hi. This time I didn't forget the Z in your name:)

> Right. I completely agree. I see closures as a means that enable to add new "statements". If you agree that Java must evolve and that the hardware is changing, you probably agree that Java will need new control statements, don't you?

Perhaps surprisingly, I don't think it will, at least not soon, and certainly not in great number. If Java does need new control statements in the future, I think we should add them one at a time, as the need arises, just as we've done in past. The one that I see as reasonably compelling (though not critical) these days is Automatic Resource Management blocks.

Also, I think the ability of BGGA to "add new 'statements'" is not at all clear! As a simple experiment, try to duplicate all of Java's existing control constructs. I think you'll find that you can't do a reasonable job for *any* of them. In my talk I showed that you can't do a reasonable job on the while loop (it looks awful) or the for-each loop (you can't make it work on primitives). I sincerely doubt that an ordinary for loop or a do-while loop would be any better.

Simply put, "new 'statements'" made with BGGA don't feel like native Java control constructs: they don't have the right syntax, semantics, or performance properties. If you can't even emulate the existing constructs properly, it seems highly unlikely that you'll be able to make nice new ones. The Automatic Resource Management example in the spec is a case in point. Even though it was a known use-case all along, it isn't very good. It doesn't handle multiple resources and it uses a colon where it ought to use an equals sign. This will be very confusing to Java programmers who use (and love) the for-each loop from Java 5. The colon means "in," and the equals sign means "assign the value on the right to the variable on the left." Any "new 'statements'" that doesn't respect this doesn't feel like Java.

If you accept this argument, then the added complexity of BGGA is not worth the gain in expressiveness.

Josh

P.S. I am not against closures as a means for providing new control structures. I think this works beautifully in languages that were designed with it in mind. But I believe, based on all I've seen to date, that this technique has no place in Java.

Anonymous said...

Speaking of X10, I have been coding a simplified implementation of X10 based on BGGA closure control abstraction. The restulting code will look something like this:

finish() {
async() {
for(Object x : xs) {
atomic() {
doStuff(x);
}
}
}
}

Unfortunately, each X10 control structure adds a level of indentation, so I guess the next logical steps are to:

1.) Drop the need of brackets {} in control structure abstraction (should be easy).

2.) Drop the () from argumentless control structures (this might lead to ambiguity while parsing).

Thus we will end with very X10-ish code looking like this:

finish async for(Object x : xs) {
atomic doStuff(x);
}

Zdeněk Troníček said...

To Joshua:

> Also, I think the ability of BGGA to "add new 'statements'" is not at all clear! As a simple experiment, try to duplicate all of Java's existing control constructs. I think you'll find that you can't do a reasonable job for *any* of them.

Perhaps I do not understand you, because it seems quite easy:

static void whileN({ ==> boolean } cond, { ==> void } body) {
if (cond.invoke()) {
body.invoke();
whileN(cond, body);
}
}

public static void main(String[] args) {
int i = 4;
whileN({ ==> i != 0 }) {
System.out.println(i--);
}
}

> In my talk I showed that you can't do a reasonable job on the while loop (it looks awful) or the for-each loop (you can't make it work on primitives).

static void forEach(Object array, { Object ==> void } body) {
if (array instanceof int[]) {
int[] p = (int[]) array;
for (int i = 0; i < p.length; i++) {
body.invoke(p[i]);
}
} else if (array instanceof double[]) {
double[] p = (double[]) array;
for (int i = 0; i < p.length; i++) {
body.invoke(p[i]);
}
}
}

public static void main(String[] args) {
int[] p = { 2, 3, 5, 7, 11 };
forEach(Object o : p) {
System.out.println(o);
}
double[] d = { 1.2, 2.3, 3.4, 4.5, 5.6 };
forEach(Object o : d) {
System.out.println(o);
}
}

> I sincerely doubt that an ordinary for loop or a do-while loop would be any better.

private static void forN2({ ==> boolean } cond, { ==> void } update, { ==> void } body) {
if (cond.invoke()) {
body.invoke();
update.invoke();
forN2(cond, update, body);
}
}

static void forN({ ==> void } init, { ==> boolean } cond, { ==> void } update, { ==> void } body) {
init.invoke();
forN2(cond, update, body);
}

public static void main(String[] args) {
int i = 0;
forN({ ==> i = 1; }, { ==> i <= 10 }, { ==> i++; }) {
System.out.printf("%d: %d%n", i, i * i);
}
}

But you are right it is ugly.

> It doesn't handle multiple resources and it uses a colon where it ought to use an equals sign.

I agree with you that the colon is strange here. But a sign should not be here because it is not assignment.

static void control(int n, { int ==> void } body) {
body.invoke(n + 1);
}

public static void main(String[] args) {
control(int i : 5) {
System.out.println(i);
}
}

Anonymous said...

I think BGGA is a good step for Java, and although not absolutely critical, I do hope that some efforts will be made when the JSR is formed, to consider adapting the syntax a little to better fit the current Java. It's an issue of style, so some will consider it outright unimportant (subjective and all), but I think it should have a place in the discussion. I do however think that the semantics proposed by BGGA are sound.

One syntax approach that might work:

int : int plus2 = (int x) { x + 2 };

The colon separates the argument type from the return type, e.g.,

int, int : int add = (int x, int y) { x + y };

If a function is being returned, some form of grouping is required to
denote this:

int, int :: int : int curry = (int x) { add.(x) };

Function that takes two ints and returns a function from int to an int.
The double colon serves to delimit the argument list when a function is
being returned.

int : int curried = curry.(2);
int z = curried.(2); // z is now 4

Passing higher order functions is similiar, except the argument function
needs to be delimited with parenthesis in the type declaration:

int, (int : boolean) : boolean filter = (int x, int : boolean pred) {
if(pred.(x))
return x
};

This is a rough sketch of what I think might be a
reasonable approach. I haven't thought extensively through
all the potential ramifications, but from the initial cursory
look I think it may be worth glancing at.

Anonymous said...

Zdeněk,

Hello again.

> Perhaps I do not understand you, because it seems quite easy:

But I think you've proven my point: it was easy but NONE of these control constructs feels right! Of course what matters is the invocation, not the definition. They're defined once and used millions of times. So let's look:

(1) while-loop

whileN({ ==> i != 0 }) {
    System.out.println(i--);
}

vs.

while (i != 0) {
    System.out.println(i--);
}

The closure-based doesn't look like Java. It's ugly, and contains boilerplate.


(2) for-each

double[] dArray = { 1.2, 2.3, 3.4, 4.5, 5.6 };
forEach(Object o : d) {
    System.out.println(o);
}

vs.

double[] dArray = { 1.2, 2.3, 3.4, 4.5, 5.6 };
for(double d : dArray) {
    System.out.println(o);
}

The closure-based version doesn't look too bad. But the closure-based version isn't typesafe! And it boxes everything so it will be very slow. I'm sorry to be so blunt, but it's a toy.

3) for-loop

int i = 0;
forN({ ==> i = 1; }, { ==> i <= 10 }, { ==> i++; }) {
    System.out.printf("%d: %d%n", i, i * i);
}

vs.

for (int i = 0; i < 10; i++) {
    System.out.printf("%d: %d%n", i, i * i);
}

The closure based version is very ugly, and less safe. The loop variable is in scope outside the loop, which is a bad thing!

4) ARM block

> It doesn't handle multiple resources and it uses a colon where it ought to use an equals sign.

I agree with you that the colon is strange here. But a sign should not be here because it is not assignment.


It *is* an assignment, in scope for the body of the construct, as is done in for and for-each. Here is how I imagine a purpose-build ARM-block:

try (BufferedReader r = new BufferedReader(new FileReader(path)) {
    String s = r.readLine();
}

The variable r is in scope only in the block, and the file is closed when you exit the block.

-------------

So near as I can tell, it's Java constructs 4, closures 0. I remain unconvinced that you can actually "add new 'statements'" with BGGA. I believe that BGGA entails all of the complexity required to enable this functionality (nonlocal returns, break, continue, access to mutable local state from elsewhere), but I don't think it really delivers. Compare this to Scala, where you can't tell the difference between a built-in construct and a library-provided one. And it's clear why this works in Scala: Scala was built with this sort of extensibility in mind.

Josh

Zdeněk Troníček said...

Hi Josh,

> But the closure-based version isn't typesafe!

I was aware of this BIG drawback. But the question (which I probably did not understand well) was if it was possible or not.

> It *is* an assignment, in scope for the body of the construct, as is done in for and for-each.

It is an assigment in CICE/ARM but not in BGGA. And I wrote on BGGA.

> I remain unconvinced that you can actually "add new 'statements'" with BGGA.

Ok, you say "we cannot add new statements in BGGA". An answer could be "we cannot add new statements now". BGGA is a proposal and it might (and very probably will) change.

Anonymous said...

Zdeněk,

>Ok, you say "we cannot add new statements in BGGA". An answer could be "we cannot add new statements now". BGGA is a proposal and it might (and very probably will) change.

I'll certainly keep my eye on it, but I remain skeptical. Up to now, each control construct in Java has been "purpose-built," so they don't have the regularity of control structures in a closure-based language.

And there is the small matter of whether we want to allow library authors to "add new statements." It has costs as well as benefits, and it isn't clear which outweighs the other. But we'll leave that discussion for another day.

Regards,

Josh

Anonymous said...

How about automatic conversion from statements to argumentless closures?

For example,

whileN(i < 2)

converts to

whileN({ => i < 2})

and

log("potential performance hit: " + obj)

converts to

log({ => "potential performance hit: " + obj})

In fact, this is how Java's own control structures if(...), while(...) etc. work. They too implicitly create closures (at compile time).

Bernard AMADE said...

Zdenek: I suggest you pick up more examples of unrestricted closure that precisely use return, break, ... and make the difference with simple closure clearer to newbies.

thanks
(PS: I really like the @Shared annotation but still being skeptic about the control statement complexity)

Zdeněk Troníček said...

Hi Bernard,

the return, break, and continue statements in unrestricted closures are not implemented in the compiler prototype. I will add some examples as soon as their implementation appears.

Bernard AMADE said...

zdenek

something is worrying me: for instance in the "controlTwo" example the parameters of the closure appear first in the call... this is illogical: if the parameters of the definition are ended with the closure why aren't the closure's parameters the last ones in the call?

plus: I find the use of ":" to separate the differents set of parameters a strange choice (why for instance not the ";" used by the old "for")....matter of taste? not really in the foreach call ":" means "in" and here I don't get this meaning.

thanks

Zdeněk Troníček said...

Hi Bernard,

thanks for the comments. I agree with you that there is much to improve in syntax.

Anonymous said...

Hey,

>We should always annotate bound variables with this annotation because bound variables are very different from ordinary local variables.

The purpos of the annotation is to tell the compiler : "Yes, I know, this variable is used as an argument for a closure, and I am aware that it may by modified there."?

Maybe you could explain what exactly means "shared" and "captured" in this context?

Thanks,
Felix

Manoj Kumar said...

Section #5 states that.. A method that accepts an unrestricted closure as the last argument can be called using the control invocation syntax.

Why do we have this restriction that only last argument can be called using the control invocation syntax? It will be nice to have any number of arguments to be like that.
Ex.
public static void DoInSequence{==>void} first, {==>void} second) {
first.invoke();
second.invoke();
}

right now we can do ..
DoInSequence(
{ =>
System.out.println("3.First");
})
{
System.out.println("3.Second");
}

but not..

DoInSequence()
{
System.out.println("3.First");
}
{
System.out.println("3.Second");
}



This will help us creating "if-then-else", "try-catch-finally" style constructs. Obviously there is no point in re-creating these basic constructs, but we can have problem domain specific constructs accepting many blocks as arguments and call such methods with simpler syntax.

Plumbing & HVAC Services San Diego said...

Compre documentos en línea, documentos originales y registrados.
Acerca de Permisodeespana, algunos dicen que somos los solucionadores de problemas, mientras que otros se refieren a nosotros como vendedores de soluciones. Contamos con cientos de clientes satisfechos a nivel mundial. Hacemos documentos falsos autorizados y aprobados como Permiso de Residencia Español, DNI, Pasaporte Español y Licencia de Conducir Española. Somos los fabricantes y proveedores de primer nivel de estos documentos, reconocidos a nivel mundial.

Comprar permiso de residencia,
permiso de residenciareal y falso en línea,
Compre licencia de conducir en línea,
Compre una licencia de conducir española falsa en línea,
Comprar tarjeta de identificación,
Licencia de conducir real y falsa,
Compre pasaporte real en línea,

Visit Here fpr more information. :- https://permisodeespana.com/licencia-de-conducir-espanola/
Address: 56 Guild Street, London, EC4A 3WU (UK)
Email: contact@permisodeespana.com
WhatsApp: +443455280186