Thursday, March 20, 2008

Method References (version 2008-03-17)

Method references (version 2008-03-17)

The compiler prototype (available at http://www.javac.info) comes with method references (also known as eta expansion). The method references are written as follows: ClassName # methodName ( listOfArgumentTypes ).

A method reference can be assigned to a function variable:

  { String => int } parseInt = Integer#parseInt(String);
  int x = parseInt.invoke("42");
  

We can use the covariant return:

  // Integer.valueOf() returns Integer
  { int => Number } p = Integer#valueOf(int);
  System.out.println(p.invoke(97));
  

And the contravariant arguments:

  class MyClass {
    static Integer print(Object o) {
      return Integer.valueOf(o.hashCode());
    }
    
    public static void main(String[] args) {
      { String => Number } pp = MyClass#print(Object);
      System.out.println(pp.invoke("hi"));
    }
  }
  

An instance method can be referenced in two ways. Either we reference a method on a given object:

  class Box {
    private int x;
    Box(int x) {
        this.x = x;
    }
    int getX() {
        return x;
    }
  }
  public class InstanceMethod {
    public static void main(String[] args) {
      Box p = new Box(10);
      { => int } getX = p#getX();
      System.out.println(getX.invoke());
    }
  }  
  

Or we reference just a method. Then the function type has an additional argument: the object on which the method is called.

  { Box => int } getX = Box#getX();  // additional argument of type Box
  Box p = new Box(10);
  System.out.println(getX.invoke(p));
  

The method is selected at runtime according to the object supplied as argument. So, in the following example, the toString method from String is called.

  { Object => String } toString = Object#toString();
  System.out.println(toString.invoke("hi"));
  

Generic methods are also supported:

  { String => Set<String> } singleton =
      Collections#<String>singleton(String);
  Set<String> set = singleton.invoke("single");
  

25 comments:

Michael Easter said...
This comment has been removed by the author.
Dani said...

Zdeněk,

I want to thanks your efforts to introduce the closure into the Java world.

I have experience using closure in other languages (Perl, Python, C++ (with Boost), ...). I think your syntax is rather clear (not so much as in Perl, but... :-) ), and not too complex.

I want to encourage you with that. The majority of opinions in other posts are, INHO, a signal of fear to change. Closure is very useful when you have to make flexible software (Yeah, we have Interfaces, but using them the software can be a lot more complex).
I know that, I had a very complex code and now I am remaking it, reducing the complexity.

I work in university, and for research the flexibility is a very important quality (to make our experiments more easily).

I have just found this page, but seeing that it is able to work with instances, I think I am going to give it an opportunity. :-)

Daniel

Konstantin Triger said...

I have some usability issues:

1. Closures can be seen as simple anonymous classes. This means, that a programmer will expect that everything possible with an anonymous class should be also possible with a closure.
Thus a restriction on private methods seems unreasonable. Can you please explain that?
2. I would also extend the capabilities and enable targeting 'super.myVirtualMethod()' to address interesting aggregation scenarios (implementation similar to that for private methods).

In addition...
What is the rational to introduce a new navigation operator '#" instead of a convenient '.'?

Thanks,
Kosta

Konstantin Triger said...

Additional usability question.

Why do you require to specify list of argument types? You may leave it optional and require only in a case of an ambiguity.

Regards,
Kosta
http://jaque.googlecode.com

TheSunToucher said...

to Konstantin:

> What is the rational to introduce a new navigation operator '#" instead of a convenient '.'?

Because using the dot would execute the Method an assign the result to the variable. This could be useful if your Method returns a Closure (what is possible)

Konstantin Triger said...

To TheSunToucher:

Only parameterless methods can be potentially executed since in other cases the method 'receives' a list of parameter types instead of actual values.

For this case I would either propose support for a simpler construct without parentheses (see my previous post) or something like: toString(Void). (The compilation should fail on ambiguity in case user does not specify Void AND the method returns Closure).

The reason I don't like '#' operator is because it has absolutely same meaning as '.' does, but is good for use in some specific case.
It's usually preferable to keep a programming language as laconic as possible, especially if there is so big overlap in the newly introduced and old capability.

Kosta
http://jaque.googlecode.com

Zdeněk Troníček said...

Hi Kosta,

in my new post "Method References (Implementation Issue)" I tried to explain how method references are implemented. This should answer your question about private methods. (Method references are transformed to closures which are then transformed to inner classes).

Regarding the '.' vs. '#': '#' is a better choice, in my opinion. Consider this:

public class Simple {
static { => { => void } } m;
static { => void } m() {
return { =>
System.out.println("hi"); };
}
public static void main(
String[] args) {
{ => { => void } }
p = Simple#m();
p.invoke().invoke();
}
}

And this:

public class Simple2 {
static { => { => void } } m;
static { => void } m(String s) {
return { =>
System.out.println(s); };
}
public static void main(
String[] args) {
String String = "hi";
{ String => { => void } }
p = Simple2#m(String);
p.invoke(String).invoke();
}
}

Konstantin Triger said...

Hi Zdenek,

1. Private methods limitation: I looked into your new post and did not find an exact problem you encounter when implementing that. Can you be more specific?
Besides, you say that Method references are transformed to closures which are then transformed to inner classes. This makes a lot of sense, but inner classes DO have access to the outer class private members.
2. I see that you addressed only the issue regarding argument list specification, since your both samples contain '#' operator.
In addition for my option you try p.invoke() without parameters, which is not correct by definition. Here is how I would like your sample to look:
public class Simple2 {
static { => { => void } } m;
static { => void } m(String s) {
return { =>
System.out.println(s); };
}
public static void main(
String[] args) {
String s = "hi";
{ String => { => void } }
p = Simple2.m;
p(s)(); //or p.invoke(s).invoke()
}
}

Regards,
Kosta
http://jaque.googlecode.com

Konstantin Triger said...

Clarification:
I don't want the full syntax to be illegal, just a simplified one to be fine. So, if the field Simpl2.m was defined as:

static {String => { => void } } m;
Then to resolve ambiguity the user will need to use full syntax:
{ String => { => void } } p = m(String); //references method.
{ String => { => void } } p = m; //references field.

Regards,
Kosta

Zdeněk Troníček said...

Hi Kosta,

1. private methods: you can access private instance methods, if they are accessable when you access them:

public class Visibility {
  private void printHi() {
    System.out.println("hi");
  }
  public static void main(String[] args) {
    Visibility v = new Visibility();
    { => void } p = v#printHi();
    p.invoke();
    { Visibility => void } pp = Visibility#printHi();
    pp.invoke(v);
  }
}

In such case, a bridge method is generated.

2. regarding your proposal: I believe this could have been done this way but the syntax would have been cumbersome. In my previous comments, I wanted to show you ambiguous examples. Not to prove that the references are not solvable using the syntax you propose. For example, following you syntax:

String String = "hi";
{ String => { => void } }
p = Simple2.m(String);

Is this a method reference or method invocation?

Konstantin Triger said...

Hi Zdenek,

Thanks for your reply.

1. private methods: thanks for clarification. Back to my original question:
class Base {
void printHi() {
System.out.println("hi base");
}
}

class Derived extends Base {
void printHi() {
System.out.println("hi derived");
}

void test() {
{ => void } p = super.printHi(); //is this somehow possible?
}

The question: can I create a reference to my super method bypassing the virtual chain?

2.
String String = "hi";
{ String => { => void } }
p = Simple2.m(String);

In this case it will be a method invocation.

To produce a method reference the user will need to write:
p = Simple2.m(java.lang.String); //or a simple 'p = Simple2.m'

And of course, I would not suggest name local variables after the class names and use Camel notation at least. IMHO any user coding this way will produce unclear code. (I would prohibit/warn during compilation about that, but this is another story.)

Regards,
Kosta
http://jaque.googlecode.com

Zdeněk Troníček said...

Hi Kosta,

1.void test() {
  { => void } p = super.printHi(); //is this somehow possible?
}


This is not possible. Do you have any use case?

2.p = Simple2.m(java.lang.String); //or a simple 'p = Simple2.m'


Ok. What if String was a class in default package?

p = Simple2.m - this is ambiguous, if there is attribute m in the Simple2 class. I know you suggest to use another syntax then. But it would have meant that Simple2.m would have refered to either a method or a field, based on that if there is a field m. And this is something I do not like.

Concerning the naming of local variables, we are not talking about guidelines how to write good code here! We are discussing the problems on the compiler level. And if one is allowed to name a variable 'String', the compiler must be able to process it.

Konstantin Triger said...

Hey Zdenek,

1. A general use case is alignment with an ability to reference private methods.
In addition, I know at least one person who used this feature, though in C#, see: http://dev.mainsoft.com/default.aspx?tabid=27&forumid=19&postid=2376&view=topic

2.
a. If String was in default package AND someone had a local variable named String, the statement would be resolved as a method invocation. To reference a method, the user will need to rename the variable.
b. Indeed, Simple2.m can be resolved as a reference to a field or method, based on the context.

Those are real arguments supporting introduction of operator '#', though I'm personally not convinced yet.

Here are my counterarguments:

a. I think it's a corner case with an easy solution - renaming. There a many opportunities to create something ambiguous, I don't think we must introduce a new language keyword for every such a case.
For example, in C# there is a possibility to override a final method with a 'new' keyword (though it's not added to the virtual chain). Java does not allow this, and I think it's reasonable.

b. This is a matter of taste, but when we move towards Functional languages with introduction of closures, when we consider adding support for properties to Java, having different access to fields and methods can be viewed even as a disadvantage. After all, method declaration can be seen as a special case of static variable, similarly to JavaScript or other dynamic languages approach.

Regards,
Kosta

Zdeněk Troníček said...

Hi Kosta,

1.A general use case is alignment with an ability to reference private methods. - I have to admit that I do not know what you mean by this sentence. Regarding the use case: ok, there is somebody who uses this feature. But we should ask 'for what?' if we want to evaluate it. Otherwise, you can ask for ANY feature from ANY language.

2. Another argument for # is readability: when I see X#m(Y), I know it is a method reference. When you see X.m(Y), it may be either a method reference or a method call.

But I agree it is a matter of personal taste. So, look at these comments as at my opinions.

As for the similarity between method and field declarations ("method declaration can be seen as a special case of static variable, similarly to JavaScript"), in JavaScript you can assign to such variable. It is interesting but I have some doubts if it was not too much for Java.

Konstantin Triger said...

Hi Zdenek,

1. I see private and super.* methods very similar, because both have similar accessibility (except inner classes), are not virtual and even invoked by the same byte-code (invokespecial) - that's why I tend treat them same way.
Use case: I cannot give you a 'perfect' one, but sometimes it can help avoid a big code change in existing system, which is probably not perfect. What will the user do? He will simply build a bridge method and reference it! So why not let the compiler do it, since I cannot see how the user can abuse this feature.
In addition, consider Sytem.identityHashCode(). This method comes to solve similar problems.

2. Yep, I agree with you, the user won't know what he references a method or a variable. Let ask ourselves should he matter, if it's a part of a public interface? See, how powerful it is: I can start with a method today, and change to a variable tomorrow. The user will need only recompile.

Regarding the ambiguity between method reference and method call: this may happen only when the user names his variables after class names. I don't think it's a common practice. The compiler should generate a method call for this case and issue a warning to user.

Regards,
Kosta

Zdeněk Troníček said...

Hi Kosta,

1. If you need to switch off virtual binding, there are very likely some defects in your design (as you say, the system "is probably not perfect"). And to manage the Java language design so that we are able to write a concise code in such a case, is not a good idea, in my opinion.

Regarding System.identityHashCode(), this method does NOT swich off virtual binding. It just returns the same value as Object.hashCode().

2. Right, it is a powerful feature. But do we want it in Java? As for me, I see many drawbacks and only a few gains.

Baldur said...

Hi,

the closures & method references looks nice, but one suggestion for the syntax:
It might be very ugly to write the {... => ...} through the complete code and it might be difficult, if you have to change the parameters of a closure or a method.

So, would it be possible to add a "alias" keyword to Java (like C++ typedef)?

For example:
alias {String => int } alias_name;

So we can use alias_name instead of the closure declaration for return types or variable names.

Zdeněk Troníček said...

Hi Baldur,

aliasing can be done even now through subclassing:

interface X extends { int, int => int } { }

static void runAndPrint(X p, int x, int y) {
  System.out.println(p.invoke(x, y));
}

Baldur said...

Ok, when this works, it would be very helpful :)

I hoped, there would come some kind of a real 'alias' keyword at some day, because I tried this trick alredy with Generics, but it doesn't work with code like
class RecordList extends Vector<? extends Record> or final classes...


Would the API make use of aliasing to provide types for closures in parameters or return values? Sure, we still have types like 'Runnable', but new closures should be provided with an alias type for easier use and understanding.

Zdeněk Troníček said...

Hi Baldur,

it works under version 2008-05-04.

As for aliasing in API, I am not convinced that aliases would improve the code. Maybe in a particular case but not in general.

ben said...

Method references sound like a great idea but I don't like the syntax. I think it should look like

#Type.method(int)

or even (the horror, the horror)

&Type.method(int)

this makes it clear that what is meant is "reference to the following method: ..."

Also, the following is very strange:

Collection#<List>(int)

shouldn't it be

Collection<List>#(int)

or better

&Collection<List>.(int)

Also, do you allow partial method references? E.g.

Type#method(int, 23)

which means the same as

lambda(int x): Type.method(x, 23)

(sorry, I haven't yet figured out the "Java" way of specifying lambda exprs)

Zdeněk Troníček said...

Hi Ben,

Collections#<String>singleton() is a reference to the singleton() generic method and Collection<String>#size() is a reference to the size() method in parameterized type Collection<String>. So, both can be used but have different meaning.

As for partial method references (currying), it is on the open issues list.

Carlo said...
This comment has been removed by the author.
Carlo said...

Instead of

{ String => int } parseInt = Integer#parseInt(String);
int x = parseInt.invoke("42");

I would prefer

int (String s) parseInt = Integer.parseInt;
int x = parseInt ("42");

Integer.parseInt returns a java.lang.ref.Method instance. parseInt ("42") automatically calls invoke on that instance.

Ok, there may be compiler problems, but in a poll among Java developer I would win hands down.

Carlo said...

{ String => int } parseInt = Integer#parseInt(String);
int x = parseInt.invoke("42");

looks like another language. That can't be in Java.