2016-03-15
Closures in Pony
Note 2017-08-01: Recent releases of Pony have changed syntax for lambda calls and partial function calls. This post has been updated to work with these changes as of Pony 0.16.1.
The Pony programming language has support for closures but the documentation hasn't caught up with changes to the implementation. This post describes some of the features of Pony closures with examples. These examples require the latest version of Pony from the github repository. I used commit e4f6f91d.
Lambda
Closures are functions combined with an environment mapping names to values. Pony has a {...} syntax that can be used to create them. It's possible to create a function that captures from the lexical scope and to create a function with no capturing at all.
The following example shows using a lambda that does not capture anything (note that this is an artificial example - the List type already has various mapping and fold functions on it):
use "collections"
actor Main
new create(env:Env) =>
let l = List[U32]
l.push(10)
l.push(20)
l.push(30)
l.push(40)
let r = reduce(l, 0, {(a:U32,b:U32): U32 => a + b})
env.out.print("Result: " + r.string())
fun reduce(l: List[U32], acc: U32, f: {(U32, U32): U32} val): U32 =>
try
let acc' = f(acc, l.shift()?)
reduce(l, acc', f)
else
acc
end
The reduce function takes an anonymous function as a parameter, named f. This is is defined as {(U32, U32): U32}. This syntax means that it must take two U32 arguments and returns a U32. A U32 is a 32-bit unsigned integer.
The val is a reference capability annotation. It states that f is an immutable value. Since nothing inside f can be mutated it is safe for f to be shared with other actors. Lambda functions that do not close over other variables default to val as they cannot change anything outside of the function. We need to explicitly annotate the val as {...} declarations default to ref. A compile error would result without it due to passing a val object to a ref parameter.
Compiling and running produces the result 100 as expected:
$ ponyc example
$ ...
$ ./example1
Result: 100
Lambda with environment
The following example closes over the env variable so that it can be accessed within the lambda to print the String contained in the List:
use "collections"
actor Main
new create(env:Env) =>
let l = List[String]
l.push("hello")
l.push("world")
for_each(l, {(s:String)(env) => env.out.print(s)})
fun for_each(l: List[String], f: {(String)}) =>
try
f(l.shift()?)
for_each(l, f)
end
Notice that the lambda has an additional set of parameters, the (env). This is an argument list of the variables that are being closed over. Variables listed there can be accessed from the body of the lambda. Pony requires you to be explicit about what gets captured vs many other languages that implicitly capture variables in scope. Variables can be renamed in this parameter list using the following syntax:
for_each(l, {(s:String)(myenv=env) => myenv.out.print(s)})
The lambda here returns None which is the default so it is left off the declaration {(String)}. This declaration requires a function that takes a single String argument and returns None. The return value is also left off the actual lambda expression. No reference capability annotations are required here as {...} defaults to ref and a lambda that closes over variables also defaults to ref.
Lambda modifying closed variables
A modification of the above example might be to keep a count of each time the lambda is called and display the count:
actor Main
new create(env:Env) =>
let l = List[String]
l.push("hello")
l.push("world")
var count = U32(0)
for_each(l, {ref(s:String)(env,count) =>
env.out.print(s)
count = count + 1})
// Displays '0' as the count
env.out.print("Count: " + count.string())
fun for_each(l: List[String], f: {ref(String)}) =>
try
f(l.shift()?)
for_each(l, f)
end
The main thing to note here is that both the lambda definition and the declaration have a ref prefixed to the argument list. This signifies a lambda that might mutate itself when called. The effect of this is that within the body of the lambda the receiver of method calls and field access (ie. the this of the lambda object) is a ref vs the default of box. A box receiver won't allow ref functions to be called and mutating requires a ref receiver.
There is an issue with this example as noted by the comment. The final count is displayed as zero. The assignment of count within the lambda changes the count field within the lambda object. The count within the lambda increments and could be displayed showing the increasing values. The count outside of this does not. There is no reference or pointer to the closed over variable. Indirection can be used to do this by using a one element array:
var count : Array[U32] = Array[U32].init(0, 1)
for_each(l, {ref(s:String)(env,count) =>
try
env.out.print(s)
count(0)? = count(0)? + 1
end})
try
env.out.print("Count: " + count(0)?.string())
end
The array access is wrapped with try as it is a partial function and can fail. Another approach could be to move the act of counting into an Actor:
use "collections"
actor Counter
var n: U32 = 0
be increment() =>
n = n + 1
be print(env:Env) =>
env.out.print("Count: " + n.string())
actor Main
new create(env:Env) =>
let l = List[String]
l.push("hello")
l.push("world")
let counter = Counter
for_each(l, {(s:String)(env,counter) =>
env.out.print(s)
counter.increment()})
counter.print(env)
fun for_each(l: List[String], f: {(String)}) =>
try
f(l.shift()?)
for_each(l, f)
end
Actors have the tag reference capability and are easier to pass around to other objects, including lambda.
The example here is contrived in that it can be done without modifying a captured variable. Sylvan produced the following approach, using the methods in the collections package:
use "collections"
actor Main
new create(env:Env) =>
let l = List[String]
l.push("hello")
l.push("world")
let count = l.fold[U32](
{(x: U32, s: String)(env): U32 =>
env.out.print(s)
x + 1}, 0)
env.out.print("Count: " + count.string())
Object Literals
Object literals are a way to create objects inline without having to name a class or actor. They look like:
object
let x = 10
fun foo() =>
...do something...
end
Lambda's are actually syntactic sugar on top of object literals. A lambda expansion looks like:
// Lambda
{(s:String)(env) => env.out.print(s)}
// Expands to:
object
var env:Env = env
fun apply(s:String) =>
env.out.print(s)
end
The apply method of an object is called when the () function call syntax is used. Keeping this syntactic transformation in mind can help solve errors that occur when using lambda. The example earlier where the count total didn't account for the increments in the lambda becomes obvious here. The transformation would be:
var count = U32(0)
for_each(l, object
var env:Env = env
var count:U32 = count
fun ref apply(s:String) =>
env.out.print(s)
count = count + 1
end)
env.out.print("Count: " + count.string())
Note the ref annotation in the apply method. This is what the {ref(...)} syntax results in. Without the ref in the {} syntax the syntactic expansion is fun apply(...). The default reference capability for fun in objects is box.
The object literal expansion helps show the difference between {(string)} ref and {ref(String))}. The first requires an object with the given reference capability to match the type. Although the object is a ref, the apply method is the default, a box. This means it cannot modify any fields in the object. In the case of a lambda it won't be able to modify the captured variables:
object ref
fun apply(...)
end
To match the second type definition it requires an object where the apply method itself has the ref capability. This allows apply to modify fields of the object:
object
fun ref apply(...)
end
Partial Application
Related to lambda is partial application. It allows supplying some arguments to a function (or constructor or behaviour) and returns something that allows supplying the other arguments later.
In cases where you would use a lambda to return a function that sets the value of some arguments you can use partial application instead. The following example creates a lambda that adds five to another number:
actor Main
new create(env:Env) =>
let add5 = {(a:U32): U32 => 5 + a}
env.out.print(add5(10).string())
env.out.print(add5(20).string())
With partial application this becomes:
actor Main
new create(env:Env) =>
let add5 = U32(5)~add()
env.out.print(add5(10).string())
env.out.print(add5(20).string())
The use of + is an alias for the add method on an object. In the example above the ~ operator represents partial application. We are binding the receiver of the add method to be the 32-bit unsigned value 5. The resulting object can be called with the remaining arguments.
Partial application allows binding any arguments, by position or by name. The tutorial goes into a lot of detail.
Accumulator Generator
Paul Graham has an accumulator generator problem with examples in various languages. The problem is defined as:
Write a function foo that takes a number n and returns a function that takes a number i, and returns n incremented by i.
A Pony implementation, using closures, could be:
actor Main
fun foo(n:U32): {ref(U32): U32} =>
var s: Array[U32] = Array[U32].init(n, 1)
{ref(i:U32)(s): U32 =>
try
s(0)? = s(0)? + i
s(0)?
else
0
end}
new create(env:Env) =>
var f = foo(5)
env.out.print(f(10).string())
env.out.print(f(20).string())
This uses the one element array trick to enable modifying the captured variable due to the requirement that the number n is incremented. It works only for U32 rather than any number which is also part of the problem definition. Pony has generics which would allow solving this but I'll leave that as an exercise or a future post.
Further reading
Some links to documentation and posts about lambda and reference capabilities to dive into more detail:
- Tutorial on reference capabilities
- Tutorial on object literals
- Tutorial on partial application
- Tutorial on reference capabilities
- Safely sharing data: Reference capabilities in Pony
