Why There Have Been No Posts Lately…

I apologize for the extended absence of posts here on the dev blog, but I’ve been busy with a new Open Source project:

  http://jschema.org

The JSchema Project is an attempt to define a very simple schema system for JSON documents. JSON, for those who are not familiar, is a subset of Javascript that has proven to be a useful and less verbose alternative to XML for data-interchange:

 {
    "group" : "Developers",
    "members" : [
      { "id" : 1, "first_name" : "Joe", "last_name" : "Smith" },
      { "id" : 2, "first_name" : "Jennifer", "last_name" : "Mitchum" }
    ]
 }

It is easy to parse and produce, and integrates well with newer front-end technologies written in Javascript.

The motivation for this project, it will not surprise you, was Gosu. An excellent engineer at Amica, JP Camara, had been playing around with a JSON-based Type Loader for Open Source Gosu. I contacted him about adopting a schema-based approach, and began looking at the industry standard, JSON Schema. It quickly became apparent that JSON Schema was too complicated, and that a small extension to JP’s simple and elegant template approach would yield an expressive schema language, with a Gosu Type Loader to boot.

To give you a taste of how simple JSchema is, here is the schema for the JSON example above:

 {
    "group" : "string",
    "members" : [
      { "id" : "integer", "first_name" : "string", "last_name" : "string" }
    ]
 }

I hope that schema makes intuitive sense to most people: JSchema was designed such that the schema corresponds closely to the documents it describes, and it should be easy to take an example JSON document and transform it into a JSchema schema.

The JSchema specification is still very young but I think it is reasonably complete. We may add some more core data types (a ‘bytes’ datatype for raw byte data seems like it might be useful, for example) but the core ideas are working out well in our Gosu implementation (I’ll blog more about that in a later post).

If you are interested in the specification and/or participating in its design (or making the website prettier), you can fork it here. Participation is very welcome!


JVMLS Talk Is Now Up

You can watch the video here:

  http://medianetwork.oracle.com/media/show/17005

Many thanks to the folks at Oracle for getting it up.


Faking Builders With Gosu

One of the features of Groovy (being aped by Kotlin) is builders, a nice syntactic way to compose nested content.

A canonical example of this, I think, would be building an HTML page. The Groovy page has an example of this:

html {
    head {
        title"XML encoding with Groovy"
    }
    body {
        h1"XML encoding with Groovy"
        p"this format can be used as an alternative markup to XML"

        / an element with attributes and text content /
        ahref:'http://groovy.codehaus.org' ["Groovy"]

        / mixed content /
        p [
            "This is some",
            b"mixed",
            "text. For more see the",
            ahref:'http://groovy.codehaus.org' ["Groovy"],
            "project"
        ]
        p "some text"
    }
}

Gosu does not support builders, but I thought it would be fun to see how close we could get to them with the syntactic tools that Gosu does have. Using names arguments and default parameters, in particular, I think we could get pretty darned close.

So, the first trick I’m going to use in faking builders is something I call the Keefer Gambit: I’m going to create an empty HtmlUtil interface and then create an enhancement of that interface that supplies all the methods you will use to build html. This allows you to mix in the HTML functionality wherever you want it by simply implementing the (empty) interface. This (hack|elegant solution) for mixins in Gosu was first pointed out by Alan Keefer, hence the name.

OK, given that general approach, the syntactic tools we can use to fake builders are named arguments and default parameters. Let’s look at implementing the html element method in our enhancement:

  function html( head : String = "", body : List<String> = null) : String {
    var bodyContent = body == null ? "" : body.join( "\n" )
    return "<html><head>\n${head}</head>\n<body>${bodyContent}</body>\n</html>"
  }

Nothing too fancy here: there are two parameters, each with a default value. The function will return these two elements, wrapped up in the appropriate tags. If the parameters are omitted then the default values are used, and you can use named parameters when invoking this method because it is defined in Gosu.

I’m not going to walk through the implementation of each method, rather I’ll just point you to the sample code I put up on GitHub here:

    https://github.com/carsongross/BuilderExample

But using that code, I can write this code in an Example client:

html(
    :head = head(
        :title = "XML encoding with Gosu"
    ),
    :body = {
        h1( "XML encoding with Gosu" ),
        p( "this format can be used as an alternative markup to XML" ),
        /* an element with attributes and text content */
        ahref( 'http://gosu-lang.org', "Gosu"),
        /* mixed content  */
        p({
            "This is some",
            b("mixed"),
            "text. For more see the",
            ahref('http://gosu-lang.org', "Gosu"),
            "project"
        }),
        p( "some text" )
    }
)

Now, I’ll admit that isn’t quite as clean as the Groovy: we violate the DRY principle when we generate the head element, for example. And things are a bit dicier around when you use parens vs. curly braces. Also, I have to admit, this is a relatively simple example. But I think that this is a pretty clean translation nonetheless, and it was doable without adding any new syntactic features to Gosu.

Gosu, unlike Groovy, Scala or Ruby, requires parentheses around method arguments, and it doesn’t support more exotic things like var-args (it has a nice list literal instead) or implicit arguments: we favor regularity over succinctness beyond a certain point. Despite this, I’m comfortable with the expressiveness of the syntax of Gosu.

I’d rather see innovations in libraries, leveraging The Open Type System rather than much continued innovation in the core language syntax.

I’ll be at OSCON this week. I’m always happy to talk gosu and/or drink beer if you are up here.


Blocks Inside Out

In my previous post I gave some practical applications of blocks (a.k.a closures) in Gosu. In this post I’m going to look at one particular method mentioned in that article, where(), and try to show how the concept of blocks emerges from code you might be writing today. I hope by the end of it you’ll understand what I mean when I say that:

    “Classes help us abstract concepts, blocks help us abstract algorithms”

As a refresher, where() is a method in Gosu available on anything implementing Iterable<T>. It takes a single block and returns a new sublist of items where the block returns true.

Now, backing up a step, let’s look at some code that filters a list written in vanilla Java:

  List<String> shortStrings = new ArrayList<String>();
  for( String s : aListOfStrings ) {
    if( s.length() < 10 ) {
      shortStrings.add( s );
    }
  }
  return shortStrings;

Very standard Java code. I want to focus on the bolded part of the algorithm: it’s a little bit of logic that actually provides the filter. Note that it is very specific to this particular application of the filter algorithm: this bit of logic is deeply baked into the for-loop. This makes it difficult to extract the code around it, which is all fairly general, into a reusable bit of software.

Now, if filtering lists of strings on length was common enough in your code, you might extract this logic into a function and make the maximum (+1) length of the strings a parameter:

  List<String> findStringsLessThanLength( List<String> aListOfStrings, int length ) {
    List<String> shortStrings = new ArrayList<String>();
    for( String s : aListOfStrings ) {
      if( s.length() < length ) {
        shortStrings.add( s );
      }
    }
    return shortStrings;
  }

This has generalized the operation a bit: you now can pass in varying lengths and get different results. However you are still stuck with the fact that this method can only filter strings with a length less than some number. If, for example, you wanted strings greater than some length, you’d need to either introduce another method or, yikes, introduce another parameter.

Ugly.

At this point, depending on how crazy you are willing to get and how much you hate copy and pasting the same algorithm everywhere, you may choose to introduce a Predicate concept to your project. This allows you to pass an anonymous inner class to a method, where the anonymous inner class expresses the filter critera:

  return Iterables.filter( aListOfStrings, new Predicate<String>() {
    public boolean apply( String s ) {
      return s.length() < length;
    }
  }

Hmmm. Well, some people might think that’s better. I’m not so sure.

The problem (in my eyes) is that the amount of syntax overwhelms the core concept: we are just filtering a list, for goodness’ sake.

Looking back at the original implementation:

  List<String> shortStrings = new ArrayList<String>();
  for( String s : aListOfStrings ) {
    if( s.length() < 10 ) {
      shortStrings.add( s );
    }
  }
  return shortStrings;

It’s really just that bolded part we care about. Wouldn’t it be nice if we could pass just that to a function?

That’s exactly what blocks let you do:

   return aListOfStrings.where( \ s -> s.length() < 10 )

The block (again, the funny bit after the ‘\‘) has a single parameter, a string named ‘s‘, and returns true if s has a length less than 10, false otherwise. You can think of the block as a mini-function, if that helps, or as an mini-anonymous-inner-class. All blocks do is take arguments and return values, in an conveniently-inlineable syntax. And, of course, you write whatever expression you’d like in the bolded area, so this is a truly general solution to filtering things.

That’s what I mean when I say that blocks help you abstract algorithms: ideally, you’ll never have to write that filtering code ever again. Or that sorting, mapping or partitioning code either.

Sweet.


Effective Blocks

One of the cool features in Gosu is closures, colloquially known as “blocks”. Blocks act kind of like inline function definitions, allowing you to plug bits of logic into other algorithms. (One my standard lines in the talks is “Classes are a tool for generalizing concepts, blocks are a tool for generalizing algorithms.” Pithy and, hey, maybe even true.)

Trying to describe blocks in general terms doesn’t appear to be a great way to explain them: people get lost in the terminology and syntax. The way that I finally “got” blocks wasn’t by staring intently at a text book or wiki page, it was by seeing them in action.

In this vein, I’m going to describe some common programming problems you might face and then show you how blocks can help you solve them elegantly. I hope this will help you understand blocks in Gosu and how use them effectively.

“I’ve got a List of X and I need a List of Y”

This is perhaps the classic problem that blocks can help solve:

  • “I’ve got a list of Claims, and I need a list of Claim IDs.”
  • “I’ve got a list of Employees, and I need a list of last names.”

To address this problem, Gosu has added the map() function to Iterable objects. Taking the two examples above, you can write this code:

  var ids = claims.map( \ c -> c.ID )
  var lastNames = employees.map( \ emp -> emp.LastName )

The block is the funny little thing starting with a ‘\‘. Between the ‘\‘ and the ‘->‘ are the arguments to the block (remember, blocks are kind of like a mini-function) and after the ‘->‘ comes the body of the block.

Taking a look at the first line in the code example, we have a block that takes an argument c of type Claim (Gosu infers this type) and that returns the ID of that claim.

The map() function then uses this block to run through all the claims in the claims list and create a new list, filled with the ID’s of each claim.

Nice, eh?

Here’s another example:

“I’d like to sort List X by attribute Y”

Some concrete examples might be:

  • “I’d like to sort a list of Policies by their effective date.”
  • “I’d like to sort a list of Contacts by their age.”

Gosu has an orderBy() function for this:

  var sortedPolicies = policies.orderBy( \ p -> p.EffectiveDate )
  var sortedContacts = contacts.orderBy( \ c -> c.Age )

In Java, you might use a Comparator (and then try to remember when to return -1, 0, or 1) in conjunction with Collections.sort(). I hope you’ll agree that the Gosu above is easier to write and also reads more clearly.

The advantage of the block-based approach becomes even more glaring when you want to add secondary sort columns:

  var sortedPolicies = policies.orderBy( \ p -> p.EffectiveDate )
                                .thenBy( \ p -> p.Status )

Imagine the Java Comparator for that. Ugh.

“I’d like all the elements in List X where Y is true”

This is another classic:

  • “I’d like all the Claims that are past due.”
  • “I’d like all the Employees that are active.”

Gosu has an where() function for exactly this:

  var pastDueClaims = claims.where( \ c -> c.DueDate < Date.Today )
  var activeEmployees = employees.where( \ emp -> emp.Active )

The where() function returns a new list where any items that do not meet the criteria expressed in the block are filtered out.

Very nice.

“I’d like to split a List of X up on attribute Y”

This is a more advanced usage of blocks, but it can come in quite handy in some situations:

  • “I’d like to split a list of automobiles up based on their color.”
  • “I’d like to split my friends up by their birthday month.”

Gosu has a partition() function for this:

  var carsByColor = cars.partition( \ c -> c.Color )
  print( "All red cars: " + carsByColor["Red"] )
  var friendsByBirthdayMonth = friends.partition( \ f -> f.Birthday.Month )

The partition() function splits a list up based on the block you pass in. So, in the first line of code in this example, we split a list of cars up by their color, and get a map, where the key is a color and the value is a list of cars of that color. In the next line we print the list of cars that are red.

You could partition on any attribute: make, model, year, whatever. Blocks help generalize the operation.

Composing Methods

Gosu really starts to sing when you compose these methods:

  • “I’d like to get the ids of claims past due, ordered by the claim due date”

We can compose three methods together to solve this quite succinctly and clearly:

  var pastDueClaimIdsByDueDate = claims.where( \ c -> c.DueDate < Date.Today )
                                       .orderBy( \ c -> c.DueDate ) 
                                       .map( \ c -> c.ID ) 

The corresponding java code would be more verbose, and would probably conflate the various operations, making it more difficult to understand what the code was doing.

“Hmm, this is all data structure manipulation…”

Yep.

The sweet spot for blocks (in Gosu, at least) is in data structure manipulation. There are other areas they can come in handy (callbacks, listeners) but usually you’ll be using them to transform one data structure into another.

Personally, not a day that I’m writing java code goes by that I don’t wish I had blocks. Once you get used to them, it can be very hard to go back. I hope that this quick overview helps you use blocks effectively in your own Gosu code and have a bit more fun when you are coding.


Gosu in Dr. Dobbs

Gosu is this month’s Dr. Dobbs Language of the Month:

  http://drdobbs.com/open-source/231001429

Scott McKinney, the creator of Gosu, talks about some of the history and features of the language, and once again points out the Open Type System as the core innovation of Gosu.


It’s the little things…

Yesterday I was writing some java code, and I needed to count the number of occurrences of a character in a String. I wanted to stay in pure java and not introduce a library dependency, so I ended up writing this code:

  int count = 0;
  for( int i = 0; i < str.length(); i++ ) {
    if( str.charAt( i ) == '.' ) {
      count++;
    }
  }

This is a preposterous amount of code to write for such a simple and common problem. There is a hilarious thread on Stack Overflow on how to do the same thing:

  http://stackoverflow.com/questions/275944/how-do-i-count-the-number-of-occurrences-of-a-char-in-a-string

In Gosu, via CoreStringEnhancement, the code is simply this:

  var count = str.countMatches( "." )

There are big features in Gosu that make development more productive that we bang on a lot publicly (type inference, closures, and the The Open Type System to name a few) and rightly so. But these small things make a huge difference in day to day coding as well. It’s not like the code for these convenience methods are hard, it’s just that the methods are there and in a place you can find them.

The culture of Java has never emphasized making the easy stuff easy. A favorite example of mine is the code to read a file. Contrast that with how you do so in Gosu:

  var content = new File( "foo.txt" ).read() 

One of my pet theories is that the reason that dynamic languages get better, easier-to-deal-with libraries is that, since they don’t have good tool support, library developers simply can’t adopt the baroque APIs that Java developers have become used to. This creates a culture and a set of expectations that a library will be simple to get going with and will scale up in complexity only as necessary: otherwise the library just never gets off the ground.

We are aiming to capture that same culture of simplicity in Gosu, while retaining the advantages of a statically typed language.


Upcoming Gosu Events

There are some Gosu-related events coming up that people might want to consider attending:

July 12, 2011 6-9PM – Sacramento Java Users Group

Exit Certified

I’ll be giving a brief talk on the experimental Protocols typeloader for Gosu and, time permitting, I’ll talk about Ronin, a small Gosu-based web framework.

July 18-20th – JVM Language Summit

Oracle Santa Clara Campus

Most of the Gosu Team will be at the JVMLS: Scott McKinney (Gosu Team Lead), Dumitru Petrusca (IDE Lead), David Green (XML Guru) and me. I’ll be giving a talk on the Protocols typeloader, and implementing language features as a library using The Open Type System.

July 25-29th – OSCON 2011

Oregon Convention Center, Portland, OR

I’ll be at OSCON this year, giving an introduction to Gosu And The Open Type System.

If you are going to be at any of these and want to meet up and talk Gosu, or just drink beer, let me know: email or twitter.


In Defense Of “Name : Type”

A big (really *the* big) syntactic departure that Gosu took from Java is around type declarations. In Java, you have the time-honored c-style of declaration: type, followed by name:

  public String aMethod(String aParameter) {
    String theString = "The String " + aParameter;
    return theString;
  }

In Gosu, you have the name first, followed by a colon, then the type:

  function aMethod(aParameter : String ) : String {
    var theString : String = "The String " + aParameter
    return theString
  }

(Note that you would typically omit the local variable’s type in Gosu and let type inference do its thing.)

This syntax comes down to us from Pascal and is the standard in ECMAScript, which is the language Scott originally based Gosu on.

Now, this is just syntax. Not really a big deal, right?

Turns out, in practice, this actually is kind of a big deal to people. Let me quote Stephen Colebourne:

@Carson, thanks for writing about Gosu. The feature literal element looks good, and fits my view of how such things should work with # for the meta level. Unfortunately, Gosu declares variables as name : type, which makes it rather unappealing to me. Fix that, and I might take a real look at the language…

Despite the fact that Gosu has everything that Stephen wants with respect to properties, he won’t take a real look at the language because of the syntax around types. So this obviously is a big deal to Stephen, and I can’t say I blame him: as Oscar Wilde said, only a very shallow person does not judge by appearances.

So, while admitting that it will look a little strange the first time to Java developers, let me try to give a small defense of the Gosu type syntax.

First off, I think that the Gosu syntax reads more naturally once you get used to it. Taking the example method above, reading it left to right in Java gives:

“Returning a String, the method ‘aMethod’ takes a String called ‘aParameter'”

In Gosu you have:

“The function ‘aMethod’ takes ‘aParameter’ of type String and returns a String”

I believe that most english speakers would find the latter a more natural way to explain the method. This is subjective of course, but, hey, we are talking about syntax here.

A less subjective advantage of Gosu’s syntax becomes apparent when you look at generic methods.

Consider this example:

   <T> T aGenericMethod(T aParameter) {
    return aParameter;
  }

Note that in Java the generic type variable must be declared first, off in space, so that it can precede the return type of the method, which could (but need not) contain a reference to the type variable.

In Gosu, this is expressed much more naturally:

   function aGenericMethod<T>(aParameter : T) : T {
    return aParameter
  }

Because the return type of the method comes at the end, the type variable can nestle comfortably after the name of the function. As a bonus, this is much closer to the syntax of generic classes, with the type variable following the name, rather than preceding it.

This has ramifications at the call site, as well. In Java, you must prefix the function call with a type if you want to give an explicit parameterization:

  anObject.<String>aGenericMethod( "Yay generics!" );

In Gosu, again, you use a more natural syntax:

  anObject.aGenericMethod<String>( "Yay generics!" )

Note how similar this is to the syntax used for class parameterization (e.g. ArrayList<String>).

As an interesting corner case, in Java you simply can’t invoke a parameterized generic method without a preceding expression:

  <String>aGenericMethod( "Yikes" );

will not compile, whereas this will:

  this.<String>aGenericMethod( "Hmmm" );

Gosu, on the other hand, has no such restriction, since the method parameterization comes comfortably after the method name:

  aGenericMethod<String>( "It just works" );

Now, these are admittedly fairly small advantages. But, nonetheless, they are advantages. I hope that outlining them will convince some of you who are skeptical about the “name : Type” syntax to look more deeply at Gosu: it is a very nice language with some interesting features. (Not the least, the Open Type System, which Scott wrote about here.)


Feature Literals + Enhancements + Blocks == Properties++

Stephen Colebourne recently commented on the lack of formal properties in Java:

  http://www.jroller.com/scolebourne/entry/beans_and_properties

I completely agree with Stephen that properties should have been introduced to Java long ago: they were one of the first things that Scott added to Gosu and they have been incredibly useful at Guidewire. They aren’t really sexy or novel, but they solve a lot of practical problems.

In the latest release of Gosu, we incorporated feature literals, which allow you to reference a property (or method) using the # operator, making properties even easier to work with. A commenter on Stephen’s blog mentioned Gosu’s feature literals and had some nice things to say about our implementation, but mentioned that they do not support a listener infrastructure, where you can register a listener with a property to be invoked when the property is written or read.

Gosu obviously can’t provide a general listener infrastructure: the Open Type System works with all sorts of different types, each with different property implementations, storage models and so on. *However*, using feature literals, blocks, generics and enhancements, you can add some pretty cool support for property listeners on types that can support them.

I’ve thrown together a quick example that shows how this can be accomplished.

The entirety of the code in this example is up on github here:

  https://github.com/carsongross/Properties-Callback-Example

First, let’s create a simple class that encapsules property storage and supports callbacks:

package props
uses java.util.HashMap
uses java.util.ArrayList

class PropHolder {
  var _props = new HashMap<String, Object>()
  var _callback = new HashMap<String, List<block(val:Object):Object>>()
                    .toAutoMap( \ s -> new ArrayList<block(val:Object):Object>() )
  
  function setProp( name : String, val : Object ) {
    for( cb in _callback[name] ) {
      val = cb( val )
    }
    _props[name] = val
  }
  
  function getProp( name : String ) : Object {
    return _props[name]
  }
  
  function addPropListener( name : String, blk : block(val:Object):Object ) {
    _callback[name].add( blk )
  }
}

This is some simple infrastructure that just wraps a hash map to store properties in, and maintains a list of listeners to execute when a property is invoked.

Next, let’s create a simple gosu class that uses this infrastructure to store a property:

package props

class ExampleGosuClass implements IHaveListenableProperties {
  var _props = new PropHolder() 
  
  property get Prop1() : String {
    return _props.getProp( "Prop1" ) as String
  }

  property set Prop1( s : String ) {
    _props.setProp( "Prop1", s )
  }
  
  override function addListener( propName : String, listener : block(Object):Object ) {
    _props.addPropListener( propName, listener )
  }
  
  property get ThisAsProp() : ExampleGosuClass {
    return this
  }
}

This class implements a simple interface, IHaveListenableProperties:

package props

interface IHaveListenableProperties {
  function addListener( propName : String, listener : block(Object):Object );
}

Finally, and this is where we get tricky, so follow along closely, we are going to enhance a specific parameterization of the return type of the expression foo#Bar, which is the API class gw.lang.reflect.features.BoundPropertyReference:

package props

uses gw.lang.reflect.features.BoundPropertyReference

enhancement BoundPropertyRefEnhancement : BoundPropertyReference<IHaveListenableProperties, Object>  
{
  function addListener( blk : block(obj:Object):Object ) {
    var root = this.Ctx as IHaveListenableProperties
    root.addListener( this.PropertyInfo.Name, blk )
  }
}

Note that this method will only appear on BoundPropertyReference‘s that have a IHaveListenableProperties object at their root, due to the particular parameterization that we have chosen to enhance. (Cool trick, eh?)

So, with all that infrastructure in place, we can now write this code:

uses props.*

var ex1 = new ExampleGosuClass()
var ex2 = new ExampleGosuClass()

ex1#Prop1.addListener( \ o -> { print( o ); return o + " Yay!" }  )
ex1.Prop1 = "Hello Listeners!" // prints "Hello Listeners!" 
print( ex1.Prop1 )             // prints "Hello Listeners! Yay!"

ex2.Prop1 = "Nope, no listener..."  // no listener, so no printing
print( ex2.Prop1 ) 

In this code, we create two instances of ExampleGosuClass and add a block-based listener to the Prop1 property on ex1. We then set the property on ex1, which invokes the block and prints out the original value, then transforms the value before storing it. Finally, we do the same with ex2 to demonstrate that the listener is bound only to the Prop1 property on ex1.

So we’ve used feature literals, enhancements and blocks to implement a slick little properties listening system in Gosu. A system like this could be integrated into, for example, an O/R framework or a web framework even more elegantly since the listener infrastructure could be entirely internal.

Just another example of the excellent tools in Gosu available for framework builders. I think you’ll see a lot of innovation in this space in the next year or so, as our tools mature and Gosu becomes more widely known.


Follow

Get every new post delivered to your Inbox.

Join 39 other followers