Why There Have Been No Posts Lately…

I apologize for the extended absence of posts here on the dev blog, but I’ve been busy with a new Open Source project:

  http://jschema.org

The JSchema Project is an attempt to define a very simple schema system for JSON documents. JSON, for those who are not familiar, is a subset of Javascript that has proven to be a useful and less verbose alternative to XML for data-interchange:

 {
    "group" : "Developers",
    "members" : [
      { "id" : 1, "first_name" : "Joe", "last_name" : "Smith" },
      { "id" : 2, "first_name" : "Jennifer", "last_name" : "Mitchum" }
    ]
 }

It is easy to parse and produce, and integrates well with newer front-end technologies written in Javascript.

The motivation for this project, it will not surprise you, was Gosu. An excellent engineer at Amica, JP Camara, had been playing around with a JSON-based Type Loader for Open Source Gosu. I contacted him about adopting a schema-based approach, and began looking at the industry standard, JSON Schema. It quickly became apparent that JSON Schema was too complicated, and that a small extension to JP’s simple and elegant template approach would yield an expressive schema language, with a Gosu Type Loader to boot.

To give you a taste of how simple JSchema is, here is the schema for the JSON example above:

 {
    "group" : "string",
    "members" : [
      { "id" : "integer", "first_name" : "string", "last_name" : "string" }
    ]
 }

I hope that schema makes intuitive sense to most people: JSchema was designed such that the schema corresponds closely to the documents it describes, and it should be easy to take an example JSON document and transform it into a JSchema schema.

The JSchema specification is still very young but I think it is reasonably complete. We may add some more core data types (a ‘bytes’ datatype for raw byte data seems like it might be useful, for example) but the core ideas are working out well in our Gosu implementation (I’ll blog more about that in a later post).

If you are interested in the specification and/or participating in its design (or making the website prettier), you can fork it here. Participation is very welcome!


JVMLS Talk Is Now Up

You can watch the video here:

  http://medianetwork.oracle.com/media/show/17005

Many thanks to the folks at Oracle for getting it up.


Faking Builders With Gosu

One of the features of Groovy (being aped by Kotlin) is builders, a nice syntactic way to compose nested content.

A canonical example of this, I think, would be building an HTML page. The Groovy page has an example of this:

html {
    head {
        title"XML encoding with Groovy"
    }
    body {
        h1"XML encoding with Groovy"
        p"this format can be used as an alternative markup to XML"

        / an element with attributes and text content /
        ahref:'http://groovy.codehaus.org' ["Groovy"]

        / mixed content /
        p [
            "This is some",
            b"mixed",
            "text. For more see the",
            ahref:'http://groovy.codehaus.org' ["Groovy"],
            "project"
        ]
        p "some text"
    }
}

Gosu does not support builders, but I thought it would be fun to see how close we could get to them with the syntactic tools that Gosu does have. Using names arguments and default parameters, in particular, I think we could get pretty darned close.

So, the first trick I’m going to use in faking builders is something I call the Keefer Gambit: I’m going to create an empty HtmlUtil interface and then create an enhancement of that interface that supplies all the methods you will use to build html. This allows you to mix in the HTML functionality wherever you want it by simply implementing the (empty) interface. This (hack|elegant solution) for mixins in Gosu was first pointed out by Alan Keefer, hence the name.

OK, given that general approach, the syntactic tools we can use to fake builders are named arguments and default parameters. Let’s look at implementing the html element method in our enhancement:

  function html( head : String = "", body : List<String> = null) : String {
    var bodyContent = body == null ? "" : body.join( "\n" )
    return "<html><head>\n${head}</head>\n<body>${bodyContent}</body>\n</html>"
  }

Nothing too fancy here: there are two parameters, each with a default value. The function will return these two elements, wrapped up in the appropriate tags. If the parameters are omitted then the default values are used, and you can use named parameters when invoking this method because it is defined in Gosu.

I’m not going to walk through the implementation of each method, rather I’ll just point you to the sample code I put up on GitHub here:

    https://github.com/carsongross/BuilderExample

But using that code, I can write this code in an Example client:

html(
    :head = head(
        :title = "XML encoding with Gosu"
    ),
    :body = {
        h1( "XML encoding with Gosu" ),
        p( "this format can be used as an alternative markup to XML" ),
        /* an element with attributes and text content */
        ahref( 'http://gosu-lang.org', "Gosu"),
        /* mixed content  */
        p({
            "This is some",
            b("mixed"),
            "text. For more see the",
            ahref('http://gosu-lang.org', "Gosu"),
            "project"
        }),
        p( "some text" )
    }
)

Now, I’ll admit that isn’t quite as clean as the Groovy: we violate the DRY principle when we generate the head element, for example. And things are a bit dicier around when you use parens vs. curly braces. Also, I have to admit, this is a relatively simple example. But I think that this is a pretty clean translation nonetheless, and it was doable without adding any new syntactic features to Gosu.

Gosu, unlike Groovy, Scala or Ruby, requires parentheses around method arguments, and it doesn’t support more exotic things like var-args (it has a nice list literal instead) or implicit arguments: we favor regularity over succinctness beyond a certain point. Despite this, I’m comfortable with the expressiveness of the syntax of Gosu.

I’d rather see innovations in libraries, leveraging The Open Type System rather than much continued innovation in the core language syntax.

I’ll be at OSCON this week. I’m always happy to talk gosu and/or drink beer if you are up here.


Blocks Inside Out

In my previous post I gave some practical applications of blocks (a.k.a closures) in Gosu. In this post I’m going to look at one particular method mentioned in that article, where(), and try to show how the concept of blocks emerges from code you might be writing today. I hope by the end of it you’ll understand what I mean when I say that:

    “Classes help us abstract concepts, blocks help us abstract algorithms”

As a refresher, where() is a method in Gosu available on anything implementing Iterable<T>. It takes a single block and returns a new sublist of items where the block returns true.

Now, backing up a step, let’s look at some code that filters a list written in vanilla Java:

  List<String> shortStrings = new ArrayList<String>();
  for( String s : aListOfStrings ) {
    if( s.length() < 10 ) {
      shortStrings.add( s );
    }
  }
  return shortStrings;

Very standard Java code. I want to focus on the bolded part of the algorithm: it’s a little bit of logic that actually provides the filter. Note that it is very specific to this particular application of the filter algorithm: this bit of logic is deeply baked into the for-loop. This makes it difficult to extract the code around it, which is all fairly general, into a reusable bit of software.

Now, if filtering lists of strings on length was common enough in your code, you might extract this logic into a function and make the maximum (+1) length of the strings a parameter:

  List<String> findStringsLessThanLength( List<String> aListOfStrings, int length ) {
    List<String> shortStrings = new ArrayList<String>();
    for( String s : aListOfStrings ) {
      if( s.length() < length ) {
        shortStrings.add( s );
      }
    }
    return shortStrings;
  }

This has generalized the operation a bit: you now can pass in varying lengths and get different results. However you are still stuck with the fact that this method can only filter strings with a length less than some number. If, for example, you wanted strings greater than some length, you’d need to either introduce another method or, yikes, introduce another parameter.

Ugly.

At this point, depending on how crazy you are willing to get and how much you hate copy and pasting the same algorithm everywhere, you may choose to introduce a Predicate concept to your project. This allows you to pass an anonymous inner class to a method, where the anonymous inner class expresses the filter critera:

  return Iterables.filter( aListOfStrings, new Predicate<String>() {
    public boolean apply( String s ) {
      return s.length() < length;
    }
  }

Hmmm. Well, some people might think that’s better. I’m not so sure.

The problem (in my eyes) is that the amount of syntax overwhelms the core concept: we are just filtering a list, for goodness’ sake.

Looking back at the original implementation:

  List<String> shortStrings = new ArrayList<String>();
  for( String s : aListOfStrings ) {
    if( s.length() < 10 ) {
      shortStrings.add( s );
    }
  }
  return shortStrings;

It’s really just that bolded part we care about. Wouldn’t it be nice if we could pass just that to a function?

That’s exactly what blocks let you do:

   return aListOfStrings.where( \ s -> s.length() < 10 )

The block (again, the funny bit after the ‘\‘) has a single parameter, a string named ‘s‘, and returns true if s has a length less than 10, false otherwise. You can think of the block as a mini-function, if that helps, or as an mini-anonymous-inner-class. All blocks do is take arguments and return values, in an conveniently-inlineable syntax. And, of course, you write whatever expression you’d like in the bolded area, so this is a truly general solution to filtering things.

That’s what I mean when I say that blocks help you abstract algorithms: ideally, you’ll never have to write that filtering code ever again. Or that sorting, mapping or partitioning code either.

Sweet.


Effective Blocks

One of the cool features in Gosu is closures, colloquially known as “blocks”. Blocks act kind of like inline function definitions, allowing you to plug bits of logic into other algorithms. (One my standard lines in the talks is “Classes are a tool for generalizing concepts, blocks are a tool for generalizing algorithms.” Pithy and, hey, maybe even true.)

Trying to describe blocks in general terms doesn’t appear to be a great way to explain them: people get lost in the terminology and syntax. The way that I finally “got” blocks wasn’t by staring intently at a text book or wiki page, it was by seeing them in action.

In this vein, I’m going to describe some common programming problems you might face and then show you how blocks can help you solve them elegantly. I hope this will help you understand blocks in Gosu and how use them effectively.

“I’ve got a List of X and I need a List of Y”

This is perhaps the classic problem that blocks can help solve:

  • “I’ve got a list of Claims, and I need a list of Claim IDs.”
  • “I’ve got a list of Employees, and I need a list of last names.”

To address this problem, Gosu has added the map() function to Iterable objects. Taking the two examples above, you can write this code:

  var ids = claims.map( \ c -> c.ID )
  var lastNames = employees.map( \ emp -> emp.LastName )

The block is the funny little thing starting with a ‘\‘. Between the ‘\‘ and the ‘->‘ are the arguments to the block (remember, blocks are kind of like a mini-function) and after the ‘->‘ comes the body of the block.

Taking a look at the first line in the code example, we have a block that takes an argument c of type Claim (Gosu infers this type) and that returns the ID of that claim.

The map() function then uses this block to run through all the claims in the claims list and create a new list, filled with the ID’s of each claim.

Nice, eh?

Here’s another example:

“I’d like to sort List X by attribute Y”

Some concrete examples might be:

  • “I’d like to sort a list of Policies by their effective date.”
  • “I’d like to sort a list of Contacts by their age.”

Gosu has an orderBy() function for this:

  var sortedPolicies = policies.orderBy( \ p -> p.EffectiveDate )
  var sortedContacts = contacts.orderBy( \ c -> c.Age )

In Java, you might use a Comparator (and then try to remember when to return -1, 0, or 1) in conjunction with Collections.sort(). I hope you’ll agree that the Gosu above is easier to write and also reads more clearly.

The advantage of the block-based approach becomes even more glaring when you want to add secondary sort columns:

  var sortedPolicies = policies.orderBy( \ p -> p.EffectiveDate )
                                .thenBy( \ p -> p.Status )

Imagine the Java Comparator for that. Ugh.

“I’d like all the elements in List X where Y is true”

This is another classic:

  • “I’d like all the Claims that are past due.”
  • “I’d like all the Employees that are active.”

Gosu has an where() function for exactly this:

  var pastDueClaims = claims.where( \ c -> c.DueDate < Date.Today )
  var activeEmployees = employees.where( \ emp -> emp.Active )

The where() function returns a new list where any items that do not meet the criteria expressed in the block are filtered out.

Very nice.

“I’d like to split a List of X up on attribute Y”

This is a more advanced usage of blocks, but it can come in quite handy in some situations:

  • “I’d like to split a list of automobiles up based on their color.”
  • “I’d like to split my friends up by their birthday month.”

Gosu has a partition() function for this:

  var carsByColor = cars.partition( \ c -> c.Color )
  print( "All red cars: " + carsByColor["Red"] )
  var friendsByBirthdayMonth = friends.partition( \ f -> f.Birthday.Month )

The partition() function splits a list up based on the block you pass in. So, in the first line of code in this example, we split a list of cars up by their color, and get a map, where the key is a color and the value is a list of cars of that color. In the next line we print the list of cars that are red.

You could partition on any attribute: make, model, year, whatever. Blocks help generalize the operation.

Composing Methods

Gosu really starts to sing when you compose these methods:

  • “I’d like to get the ids of claims past due, ordered by the claim due date”

We can compose three methods together to solve this quite succinctly and clearly:

  var pastDueClaimIdsByDueDate = claims.where( \ c -> c.DueDate < Date.Today )
                                       .orderBy( \ c -> c.DueDate ) 
                                       .map( \ c -> c.ID ) 

The corresponding java code would be more verbose, and would probably conflate the various operations, making it more difficult to understand what the code was doing.

“Hmm, this is all data structure manipulation…”

Yep.

The sweet spot for blocks (in Gosu, at least) is in data structure manipulation. There are other areas they can come in handy (callbacks, listeners) but usually you’ll be using them to transform one data structure into another.

Personally, not a day that I’m writing java code goes by that I don’t wish I had blocks. Once you get used to them, it can be very hard to go back. I hope that this quick overview helps you use blocks effectively in your own Gosu code and have a bit more fun when you are coding.


Gosu in Dr. Dobbs

Gosu is this month’s Dr. Dobbs Language of the Month:

  http://drdobbs.com/open-source/231001429

Scott McKinney, the creator of Gosu, talks about some of the history and features of the language, and once again points out the Open Type System as the core innovation of Gosu.


It’s the little things…

Yesterday I was writing some java code, and I needed to count the number of occurrences of a character in a String. I wanted to stay in pure java and not introduce a library dependency, so I ended up writing this code:

  int count = 0;
  for( int i = 0; i < str.length(); i++ ) {
    if( str.charAt( i ) == '.' ) {
      count++;
    }
  }

This is a preposterous amount of code to write for such a simple and common problem. There is a hilarious thread on Stack Overflow on how to do the same thing:

  http://stackoverflow.com/questions/275944/how-do-i-count-the-number-of-occurrences-of-a-char-in-a-string

In Gosu, via CoreStringEnhancement, the code is simply this:

  var count = str.countMatches( "." )

There are big features in Gosu that make development more productive that we bang on a lot publicly (type inference, closures, and the The Open Type System to name a few) and rightly so. But these small things make a huge difference in day to day coding as well. It’s not like the code for these convenience methods are hard, it’s just that the methods are there and in a place you can find them.

The culture of Java has never emphasized making the easy stuff easy. A favorite example of mine is the code to read a file. Contrast that with how you do so in Gosu:

  var content = new File( "foo.txt" ).read() 

One of my pet theories is that the reason that dynamic languages get better, easier-to-deal-with libraries is that, since they don’t have good tool support, library developers simply can’t adopt the baroque APIs that Java developers have become used to. This creates a culture and a set of expectations that a library will be simple to get going with and will scale up in complexity only as necessary: otherwise the library just never gets off the ground.

We are aiming to capture that same culture of simplicity in Gosu, while retaining the advantages of a statically typed language.


Follow

Get every new post delivered to your Inbox.

Join 39 other followers