Release notes for Groovy 2.5

Groovy 2.5 comes with a host of new features and capabilities.

Jar packaging and JDK9 support

Groovy versions prior to 2.5 included a core groovy jar and numerous "module" jars, e.g. groovy-xml.jar. Note: Groovy’s existing "modules" aren’t (currently) the same as Java9+ modules (more on that later).

We also provided a convenience "all" jar by jarjar’ing the core and "module" jars. As part of moving to better JDK9+ support, the "all" jar is no longer available but we do provide an "all" pom which brings in the equivalent component jars.

For most users referencing Groovy’s "all" dependency, simply bumping the version number should be sufficient for you to get all the same class files and transitive dependencies as before but there are some slight differences discussed below. For users with complex builds, which might (perhaps implicitly) reference the actual "all" jar itself, perhaps more remediation work might be required.

There are some caveats:

the groovy-all jar used to have all the transitive dependencies (e.g. bsf.jar, testng.jar, ant.jar) listed as optional when referenced via the all dependency but included when referenced by the module dependencies (e.g. groovy-bsf, groovy-testng, groovy-ant). Now with the groovy-all pom since it references the module dependencies, those transitive dependencies are all now automatically added. This means that you might be able to delete explicit references to those dependencies if you are actually using them, and conversely, you might wish to explicitly exclude the respective groovy-module (or transitive dependency) for those modules you aren’t in fact using.
please read the breaking changes section for unrelated changes to the included classes referenced by the "all" pom
also note that since the current jars don’t comply fully yet with JDK9+ requirements (e.g. split package requirements) you may not be able to place the Groovy jars on your module path. Instead, place them on the classpath. Ongoing work for Groovy 3.0 will address split package issues (see https://issues.apache.org/jira/browse/GROOVY-8647). If the Groovy "modules" you want to use aren’t in the list being rectified for Groovy 3.0, then you can probably use them on the module path. The Groovy build currently doesn’t test such usage.

Macro support

Groovy is well known for its extensible compilation process. The compiler supports a compile-time metaprogramming extension mechanism which allows the compilation process to be enhanced with new functionality. The mechanism allows the definition and execution of AST transformations at various stages of the compilation process.

Groovy comes with a number of built-in AST transformations and, importantly, you can write your own. To write such transformations until now, you have had to become familiar with the compiler’s internal representation of Groovy syntactic structures. But that has now changed.

Groovy 2.5+ supports macros. Macros let you use Groovy syntax directly rather than using the internal compiler representations when creating compile-time metaprogramming extensions. This puts the creation of transformations in the hands of all Groovy programmers not just Groovy compiler gurus.

Expressions and statements

Suppose we want to create an @Info transform which when placed on a class creates a getInfo() method. Calling the getInfo() method will display some information about the class. The traditional way to write the transformation code might be something like:

...
def clazz = new MethodCallExpression(new VariableExpression("this"), "getClass", EMPTY_ARGUMENTS)
def body = new ExpressionStatement(new MethodCallExpression(clazz, "getName", EMPTY_ARGUMENTS))
classNode.addMethod('getInfo', ACC_PUBLIC, STRING_TYPE, EMPTY_ARRAY, ClassNode.EMPTY_ARRAY, body)
...

This will print out the class name when the getInfo() method is called.

With macros, the first two lines become:

def body = macro(true) { getClass().name }

Creation of both expressions and statements are supported. The true parameter to the macro method ensures we get a statement here which is what addMethod needs.

Frequently you will want to receive parameters or resolve surrounding variables. This is supported using a special $v notation. Suppose we want to augment the getInfo method to additionally output the time the class was compiled and the Groovy version used. Our code might look like this:

def date = new ConstantExpression(new Date().toString())
def version = new ConstantExpression(GroovySystem.version)
def body = macro(true) {
    """\
    Name: ${getClass().name}
    Compiled: ${$v{ date }}
    Using Groovy version: ${$v{ version }}""".stripIndent()
}

What the macro does is convert any Groovy code inside the closure argument into its internal representation. When it gets to the $v placeholders it does no conversion and just inserts those sections directly. This is exactly what we want here since we wrote those parts old-school style using the internal AST data structures.

Macro classes

In addition to the ability to create statements and expressions, as supported by macro, you might want to create whole classes. The MacroClass capability allows you to do that. You supply the class you are wanting to create in a similar fashion to creating an anonymous inner class in Java. The following example shows the approach for a scenario where you might want to extract the information collected in previous examples in a class of its own:

ClassNode buildInfoClass(ClassNode reference) {
    def date = new ConstantExpression(new Date().toString())
    def vers = new ConstantExpression(GroovySystem.version)
    def name = new ClassExpression(reference)
    ClassNode infoClass = new MacroClass() {
        class DummyName {
            java.lang.String getName() { $v{ name }.name }
            java.lang.String getVersion() { $v{ vers } }
            java.lang.String getCompiled() { $v{ date } }
        }
    }
    infoClass.name = reference.name + 'Info'
    return infoClass
}

Note	Types inside the `MacroClass` implementation should be resolved inside, that’s why we had to write `java.lang.String` instead of simply writing `String`. See the documentation for further details.

The result of all this is that if we annotate a Foo class with @Info, then it is as if we also typed the following source code in as well:

class FooInfo {
    String getName() { 'Foo' }
    String getVersion() { '2.5.0' }
    String getCompiled() { 'Mon May 28 02:09:35 AEST 2018' }
}

AST matching

AST transforms typically add additional ASTNodes into the AST tree or change some existing subtrees of nodes with a transformed subtree. Macros are great for allowing us to succinctly describe the added or transformed subtree of nodes but don’t offer us anything for identifying the subtree which might be the candidate to be replaced. This is where ASTMatcher comes to the rescue. ASTMatcher compares two expressions and each expression can be created with or without macros.

Here’s a trivial example of a transformer that looks for binary expressions of the form 1 + 1 in our code and if it finds it, replaces it with the constant 2:

Expression transform(Expression exp) {
    Expression pattern = macro { 1 + 1 }
    if (ASTMatcher.matches(exp, pattern)) {
        return macro { 2 }
    }
    return super.transform(exp)
}

This happens at compile time, so speeds up our code at runtime.

We can generalise this a bit more since ASTMatcher supports constraints:

Expression pattern = macro { a + b }.withConstraints {
    placeholder a
    placeholder b
    anyToken()
}

The placeholder constraint allows any term to appear and the anyToken allows '-', '*', '/', '<' and other tokens to replace the '+'.

We can make our transform a bit smarter too as follows:

Expression transform(Expression exp) {
    if (ASTMatcher.matches(exp, pattern)) {
        BinaryExpression be = exp
        Expression lhs = be.leftExpression
        Expression rhs = be.rightExpression
        if (lhs instanceof ConstantExpression && rhs instanceof ConstantExpression) {
            def left = lhs.value
            def right = rhs.value
            if ((left instanceof String && right instanceof String)||
                    (left instanceof Integer && right instanceof Integer)) {
                def op = be.operation.text
                left = left instanceof String ? "'" + left + "'" : left
                right = right instanceof String ? "'" + right + "'" : right
                def result = new GroovyShell().evaluate "$left $op $right"
                return constX(result)
            }
        }
    }
    return exp.transformExpression(this)
}

Now we can use our transform, e.g.:

@SmartOps
class Foo {
    int theAnswer = 40 + 2
    int eight = 4 * 2
    def foobar = 'foo' + 'bar'
    def test() {
        assert 3 < 4
        assert foobar.size() == 6
        assert theAnswer + eight == 25 + 25
    }
}

If we look at this class at the end of the SEMANTIC_ANALYSIS phase in the AST browser, we’ll indeed see that the initial value expression for theAnswer, eight and foobar are binary expressions as is the expression for the first assert and the right-hand side of the == expression for the third assert`. If we move forward to the end of the CANONICALIZATION phase we’ll see that all 5 of those binary expressions are now constant expressions. It is as if we had typed our source code in as:

class Foo {
    int theAnswer = 42
    int eight = 8
    def foobar = 'foobar'
    def test() {
        assert true
        assert foobar.size() == 6
        assert theAnswer + eight == 50
    }
}

Macro methods

The compiler capability to expand macros with their replacements can also be enhanced by your own methods. Consider the following definition:

class StringMacroMethods {
    @Macro
    static Expression upper(MacroContext macroContext, ConstantExpression constX) {
        if (constX.value instanceof String) {
            return new ConstantExpression(constX.value.toUpperCase())
        }
        macroContext.sourceUnit.addError(new SyntaxException("Can't use upper with non-String", constX))
    }
}

If you register the method in the same way as you would with extension methods (by creating a reference to the class in a META-INF/groovy/org.codehaus.groovy.runtime.ExtensionMethods file).

Now, assuming the META-INF file and class are on your classpath, you can use the upper method in your code such as shown in the following test code:

assertScript '''
    def foo = upper('Foo')
    assert foo == 'FOO'
'''
def msg = shouldFail '''
    def foo = upper(42)
'''
assert msg.contains("Can't use upper with non-String")

It’s important to realise that use of upper doesn’t cause a call to toUpperCase to be embedded in the bytecode but rather causes toUpperCase to be called at compile time.

New AST Transformations

@AutoFinal automatically applies the "final"-keyword to every parameter/field of an annotated class/method/closure/ctor. (GROOVY-8300).
@AutoImplement allows you to provide dummy implementations of any abstract methods that might be inherited from super classes or interfaces (GROOVY-7860).
@ImmutableBase checks the validity of an immutable class and makes some preliminary changes to the class to support immutability. Normally not used directly but brought in automatically by the @Immutable meta annotation.
@ImmutableOptions allows known immutable properties/classes to be declared. Normally not used directly but brought in automatically by the @Immutable meta annotation.
@KnownImmutable is a marker interface used to designate a class as being immutable. Not usually used explicitly but rather implicitly added via the @Immutable meta annotation. If you create your own Java or Groovy immutable class manually, you can add this annotation to save you having to list your class as one of the known immutable classes.
@MapConstructor adds a Map-based constructor to a class. This allows a usage style similar to Groovy’s named parameters but doesn’t use the no-arg constructor and then call setters. This is useful if you have final properties in your class or you need the class file to have the Map constructor for polyglot integration purposes (GROOVY-7353).
@NamedDelegate is a marker interface used to indicate that the property names of the annotated parameter represent valid key names when using named arguments and that the property types are applicable for type checking purposes
@NamedParam is a marker interface used to indicate that the name of the annotated parameter (or specified optional name) is a valid key name when using named arguments and that the parameter type is applicable for type checking purposes
@NamedParams is the collector annotation for @NamedParam
@NamedVariant allows construction of a named-arg equivalent of a method or constructor. This allows the creation of methods or constructors which can be used with Groovy’s named-argument syntax yet still retain good type checking capabilities. Combining @NamedDelegate and @NamedParam when using @NamedVariant can be quite powerful. For example, given these class definitions:

class Animal {
    String type, name
}

@ToString(includeNames=true)
class Color {
    Integer r, g, b
}

@NamedVariant
String foo(String s1, @NamedParam String s2,
           @NamedDelegate Color shade,
           @NamedDelegate Animal pet) {
    "$s1 $s2 ${pet.type?.toUpperCase()}:$pet.name $shade"
}

the constructed foo method will look like:

String foo(@NamedParam(value = 's2', type = String)
           @NamedParam(value = 'r', type = Integer)
           @NamedParam(value = 'g', type = Integer)
           @NamedParam(value = 'b', type = Integer)
           @NamedParam(value = 'type', type = String)
           @NamedParam(value = 'name', type = String)
           Map __namedArgs, String s1) {
    // some key validation code ...
    return this.foo(s1, __namedArgs.s2,
            ['r': __namedArgs.r, 'g': __namedArgs.g, 'b': __namedArgs.b] as Color,
            ['type': __namedArgs.type, 'name': __namedArgs.name] as Animal)
}

@PropertyOptions a marker annotation used to indicate that special property handling code will be generated for this class. It can be used to override how properties are set within the constructor or accessed via getters. This lets you customize for instance how @Immutable classes are generated.
@VisibilityOptions is a marker annotation used in the context of AST transformations to provide a custom visibility. One example of its use would be when you want to create a private constructor that might be called only from within a static factory method of the class:

import groovy.transform.*
import static groovy.transform.options.Visibility.PRIVATE

@TupleConstructor
@VisibilityOptions(PRIVATE)
class Person {
    String name
    static makePerson(String first, String last) {
        new Person("$first $last")
    }
}

Here using @TupleConstructor saves us the effort of writing the typical boilerplate code within the constructor but we don’t need to make it public but instead use it within our handwritten factory method which can focus on our particular business logic relevant to that factory method.

AST Transformation improvements

@Canonical is now a meta-annotation (GROOVY-6319) allowing more flexible usage of the annotation attributes from its constituent annotations and allowing you to define an alternative custom meta-annotation. @Canonical expands into the @TupleConstructor, @EqualsAndHashCode and @ToString annotations. Any annotation attributes are automatically distributed to the component annotations that support them as shown in the following example:

@Canonical(cache = true, useSetters = true, includeNames = true)
class Point {
  int x, y
}

is expanded into:

@ToString(cache = true, includeNames = true)
@TupleConstructor(useSetters = true)
@EqualsAndHashCode(cache = true)
class Point {
  int x, y
}

@Immutable is now a meta-annotation (GROOVY-8440) with the same advantages as described for @Canonical. It expands into numerous other annotations as shown in the following example:

@Immutable
class Point {
  int x, y
}

is equivalent to:

@ToString(includeSuperProperties = true, cache = true)
@EqualsAndHashCode(cache = true)
@ImmutableBase
@ImmutableOptions
@PropertyOptions(propertyHandler = groovy.transform.options.ImmutablePropertyHandler)
@TupleConstructor(defaults = false)
@MapConstructor(noArg = true, includeSuperProperties = true, includeFields = true)
@KnownImmutable
class Point {
  int x, y
}

This might seem like quite a few component annotations but you rarely see the expanded list and having these annotations gives you fine-grained control when combing the bits you want, for example you can create a dependency injection (one constructor only) friendly immutable class by using this combination:

@ImmutableBase
@PropertyOptions(propertyHandler = ImmutablePropertyHandler)
@Canonical(defaults=false)
class Shopper {
  String first, last
  Date born
  List items
}

@Immutable now supports Java’s Optional container class (GROOVY-7600).
@Immutable handles inheritance hierarchies (GROOVY-7162).
@Immutable handles JSR-310 java.time classes (GROOVY-7599).
@Delegate can now be used on getters (GROOVY-7769).
@TupleConstructor now supports pre and post closure conditions to match the functionality provided by @MapConstructor (GROOVY-7769).
@TupleConstructor and @Builder should be able to use defined setters rather than the field directly (GROOVY-7087).
@Newify supports an additional attribute that allows selecting the classes whose constructors can be invoked without the new keyword using a regex pattern for the class name:

@Newify(pattern="[A-Z].*")}
class MyTreeProcessor {
    final myTree = Tree(Tree(Leaf("A"), Leaf("B")), Leaf("C"))
    ...
}

Most annotations check property and field names provided to annotation attributes (GROOVY-7087).

Tool improvements

Some improvements were made to various tools:

groovy and groovyConsole now let you run JUnit 5 tests directly:

import org.junit.jupiter.api.*
// other imports not shown ...

class MyTest {
  @Test
  void streamSum() {
    assert Stream.of(1, 2, 3).mapToInt{ i -> i }.sum() > 5
  }

  @RepeatedTest(value=2, name = "{displayName} {currentRepetition}/{totalRepetitions}")
  void streamSumRepeated() {
    assert Stream.of(1, 2, 3).mapToInt{i -> i}.sum() == 6
  }

  private boolean isPalindrome(s) { s == s.reverse()  }

  @ParameterizedTest // requires org.junit.jupiter:junit-jupiter-params
  @ValueSource(strings = [ "racecar", "radar", "able was I ere I saw elba" ])
  void palindromes(String candidate) {
    assert isPalindrome(candidate)
  }

  @TestFactory
  def dynamicTestCollection() {[
    dynamicTest("Add test") { -> assert 1 + 1 == 2 },
    dynamicTest("Multiply Test") { -> assert 2 * 3 == 6 }
  ]}
}

which when run will show:

JUnit5 launcher: passed=8, failed=0, skipped=0, time=246ms

with additional information available via logging if needed.

groovysh offers easier access to grapes (GROOVY-6514).

groovy:000> :grab 'com.google.guava:guava:24.1-jre'
groovy:000> import com.google.common.collect.ImmutableBiMap
===> com.google.common.collect.ImmutableBiMap
groovy:000> m = ImmutableBiMap.of('foo', 'bar')
===> [foo:bar]
groovy:000> m.inverse()
===> [bar:foo]
groovy:000>

groovyConsole now provides an ASMifier tab within the AstBrowser (GROOVY-8091).

CliBuilder changes

Groovy’s CliBuilder now supports annotation style definitions (GROOVY-7825).
Revamped versions of CliBuilder now exist supporting Commons CLI and Picocli backed implementations. See this blog post for more details.

Other improvements

Repeated annotation support has been added
Alternative to with called tap that has an implicit return delegate (GROOVY-3976).
Various JSON customization options are now supported (GROOVY-6975 and GROOVY-6854).
Method parameter names are now accessible at runtime (GROOVY-7423).

Breaking changes

A few issues fixed might also be considered breaking changes in some situations:

The extension methods for the java.util.Date class are now in a separate groovy-dateutil module which isn’t included by default when using the groovy-all pom dependency. Add the additional module as a dependency if you need it or consider migrating to the java.time JSR-310 classes (similar Groovy extension methods exist for those classes and they are included by default when using the groovy-all pom dependency).
@TupleConstructor could use the order of properties listed in 'includes' when that option is used (GROOVY-8016)
@ToString could output properties in a predefined order when 'includes' is used (GROOVY-8014)
AstNodeToScriptAdapter should output source using the recommended modifier order (GROOVY-7967)
ObjectRange iterator returns null instead of NoSuchElementException (GROOVY-7961)
IntRange iterator returns null instead of NoSuchElementException (GROOVY-7960) (GROOVY-7937)
o.c.g.r.t.DefaultTypeTransformation does not apply the right toString on primitive arrays when transforming to String (GROOVY-7853)
Remove synchronized methods of groovy.sql.Sql and document it as not thread-safe (GROOVY-7673)
InvokerHelper formatting methods have inconsistent API (GROOVY-7563)
Fix up transforms (apart from TupleConstructor) which are affected by empty includes default (GROOVY-7529)
TupleConstructor with empty includes includes all (GROOVY-7523)
TupleConstructor overwrites empty default constructors (GROOVY-7522)
ResourceGroovyMethods/NioGroovyMethods BOM behavior is inconsistent (GROOVY-7465)
API inconsistency between takeWhile, dropWhile and collectReplacements for CharSequences (GROOVY-7433)
@ToString could support non-field properties (GROOVY-7394)
same linkedlist code different behavior between groovy and java (GROOVY-6396)
CLONE - same linkedlist code different behavior between groovy and java (fix priority of DGM methods vs actual methods on an object)
Accessing private methods from public ones using categories and inheritance causes MissingMethodException (GROOVY-6263)
Have the elvis operator (?:) support the Optional type in Java 8 (GROOVY-6744)
java.util.Optional should evaluate to false if empty (GROOVY-7611)
If you use the FileSystemCompiler class programmatically (rather than via the groovyc commandline) and you use the part of it for handling commandline processing, then you might notice that it has been converted to picocli and usage of a handful of methods will throw a DeprecationException which mentions the alternative approach you should use.

Known issues

The GDK documentation for the java.time extensions wasn’t included in the 2.5.0 release (please see 2.5.1+).
Users of groovy.util.CliBuilder need to also include the org.codehaus.groovy:groovy-cli-commons dependency on their compile classpath in addition to groovy or groovy-all. This won’t be required after the next maintenance release but users should migrate away from that class in any case as it will be removed from the next major version of Groovy.
Users of Spock 1.1-groovy-2.4 may find strange ClassCastException errors with some tests, e.g. with cleanup: clauses. Using Spock 1.2-groovy-2.4-SNAPSHOT from the https://oss.sonatype.org/content/repositories/snapshots/ repo may help but work is ongoing to improve Spock support.
Users combining final and @CompileStatic or final and Spock may see errors from the final variable analyzer. Work is underway to resolve those error messages. You may need to temporarily remove the final modifier in the meantime.
Users needing the groovy-xml module and running on JDK9 and above may need to use the --add-modules java.xml.bind command-line option to fix the break in backwards compatibility caused by JDK9+.
JDK9+ produces warnings with many libraries including Groovy due to some planned future restrictions in the JDK. Work is underway to re-engineer parts of Groovy to reduce/remove those warnings. Users wanting to hush the warnings as an interim measure may consider using the --add-opens escape clause offered by JDK9+. See commit 92bd96f (currently reverted) on the Groovy master branch for a potential list to add.
Users of the Spring Boot Gradle plugin wanting to upgrade to Groovy 2.5.0 might like to re-check the plugin doco if you have any troubles. More comments here: https://github.com/spring-projects/spring-boot/issues/13444
proper OSGi operation was inadvertently broken due to split package issues. This should be fixed in 2.5.1. You can only use combinations of Groovy classes/modules which avoid the split package problem in the meantime. See https://issues.apache.org/jira/browse/GROOVY-8647 for the list of work being done on split packages to identify the affected modules/classes.

JDK requirements changes

Groovy 2.5 requires JDK8+ to build and JDK7 is the minimum version of the JRE that we support.

More information

You can browse all the tickets closed for Groovy 2.5.0 in JIRA.

Addendum for 2.5.1

We normally try very hard not to have breaking changes in minor releases, and that remains the case for the APIs within 2.5.1, but due to some bugs that slipped through 2.5.0 release candidate shakedown and also some usability feedback, we have made the following breaking packaging changes in 2.5.1:

groovy-bsf is now an optional module and isn’t referenced by default when using the groovy-all pom dependency (this helps with an OSGi issue)
Groovy’s extension methods for JAXB are now in the optional groovy-jaxb module. This means that you aren’t using JAXB, you don’t need to worry about the JDK JAXB changes even if you are using the groovy-all pom, you should have nothing to do.
If you do need the JAXB extension methods, please add that module to your dependency list, and depending on your JDK version (see table below), you may have some further steps, e.g. add additional dependencies or command-line switches, or set one of the JAVA_OPTS or JDK_JAVA_OPTIONS environment variables.

JDK Version

Options

JDK8

Do nothing

JDK9/10

Command-line switch/environment variable value:
--add-modules java.xml.bind
when starting the JDK that Groovy will use or see solution for JDK11. For a gradle build that needs to work across JDK versions, you might need something like:

if (JavaVersion.current().isJava9Compatible()) {
   def jaxbJvmArg = '--add-modules java.xml.bind'
   tasks.withType(GroovyCompile) {
     groovyOptions.fork = true
     groovyOptions.forkOptions.jvmArgs.add(jaxbJvmArg)
   }
   tasks.withType(Test) {
     jvmArgs jaxbJvmArg
   }
}

The command-line switch approach is automatically done for the Groovy command-line tools, e.g. groovyc, groovy, groovyConsole, etc. since groovy-jaxb is currently bundled in the Groovy install.

JDK11

Add these dependencies:
    javax.xml.bind:jaxb-api:2.3.0
    com.sun.xml.bind:jaxb-core:2.3.0.1
    com.sun.xml.bind:jaxb-impl:2.3.0.1
    javax.activation:activation:1.1.1
The first is required at compile time, the last three at runtime, the last only required for JDK11. For an example Gradle build that works JDK8 through 11, see build.gradle in the groovy-jaxb subproject.

Note: The above steps aren’t our final plan in this space. For 2.5.1, we focused just on making life better for non-JAXB users using Groovy via Gradle or Maven. For 2.5.2, we hope to make much of the above details as hidden as possible and certainly hidden for users of the groovy command-line tools for JDK9+.

If you are using JDK9+ and the command-line tools (e.g. groovy, groovyc, groovyConsole) and want to make your life easier on 2.5.1 in the meantime, please consider doing one of the following:

remove groovy-jaxb jar from your Groovy distribution’s lib directory
copy the startGroovy or startGroovy.bat script files from master
copy the four jars mentioned in the above JDK11 table entry into your lib directory

Addendum for 2.5.10

Known issues

2.5.10 may fail to compile code containing a switch statement under certain conditions. It only affects code where the switch statement is the last statement in a method and the switch statement includes a case statement with a break statement and no other statements. See (GROOVY-9424) for known workarounds.

Addendum for 2.5.13

Breaking changes

If you are using the SecureASTCustomizer and relying on the exact wording of error messages, e.g. in tests, then you may need to tweak the wording in those tests. See (GROOVY-9594) for more details.