compiler construction - Compiling a Java Class in memory with `lombok` annotations and Java JDK 8

Question

I'm trying to retrieve the description of a few Java Beans from an XML file. I'd like to annotate them with @Data from project lombok to automatically include constructor, equals, hashCode, getters, setters and toString. I'd like to compile them in memory, generate a few instances (with data from the same XML file) and add them to Drools to eventually do some reasoning on that data.

Unfortunately, I cannot compile those classes and so I am asking for your help!

The following code shows how to programmatically compile Java classes in memory:

package example;

import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;

import javax.tools.JavaCompiler;
import javax.tools.JavaFileManager;
import javax.tools.JavaFileObject;
import javax.tools.ToolProvider;

public class Simple {

    public static void main(String[] args) throws Exception {
        String name = "Person";
        String content = //
            "public class " + name + " {
" + //
            "    @Override
" + //
            "    public String toString() {
" + //
            "        return "Hello, world!";
" + //
            "    }
" + //
            "}
";
        System.out.println(content);

        JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
        JavaFileManager manager = new MemoryFileManager(compiler.getStandardFileManager(null, null, null));

        List<String> options = new ArrayList<String>();
        options.addAll(Arrays.asList("-classpath", System.getProperty("java.class.path")));

        List<JavaFileObject> files = new ArrayList<JavaFileObject>();
        files.add(new MemoryJavaFileObject(name, content));

        compiler.getTask(null, manager, null, options, null, files).call();

        Object instance = manager.getClassLoader(null).loadClass(name).newInstance();
        System.out.println(instance);
    }

}

where MemoryFileManager is:

package example;

import java.io.IOException;
import java.security.SecureClassLoader;

import javax.tools.FileObject;
import javax.tools.ForwardingJavaFileManager;
import javax.tools.JavaFileObject;
import javax.tools.JavaFileObject.Kind;
import javax.tools.StandardJavaFileManager;

public class MemoryFileManager extends ForwardingJavaFileManager<StandardJavaFileManager> {

    private MemoryJavaClassObject object;

    public MemoryFileManager(StandardJavaFileManager manager) {
        super(manager);
    }

    @Override
    public ClassLoader getClassLoader(Location location) {
        return new SecureClassLoader() {
            @Override
            protected Class<?> findClass(String name) throws ClassNotFoundException {
                byte[] b = object.getBytes();
                return super.defineClass(name, object.getBytes(), 0, b.length);
            }
        };
    }

    @Override
    public JavaFileObject getJavaFileForOutput(Location location, String name, Kind kind, FileObject sibling) throws IOException {
        object = new MemoryJavaClassObject(name, kind);
        return object;
    }

}

and MemoryJavaClassObject is:

package example;

import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.OutputStream;
import java.net.URI;

import javax.tools.SimpleJavaFileObject;

public class MemoryJavaClassObject extends SimpleJavaFileObject {

    protected final ByteArrayOutputStream stream = new ByteArrayOutputStream();

    public MemoryJavaClassObject(String name, Kind kind) {
        super(URI.create("string:///" + name.replace('.', '/') + kind.extension), kind);
    }

    public byte[] getBytes() {
        return stream.toByteArray();
    }

    @Override
    public OutputStream openOutputStream() throws IOException {
        return stream;
    }

}

and finally MemoryJavaFileObject is:

package example;

import java.net.URI;

import javax.tools.SimpleJavaFileObject;

public class MemoryJavaFileObject extends SimpleJavaFileObject {

    private CharSequence content;

    protected MemoryJavaFileObject(String className, CharSequence content) {
        super(URI.create("string:///" + className.replace('.', '/') + Kind.SOURCE.extension), Kind.SOURCE);
        this.content = content;
    }

    @Override
    public CharSequence getCharContent(boolean ignoreEncodingErrors) {
        return content;
    }

}

If I run the example in the first code block, I get the following output, as expected:

public class Person {
    @Override
    public String toString() {
        return "Hello, world!";
    }
}

Hello, world!

Now, if I add the lombok.jar into my project and I include the following example:

package example;

import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;

import javax.tools.JavaCompiler;
import javax.tools.JavaFileManager;
import javax.tools.JavaFileObject;
import javax.tools.ToolProvider;

public class Lombok {

    public static void main(String[] args) throws Exception {
        String name = "Person";
        String content = //
            "import lombok.Data;
" + //
            "public @Data class " + name + " {
" + //
            "    private String name;
" + //
            "}
";
        System.out.println(content);

        JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
        JavaFileManager manager = new MemoryFileManager(compiler.getStandardFileManager(null, null, null));

        List<String> options = new ArrayList<String>();
        options.addAll(Arrays.asList("-classpath", System.getProperty("java.class.path")));

        List<JavaFileObject> files = new ArrayList<JavaFileObject>();
        files.add(new MemoryJavaFileObject(name, content));

        compiler.getTask(null, manager, null, options, null, files).call();

        Object instance = manager.getClassLoader(null).loadClass(name).newInstance();
        System.out.println(instance);
    }

}

unfortunately I don't get the expected output but rather:

import lombok.Data;
public @Data class Person {
    private String name;
}

/Person.java:2: warning: Can't initialize javac processor due to (most likely) a class loader     problem: java.lang.NoClassDefFoundError: com/sun/tools/javac/processing/JavacProcessingEnvironment
public @Data class Person {
             ^
      at lombok.javac.apt.Processor.init(Processor.java:84)
      at lombok.core.AnnotationProcessor$JavacDescriptor.want(AnnotationProcessor.java:87)
      at lombok.core.AnnotationProcessor.init(AnnotationProcessor.java:141)
      at com.sun.tools.javac.processing.JavacProcessingEnvironment$ProcessorState.<init>(JavacProcessingEnvironment.java:500)
      at com.sun.tools.javac.processing.JavacProcessingEnvironment$DiscoveredProcessors$ProcessorStateIterator.next(JavacProcessingEnvironment.java:597)
      at com.sun.tools.javac.processing.JavacProcessingEnvironment.discoverAndRunProcs(JavacProcessingEnvironment.java:690)
      at com.sun.tools.javac.processing.JavacProcessingEnvironment.access$1800(JavacProcessingEnvironment.java:91)
      at com.sun.tools.javac.processing.JavacProcessingEnvironment$Round.run(JavacProcessingEnvironment.java:1035)
      at com.sun.tools.javac.processing.JavacProcessingEnvironment.doProcessing(JavacProcessingEnvironment.java:1176)
      at com.sun.tools.javac.main.JavaCompiler.processAnnotations(JavaCompiler.java:1173)
      at com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:859)
      at com.sun.tools.javac.main.Main.compile(Main.java:523)
      at com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:129)
      at com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:138)
      at example.Lombok.main(Lombok.java:42)
  Caused by: java.lang.ClassNotFoundException: com.sun.tools.javac.processing.JavacProcessingEnvironment
      at java.net.URLClassLoader$1.run(URLClassLoader.java:372)
      at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
      at java.security.AccessController.doPrivileged(Native Method)
      at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
      at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
      ... 15 more
1 warning
Person@39aeed2f

Notice that the class gets compiled and the default toString() method is executed since the typical output is displayed. Also notice that if I run the former example, now I get the following:

public class Person {
    @Override
    public String toString() {
        return "Hello, world!";
    }
}

/Person.java:1: warning: Can't initialize javac processor due to (most likely) a class loader problem: java.lang.NoClassDefFoundError: com/sun/tools/javac/processing/JavacProcessingEnvironment
public class Person {
       ^
      at lombok.javac.apt.Processor.init(Processor.java:84)
      at lombok.core.AnnotationProcessor$JavacDescriptor.want(AnnotationProcessor.java:87)
      at lombok.core.AnnotationProcessor.init(AnnotationProcessor.java:141)
      at com.sun.tools.javac.processing.JavacProcessingEnvironment$ProcessorState.<init>(JavacProcessingEnvironment.java:500)
      at com.sun.tools.javac.processing.JavacProcessingEnvironment$DiscoveredProcessors$ProcessorStateIterator.next(JavacProcessingEnvironment.java:597)
      at com.sun.tools.javac.processing.JavacProcessingEnvironment.discoverAndRunProcs(JavacProcessingEnvironment.java:690)
      at com.sun.tools.javac.processing.JavacProcessingEnvironment.access$1800(JavacProcessingEnvironment.java:91)
      at com.sun.tools.javac.processing.JavacProcessingEnvironment$Round.run(JavacProcessingEnvironment.java:1035)
      at com.sun.tools.javac.processing.JavacProcessingEnvironment.doProcessing(JavacProcessingEnvironment.java:1176)
      at com.sun.tools.javac.main.JavaCompiler.processAnnotations(JavaCompiler.java:1173)
      at com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:859)
      at com.sun.tools.javac.main.Main.compile(Main.java:523)
      at com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:129)
      at com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:138)
      at example.Simple.main(Simple.java:44)
  Caused by: java.lang.ClassNotFoundException: com.sun.tools.javac.processing.JavacProcessingEnvironment
      at java.net.URLClassLoader$1.run(URLClassLoader.java:372)
      at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
      at java.security.AccessController.doPrivileged(Native Method)
      at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
      at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
      ... 15 more
1 warning
Hello, world!

Apparently, by looking at the warning message passed by the exception, lombok doesn't hook the given compiler properly. Unfortunately I was not able to find any useful bit of information. I can only think it could be lombok not dealing properly with Java JDK 8. Am I right?

Do you know any other way to work around this problem?

See Question&Answers more detail:<a href="https://stackoverflow.com/questions/25607272/compiling-a-java-class-in-memory-with-l

Answer 1

Thanks to Holger, I successfully solved the problem.

The issue was caused by the absence of tools.jar in the class path. This is due to the fact that Eclipse by default recognises the Java environment as a JRE instead of a JDK.

On top of that, Java JDK may - or may not, depending on which version you have - have the tools.jar file.

If you have Java 7 or 8, you should have such library in $JAVA_HOME/lib/tools.jar.

If you have Java 6, the file is not present but the same functionality is provided by $JAVA_HOME/Classes/classes.jar.

The compiler is a feature added with Java 6, so if you want to use it and you have an older version of Java, you should update your environment first.

Now, there are several ways to include tools.jar (or classes.jar) into your project's class path; since I use gradle, I decided to introduce it as a dependency, as you can see in the following snippet of code:

dependencies {
    compile files("${System.properties['java.home']}/../lib/tools.jar")
    compile 'org.projectlombok:lombok:1.14.4'
    testCompile 'junit:junit:4.11'
}

Hope this little explanation might help other people facing a similar problem!

Cheers!