java - javac cannot compile upper bounded wildcard but Eclipse can

Question

I am working with a code base that has relied on Eclipse for compilation until now. My objective is to compile it with javac (via ant) to simplify the build process. The project compiles without complaint in Eclipse (version 2019-12 (4.14.0)), but javac (OpenJDK, both versions 1.8.0_275 and 14.0.2) produces method ... cannot be applied to given types errors involving upper bounded wildcards.

Steps to reproduce

Note that the repository is 64 MB at time of writing:

git clone [email protected]:jamesdamillington/CRAFTY_Brazil.git
cd CRAFTY_Brazil && git checkout 550e88e
javac -Xdiags:verbose 
    -classpath bin:lib/jts-1.13.jar:lib/MORe.jar:lib/ParMa.jar:lib/ModellingUtilities.jar:lib/log4j-1.2.17.jar:lib/repast.simphony.bin_and_src.jar 
    src/org/volante/abm/agent/DefaultSocialLandUseAgent.java

Error message output:

src/org/volante/abm/agent/DefaultSocialLandUseAgent.java:166: error: method removeNode in interface MoreNetworkModifier<AgentType,EdgeType> cannot be applied to given types;
                        this.region.getNetworkService().removeNode(this.region.getNetwork(), this);
                                                       ^
  required: MoreNetwork<SocialAgent,CAP#1>,SocialAgent
  found: MoreNetwork<SocialAgent,MoreEdge<SocialAgent>>,DefaultSocialLandUseAgent
  reason: argument mismatch; MoreNetwork<SocialAgent,MoreEdge<SocialAgent>> cannot be converted to MoreNetwork<SocialAgent,CAP#1>
  where AgentType,EdgeType are type-variables:
    AgentType extends Object declared in interface MoreNetworkModifier
    EdgeType extends MoreEdge<? super AgentType> declared in interface MoreNetworkModifier
  where CAP#1 is a fresh type-variable:
    CAP#1 extends MoreEdge<SocialAgent> from capture of ? extends MoreEdge<SocialAgent>
1 error

For completeness the method containing the offending line is:

public void die() {
    if (this.region.getNetworkService() != null && this.region.getNetwork() != null) {
        this.region.getNetworkService().removeNode(this.region.getNetwork(), this);
    }

    if (this.region.getGeography() != null
            && this.region.getGeography().getGeometry(this) != null) {
        this.region.getGeography().move(this, null);
    }
}

Analysis of error message

1. We're told the compiler found type MoreNetwork<SocialAgent,MoreEdge<SocialAgent>> where it required MoreNetwork<SocialAgent,CAP#1>.
2. This implies the inferred value of CAP#1 is incompatible with MoreEdge<SocialAgent>
3. However we're told CAP#1 extends MoreEdge<SocialAgent> from capture of ? extends MoreEdge<SocialAgent>

The Java documentation on Upper Bounded Wildcards states that

The upper bounded wildcard, <? extends Foo>, where Foo is any type, matches Foo and any subtype of Foo.

I cannot see why MoreEdge<SocialAgent> doesn't match <? extends MoreEdge<SocialAgent>>, and consequently cannot reconcile 2 and 3.

Efforts made to resolve the problem

While my objective is to compile for Java 8, I am aware that there have been bugs found in javac related to generics and wildcards in the past (see discussion around this answer). However, I find the same problem with both javac 1.8.0_275 and javac 14.0.2.

I also thought about whether some form of explicit cast might provide the compiler with sufficient hints. However I can't think what to change since the type of this.region.getNetwork() is reported as MoreNetwork<SocialAgent,MoreEdge<SocialAgent>> in the error message as expected.

Generalisation of the problem (EDIT)

@rzwitserloot rightly pointed out that I hadn't included enough information about the dependencies in the code above to properly debug. Copying all the dependencies (including some code from libraries I don't control) would get very messy so I have distilled the problem into a self-contained program that produces an analogous error.

import java.util.HashMap;
import java.util.Map;
import java.util.List;
import java.util.ArrayList;

public class UpperBoundNestedGenericsDemo {

    public static void main(String[] args) {
        MapContainerManagerBroken mapContainerManager = new MapContainerManagerBroken();
        MapContainer<A, B<A>> mapContainer = new MapContainer<>();
        mapContainerManager.setMapContainer(mapContainer);

        Map<A, B<A>> aMap = new HashMap<>();
        aMap.put(new A(), new B<A>());
        
        mapContainerManager.getMapContainer().addMap(aMap);
        mapContainerManager.getMapContainer().removeMap(aMap);
    }

}

/**
 * Analogue of Region
 */
class MapContainerManagerBroken {
    private MapContainer<A, ? extends B<A>> mapContainer;

    void setMapContainer(MapContainer<A, ? extends B<A>> mapContainer) {
        this.mapContainer = mapContainer;
    }

    MapContainer<A, ? extends B<A>> getMapContainer() {
        return this.mapContainer;
    }
}

/**
 * Analogue of MoreNetworkService
 */
class MapContainer<T1, T2> {

    List<Map<T1, T2>> listOfMaps = new ArrayList<>();

    void addMap(Map<T1, T2> map) {
        listOfMaps.add(map);
    }

    boolean removeMap(Map<T1, T2> map) {
        return listOfMaps.remove(map);
    }

}

class A {
}

class B<T> {
}

This compiles in Eclipse, but upon compiling with

javac -Xdiags:verbose UpperBoundNestedGenericsDemo.java 

produces the error message

UpperBoundNestedGenericsDemo.java:18: error: method addMap in class MapContainer<T1,T2> cannot be applied to given types;
                mapContainerManager.getMapContainer().addMap(aMap);
                                                     ^
  required: Map<A,CAP#1>
  found: Map<A,B<A>>
  reason: argument mismatch; Map<A,B<A>> cannot be converted to Map<A,CAP#1>
  where T1,T2 are type-variables:
    T1 extends Object declared in class MapContainer
    T2 extends Object declared in class MapContainer
  where CAP#1 is a fresh type-variable:
    CAP#1 extends B<A> from capture of ? extends B<A>
UpperBoundNestedGenericsDemo.java:19: error: method removeMap in class MapContainer<T1,T2> cannot be applied to given types;
                mapContainerManager.getMapContainer().removeMap(aMap);
                                                     ^
  required: Map<A,CAP#1>
  found: Map<A,B<A>>
  reason: argument mismatch; Map<A,B<A>> cannot be converted to Map<A,CAP#1>
  where T1,T2 are type-variables:
    T1 extends Object declared in class MapContainer
    T2 extends Object declared in class MapContainer
  where CAP#1 is a fresh type-variable:
    CAP#1 extends B<A> from capture of ? extends B<A>
2 errors

Partial solution

The program in the previous section can be modified such that it compiles under both Eclipse and javac by replacing MapContainerManagerBroken with

class MapContainerManagerNoWildcards {
    private MapContainer<A, B<A>> mapContainer;
    
    void setMapContainer(MapContainer<A, B<A>> mapContainer) {
        this.mapContainer = mapContainer;
    }
    
    MapContainer<A, B<A>> getMapContainer() {
        return this.mapContainer;
    }
}

That is, by removing the wildcard type bounds for MapContainer. This solves the immediate practical problem, but limits the flexibility of MapContainerManagerNoWildcards compared to MapContainerManagerBroken. An alternative would be to make this class generic, e.g.

class MapContainerManagerFixedGeneric<T extends B<A>> {
    private MapContainer<A, T> mapContainer;

    void setMapContainer(MapContainer<A, T> mapContainer) {
        this.mapContainer = mapContainer;
    }

    MapContainer<A, T> getMapContainer() {
        return this.mapContainer;
    }
}

However, this does not explain why the line mapContainerManager.getMapContainer().addMap(aMap) is a compiler error when mapContainerManager is a MapContainerManagerBroken (such as in the example program). Specifically, why is the preceding line an error, but the following compiles?

MapContainer<A, ? extends B<A>> mapContainer = new MapContainer<A, B<A>>();

Answer 1

You've misunderstood the rules on what ? extends means as far as type compatibility is concerned.

Any two occurrences of ? extends Number are not compatible with each other, nor is ? extends Number compatible with Number itself. Here is a trivial 'proof' of why that is:

List<Integer> ints = new ArrayList<Integer>();
List<? extends Number> numbers1 = ints; // legal
List<Number> numbers2 = numbers1; // legal?
numbers2.add(new Double(5.0)); // oh whoopsie

If the above compiles, then there is a non-int in ints, and that's no good. Fortunately, it doesn't compile. Specifically, the third line is a compiler error.

The segment of the JLS you are talking about is talking about a one-way street. You can assign a List<Number> to a variable of type List<? extends Number> (or pass a List<Number> as argument when the argument is of that type), but not the other way around.

? is the same as 'imagine I used a letter here and that I use this letter only here and nowhere else'. Thus, if you have two ? involved, they may not be equal to each other. Therefore, for type compatibility purposes, they aren't compatible. This makes sense; imagine you have:

void foo(List<? extends Number> a, List<? extends Number b>) {}

then it stands to reason that the point is that I can invoke this passing some List<Integer> for a and List<Double> for b: Each ? gets to be whatever it wants as long as it fits the bounds. Which also means that it is impossible to invoke add on either of these lists, as the thing you add must be of type ?, and you can't make that happen (except, trivially, by writing .add(null), as null is every type for such purposes), but that's not very useful). It also explains why you can't write a = b; and that goes to the heart of your problem here. Why can you not assign a to b? They are the same type, after all! - No, they are not, and that CAP stuff captures this: a is of type CAP#1 and b is of type CAP#2. That's how javac (and ecj, presumably) sort this out, and that's why this CAP stuff is showing up. It's not a matter of the compiler being intentionally dense or underspecced. It's inherent in the complexities of generics.

Thus, yes: CAP#1 is not the same as ? extends Number, it is merely one capture of that (and any further ? extends Number would then be referred to as CAP#2, and CAP#1 and CAP#2 are not compatible; one may be Integer and one may be Double, after all). The error message itself is sensible.

Normally if ecj and javac disagree, usually ecj is correct and javac is not, based on personal experience (I recount about 10 times I ran into a situation where ecj and javac disagreed, and 9 out of the 10 times, ecj was more correct than javac; though often it is an ambiguity in the JLS that I then report and which have been resolved). Nevertheless, given that JDK14 still has the issue, and trying to interpret these error messages (this is quite difficult without all the signatures involved here, you haven't pasted the useful parts of your codebase), it does look like javac is correct.

The usual fix is to toss more ? in there. In particular, a removeEdge sure sounds like it should accept either Object or ? extends T and not T. After all, arraylist's .remove() method accepts any object, not a T - trying to remove some double from a list of ints simply doesn't do anything, as per spec: Asking a list to remove a thing that isn't inside is a noop. No reason to limit the parameter, then. That solves many problems right there.

---

EDIT, after you've significantly updated your question with way more detail.

MapContainer<A, ? extends B> mapContainer = new MapContainer<A, B>();

Because that's what that means. Remember, MapContainer<? extends Number> does not represent a type. It represents a whole dimension worth of types. It's saying that mapContainer is a reference that can point at almost anything, as long as it is a MapContainer, of any 'tag' (the stuff in the <> you please, long as the thing that is in between is either Number or any subtype thereof. The only methods you can invoke on this exploded 'could be so many things' type is what ALL possible things it could be have in command, and no addMap method of any stripe is a part of the intersection. The addMap method's parameter involves an A, as in, in this example, the same thing as ? extends Number and the compiler says: Well, I don't know. There is no type that fits. I can't go with Number; what if you have a MapContainer<Integer>? If I let you call addMap using any Number, you could put a Double in there, and that's not allowed. The fact that eclipse allows it at all is bizarre.

Here is a borderline trivial example:

Map<? extends Number, String> x = ...;
x.put(A, B);

In the above example, nothing can be written on either the 3 dots, or in place of A to make that ever compile. ? extends is shorthand for: No add/put. Period.

There is actually one thing that will work: x.put(null, B);, because null 'fits' every type. But this is a copout and not at all useful for serious code.

Once you fully grok this, the problem is explained. More generally, given that you have a MapContainer<? extends something>, you can't call addMap on that thing. Period. You can't call 'write' operations on extends style typebounds.

I've explained why that is at the very top of this answer.