jakarta ee - EclipseLink "create-or-extend-tables" not working: "...unknown column..."

Question

I am using EclipseLink 2.4.1 with Glassfish and a MySQL database for persisting entities.
I added a field to an entity, and when I try to persist this entity it says the new field is 'unknown'.
How are we supposed to use the new 'create-or-extend-tables' feature? I would have thought it would create a new column for this field.

Some relevant information is below, let me know if you want more.

Thanks in advance!

Stack Trace

...
Caused by: Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.4.1.v20121003 ad44345): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Unknown column 't0.TESTFIELD' in 'field list'
Error Code: 1054
at org.eclipse.persistence.exceptions.DatabaseException.sqlException(DatabaseException.java:333)
...

persistence.xml:

<persistence-unit name="myApp" transaction-type="JTA">
<provider>org.eclipse.persistence.jpa.PersistenceProvider</provider>
<jta-data-source>jdbc/mysql</jta-data-source>
<mapping-file>META-INF/orm.xml</mapping-file>
<properties>
  <property name="eclipselink.ddl-generation.output-mode" value="database"/>
  <property name="eclipselink.jdbc.batch-writing" value="Buffered"/>
  <property name="eclipselink.logging.level" value="INFO"/>
  <property name="eclipselink.ddl-generation" value="create-or-extend-tables"/>
</properties>

Extra Info

drop-and-create-tables does work.
I am using merge(entity) to persist new entities (as they have lots of many to one fields which causes duplicate primary id issues otherwise) - I feel this may be the issue.
Looking at the MySQL log, and the finest log level from EclipseLink, EclipseLink first tries to select the entity from the database as follows:
mysql_log:

121115 10:49:03 9 Query SELECT t0.LISTINGID, t0.DTYPE, ... , t0.TESTFIELD FROM the_entity...

This is the last mysql log entry, meaning that it crashes here, it never tries to drop the table etc. Does this mean you cannot use merge with drop-and-create or create-or-extend? I just did a google and didn't find any info on this.

EclipseLink Logging:

FINER: client acquired: 64279491
FINER: TX binding to tx mgr, status=STATUS_ACTIVE
FINER: acquire unit of work: 161130796
FINEST: Merge clone with references nz.co.site.api.v1.ListedItemDetail@8d88ca1
FINEST: Execute query ReadObjectQuery(referenceClass=ListedItemDetail )
FINEST: Connection acquired from connection pool [read].
FINEST: reconnecting to external connection pool
FINE: SELECT t0.LISTINGID, t0.DTYPE, ... , t0.TESTFIELD, ... , FROM ITEM t0, LISTEDITEMDETAIL t1 WHERE ((t0.LISTINGID = ?) AND ((t1.LISTINGID = t0.LISTINGID) AND (t0.DTYPE = ?)))
bind => [2 parameters bound]
FINE: SELECT 1
FINEST: Connection released to connection pool [read].
WARNING: Local Exception Stack: 
Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.4.1.v20121003-ad44345): org.eclipse.persistence.exceptions.DatabaseException

I use bean transaction management to merge new entities:

userTransaction.begin();
entityManager.merge(entity);
entityManager.flush();
userTransaction.commit();

Answer 1

Glassfish intercepts the eclipselink.ddl-generation.output-mode property and forces it write out to file as described in an answer here: eclipselink does not generate tables from annotated JPA classes So you will need to check that you've enabled java2db in Glassfish for "drop-and-create-tables" to work.

"create-or-extend-tables" though requires database connections to see what is in the database, and so does not currently work with the sql-script output mode. I am not sure how to force Glassfish to not overwrite the output-mode property, but if it cannot be done, you will need to run your persistence unit side glassfish to use this feature.

A workaround in glassfish would be to try something like this:

Map properties = new HashMap();
properties.put(PersistenceUnitProperties.DDL_GENERATION, PersistenceUnitProperties.CREATE_OR_EXTEND);
//create-or-extend only works on the database
properties.put(PersistenceUnitProperties.DDL_GENERATION_MODE, PersistenceUnitProperties.DDL_DATABASE_GENERATION);
//this causes DDL generation to occur on refreshMetadata rather than wait until an em is obtained
properties.put(PersistenceUnitProperties.DEPLOY_ON_STARTUP, "true");
JpaHelper.getEntityManagerFactory(em).refreshMetadata(properties);