java - JMS DefaultMessageListenerContainer not able to shutdown on Spring application context.close()

Question

I am listening to the Solace dynamic topic using spring integration as below:

<int-jms:message-driven-channel-adapter
    id="myJmsAdapter" channel="receiveChannel" container="jmsContainer" />

<bean id="jmsContainer"
    class="org.springframework.jms.listener.DefaultMessageListenerContainer">
    <property name="connectionFactory" ref="solaceConnectionFactory" />
      <property name="destinationName" value="${my.topic}/#{main.getCar_type_value()}" />
    <property name="pubSubDomain" value="true" />        
    <property name="sessionTransacted" value="false" />
    
</bean>

I am able to listen to the topic, consume the message, and process it. But on System.exit(0), even though the spring application context is closed inside the ShutdownHook, the JMS listener thread is hanging and not able to gracefully shutdown:

ctxt.registerShutdownHook();
...
  trigger = new ShutdownTrigger(maxWaitForResponseMillis, ctxt);
  trigger.startShutdownTimer();

    Runtime.getRuntime().addShutdownHook(new Thread() {
        public void run() {
            log.info("Exiting Main Thread!");
            ctxt.close();
        }
    });

The application shuts down properly on timeout, but when I try to exit before the timer using System.exit(0) form other class, it doesn't shutdown and the thread is hanging.

The timer function is as below:

   public void startShutdownTimer() {
      timer.schedule(new TimerTask() {
        @Override
        public void run() {
            log.info("Timer max wait has elapsed - triggering graceful 
     shutdown");
            
            // perform some task here on timeout..
            }
            log.info("Returning with exit code: " + exitCode);
            
            System.exit(exitCode); //this one works and it properly shutsdown
        }
    }, maxWaitForResponseMillis);
}

I also tried adding Timer.cancel before closing the context on the main class as below and it didnt work:

     Runtime.getRuntime().addShutdownHook(new Thread() {
    public void run() {
    log.info("Cancelling current timer task..")
        trigger.getTimer().cancel(); //it didnt work and the thread still hangs
        log.info("Exiting Main Thread!");
        ctxt.close();
    }
});

I tried taking out the Timer function and all its references to see if its this timer thread that is causing the issue, but no.. the application hangs even though the timer task is deleted.

Below is the thread stack during it was hanging:

"Thread-3" #26 prio=5 os_prio=0 tid=0x00007fcf48024800 nid=0x60ad in Object.wait() [0x00007fd026ef1000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at org.springframework.jms.listener.DefaultMessageListenerContainer.doShutdown(DefaultMessageListenerContainer.java:568)
- locked <0x00000000ee0c9ec0> (a java.lang.Object)
at org.springframework.jms.listener.AbstractJmsListeningContainer.shutdown(AbstractJmsListeningContainer.java:237)

Please help to figure out the issue and the solution...

question from:https://stackoverflow.com/questions/65908509/jms-defaultmessagelistenercontainer-not-able-to-shutdown-on-spring-application-c

Answer 1

You are calling System.exit() on the listener container thread

"jmsContainer-1" #24 prio=5 os_prio=0 tid=0x00007fdbd8e50800 nid=0xb77 in Object.wait() [0x00007fdb9f1bc000]
   java.lang.Thread.State: WAITING (on object monitor)
    at java.lang.Object.wait(Native Method)
    - waiting on <0x00000000f02cfb68> (a com.tdsecurities.fxts.vs.FxVsCashBalanceApp$1)
    at java.lang.Thread.join(Thread.java:1245)
    - locked <0x00000000f02cfb68> (a com.tdsecurities.fxts.vs.FxVsCashBalanceApp$1)
    at java.lang.Thread.join(Thread.java:1319)
    at java.lang.ApplicationShutdownHooks.runHooks(ApplicationShutdownHooks.java:106)
    at java.lang.ApplicationShutdownHooks$1.run(ApplicationShutdownHooks.java:46)
    at java.lang.Shutdown.runHooks(Shutdown.java:123)
    at java.lang.Shutdown.sequence(Shutdown.java:167)
    at java.lang.Shutdown.exit(Shutdown.java:212)
    - locked <0x00000000c04335c8> (a java.lang.Class for java.lang.Shutdown)
    at java.lang.Runtime.exit(Runtime.java:109)
    at java.lang.System.exit(System.java:971)

This is causing a deadlock in the Application context (Thread-3 and Thread-5) (shutting down the container).

You probably need to hand off the System.exit() call to another thread, but I'd need to know what version of Spring you are using to analyze further. Otherwise it's hard to correlate the line numbers in the dump.