某个业务采用 BoneCP 连接池遇到的情况,200个dubbo线程都阻塞在 com.jolbox.bonecp.BoneCPDataSource.maybeInit 方法上,这里wait的是同一个锁:

"DubboServerHandler-thread-203" daemon prio=10 tid=0x00007fc2400e7800 nid=0x658c waiting on condition [0x00007fc233dbb000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
    - parking to wait for  <0x00000000c071f518> (a java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:964)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1282)
at java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:731)
at com.jolbox.bonecp.BoneCPDataSource.maybeInit(BoneCPDataSource.java:133)
at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:112)
at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111)
at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77)
at org.mybatis.spring.SqlSessionUtils.getSqlSession(SqlSessionUtils.java:116)
at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:333)
at com.sun.proxy.$Proxy11.selectList(Unknown Source)
at org.mybatis.spring.SqlSessionTemplate.selectList(SqlSessionTemplate.java:189)
at org.apache.ibatis.binding.MapperMethod.executeForList(MapperMethod.java:100)
at org.apache.ibatis.binding.MapperMethod.execute(MapperMethod.java:70)
at org.apache.ibatis.binding.MapperProxy.invoke(MapperProxy.java:38)

从BoneCP的代码里看不出直接问题,是一个ReentrantReadWriteLock去获取锁的逻辑:

private void maybeInit() throws SQLException {
    this.rwl.readLock().lock();  // ------->  133行,所有线程都阻塞在这里
    if (this.pool == null){
        this.rwl.readLock().unlock();
        this.rwl.writeLock().lock();
        if (this.pool == null){ //read might have passed, write might not
            try {
                if (this.getDriverClass() != null){
                    loadClass(this.getDriverClass());
            catch (ClassNotFoundException e) {
                throw new SQLException(PoolUtil.stringifyException(e));
            logger.debug(this.toString());
            this.pool = new BoneCP(this);
        this.rwl.writeLock().unlock(); // Unlock write
    } else {
        this.rwl.readLock().unlock(); // Unlock read

按说通常如果是死锁的话,应该是某个线程A先获取到锁X,然后再去获取另一个锁Y时发被线程B占用,且线程B又要获取锁X,但这里的现象却是都在等同一把锁,并且一直处于这种状态导致服务不可用。jstack -F看看是否能检测到死锁,输出也是No deadlocks found. (也可能是ReentrantLock的读锁没有ownership的原因,jvm无法跟踪并检测出来,参考这篇这个bug里Doug Lea的回复),怀疑也可能跟jdk和os的版本有关,可能是该版本的bug(或其他我不了解的)。

This entry was posted in java and tagged , by hongjiang. Bookmark the permalink.

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *