Pgpool-II Regression Test 001.load_balance Timeout Issue

[Akashkumar1208]

Using Pgpool-II 4.7devel with PostgreSQL 14 on Ubuntu 22.04.
When running:

./regress.sh -p /usr/lib/postgresql/14/bin -b /usr/lib/postgresql/14/bin -t 300 -d 001.load_balance

creating pgpool-II temporary installation ...
moving pgpool_setup to temporary installation path ...
moving watchdog_setup to temporary installation path ...
using pgpool-II at /home/qa_admin/pgpool2/src/test/regression/temp/installed

---

REGRESSION MODE : install
Pgpool-II version : pgpool-II version 4.7devel (tasukiboshi)
Pgpool-II install path : /home/qa_admin/pgpool2/src/test/regression/temp/installed
PostgreSQL bin : /usr/lib/postgresql/14/bin
PostgreSQL Major version : 14
pgbench : /usr/lib/postgresql/14/bin
PostgreSQL jdbc : /usr/share/java/postgresql-9.2-1003-jdbc4.jar

---

testing 001.load_balance...timeout.
out of 1 ok:0 failed:0 timeout:1

001.load_balance.txt

the test loops during test8 after executing:

pcp_promote_node -w -p 11001 --switchover 1

=== test8 started === /home/qa_admin/pgpool2/src/test/regression/temp/installed/bin/pcp_promote_node -w -p 11001 --switchover 1 pcp_promote_node -- Command Successful
0 | localhost | 11002 | down | down | 0.000000 | standby | unknown | 0 | false | 0 | | | 2025-11-03 20:48:08
0 | localhost | 11002 | down | down | 0.000000 | standby | unknown | 0 | false | 0 | | | 2025-11-03 20:48:08
0 | localhost | 11002 | down | down | 0.000000 | standby | unknown | 0 | false | 0 | | | 2025-11-03 20:48:08
0 | localhost | 11002 | down | down | 0.000000 | standby | unknown | 0 | false | 0 | |

Logs show:

recovery node 1...ERROR: recovery is checking if postmaster is started
DETAIL: postmaster on hostname:"localhost" user:"qa_admin" failed to start in 90 seconds

The node remains in a down/unknown state, and the test times out.

Any idea why the recovery node fails to start or loops in regression mode?

[pengbo0328]

It seems that pcp_recovery_node failed.
Did other test cases also fail, or was it only Test 001.load_balance?
Could you check whether any errors occur in the following log files?

/home/qa_admin/pgpool2/src/test/regression/tests/001.load_balance/testdir/log/pgpool.log
/home/qa_admin/pgpool2/src/test/regression/tests/001.load_balance/testdir/data0/log/

[Akashkumar1208]

It seems that pcp_recovery_node failed. Did other test cases also fail, or was it only Test 001.load_balance? Could you check whether any errors occur in the following log files?

/home/qa_admin/pgpool2/src/test/regression/tests/001.load_balance/testdir/log/pgpool.log /home/qa_admin/pgpool2/src/test/regression/tests/001.load_balance/testdir/data0/log/

I ran the Pgpool regression tests, and all the test cases that use pcp_recovery_node are failing with the same error. However, the same test cases run successfully when pcp_recovery_node is not used.

recovery node 1...ERROR: recovery is checking if postmaster is started
DETAIL: postmaster on hostname:"localhost" user:"qa_admin" failed to start in 90 seconds

When I checked the failover log (~/pgpool2/src/test/regression/tests/001.load_balance/testdir/log/failover.log), I saw the following:

waiting for server to start....989886 2025-11-06 09:12:41.081 IST LOG: redirecting log output to logging collector process
989886 2025-11-06 09:12:41.081 IST HINT: Future log output will appear in directory "log".
stopped waiting
pg_ctl: could not start server
Examine the log output.
pg_ctl: no server running
restarting /home/qa_admin/pgpool2/src/test/regression/tests/001.load_balance/testdir/data0 failed
fallback to pcp_recovery_node
ERROR: recovery is checking if postmaster is started
DETAIL: postmaster on hostname:"localhost" database:"template1" user:"qa_admin" failed to start in 90 second

data1_Thursday.log
data0_Thursday.log
failover.log
recovery.log

[Akashkumar1208]

It seems that pcp_recovery_node failed. Did other test cases also fail, or was it only Test 001.load_balance? Could you check whether any errors occur in the following log files?
/home/qa_admin/pgpool2/src/test/regression/tests/001.load_balance/testdir/log/pgpool.log /home/qa_admin/pgpool2/src/test/regression/tests/001.load_balance/testdir/data0/log/

I ran the Pgpool regression tests, and all the test cases that use pcp_recovery_node are failing with the same error. However, the same test cases run successfully when pcp_recovery_node is not used.

recovery node 1...ERROR: recovery is checking if postmaster is started DETAIL: postmaster on hostname:"localhost" user:"qa_admin" failed to start in 90 seconds

When I checked the failover log (~/pgpool2/src/test/regression/tests/001.load_balance/testdir/log/failover.log), I saw the following:

waiting for server to start....989886 2025-11-06 09:12:41.081 IST LOG: redirecting log output to logging collector process 989886 2025-11-06 09:12:41.081 IST HINT: Future log output will appear in directory "log". stopped waiting pg_ctl: could not start server Examine the log output. pg_ctl: no server running restarting /home/qa_admin/pgpool2/src/test/regression/tests/001.load_balance/testdir/data0 failed fallback to pcp_recovery_node ERROR: recovery is checking if postmaster is started DETAIL: postmaster on hostname:"localhost" database:"template1" user:"qa_admin" failed to start in 90 second

data1_Thursday.log data0_Thursday.log failover.log recovery.log

I updated the create_recovery_conf function in pgpool_setup as follows:

create_recovery_conf {
if [ $PGVERSION -ge 120 ]; then
fname=myrecovery.conf
elif [ $2 = "0" ]; then
fname=recovery.done
else
fname=recovery.conf
fi

# PRIMARY NODE
if [ $2 = "0" ]; then
    if [ $PGVERSION -lt 120 ]; then
        cat > $1/$fname <<EOF

standby_mode = 'on'
EOF
else
touch $1/$fname
rm -f $1/standby.signal 2>/dev/null
fi
return
fi

# STANDBY NODES
if [ $PGVERSION -lt 120 ]; then
    cat > $1/$fname <<EOF

standby_mode = 'on'
EOF
fi

cat > $1/$fname <<EOF

primary_conninfo = 'host=localhost port=$BASEPORT user=$WHOAMI application_name=''server$2'''
recovery_target_timeline = 'latest'
EOF

if [ $USE_REPLICATION_SLOT = "true" ] || [ $TEST_SAMPLES = "true" ]; then
    echo "primary_slot_name = 'pgpool_setup_slot$2'" >> $1/$fname
else
    cat >> $1/$fname <<EOF

restore_command = 'cp $BASEDIR/archivedir/data0/%f "%p" 2> /dev/null'
EOF
fi

if [ $PGVERSION -ge 120 ]; then
    touch $1/standby.signal
fi

}

After making this change, the 001.load_balance test case passed successfully.

However, I still noticed the following warning in the logs:

recovery node 1...ERROR: recovery is checking if postmaster is started
DETAIL: postmaster on hostname:"localhost" database:"template1" user:"qa_admin" failed to start in 90 seconds

When testing 034.promote_node, the test also failed with a similar behavior — the nodes repeatedly show as down after promotion even though the process logs indicate the servers started successfully:

server started
localhost 11002 2 0.250000 up up primary primary 0 none none 2025-11-06 11:16:17
localhost 11003 2 0.250000 up up standby standby 0 streaming async 2025-11-06 11:16:17
localhost 11004 2 0.250000 up up standby standby 0 streaming async 2025-11-06 11:16:17
localhost 11005 2 0.250000 up up standby standby 0 streaming async 2025-11-06 11:16:17
pcp_promote_node -- Command Successful
starting to check follow primary results: Thu Nov 6 11:16:20 IST 2025
0 | localhost | 11002 | down | down | 0.250000 | standby | unknown | 0 | false | 0 | | | 2025-11-06 11:16:19
1 | localhost | 11003 | down | up | 0.250000 | standby | standby | 0 | false | 0 | streaming | async | 2025-11-06 11:16:19
2 | localhost | 11004 | down | up | 0.250000 | standby | standby | 0 | false | 0 | streaming | async | 2025-11-06 11:16:19
cnt: 119
1 | localhost | 11003 | down | down | 0.250000 | standby | unknown | 0 | false | 0 | streaming | async | 2025-11-06 11:16:19
2 | localhost | 11004 | down | up | 0.250000 | standby | standby | 0 | false | 0 | streaming | async | 2025-11-06 11:16:19
cnt: 118
1 | localhost | 11003 | down | down | 0.250000 | standby | unknown | 0 | false | 0 | streaming | async | 2025-11-06 11:16:19
2 | localhost | 11004 | down | up | 0.250000 | standby | standby | 0 | false | 0 | streaming | async | 2025-11-06 11:16:19
cnt: 117
1 | localhost | 11003 | down | down | 0.250000 | standby | unknown | 0 | false | 0 | streaming | async | 2025-11-06 11:16:19
2 | localhost | 11004 | down | up | 0.250000 | standby | standby | 0 | false | 0 | streaming | async | 2025-11-06 11:16:19
cnt: 116

log
tests/034.promote_node/testdir/log/failover.log

waiting for server to start....2612857 2025-11-06 11:16:20.536 IST LOG: redirecting log output to logging collector process
2612857 2025-11-06 11:16:20.536 IST HINT: Future log output will appear in directory "log".
stopped waiting
pg_ctl: could not start server
Examine the log output.
pg_ctl: no server running
restarting /home/qa_admin/pgpool2/src/test/regression/tests/034.promote_node/testdir/data1 failed
fallback to pcp_recovery_node

[liujinyang-highgo]

Is any progress on this issue? I also met same thing.