mgmtd deadloops on start

[vgrebenschikov]

Description

FreeBSD 14.3-RELEASE-p7 frr10-10.5.0
just after upgrade to p7 (not changing frr version)

on restart:

Starting watchfrr.
2025-12-18T10:28:30.046724+01:00 ha.sunrise watchfrr 1081 - - [T83RR-8SM5G] watchfrr 10.5.0 starting: vty@0
2025-12-18T10:28:30.047472+01:00 ha.sunrise watchfrr 1081 - - [ZCJ3S-SPH5S] zebra state -> down : initial connection attempt failed
2025-12-18T10:28:30.047541+01:00 ha.sunrise watchfrr 1081 - - [ZCJ3S-SPH5S] ospfd state -> down : initial connection attempt failed
2025-12-18T10:28:30.047572+01:00 ha.sunrise watchfrr 1081 - - [ZCJ3S-SPH5S] staticd state -> down : initial connection attempt failed
2025-12-18T10:28:30.047601+01:00 ha.sunrise watchfrr 1081 - - [ZCJ3S-SPH5S] bgpd state -> down : initial connection attempt failed
2025-12-18T10:28:30.047628+01:00 ha.sunrise watchfrr 1081 - - [ZCJ3S-SPH5S] mgmtd state -> down : initial connection attempt failed
2025-12-18T10:28:30.047656+01:00 ha.sunrise watchfrr 1081 - - [ZCJ3S-SPH5S] bfdd state -> down : initial connection attempt failed
2025-12-18T10:28:30.047684+01:00 ha.sunrise watchfrr 1081 - - [YFT0P-5Q5YX] Forked background command [pid 1082]: /usr/sbin/service frr restart all
2025-12-18T10:28:30.319030+01:00 ha.sunrise watchfrr 1081 - - [VTVCM-Y2NW3] Configuration Read in Took: 00:00:00
igb0: link state changed to UP
igb1: link state changed to UP
vlan20: link state changed to UP
2025-12-18T10:28:34.981505+01:00 ha.sunrise watchfrr 1081 - - [QDG3Y-BY5TN] staticd state -> up : connect succeeded
2025-12-18T10:28:35.023868+01:00 ha.sunrise watchfrr 1081 - - [QDG3Y-BY5TN] zebra state -> up : connect succeeded
2025-12-18T10:28:35.032542+01:00 ha.sunrise watchfrr 1081 - - [QDG3Y-BY5TN] mgmtd state -> up : connect succeeded
2025-12-18T10:28:35.074768+01:00 ha.sunrise watchfrr 1081 - - [QDG3Y-BY5TN] bfdd state -> up : connect succeeded
2025-12-18T10:28:35.136330+01:00 ha.sunrise watchfrr 1081 - - [QDG3Y-BY5TN] ospfd state -> up : connect succeeded
2025-12-18T10:28:35.162841+01:00 ha.sunrise watchfrr 1081 - - [QDG3Y-BY5TN] bgpd state -> up : connect succeeded
2025-12-18T10:28:35.162961+01:00 ha.sunrise watchfrr 1081 - - [KWE5Q-QNGFC] all daemons up, doing startup-complete notify

here it waits for a long time (30-60s)
then

2025-12-18T10:30:00.283570+01:00 ha.sunrise watchfrr 1081 - - [ZE9RA-19PS5] restart all child process 1082 still running after 90 seconds, sending signal 15
2025-12-18T10:30:00.284220+01:00 ha.sunrise watchfrr 1081 - - [SK7QP-A2GT9] restart all process 1082 terminated due to signal 15
[1161|mgmtd] sending configuration
[1162|zebra] sending configuration
% Can't enter config; candidate datastore locked by another session
[1165|ospfd] sending configuration
[1168|bgpd] sending configuration
[1176|watchfrr] sending configuration
[1162|zebra] done
[1178|staticd] sending configuration
[1179|bfdd] sending configuration
Waiting for children to finish applying config...
For this router-id change to take effect, use "clear ip ospf process" command
2025-12-18T10:30:00.550083+01:00 ha.sunrise watchfrr 1081 - - [VTVCM-Y2NW3] Configuration Read in Took: 00:00:00
[1165|ospfd] done
[1176|watchfrr] done
[1178|staticd] done
[1179|bfdd] done
[1168|bgpd] done
% Can't enter config; candidate datastore locked by another session
% Can't enter config; candidate datastore locked by another session
% Can't enter config; candidate datastore locked by another session
% Can't enter config; candidate datastore locked by another session
% Can't enter config; candidate datastore locked by another session
% Can't enter config; candidate datastore locked by another session
% Can't enter config; candidate datastore locked by another session
% Can't enter config; candidate datastore locked by another session
% Can't enter config; candidate datastore locked by another session
% Can't enter config; candidate datastore locked by another session
% Can't enter config; candidate datastore locked by another session
% Can't enter config; candidate datastore locked by another session
% Can't enter config; candidate datastore locked by another session
% Can't enter config; candidate datastore locked by another session
% Can't enter config; candidate datastore locked by another session
<deadloop>

after stop cleanup start

Dec 18 10:40:06 ha mgmtd[2513]: [HD3VA-5FCFX] mgmt_ds_lock: ERROR: lock already taken on DS:running by session-id 2
Dec 18 10:40:06 ha mgmtd[2513]: [VNSS1-Q9XX7] XXX txn in progress, retry init
Dec 18 10:40:06 ha mgmtd[2513]: [HD3VA-5FCFX] mgmt_ds_lock: ERROR: lock already taken on DS:running by session-id 2
Dec 18 10:40:06 ha mgmtd[2513]: [VNSS1-Q9XX7] XXX txn in progress, retry init
Dec 18 10:40:06 ha mgmtd[2513]: [HD3VA-5FCFX] mgmt_ds_lock: ERROR: lock already taken on DS:running by session-id 2
Dec 18 10:40:06 ha mgmtd[2513]: [VNSS1-Q9XX7] XXX txn in progress, retry init
Dec 18 10:40:06 ha mgmtd[2513]: [HD3VA-5FCFX] mgmt_ds_lock: ERROR: lock already taken on DS:running by session-id 2
Dec 18 10:40:06 ha mgmtd[2513]: [VNSS1-Q9XX7] XXX txn in progress, retry init
Dec 18 10:40:06 ha mgmtd[2513]: [HD3VA-5FCFX] mgmt_ds_lock: ERROR: lock already taken on DS:running by session-id 2
Dec 18 10:40:06 ha mgmtd[2513]: [VNSS1-Q9XX7] XXX txn in progress, retry init
Dec 18 10:40:06 ha mgmtd[2513]: [HD3VA-5FCFX] mgmt_ds_lock: ERROR: lock already taken on DS:running by session-id 2
Dec 18 10:40:06 ha mgmtd[2513]: [VNSS1-Q9XX7] XXX txn in progress, retry init
Dec 18 10:40:06 ha mgmtd[2513]: [HD3VA-5FCFX] mgmt_ds_lock: ERROR: lock already taken on DS:running by session-id 2
Dec 18 10:40:06 ha mgmtd[2513]: [VNSS1-Q9XX7] XXX txn in progress, retry init
Dec 18 10:40:06 ha mgmtd[2513]: [HD3VA-5FCFX] mgmt_ds_lock: ERROR: lock already taken on DS:running by session-id 2
Dec 18 10:40:06 ha mgmtd[2513]: [VNSS1-Q9XX7] XXX txn in progress, retry init
Dec 18 10:40:06 ha mgmtd[2513]: [HD3VA-5FCFX] mgmt_ds_lock: ERROR: lock already taken on DS:running by session-id 2
<in dead loop>

% cat fgrep frr /etc/rc.conf
frr_enable="YES"
frr_daemons="zebra ospfd staticd bgpd bfdd mgmtd"

then only way how I was able to fix - remove mgmtd from startup
was introduced due to #17749 (comment)

probably connected - #19359

Version

frr10-10.5.0
14.3-RELEASE-p7

How to reproduce

upgrade to 14.3-RELEASE-p7 on FreeBSD system with frr10-10.5.0 and config above
only one of two my system was affected by this

Expected behavior

no deadloops on startup, preventing normal boot

Actual behavior

deadloops on startup

Additional context

No response

Checklist

• I have searched the open issues for this bug.
• I have not included sensitive information in this report.