AMP systemd service restart causes all managed instances to stop during unattended package maintenance

Technical Support
1

Environment

AMP Version: 2.8
Build: 25/06/2026 23:15
OS: Ubuntu Server 26.04 LTS
Install method: Linux systemd service
Run user: amp

Problem description

Multiple production AMP hosts experienced all instances shutting down simultaneously. The instances were not crashing; AMP received a systemd stop request, executed its configured ExecStop command, gracefully stopped all instances, and then restarted normally.

Evidence

[Service]
Type=oneshot
RemainAfterExit=yes
User=amp
Group=amp
ExecStart=/opt/cubecoders/amp/ampinstmgr startboot true
TimeoutSec=180

ExecStop=/opt/cubecoders/amp/ampinstmgr stopall
TimeoutStopSec=180

Log example

Jul 07 06:40:58 anarchy systemd[1]: Stopping ampinstmgr.service - AMP Instance Manager...

Jul 07 06:40:58 anarchy ampinstmgr:
Stopping Instance: 'ADS01'

Jul 07 06:40:59 anarchy ampinstmgr:
Instance ADS01 has stopped.

Jul 07 06:40:59 anarchy ampinstmgr:
Stopping Instance: 'Anarchy01'

Jul 07 06:40:59 anarchy ampinstmgr:
Instance Anarchy01 has stopped.

Jul 07 06:41:02 anarchy systemd[1]:
Starting ampinstmgr.service - AMP Instance Manager...

Jul 07 06:41:03 anarchy:
AMP instance ADS01 is now running.

  • stop request comes from systemd

  • AMP shuts down cleanly

  • AMP starts again

  • Individual instances do not start

Pattern observed

This occurred on multiple AMP hosts at the same time. It happened repeatedly on July 2, July 3, and July 7, always near the same time, which coincides with ubuntu automatic update periods.

July 2 ~06:24
July 3 ~06:35
July 7 ~06:41

Trigger investigation

The events coincided with Ubuntu package maintenance activity. PackageKit/apt/needrestart activity was observed immediately before the AMP service restart. Disabling unattended upgrades stopped the issue.

The following were disabled as a workaround:

apt-daily.timer
apt-daily-upgrade.timer
unattended-upgrades.service

Feature request / question

Is there a recommended AMP systemd configuration that prevents a routine service restart from executing ampinstmgr stopall?

Would CubeCoders consider changing the service architecture so restarting the AMP manager does not terminate all managed instances?

Possible solutions:

  • Separate AMP management daemon from instance lifecycle

  • Add a reload operation

  • Make ExecStop conditional

  • Provide a recommended systemd override for production hosts

  • Support systemctl reload ampinstmgr.service

2

AMP 3 won’t have this issue, but because of how AMP 2 is built you have to pull the instances down temporarily to update the core packages because the running instances rely on some of the files that will be touched by package updates.

AMP 3 uses a different design that allows everything to keep running during updates (and this difference is part of why it’s being built in the first place) - but AMP 2 has architectural limitations stopping this from being the case.

So the answer for the second is to pin your ampinstmgr version so it doesn’t automatically update except during scheduled downtime.

3

“pin your ampinstmgr version so it doesn’t automatically update”

Ampinstmgr itself was not being updated on July 2, 3, or 7. Only packages like the following were being upgraded at the time:

  • curl

  • libcurl

  • vim

  • libnghttp2

What I am basically getting from your response is the following:

Don’t let package maintenance restart the AMP service automatically.
Update AMP and the OS during scheduled maintenance.

If so, that is what we have been doing all along… Amp was updated to 2.8.0.0 via command line, across all nodes, about a week ago. This is when all of this began happening.

4

If the systemd unit didn’t run stopall then game servers wouldn’t shut down cleanly when the host system was shut down. They’d get killed and you’d risk data corruption or loss. That’s the purpose of the system unit, to bring them up and down with the host.

I’d recommend against running unattended updates if they are shutting down unrelated systemd services, or investigate why it was decided that the ampinstmgr service had to be stopped when it wasnt updated.