Startup Issue with Application Deployment - All instances have disappeared like it's a fresh install

Technical Support
,
1

System Information

Field Value
Operating System Linux - Ubuntu 22.04.4 on x86_64
Product AMP ‘Callisto’ v2.5.0.4 (Mainline)
Virtualization QEMU_KVM
Application Application Deployment
Module ADSModule
Running in Container No
Current State Indeterminate

Problem Description

Issue

I lost all instance. I can see them in the files still. The minecraft server backups are still in the file tree. After logging in the web interface the instance are all missing.

Reproduction Steps

  • checked console logs
  • started a ssh terminal and rebooted server
  • ssh console and did update via command line

21:14:32

Returned exception from API call Core/GetUpdates

IOException

[0] (IOException) : Too many open files

at IO.FileStream…ctor (String path, IO.FileMode mode, IO.FileAccess access, IO.FileShare share, Int32 bufferSize, Boolean anonymous, IO.FileOptions options) at IO.FileStream…ctor (String path, IO.FileMode mode, IO.FileAccess access, IO.FileShare share, Int32 bufferSize, IO.FileOptions options) at (wrapper remoting-invoke-with-check) IO.FileStream…ctor(string,IO.FileMode,IO.FileAccess,IO.FileShare,int,IO.FileOptions) at IO.StreamReader…ctor (String path, Text.Encoding encoding, Boolean detectEncodingFromByteOrderMarks, Int32 bufferSize) at IO.StreamReader…ctor (String path, Text.Encoding encoding) at (wrapper remoting-invoke-with-check) IO.StreamReader…ctor(string,Text.Encoding) at Net.NetworkInformation.MibIPGlobalProperties.GetRows (String file, Collections.Generic.List`1[T] list) at Net.NetworkInformation.MibIPGlobalProperties.GetActiveTcpListeners () at GSMyAdmin.WebServer.WebMethods.GetPortSummaries () at GSMyAdmin.WebServer.WebMethods.GetUpdates () at GSMyAdmin.WebServer.WebAttributes.InvokeMethod (String MethodName, Newtonsoft.Json.Linq.JObject Data, m.Http.IHttpRequest request, ModuleShared.IWebSession Session, ModuleShared.WebMethodsBase MethodsClass, String RawRequest) at GSMyAdmin.WebServer.APIService.InvokeAPI (m.Http.IHttpRequest request, ModuleShared.IWebSession Session, Newtonsoft.Json.Linq.JObject Data, String RequestModule, String RequestMethod, String RawRequest)

21:14:33

Returned exception from API call Core/GetUpdates

IOException

[0] (IOException) : Too many open files

at IO.FileStream…ctor (String path, IO.FileMode mode, IO.FileAccess access, IO.FileShare share, Int32 bufferSize, Boolean anonymous, IO.FileOptions options) at IO.FileStream…ctor (String path, IO.FileMode mode, IO.FileAccess access, IO.FileShare share, Int32 bufferSize, IO.FileOptions options) at (wrapper remoting-invoke-with-check) IO.FileStream…ctor(string,IO.FileMode,IO.FileAccess,IO.FileShare,int,IO.FileOptions) at IO.StreamReader…ctor (String path, Text.Encoding encoding, Boolean detectEncodingFromByteOrderMarks, Int32 bufferSize) at IO.StreamReader…ctor (String path, Text.Encoding encoding) at (wrapper remoting-invoke-with-check) IO.StreamReader…ctor(string,Text.Encoding) at Net.NetworkInformation.MibIPGlobalProperties.GetRows (String file, Collections.Generic.List`1[T] list) at Net.NetworkInformation.MibIPGlobalProperties.GetActiveTcpListeners () at GSMyAdmin.WebServer.WebMethods.GetPortSummaries () at GSMyAdmin.WebServer.WebMethods.GetUpdates () at GSMyAdmin.WebServer.WebAttributes.InvokeMethod (String MethodName, Newtonsoft.Json.Linq.JObject Data, m.Http.IHttpRequest request, ModuleShared.IWebSession Session, ModuleShared.WebMethodsBase MethodsClass, String RawRequest) at GSMyAdmin.WebServer.APIService.InvokeAPI (m.Http.IHttpRequest request, ModuleShared.IWebSession Session, Newtonsoft.Json.Linq.JObject Data, String RequestModule, String RequestMethod, String RawRequest)

21:14:34

Returned exception from API call Core/GetUpdates

IOException

[0] (IOException) : Too many open files

at IO.FileStream…ctor (String path, IO.FileMode mode, IO.FileAccess access, IO.FileShare share, Int32 bufferSize, Boolean anonymous, IO.FileOptions options) at IO.FileStream…ctor (String path, IO.FileMode mode, IO.FileAccess access, IO.FileShare share, Int32 bufferSize, IO.FileOptions options) at (wrapper remoting-invoke-with-check) IO.FileStream…ctor(string,IO.FileMode,IO.FileAccess,IO.FileShare,int,IO.FileOptions) at IO.StreamReader…ctor (String path, Text.Encoding encoding, Boolean detectEncodingFromByteOrderMarks, Int32 bufferSize) at IO.StreamReader…ctor (String path, Text.Encoding encoding) at (wrapper remoting-invoke-with-check) IO.StreamReader…ctor(string,Text.Encoding) at Net.NetworkInformation.MibIPGlobalProperties.GetRows (String file, Collections.Generic.List`1[T] list) at Net.NetworkInformation.MibIPGlobalProperties.GetActiveTcpListeners () at GSMyAdmin.WebServer.WebMethods.GetPortSummaries () at GSMyAdmin.WebServer.WebMethods.GetUpdates () at GSMyAdmin.WebServer.WebAttributes.InvokeMethod (String MethodName, Newtonsoft.Json.Linq.JObject Data, m.Http.IHttpRequest request, ModuleShared.IWebSession Session, ModuleShared.WebMethodsBase MethodsClass, String RawRequest) at GSMyAdmin.WebServer.APIService.InvokeAPI (m.Http.IHttpRequest request, ModuleShared.IWebSession Session, Newtonsoft.Json.Linq.JObject Data, String RequestModule, String RequestMethod, String RawRequest)

2

I am seeing similar with the addition that the web interface crashes after a minute or two.

OS: Linux - Debian 12 (Kernel 6.1.0-21-amd64)
Product: AMP Instance Manager v2.5.0.8
Virtualization: QEMU (on UnRaid)

Did my normal monthly OS & AMP updates, rebooted system and could not login to web interface after several minutes. Restarted instance with systemctl restart ampinstmgr.service which recovered the web interface. Logged in, found no instances listed, then web interface crashed again.

Checking with ampinstmgr --LatestLog found the following:

[15:37:48] [Core Info]            : Starting AMP version 2.5.0.8 (Callisto), built 02/05/2024 04:53
[15:37:48] [Core Info]            : Stream: Mainline / Release - built by CUBECODERS/buildbot on CCL-DEV
[15:37:48] [Core Info]            : Running in a QEMU KVM environment.
[15:37:48] [Core Warning]         : Current time zone is set to CST - It is recommended you switch to UTC+0 to avoid time-zone related issues
[15:37:48] [Core Info]            : OS: Linux / x86_64
[15:37:48] [Core Info]            : CPU: AMD Ryzen 9 5900X 12-Core Processor (10C/20T)
[15:37:48] [Core Info]            : RAM: 48166MB
[15:37:48] [Core Info]            : AMP Instance ID: e504e4c9-5a0b-4a42-afd7-2c3fda68695e
[15:37:48] [Loader Info]          : Loaded ADSModule version 1.0.0.0 by CubeCoders Limited
[15:37:48] [Loader Info]          : Loaded FileManagerPlugin by CubeCoders Limited
[15:37:48] [Loader Info]          : Loaded EmailSenderPlugin by CubeCoders Limited
[15:37:48] [Loader Info]          : Loaded WebRequestPlugin by CubeCoders Limited
[15:37:48] [Loader Info]          : Loaded LocalFileBackupPlugin by CubeCoders Limited
[15:37:48] [Loader Info]          : Loaded CommonCorePlugin by CubeCoders Limited
[15:37:48] [Loader Info]          : ADSModule requests dependency InstanceManagerPlugin...
[15:37:48] [Loader Info]          : Loaded InstanceManagerPlugin by CubeCoders Limited
[15:37:48] [Loader Info]          : ADSModule requests dependency SystemUserManagerPlugin...
[15:37:48] [Loader Info]          : Loaded SystemUserManagerPlugin by CubeCoders Limited
[15:37:48] [Scheduler Warning]    : Schedule trigger 52bd537c-f826-4f0c-b866-442de1243983 has a mismatched ID!
[15:37:48] [Loader Info]          : Loaded steamcmdplugin by CubeCoders Limited
[15:37:48] [Core Info]            : Metrics server started OK on port 12820
[15:37:48] [ADS Info]             : ADS startup complete in 75ms
[15:37:48] [ADS Info]             : Updating remote source CubeCoders/AMPTemplates
[15:37:48] [ADS Info]             : Updating existing remote source https://github.com/CubeCoders/AMPTemplates.git...
[15:37:48] [FileManager Notice]   : Using keypair with fingerprint qlOlYdsJu3egHguodC/EmVZNad5TbtodaI4TGRU1WV0=
[15:37:48] [FileManager Info]     : SFTP Server started on 0.0.0.0:2223
[15:37:48] [Webserver Info]       : Websockets are enabled.
Already up to date.
[15:37:49] [Webserver Info]       : Webserver started on http://0.0.0.0:8080
[15:37:49] [Logger Info]          : RouterTimer@10Hz with 2 jobs started
[15:37:49] [Core Info]            : Checking for AMP updates...
[15:37:49] [Core Info]            : Startup mode is UpdateAndStart.
[15:37:49] [Core Info]            : AMP is up to date.
[15:38:13] [Core Warning]         : Slow method invocation: Login took 2939ms to complete.
[15:38:13] [Core Warning]         : Slow response: Core.Login took 2939ms to complete.
[15:38:40] [Scheduler Error]      : Exception executing global interval tick. Too many open files
[15:38:40] [ErrorReporter Error]  : Unhandled Exception: Please report this to the AMP developers.
[15:38:40] [ErrorReporter Error]  : [Error] --- Begin Exception Data ---
[Error] OS: Linux (x86_64/QEMU_KVM)
[Error] Module: ADSModule, Version: AMP 2.5.0.8
[Error] Release spec: Mainline - built by CUBECODERS/buildbot on CCL-DEV
[Error] Instance ID: e504e4c9-5a0b-4a42-afd7-2c3fda68695e
[Error] IOException
[Error] [0] (IOException) : Too many open files
[Error]   at Diagnostics.Process.CreatePipe (IntPtr& read, IntPtr& write, Boolean writeDirection)
  at Diagnostics.Process.StartWithCreateProcess (Diagnostics.ProcessStartInfo startInfo)
3

Of note, my game instances are actually running, it is only the management interface that is not showing the instances and crashing.

4

My web interface crashes also. It takes many attempts to get in. The platform has been stable for many updates. Seems like this only occurred after the last update and a period of time of running. I paid for the Advanced Edition License and thought that came with actual support from CubeCoders. I’m only running one server with three modded minecraft servers on it.

5

Clear your browser cache. It needs it. If it’s still not showing, please let us know and we can check a few things

6

Attempting to clear the cache did not initially change anyting in my instance, but I have resolved with the following. Not saying all of this was required to solve the issue, but something in this got it to boot and stay up.

  • Cleared cache in browser, attempted connection - Crashed on login
  • Restarted web server with systemctl restart ampinstmgr.service
  • Attempted to connect with Incognito window - Crashed before entering credentials
  • Shut down VM & Rebooted entire UnRaid server
  • Attempted connection in standard web browser window - Connection rejected but did not crash
  • Rebooted VM that AMP is installed in
  • Cleared browser cache again, successfully connected

So far instances are showing & I’m not crashing out of the management pannel.

7

Go to the amp user sudo su -l amp and show what it says for ls -al .ampdata

1 Like
8

clearing cache sometimes lets me in. Otherwise I enter credentials and then the credentials boxes disappears and it just sits. I can hit refresh a few times and then it loads in. My instances are still missing when I get in.

here is that command you wanted to have run:

 CubeCoders AMP  :computer:amp@Ubuntu01  :file_folder:~  ls -al .ampdata
total 136
drwxrwxr-x 4 amp amp 4096 Apr 11 17:45 .
drwxr-x— 4 amp amp 4096 Oct 29 2023 …
drwxrwxr-x 7 amp amp 4096 Feb 8 15:20 instances
-rw-r–r-- 1 amp amp 10449 Apr 11 18:37 instances.json
-rw-rw-r-- 1 amp amp 1902 Oct 29 2023 instances.json-20231029.bak
-rw-rw-r-- 1 amp amp 4140 Oct 29 2023 instances.json-20231105.bak
-rw-r–r-- 1 amp amp 6187 Nov 5 2023 instances.json-20231118.bak
-rw-r–r-- 1 amp amp 6187 Nov 17 23:57 instances.json-20231216.bak
-rw-r–r-- 1 amp amp 6266 Dec 16 16:33 instances.json-20240104.bak
-rw-r–r-- 1 amp amp 8334 Jan 4 17:57 instances.json-20240120.bak
-rw-r–r-- 1 amp amp 8334 Jan 20 10:28 instances.json-20240204.bak
-rw-r–r-- 1 amp amp 8334 Feb 4 11:23 instances.json-20240208.bak
-rw-r–r-- 1 amp amp 10449 Feb 8 15:20 instances.json-20240213.bak
-rw-r–r-- 1 amp amp 10449 Feb 13 16:45 instances.json-20240401.bak
-rw-r–r-- 1 amp amp 10924 Apr 1 11:55 instances.json-20240411.bak
drwxrwxr-x 3 amp amp 4096 Oct 29 2023 Versions
 CubeCoders AMP  :computer:amp@Ubuntu01  :file_folder:~ 

10

Err, no? Third line shows the dir

11

you’re right. sorry just a long week

12

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.