OS Name/Version: Debian 10 on a KVM server
Product Name/Version: AMP version 2.4.4.0 (Decadeus)
Problem Description:
We have 2FA activated and enforced by default. Which is mandatory for us. When I tried to log in today everything was normal just like yesterday but now then I try to manage any instances which where created before today aren’t accessible anymore with the following error message:
Instance logs seem to be normal but there you go:
[00:00:00] [Logger Info] : Rollover from previous log file: AMP_Logs/AMPLOG 2023-06-08 00-00-00.log
[03:44:00] [Core Info] : Checking for AMP updates...
[03:44:00] [Core Info] : AMP is up-to-date.
[09:44:00] [Core Info] : Checking for AMP updates...
[09:44:00] [Core Info] : AMP is up-to-date.
[15:44:00] [Core Info] : Checking for AMP updates...
[15:44:00] [Core Info] : AMP is up-to-date.
[17:41:20] [Logger Activity] : Authentication attempt for user block from 127.0.0.1
[17:41:20] [Logger Activity] : Authentication attempt for user block from 127.0.0.1
[17:41:20] [Core Activity] : Authentication success
[17:41:20] [Logger Activity] : Authentication failure for user block from 127.0.0.1 - TwoFactorChallenge
[17:43:00] [Core Error] : SocketException
[17:43:00] [Logger Error] : [0] (SocketException) : Connection refused
[17:43:00] [Core Error] : at Net.Sockets.Socket.Send (Byte[] buffer, Int32 offset, Int32 size, Net.Sockets.SocketFlags socketFlags)
at Net.Sockets.UdpClient.Send (Byte[] dgram, Int32 bytes)
at ModuleShared.MetricsInfoPublisher.ScheduleHandler (Object sender, EventArgs e)
[18:15:29] [Logger Activity] : Authentication attempt for user Jens from 127.0.0.1
[18:15:29] [Core Activity] : Authentication success
[18:15:29] [Logger:Anonymous Activity] : Authentication attempt for user Jens from 127.0.0.1
[18:15:30] [Logger:Anonymous Activity] : Authentication failure for user Jens from 127.0.0.1 - TwoFactorChallenge
[18:15:49] [Logger Activity] : Authentication attempt for user Jens from 127.0.0.1
[18:15:49] [Core Activity] : Authentication success
[18:15:49] [Logger:Anonymous Activity] : Authentication attempt for user Jens from 127.0.0.1
[18:15:49] [Logger:Anonymous Activity] : Authentication failure for user Jens from 127.0.0.1 - TwoFactorChallenge
[18:18:38] [Logger Activity] : Authentication attempt for user Jens from 127.0.0.1
[18:18:38] [Core Activity] : Authentication success
[18:18:38] [Logger:Anonymous Activity] : Authentication attempt for user Jens from 127.0.0.1
[18:18:38] [Logger:Anonymous Activity] : Authentication failure for user Jens from 127.0.0.1 - TwoFactorChallenge
And this is the log from the panel itself
[18:11:47] [Logger Info] : Loading locale data for xx from xx.json...
[18:11:47] [Logger Info] : Starting AMP version 2.4.4.0 (Decadeus), built 17/05/2023 17:54
[18:11:47] [Logger Info] : Stream: Mainline (Release) - built by CUBECODERS/buildbot on CCL-DEV
[18:11:47] [Logger Info] : Running in a QEMU KVM environment.
[18:11:47] [Logger Warning] : Current time zone is set to CET - It is recommended you switch to UTC+0 to avoid time-zone related issues
[18:11:47] [Logger Info] : OS: Linux / x86_64
[18:11:47] [Logger Info] : CPU: Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz (16C/16T)
[18:11:47] [Logger Info] : RAM: 64424MB
[18:11:47] [Logger Info] : AMP Instance ID: 10de0028-8577-44dc-80a9-861c77323c85
[18:11:48] [Logger Info] : Loaded ADSModule version 1.0.0.0 by CubeCoders Limited
[18:11:48] [Logger Info] : Loaded FileManagerPlugin by CubeCoders Limited
[18:11:48] [Logger Info] : Loaded EmailSenderPlugin by CubeCoders Limited
[18:11:48] [Logger Info] : Loaded WebRequestPlugin by CubeCoders Limited
[18:11:48] [Logger Info] : Loaded LocalFileBackupPlugin by CubeCoders Limited
[18:11:48] [Logger Info] : Loaded CommonCorePlugin by CubeCoders Limited
[18:11:48] [Logger Info] : ADSModule requests dependency InstanceManagerPlugin...
[18:11:48] [Logger Info] : Loaded InstanceManagerPlugin by CubeCoders Limited
[18:11:48] [Logger Info] : ADSModule requests dependency SystemUserManagerPlugin...
[18:11:48] [Logger Info] : Loaded SystemUserManagerPlugin by CubeCoders Limited
[18:11:49] [Logger Info] : Loaded steamcmdplugin by CubeCoders Limited
[18:11:49] [Logger Info] : Metrics server started OK on port 12820
[18:11:49] [Logger Info] : ADS startup complete in 290ms
[18:11:49] [Logger Info] : Updating remote source CubeCoders/AMPTemplates
[18:11:49] [Logger Info] : Updating existing remote source https://github.com/CubeCoders/AMPTemplates.git...
[18:11:49] [Logger Notice] : Using keypair with fingerprint PtBXOTbMS/2ZdjrNdJd2jSgsoyTBFzAxQTGZ3LX+1P0=
[18:11:49] [Logger Info] : SFTP Server started on 0.0.0.0:2223
[18:11:49] [Webserver Info] : Websockets are enabled.
[18:11:49] [Logger Info] : RouterTimer@10Hz with 2 jobs started
[18:11:49] [Logger Info] : Webserver started on http://0.0.0.0:8080
[18:11:49] [Core Info] : Checking for AMP updates...
[18:11:49] [Logger Info] : Startup mode is StartApplication.
Bereits aktuell.
[18:11:50] [Core Info] : AMP is up-to-date.
[18:11:53] [Logger:Jens Error] : Error handling request:[POST:/API/ADSModule/GetInstances] - [InvalidOperationException]: System.InvalidOperationException: Collection was modified; enumeration operation may not execute.
at System.Collections.Generic.List`1+Enumerator[T].MoveNextRare () [0x00013] in <fbab4baee7c44bd8a064d25f6d86a08d>:0
at System.Collections.Generic.List`1+Enumerator[T].MoveNext () [0x0004a] in <fbab4baee7c44bd8a064d25f6d86a08d>:0
at System.Linq.Enumerable+WhereListIterator`1[TSource].MoveNext () [0x0004e] in <20af23a0769a48079b9e6e196c7a1c9e>:0
at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeList (Newtonsoft.Json.JsonWriter writer, System.Collections.IEnumerable values, Newtonsoft.Json.Serialization.JsonArrayContract contract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract collectionContract, Newtonsoft.Json.Serialization.JsonProperty containerProperty) [0x000e1] in <dd21e08747d94bcebed8b56112226337>:0
at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeValue (Newtonsoft.Json.JsonWriter writer, System.Object value, Newtonsoft.Json.Serialization.JsonContract valueContract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract containerContract, Newtonsoft.Json.Serialization.JsonProperty containerProperty) [0x000d6] in <dd21e08747d94bcebed8b56112226337>:0
at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject (Newtonsoft.Json.JsonWriter writer, System.Object value, Newtonsoft.Json.Serialization.JsonObjectContract contract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract collectionContract, Newtonsoft.Json.Serialization.JsonProperty containerProperty) [0x0008d] in <dd21e08747d94bcebed8b56112226337>:0
at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeValue (Newtonsoft.Json.JsonWriter writer, System.Object value, Newtonsoft.Json.Serialization.JsonContract valueContract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract containerContract, Newtonsoft.Json.Serialization.JsonProperty containerProperty) [0x000b2] in <dd21e08747d94bcebed8b56112226337>:0
at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeList (Newtonsoft.Json.JsonWriter writer, System.Collections.IEnumerable values, Newtonsoft.Json.Serialization.JsonArrayContract contract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract collectionContract, Newtonsoft.Json.Serialization.JsonProperty containerProperty) [0x000d6] in <dd21e08747d94bcebed8b56112226337>:0
at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeValue (Newtonsoft.Json.JsonWriter writer, System.Object value, Newtonsoft.Json.Serialization.JsonContract valueContract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract containerContract, Newtonsoft.Json.Serialization.JsonProperty containerProperty) [0x000d6] in <dd21e08747d94bcebed8b56112226337>:0
at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject (Newtonsoft.Json.JsonWriter writer, System.Object value, Newtonsoft.Json.Serialization.JsonObjectContract contract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract collectionContract, Newtonsoft.Json.Serialization.JsonProperty containerProperty) [0x0008d] in <dd21e08747d94bcebed8b56112226337>:0
at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeValue (Newtonsoft.Json.JsonWriter writer, System.Object value, Newtonsoft.Json.Serialization.JsonContract valueContract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract containerContract, Newtonsoft.Json.Serialization.JsonProperty containerProperty) [0x000b2] in <dd21e08747d94bcebed8b56112226337>:0
at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.Serialize (Newtonsoft.Json.JsonWriter jsonWriter, System.Object value, System.Type objectType) [0x00079] in <dd21e08747d94bcebed8b56112226337>:0
at Newtonsoft.Json.JsonSerializer.SerializeInternal (Newtonsoft.Json.JsonWriter jsonWriter, System.Object value, System.Type objectType) [0x0023a] in <dd21e08747d94bcebed8b56112226337>:0
at Newtonsoft.Json.JsonSerializer.Serialize (Newtonsoft.Json.JsonWriter jsonWriter, System.Object value, System.Type objectType) [0x00000] in <dd21e08747d94bcebed8b56112226337>:0
at Newtonsoft.Json.JsonConvert.SerializeObjectInternal (System.Object value, System.Type type, Newtonsoft.Json.JsonSerializer jsonSerializer) [0x00028] in <dd21e08747d94bcebed8b56112226337>:0
at Newtonsoft.Json.JsonConvert.SerializeObject (System.Object value, System.Type type, Newtonsoft.Json.JsonSerializerSettings settings) [0x00007] in <dd21e08747d94bcebed8b56112226337>:0
at Newtonsoft.Json.JsonConvert.SerializeObject (System.Object value, Newtonsoft.Json.JsonSerializerSettings settings) [0x00000] in <dd21e08747d94bcebed8b56112226337>:0
at GSMyAdmin.ExtensionMethods.SerializeToJSON (System.Object source) [0x0000f] in <634a46d0925d4a5f95f08b26a98b1b87>:0
at GSMyAdmin.WebServer.APIService.HandleRequest (m.Http.IHttpRequest request) [0x00281] in <634a46d0925d4a5f95f08b26a98b1b87>:0
at m.Http.Handlers.Handler+<>c__DisplayClass6_0.<From>b__0 (m.Http.IHttpRequest req) [0x00000] in <6ba0f115d4834d51b3526e12d15cee09>:0
at m.Http.Routing.RateLimitedEndpoint+<>c__DisplayClass4_0.<Wrap>b__0 (m.Http.IHttpRequest request) [0x00019] in <6ba0f115d4834d51b3526e12d15cee09>:0
at m.Http.Router.HandleRequest (m.Http.Backend.HttpRequest httpReq, System.DateTime requestArrivedOn) [0x000cd] in <6ba0f115d4834d51b3526e12d15cee09>:0
[18:12:57] [Logger:Jens Activity] : Managed remote instance Farmwelt01 at http://127.0.0.1:8082/
[18:12:57] [Logger:Jens Activity] : Authentication token for Jens requested by ManageInstance on behalf of Jens
[18:13:42] [InstanceManager:Jens Info] : bfbd3d64086805f25c84a78d6135f1be1bb759775f7c84fead0a0899a5d892c5
[18:13:43] [Logger:Jens Activity] : Managed remote instance ark01 at http://127.0.0.1:8081/
[18:13:43] [Logger:Jens Activity] : Authentication token for Jens requested by ManageInstance on behalf of Jens
[18:13:43] [Logger Error] : LIM - Failure to make API call to ark01: Connection refused
[18:13:52] [Logger:Hannes Activity] : Managed remote instance HannesTestServer01 at http://127.0.0.1:8086/
[18:13:52] [Logger:Hannes Activity] : Authentication token for Hannes requested by ManageInstance on behalf of Hannes
[18:13:55] [Logger:Jens Activity] : Managed remote instance ark01 at http://127.0.0.1:8081/
[18:13:55] [Logger:Jens Activity] : Authentication token for Jens requested by ManageInstance on behalf of Jens
[18:14:05] [Logger:Jens Info] : Stopping instance ark01...
[18:14:10] [InstanceManager:Jens Info] : Error response from daemon: Cannot kill container: AMP_ark01: No such container: AMP_ark01
[18:14:10] [Logger:Jens Info] : Instance ark01 has stopped.
[18:14:47] [Logger:Jens Activity] : Managed remote instance Farmwelt01 at http://127.0.0.1:8082/
[18:14:47] [Logger:Jens Activity] : Authentication token for Jens requested by ManageInstance on behalf of Jens
[18:14:49] [Logger:Hannes Activity] : Managed remote instance Waterfall01 at http://127.0.0.1:8083/
[18:14:49] [Logger:Hannes Activity] : Authentication token for Hannes requested by ManageInstance on behalf of Hannes
[18:14:54] [Logger:Jens Activity] : Managed remote instance Farmwelt01 at http://127.0.0.1:8082/
[18:14:54] [Logger:Jens Activity] : Authentication token for Jens requested by ManageInstance on behalf of Jens
[18:14:58] [Logger:Jens Activity] : Managed remote instance HannesTestServer01 at http://127.0.0.1:8086/
[18:14:58] [Logger:Jens Activity] : Authentication token for Jens requested by ManageInstance on behalf of Jens
[18:15:00] [Logger:Jens Activity] : Managed remote instance Citybuild01 at http://127.0.0.1:8084/
[18:15:00] [Logger:Jens Activity] : Authentication token for Jens requested by ManageInstance on behalf of Jens
[18:15:17] [Logger:Jens Activity] : Managed remote instance Waterfall01 at http://127.0.0.1:8083/
[18:15:17] [Logger:Jens Activity] : Authentication token for Jens requested by ManageInstance on behalf of Jens
[18:15:22] [Logger:Hannes Activity] : Managed remote instance HannesTestServer01 at http://127.0.0.1:8086/
[18:15:22] [Logger:Hannes Activity] : Authentication token for Hannes requested by ManageInstance on behalf of Hannes
[18:15:23] [Logger:Jens Activity] : Managed remote instance TeamSpeak01 at http://127.0.0.1:8085/
[18:15:23] [Logger:Jens Activity] : Authentication token for Jens requested by ManageInstance on behalf of Jens
[18:15:29] [Logger:Jens Activity] : Managed remote instance M120Testserver01 at http://127.0.0.1:8087/
[18:15:29] [Logger:Jens Activity] : Authentication token for Jens requested by ManageInstance on behalf of Jens
[18:15:41] [Logger:Jens Activity] : Managed remote instance Citybuild01 at http://127.0.0.1:8084/
[18:15:41] [Logger:Jens Activity] : Authentication token for Jens requested by ManageInstance on behalf of Jens
[18:15:45] [Logger:Jens Activity] : Managed remote instance Citybuild01 at http://127.0.0.1:8084/
[18:15:45] [Logger:Jens Activity] : Authentication token for Jens requested by ManageInstance on behalf of Jens
[18:15:48] [Logger:Jens Activity] : Managed remote instance M120Testserver01 at http://127.0.0.1:8087/
[18:15:48] [Logger:Jens Activity] : Authentication token for Jens requested by ManageInstance on behalf of Jens
[18:15:51] [Logger:Jens Activity] : Managed remote instance Waterfall01 at http://127.0.0.1:8083/
[18:15:51] [Logger:Jens Activity] : Authentication token for Jens requested by ManageInstance on behalf of Jens
[18:16:00] [Logger:Hannes Activity] : Managed remote instance Waterfall01 at http://127.0.0.1:8083/
[18:16:00] [Logger:Hannes Activity] : Authentication token for Hannes requested by ManageInstance on behalf of Hannes
[18:16:26] [Logger:Jens Info] : Stopping instance Farmwelt01...
[18:16:32] [InstanceManager:Jens Info] : Error response from daemon: Cannot kill container: AMP_Farmwelt01: No such container: AMP_Farmwelt01
[18:16:32] [Logger:Jens Info] : Instance Farmwelt01 has stopped.
[18:16:43] [Logger:Hannes Activity] : Managed remote instance HannesTestServer01 at http://127.0.0.1:8086/
[18:16:43] [Logger:Hannes Activity] : Authentication token for Hannes requested by ManageInstance on behalf of Hannes
[18:18:00] [Logger:Hannes Activity] : Managed remote instance HannesTestServer01 at http://127.0.0.1:8086/
[18:18:00] [Logger:Hannes Activity] : Authentication token for Hannes requested by ManageInstance on behalf of Hannes
[18:18:37] [Logger:Jens Activity] : Managed remote instance M120Testserver01 at http://127.0.0.1:8087/
[18:18:37] [Logger:Jens Activity] : Authentication token for Jens requested by ManageInstance on behalf of Jens
[18:18:41] [Logger:Hannes Activity] : Managed remote instance HannesTestServer01 at http://127.0.0.1:8086/
[18:18:41] [Logger:Hannes Activity] : Authentication token for Hannes requested by ManageInstance on behalf of Hannes
[18:32:44] [Logger:Alpha Activity] : Managed remote instance HannesTestServer01 at http://127.0.0.1:8086/
[18:32:44] [Logger:Alpha Activity] : Authentication token for Alpha requested by ManageInstance on behalf of Alpha
Steps to reproduce:
We created a new user and tried it = same error
We disabled 2FA = same error (even though the 2FA was not enforced anymore)
We checked the permissions, cleared user permission, checked in a different browser, deleted the cache and restarted the panel itself only to get greeted by the same problem that we are unable to mange our instances because of a 2FA check or something
Actions taken to resolve so far:
see actions taken above
We hope you can resolve this issue so we can use our instances again