This incident has been resolved and the platform is fully operational.
Posted Sep 05, 2020 - 17:13 CEST
We have deployed a fix in production and the system is back to normal. We are monitoring the stability of the platform.
Posted Sep 05, 2020 - 13:18 CEST
We are continuing to monitor for any further issues.
Posted Sep 05, 2020 - 09:44 CEST
We have deployed a temporary fix that makes the runtime (/dialog endpoint) fully operational again. Users will be able to have fully functional conversations with the bots. The temporary fix comes with the following limitations: * New conversations will not appear in the conversation logs (Monitor tab) * New conversations will not be reflected in the usage metrics (Monitor tab) * Conversation logs and usage metrics cannot currently be retrieved. Old conversation logs and usage metrics will become available again once the issue is fully resolved.
We continue to work on the DB volume increase, which should be finished in about 20 minutes and would remove the aforementioned limitations. We will monitor the system and keep you updated on the progress.
Posted Sep 05, 2020 - 09:40 CEST
The volume increase is ongoing - at this point we are not able to give an estimate when it will complete. We are investigating options to recover the runtime (/dialog endpoint) before the volume increase concludes. We apologize for this outage and will keep you updated on the progress.
Posted Sep 05, 2020 - 07:47 CEST
Our teams are continuing to work on a solution. Our storage for conversation logs ran out of disk space unexpectedly. We will now increase the volume size and restart the corresponding database to bring the /dialog endpoint back up. We will keep you updated on the progress.
Posted Sep 05, 2020 - 06:03 CEST
We have identified the root cause of the outage and we are working on a fix. Fix will be implemented as soon as possible.
Posted Sep 05, 2020 - 05:10 CEST
We are currently experiencing outages on our /dialog endpoint. Our team is working on identifying and fixing the issue. We apologize for the inconvenience.
Posted Sep 05, 2020 - 04:50 CEST
This incident affected: Bot Run (Connector, NLU Analysis (French), NLU Analysis (English), NLU Analysis (Spanish), NLU Analysis (German)) and Bot Management (Monitor).