[Investigating] Rayos suffered database issues start around 1AM CDT in us-east-1.
We're currently investigating the cause, but services have been restored at this time.
June 3, 2020 2:23AM CDT June 3, 2020 7:23AM UTC
[Monitoring] The issue appears to be related to an automated task that optimizes tables. In this case, it caused a transaction deadlock on the database system.
We're looking into changes this process to prevent the deadlocks in the future.
June 3, 2020 2:26AM CDT June 3, 2020 7:26AM UTC
June 3, 2020 12:38PM CDT June 3, 2020 5:38PM UTC
[Monitoring] At 1pm CDT today, we will be making a software release that should prevent the specific database deadlock from occurring again.
[Identified] The RAYOS 0.12.0 release that was made earlier has an upgrade issue with Gravity Forms. Consumers attempting to submit forms are reporting issues posting. An upgrade script needs to be run but will require making an out-of-band release of 0.12.1 to address the issue. The upgrade script will be deployed and ran as soon as possible today.
June 3, 2020 5:47PM CDT June 3, 2020 10:47PM UTC
[Resolved] Gravity forms database upgrade script has been run on all sites. Form submissions and capturing of all fields should be functioning once more on all sites.
[Identified] Imported videos from the CNN Video Affiliate Network are no longer functioning properly. Customers with access to and have configured CNN Full Bundle imports do not appear to be affected. We are working with CNN to resolve this issue and will update this advisory when a resolution is available.
June 2, 2020 11:23AM CDT June 2, 2020 4:23PM UTC
[Resolved] We've migrated affected customers to an alternative feed choice and are still working with CNN behind the scenes to deal with other potential feed API changes.
[Identified] Semantic keyword tagging using OpenCalais is presently down and appears to be related to an internal API issue within Reuter's systems regarding invalid SSL certificates. The actual public facing end-point is available. A support ticket has been opened with Reuter's to investigate the issue. There is no ETA on a fix at this time.
Customers making use of BLOX smart tagging are unaffected as that uses another provider.
June 1, 2020 2:52PM CDT June 1, 2020 7:52PM UTC
[Monitoring] Reuters has responded and has confirmed an issue with SSL certificates on their end. They are working on attempting to resolve the issue.
June 2, 2020 11:04AM CDT June 2, 2020 4:04PM UTC
[Resolved] Reuters has fixed the certificate issue with their domains and keyword suggestion should now be functioning again.
[Identified] Beginning June 02, 2020 all email accounts hosted on Townnews will be required to have the passwords changed every 60 days.
Beginning May 26th these email accounts will receive a daily reminder about the password expiration, including instructions on how to change your password.
Please contact Townnews if you have questions. Users can change their email password at any time by clicking the "Change password" link on our webmail portal: https://webmail.tn-cloud.net/