2018/02/07 5PM EDT to 2018/02/07 9:00PM EDT (~4 hours)
Dynalist service is intermittent, especially regards to syncing. Several users reported seeing internal server errors. No data was lost.
Disk space was full on our web server, causing large web-requests to be dropped as temp files couldn’t be stored for processing. This was mostly taken by
/tmp and restarted server.
Prevention going forward
Installed automated scripts to clean
Why this hasn’t been caught/fixed sooner
Our disk space monitoring graph was just slightly out of view on our main dashboard, so we did not noticed while it was getting full. We’ve rearranged our dashboard as a result to better monitor the disk space.
The reason why the service was down for ~4 hours was because all of our team members were unavailable at the time of outage.
Sorry for the inconvenience, and thanks to all those who reached out to report the problem!