Through hard experience, I’ve learned the importance of backing up all of my data. On my hard drive, this is easy: I use an Internet-based backup service which works automatically in the background. As I begin to store important data in Dynalist, I wonder how confident I can be that, no matter what bugs may bite Dynalist, my data will remain safe and accessible.
I’ve configured automatic backups to my Google Drive, but the method is rather inefficient as it simply creates a new backup every day instead of storing only deltas. It’s reasonable for the platform, though. Once the API becomes available, I’ll be able to write a script to copy my data to my hard drive, thus entering it into my regular backup scheme.
On the sync side, we compute a diff on the browser to send to the server. Once the server acknowledges, the user should see a ‘synced’ text on the top right corner, which is an aggregate of the sync status of all documents currently loaded. We’re still working to get a status indicator to mobile versions of Dynalist.
Once the server acknowledges that the sync succeeded, data is kept in two formats in a mysql database:
The latest version of the document, as it is right now.
An incremental list of diffs with timestamps, which we may or may not keep forever (we haven’t made a decision about this yet). This is essentially the version history, which we’re keeping now so that it can be used in the future once we roll out the version history feature along with other collaboration-related improvements.
The entire mysql database is on a primary-slave setup with a completely separate server, currently in the same data-center. This gets us a rolling backup of anything that makes it to Dynalist’s main DB. We have plans to add another backup server in a different data center in the future.
That’s the current state of our setup as of November 2016. I believe that we’re not yet at a point where we can afford expensive enterprise data protection systems, but as we grow, we will definitely be upgrading our systems for improved security and integrity.
I hope that answers your question. Of course, if you have any suggestions, feel free to let us know!
I don’t know the specifics here, but I’m wondering whether having the slave server on a service such as AWS would cost about the same as your current setup with the added benefits of an offsite location and a highly experienced team to protect against data loss associated with hardware failures and the like.
I’ve never used AWS so I don’t know the cost, but I’ve heard that it’s one of the lowest-priced options out there.
We’re currently hosting our servers on Digital Ocean, but our plan to add new backup servers would likely be on a different provider, possibly AWS. Currently the consideration for adding new slave servers outside is performance and bandwidth cost. Synchronizing change within a data-center is blazing fast, while going out is less so. Cloud service providers usually make bandwidth within-data-center to be free of charge, while anything transferring in and out is at a cost.
