Sync deletes stuff

If you have multiple users or multiple devices, stuff you type can be deleted without chance of recovery! If this matters to you, please Like this post!

Steps to reproduce

  1. Create an empty node, and sync.
  1. Go to a second computer, sync.
  2. Disconnect from the network. With that same computer, edit that node, say typing:
  • a
    • b
    • paste a TON of stuff here.
  1. Back to the first computer, delete that empty node.

  2. Back to the second computer, reconnect to the net, and sync.

Understand, the sequencing in practice might not be intended. It could be the result of simply working with 2 devices concurrently, and having a momentary internet disruption.

Actual result

The entire tree vanishes. Depending on the sequence of events, the vanished tree may not appear in the version history of the document.

At first, this might seem logical, but it is clearly not the intent of the two users viewed in isolation. Also, deleting an empty node was never intended to delete future development of an empty node into relevant data.

Expected result

I expect those edits to be retained. Especially because it was such a trivial deletion, it shouldn’t override the edit, but even if it’s not a trivial thing that is deleted, the stuff created on the other computer should not be deleted by a delete taking place elsewhere. They don’t know what all was deleted.

If both nodes are edited, the sync process should know both nodes are edited, and create an extra conflict node so we can see both versions. A tag gets attached to each version saying we have a sync issue, and the users can review both and integrate. (Searching for #syncissue could be very helpful for a user trying to resolve the confusion.)

If one node is deleted and the other edited, the edited one should remain, and (optional) mark as a #syncissue.

Environment

Tested with Windows, Chrome, and iOS app as my two devices. I had wifi not connected for the computer. (was at Starbucks, connection not opened yet).

Additional information

No, the item that got deleted never made it into the History.

Additional comments

A very important point here: I might not know the other computer didn’t sync yet. From the current screen it says synced, but if they other was turned off, then I turn it on and it syncs, accidental deletion may occur.

Points 3 and 4 above can (probably) happen in reverse. That is, everything is synced and I delete a node. Later I go to a computer which I thought was synced, do a whole bunch of work, and then SURPRISE it all gets erased.

Implementing is a bit tricky of course. In detail, what you should have is timestamping or version stamping of nodes. When syncing a change, you need to detect not the just exact same node, but any children of that node in the case of deletion.

I am all but certain this bug has caused me to lose a non-trivial amount of data in the past. Trouble is it can be subtle - something disappears that you aren’t looking at now. And afterwards you wonder if you really typed it where you thought you did.

7 Likes

The root cause of this issue, I think, is that the editing action doesn’t re-add an item if it’s already deleted. In theory, the now detached item was still successfully modified, it’s just that nobody can see it.

Is this a surprise though? By the last-sync-wins rule, if the last person decides to delete it, the item will get deleted, otherwise we’re basically invalidating the user’s actions, which might create inconveniences too.

When point 3 and 4 happen in reverse, this happens: when the first computer gets connection, changes from the 2nd computer come first and get applied. Then the item gets deleted. This means that changes from the first computer are retrievable in version history.

This is not ideal, but I think it’s better than preventing the deleting from happening at all. It’s a balance between accuracy and security. When point 3 and 4 are reversed, the current model is more accurate, but it runs the risk of losing some data (still retrievable in version history but you might not realize it). It’s the choice of “do I prefer to occasionally [unintentionally delete data] or [fail to delete data]”?

Generating conflict copies is probably the most fail-safe way though.

@Alan I know you’ve read this, just posting this as a reference: Dynalist concurrency model

If I understood it correctly, the case you’re raising is different from what’s being discussed in that post because the order of 3 and 4 are reversed.

I emphatically disagree. In all circumstances, to minimize accidentally deleted information is much more important than any other consideration. If I fail to delete data, I will see the data and can delete it again. If I accidentally delete data, I don’t see it missing and I cannot restore it.

“Last sync wins” is not a good rule. A sync should consider both the server and the client version equally valid, not preferring the client over the server. But it should be able to know that when the client version (portions of it) are equal or newer than the server.

I need to press this issue. So long as this bug exists, I cannot be comfortable moving forward with Dynalist. All other bugs and feature requests won’t matter.

I use multiple many devices in many locations. Hypothetically, I could one day turn on a laptop I hadn’t used in months, a sync happens, and a huge amount of data gets erased.

If a full conflict resolution model is beyond your resolution to implement, can we at least get an ‘orphans’ feature which will store things that fell away, which I can refer to, to find something I unintentionally deleted. (Such might not only be about sync problems, but include accidental deletions.)

Thoughts?

Very disappointed about lack of response on this.

My concern is somewhat relieved since I discovered the daily offsite backup feature. (When was this added?). This limits the amount of info that might be lost to levels I can live with.

(Fortunately for you, I failed to find a comparable cloud-based multi-platform outliner. Omni is very nice looking, a worthy source of ideas, but no Windows or browser support.)

Another thing thst would mitigate this issue: better sync indicators. On the ios app there is no indication at all if things are synced. I tested with airplane mode and nothing happened at all. Very dangerous. On Windows chrome app, the words changes are subtle. I would much appreciate a bolder indicator including color and / or icon changes. MS OneNote does this well.

IT HAPPENED AGAIN. I was typing some new stuff, realized i had no internet connection, so I fixed that. Lost about 4 lines of text as the sync kicked in and deleted the stuff I just created.

DOES nobody else have this issue? For anybody trying to do multi-user editing I can’t believe it doesn’t come up more frequently.

It’s not a common occurrence for me because I am aware of the bug and try to avoid it, but it does happen, and this is still my A1 priority.

I thought of a simple solution, unless I’m wrong:

Make the Sync operation always operate as

  1. local edits sent to server and merged.
  2. server data sent to client.

Now with 1, you need to ensure every edit or add arrives at the server. If a parent node was erased, it needs to be brought back to service the current edit. Take the version from the client.

The value of this process is you have the version history on the server, so if anything of the result is undesired, you can go to the history to recover it.

As things are currently, local edits can be erased by the sync process and no history exists.

I just realised that I lost important notes by accidentally editing and syncing a document from my phone BEFORE retrieving the edits on the same document from my laptop.

@Masato_Kakinoki, is it possible your notes are still in the version history? Even if they got overwritten with the server version, they might be visible as a previous version of the doc.

Craig, you understand the problem in the word “might”? It depends on which order it happens, and if the order is wrong, the data is just erased with no recourse.

Yesterday I upgraded my account to PRO just to retrieve the lost notes, but they were not there. I hope support team can retrieve them. I wish Dynalist had an Evernote-like solution (Evernote keeps conflicts as ‘Conflicting modification’). Now I am considering switching back to WorkFlowy…

1 Like

Which service are you using now to avoid this risk? WorkFlowy?

Aw, bummer. So sorry to hear that. :frowning: I hope the Dynalist team can help you out.

1 Like

Thank you Craig.

This, or something just like it, just happened to me. Yesterday I was in a location with poor service and today when the sync occurred I could see an item I entered yesterday was visible and then disappeared. What makes me wonder if this is slightly different is that then a dialogue appeared saying I didn’t have a document selected and that I needed to choose one in the documents list. After choosing the same document I previously had active (I mainly work on only one) the entry I made yesterday was definitely gone.

I ran into 2 data loss issues with Dynalist so far. Once where a node was missing on mobile, then when I looked on web, it was also gone there. Once where a note on an item went missing.

I love dynalist as a product and was considering moving more stuff over to it but data loss is very scary.

2 Likes

This is still an issue. Syncing regularly deletes data for me. Please fix it if you can.

I agree it’s an issue. Since then I have learned a meticulous way of working to be sure this never comes up. I always ensure documents are synced before I delete anything. It’s the only way.

I have no issues. But I am super curious about folks who do. There must be some common denominator?

I’ve used dynalist daily for 4+ years now and never had data loss. I use the mac app and the browser website, and the API thru various scripts, and android apps.

Theories:

  • perhaps I tend to write in list form, so I generally start and finish unique NodeIDs in one go, so there’s no “conflict resolution” coming into play between devices.
  • I wouldn’t say my internet is particularly reliable. I use dynalist for hours on busses with spotty internet access.
  • I use dynalist and have it open on many devices, linux, windows, mac, android, ios, actually all of them pretty equally.

As I vaguely understand it, each bullet point has a unique ID number and last modified date. So for a sync issue to occur you must:

  1. Create a new bullet point on one device, then
  2. Edit that same bullet point on another device.
  3. Then the server then gets sync data from both devices of Node ID # and last modified date, and discards the data from the device with the earlier timestamp.
  4. The user should be happy since their last edit of the day is what “won” the conflict resolution, and was ultimately saved. I can’t think of a reason why you’d want the older edit to win.

I’m kind of foggy of what exactly is happening to folk - items are disappearing? Are you sure you didn’t just move or delete it on another device, and you’re witnessing that sync? Or it just got pushed off the page by everything else that synced? Anyone have guesses what’s going on technically, with the folks having issues?

1 Like