Lost Data from iOS (related to sharing extension)

Kerim · March 3, 2020, 2:43am

Steps to reproduce

Starting from scratch, what are the steps to make the bug happen? The fewer the steps, the better.

Save link to iOS app from Safari using Sharing extension

Expected result

What do you expect to see after carrying out the steps above?

Link should be synced to web

Actual result

Instead of the expected result, what happened?

Links were not synced. Later I tried to force a refresh and all my saved links (about 5 of them, saved over the past week or so) were deleted, overwriting the file with the version on the web.

Environment

Which operating system are you using? iOS 13.3
Which browser are you using? Safari Mobile
If you’re using a desktop or mobile app, what’s the version number of Dynalist? I don’t see this information in the iOS settings or help screen, but I’m up to date with the latest version.
Are you using any third-party scripts for Dynalist, e.g. PowerPack? no.

Additional information

I did some edits in the inbox on the web as well before I noticed the saved links were missing.

Additional comments

none.

Shida · January 1, 2020, 3:41pm

May I ask, how long did you keep the app open after performing the share operation?

We’ve gotten multiple reports over the last few months but somehow I am still unable to reproduce this exact problem on our iOS device. If you have a way to reproduce the issue consistently, it would be really helpful if you can record a video of the exact steps so we can try to follow w.r.t. timing, etc.

Alan · January 1, 2020, 7:04pm

Its possible those edits get synced to the server as later than the added links, and thus overwrites them. To avoid this it is absolutely necessary to be sure your edits are in a distinct part of your outline from where the links are sent. OR, be diligent about syncing.

Kerim · January 2, 2020, 2:23am

The share extension now launches the app when creating the links. I usually switch back to whatever I’m doing. But I’ve opened and worked on other documents in Dynalist for hours afterwords. It seems strange that the inbox wouldn’t sync while I’m doing that.

I’m curious why the extension works this way. When I use the API via Drafts or the Dynalist Bookmarker Chrome extension I don’t seem to have these problems. Moreover, I don’t even have to launch Dynalist. (And as an added bonus I can even save things to other documents besides the inbox.) Why are these 3rd party solutions so much more robust than the Dyanlist share extension?

Shida · January 2, 2020, 3:45am

The Share extension is native iOS code that pushes the text to the (cordova) app, which inserts it as if you’re editing it using the app. I still haven’t been able to find out why items inserted using this method doesn’t sync whereas items inserted traditionally by typing syncs.

3rd party methods rely on the API which pushes the text directly to the server, then the apps would pull that change from remote.

Alan · January 2, 2020, 5:27am

This extension aside, Sync doesn’t happen on an iPhone when DynaList isn’t loaded and active. In particular, if I’m working disconnected, then leaving the app then later connecting; such is not a valid multi-device process with DynaList. You can do the above, but you must go back into the app to give it time to sync. (Also, sync doesn’t happen every millisecond; there’s always a notable delay.)

My guess is this extension doesn’t keep dynalist open for long enough to allow the sync process to complete.

Kerim · January 2, 2020, 8:01am

This doesn’t explain it, because the app is open otherwise and should have time to sync later on.

Alan · January 2, 2020, 3:49pm

Oh, you are actively working in DynaList? Okay then.

Panzerboy · February 7, 2020, 9:41am

I got bitten by this bug today as well. I shared a link from Overcast to Dynalist, I could see it in Inbox on the iPhone, but on the web it never appeared.

I added then some other text in the Inbox on the iPhone which, after a while, appeared in the web. But the link I originally shared from Overcast never appeared.

This is a deal breaker for me. Things like this make me not trust Dynalist at all, as I am now wondering what other things are just on the iPhone/iPad and not on the web.

Is this something the devs are looking into?

Shida · February 7, 2020, 9:54am

It is something we’re aware, but so far we haven’t been able to find out the root cause yet. We’re still looking for more reports of how it can happen, any data point helps!

If you notice any specific behavior that gets the bug to reproduce reliably please share it with us.

Panzerboy · February 7, 2020, 9:57am

Of course, now when I tried to reproduce it, it worked

Shida · February 7, 2020, 10:03am

Haha, yeah, we’re pretty frustrated about this issue too. As far as all the support channels we’re listening on, this seems to be the only remaining synchronization bug we have, so I’d love to have it fixed. Thanks in advance!

Alex_Avenarius · February 7, 2020, 11:03am

All I can say is that, because of these concerns, I don’t use the sharing extensions on iOS or Android but, instead, the specialized app Dynawrite on both platforms. No lost data when submitting Inbox items that way, as far as I’m aware. (I think they get submitted to the server directly from Dynawrite.)

I realize it’s a workaround and doesn’t help fixing the current issue with extensions. But it’s something I’ve got used to over time, and I have Dynawrite always within easy reach (for example, on my Android lock screen).

Alan · February 7, 2020, 1:41pm

You have never yet addressed the problem of offline work. I can’t safely work on a document offline because when it comes online there’s a chance that unknown to me a node was erased on another machine and when I go online the sync erases all my work. You might say “Expected behavior” but that’s not close to good enough.

I don’t know, but it’s possible the sharing mechanism interacted with this misfeature of the sync mechanism and caused the lost data the OP writes about here.

Shida · February 7, 2020, 5:38pm

I believe that’s correct, they use the API to send directly to the server.

That’s generally dependent on how you work - If your work isn’t shared with other users there’s less chance of this happening. I think this is more of an edge case that most people can keep track to avoid. A more extreme example could be if you delete a document from another machine, but still expecting offline changes to sync to the deleted document. While I think it’s still reasonable, most people would consider that a pretty rare edge-case.

Alan · February 7, 2020, 8:05pm

It’s true that I am not sharing my work, but I still feel unsafe working offline. I have been bitten multiple times by this in the past, and so I simply resolve to always make sure I sync before leaving a device and sync before starting work on a different device. It’s the only way to be safe, and that means never going a place where there’s no wifi to work.

Shida · February 12, 2020, 8:51pm

Just to update all of you on this issue: I have reproduced the problem, nailed down the exact cause of the issue, came up with a patch which fixes the issue and verified the fix to be working.

We will be pushing an update out soon!

For anyone interested in a technical overview of the bug:

Internally, dynalist uses a “diff” system to sync changes to the server. We store a local copy of your current Dynalist file, and a local copy of the “last synced” Dynalist file. When it’s time to send changes to the server, a “diff” is created by comparing these files and summarizing the difference.

When we compute the diff, an additional step of optimization is done to avoid scanning the whole file: we mark each item with a version number, and when they’re edited locally, the version number is bumped to indicate that it’s been changed. When computing a diff, any item with version numbers below the last synced number is ignored.

When you capture an item right after starting the app (i.e. when you share from another app), there’s a chance you click the button before the destination inbox document is loaded. We wait until it’s loaded before sending the item and showing a confirmation dialog.

The issue is caused by the fact that

We insert the captured item when an event fires that the document is now available.
The event is fired before the version number tags are loaded.
This means the inserted item is tagged with a version number of 0, so the sync algorithm ignores it.

To fix the issue, I’ve made 2 changes. Either one of them should fix the issue, but I want to make things right:

When inserting the captured item, we wait until the full document loading cycle is over, rather than the moment when the document is available (event/signal).
When loading a document, its version tags are loaded first, before the document is made available.

The reason why it’s difficult to reproduce this issue is that most of the time, the document is already loaded. Or if your phone is fast enough, it might be loaded before you click on the capture button.

Kerim · February 13, 2020, 3:16am

Wow. Great news on figuring this out! (Also thanks for the clear explanation which makes total sense to me even though I am not myself a software developer.)

Kerim · February 19, 2020, 2:19am

Hmm. Don’t know if this should be a new thread or not, but I found that items can be slow to show up (although they do eventually), if I move them after adding to inbox.

Share to inbox from iOS share menu
Click move to X document

The changes to document X are not synced. I then open that target document on iOS and force refresh. It syncs, but at first the newly added item doesn’t appear. Later, however, it finally shows up.

Not exactly sure what is going on here, but sync is far from as smooth or responsive as it should be so I thought I’d mention it just in case anyone else here sees something similar happening.

Erica · March 3, 2020, 2:39am

@Shida: should this get a fixed tag, or are you waiting for the iOS app release?