Collapsing item on mobile not getting synced to web

[Jayden_Navarro]

Steps to reproduce

Collapse an item in the mobile app.

Expected result

The item to be collapsed when viewed from desktop web.

Actual result

The item is not collapsed on desktop. Even if I reopen the Android app, wait 10 seconds (hoping that this eventually gets synced), then refresh the desktop page, it still isn’t collapsed. The mobile and desktop views become out of sync, and I have to manually collapse the item on desktop in order for it to actually reflect on all devices.

I’ve seen this happen a few times before, but over the past week it’s been happening much more frequently than in the past. Given our previous conversations regarding item collapses not following the same “item change/move” sync path I understand that it’s not guaranteed to be synced when “Synced” is displayed, but I’ve kept the mobile app open for much longer after it says “synced” and the change is still not persisted.

Do you know if this is a recent regression or if something else has changed in the way that view mode changes (e.g. collapse) are synced from mobile?

[Jayden_Navarro]

It just happened to me again, and I think what I described before was wrong.

It looks like the change is being synced from mobile, but the desktop page will not sync down the changes, even if I use ctrl+s multiple times. But if I do a full refresh of the webpage it properly shows the item as collapsed.

So it seems that sometimes the desktop page will think it doesn’t have any changes to sync down even if an item is collapsed. When this occurred right now I should point out that my computer was asleep for a while, including when I collapsed the item on mobile, and then it wouldn’t properly sync the collapsed item after coming back online. Note that it did sync actual items, just not the collapsed view (the items were added, but uncollapsed).

I think the code path where view changes are pulled down should be investigated, as it seems this is where the issue is.

[Shida]

Will take a look and try to repro, thanks for the report!

[Shida]

@Jayden_Navarro

So it seems that sometimes the desktop page will think it doesn’t have any changes to sync down even if an item is collapsed.

So what I’m seeing right now is - if there’s no document “content” change, then there is no notification of any document changes, thus the desktop app will not attempt to sync, unless there’s local changes. Since collapsed states are considered optional, per user data, it only gets synced down to the desktop app when actual changes happens to the document.

In my testing, if you collapse an item on the web app, ctrl-s to force sync, go to the desktop app and ctrl-s to force sync on the same document, the collapse comes through. If on the web app, you collapse, then make a change (edit an item or move an item), then the collapse also gets synced down to the desktop app.

[Jayden_Navarro]

That’s interesting. It makes sense that the web app wouldn’t pull down the collapsed view by itself. But what was strange was that it was pulling down new items that were added after I had collapsed an existing item, but it still wasn’t displaying the existing item as collapsed. The bigger issue though was that ctrl+s didn’t even pull down the collapsed view and I had to perform a full page refresh to see it (from a UX perspective, ideally ctrl+s and page refresh would be equivalent).

I’m assuming there are different code paths for a full sync when you refresh (or open) the web app, versus using ctrl+s, for performance reasons. Could it be that the web app thinks it’s synchronized the user data, and so it doesn’t attempt to pull this down even when ctrl+s is manually invoked?

Unfortunately it is pretty rare, so I can’t reliably repro. If I see the issue again where the web app doesn’t reflect the collapsed view, even after ctrl+s, is there some diagnostic information I can collect and send over, perhaps from the console?

[Shida]

I see. That does sound like it shouldn’t happen.

If I understand correctly: When the issue happens, and you open a 2nd tab, you’ll see a different collapsed state, even after you force-sync the first tab. I can’t think of a reason why that would happen; the sync state is controlled by a per-user per-document version number that is incremented any time collapse state changes, and is checked when syncing.

As for debugging info, I’m not quite sure - the ideal case would be if you can identify a guaranteed way to repro. The only alternative I can think of would be to ask you to dump a bunch of network requests which would be helpful but still quite time consuming to comb through (some of it would be compressed data too).

[Jayden_Navarro]

To verify the collapsed state from collapsing an item on mobile (native Android app) was synced to the servers I opened Dynalist in Chrome on my phone and saw the proper state (same phone where I collapsed the item), but I didn’t view it in a 2nd tab on my PC.

I’ll try to come up with a reliable repro Thanks for looking into this!

[Jayden_Navarro]

@Shida This just happened to me again, and I currently have Dynalist open in a browser tab that does not reflect the collapsed view even after repeated ctrl+s. Opening another tab on the same machine shows the item collapsed so it definitely is fully synced to the server.

Since my system is in this inconsistent state I’m trying a couple of experiments. I’ve tried collapsing a different item on mobile, and this one properly reflects on the web, even though the old one remains uncollapsed (implying it has recognized the document version has changed, but it just doesn’t sync down the state correctly for the problem item?). I’ve tried collapsing and uncollapsing the “problem item” multiple times, but it still doesn’t reflect as collapsed on the web.

However…if I add an item below the “problem item” it then reflected the item as collapsed on web!

I should note that in ALL instances where I’ve observed this issue the uncollapsed item is the last item in the list (indented one level under the document root). Specifically the hierarchy in the document looks like this:

- Foo
- Bar
- Baz
  + Problem Item (collapsed, with at least one item underneath)

My theory is that there is some off by one bug that can cause the item collapse state to become inconsistent for the last item in the document, under specific edge cases (not sure what the edge cases are though…). This could explain why placing an item below it would then force the item collapse state to be corrected, as it would no longer be the last item in the document.

[Jayden_Navarro]

@Shida

I can’t think of a reason why that would happen; the sync state is controlled by a per-user per-document version number that is incremented any time collapse state changes, and is checked when syncing.

After the web app recognizes the document version number has changed, how does it figure out what has changed? Is there a log of changes that have occurred in each document version that are replayed to reach the final state, or does it simply update its current state to the latest user data state (for the latest document version)?

[Shida]

I can think of one case where it could fail - If at the time of syncing, the item on the receiving client’s side doesn’t have any child item, then it fails to collapse. Later once the child items are added, it is not collapsed since it’s considered synced.

The only issue is - when syncing a document, the contents are synced first, then the collapsed states are applied. This means that case shouldn’t happen.

In either case, to answer your question, each node id is tagged with a version ID and it’s updated to a new version when a client changes the collapsed state. The sync protocol sends out a list of items for which the collapsed states have changed, and their resulting values (true/false). The version is per-user per-document, and is different from the document version because it’s different for each individual user.

[Jayden_Navarro]

@Shida

I can think of one case where it could fail - If at the time of syncing, the item on the receiving client’s side doesn’t have any child item, then it fails to collapse.

As it still was inconsistent after uncollapsing/recollapsing repeatedly on the mobile side, I don’t think this could be the issue.

@Shida

The sync protocol sends out a list of items for which the collapsed states have changed, and their resulting values (true/false).

Is this server side pushed to each syncing client, or on the mobile client it sends out a list of items where the collapsed states have changed which then are replayed by the server to each syncing client?

If it’s server side, how does it determine which items it should send to the web client, does it internally track the synced node version ID for each connected session to identify which nodes differ and which updates to send? If so, if the client was offline for a while could this client session be cleared from the server side, leading to inconsistency?

If these updates are pushed from the mobile side, is it possible if I closed the app too quickly they weren’t sent out? Are these updates the only way that the server detects collapse state? If so then this argument doesn’t hold water as new clients showed the proper collapse state.

[Shida]

On the server side, we store the latest collapsed/expanded state and a version number that it was changed in. Each client holds its own “last synced version” which is presented to the server on sync. The server selects all collapse/expand states from the document that has version number strictly above the one presented and sends the list back to the client, as well as the current version number.

[Jayden_Navarro]

@Shida

Each client holds its own “last synced version” which is presented to the server on sync.

This “last synced version” is stored per item/node right? And this per-node version is sent to the server for all nodes in the document on every sync? Or are the version numbers of only a subset of nodes sent?

Also, is this version number specifically tied to collapse state changes, or is it a more general version number for the node?

[Shida]

No, only one “last synced version” is stored on the client per document. It’s only used for syncing collapsed changes, and it is different than the versioning/syncing structures used for syncing document content.

The last synced version for collapse state is sent to the server, and the server returns all nodes that have collapsed version numbers higher than the client’s last version, which is the subset of all nodes that had their collapsed state changed since last time the client had synced with the server.

[Jayden_Navarro]

@Shida I still experience this frequently and I’m really curious what’s causing it. Based on the protocol described I can think of a few possible scenarios:

1. There’s some race condition, like you described above, where the collapse state is applied before the item’s children are synced down. In this case, is there a console error message when a collapse is applied to an empty item, or is it silently ignored? You noted that this shouldn’t happen, but maybe there’s some small edge case here related to Chrome/Windows backgrounding a tab (I’ve only seen this bug when resuming my PC after being asleep for a while).

2. The client side last synced version is somehow bumped without getting the collapse state changes.

3. There’s some client side bug that has to do with it being the last item in the list combined with some race condition causing its collapse state to be out of sync.

The first two scenarios don’t line up with the behavior I observed where collapsing/recollapsing the “problem item” doesn’t sync to the broken client, as those should be properly reflected on update even if we missed the initial update.

I’m going to leave the network tab open for a few days and hopefully this repros, then I’ll be able to see if there are collapse_changes in the update bundle_binary which should determine whether it’s a server side or client side issue.

[Jayden_Navarro]

@shida I finally have a repeatable repro! It’s a fun little edge case that I backed into from thinking about the protocol (thanks for going into the details there, it wouldn’t have been possible without that!).

Have Dynalist open in two tabs, A and B. Open dev console in tab A.

1. Create item “Foo” in A with child item “Bar” and collapse “Foo”. Make sure it syncs to B.
2. In the “network” tab of dev console on A change from “Online” to “Offline”.
3. In A, zoom into the “Foo” item and delete “Bar”. The item is now no longer collapsed in A since it has no children.
4. In B, zoom into the “Foo” item and add a new item “Baz”. It should remain collapsed and now have two children.
5. Change A back to “Online” and sync changes (e.g. Ctrl-s).

Tab A should now be in an inconsistent state where it’s uncollapsed with child “Baz”, as opposed to B and and other clients which will have “Baz” and be collapsed. No matter how many times you sync or perform new changes tab A will remain in this inconsistent state.

The root cause is that an offline tab can move into an uncollapsed state while other clients (and thus the server) never become uncollapsed. This means there are no collapse_changes to sync down to the uncollapsed client since from the perspective of the server the item never became uncollapsed (all other clients always had at least one child).

When I’ve experienced this problem I don’t think it was exactly like this repro, but it likely has the same root cause, so if the fix is fundamental enough it should hopefully resolve it (as opposed to just patching this one edge case). I still need to figure out what the exact case is when I’ve hit this in the past…

One robust solution is to always provide syncing clients the collapse state of new items’ parents, which should hopefully resolve this issue.

EDIT: Thinking about this a bit more, it’s likely that when I’ve experienced this I’ve quickly deleted the last item in a collapsed child and then put my computer to sleep before the collapsed change is synced, but after the item deletion synced (was able to repro locally that that can occur). I think even switching tabs is sometimes sufficient to prevent the collapsed state syncing from occurring.

[Shida]

Wow nice debugging. I think that explains the problem well and is probably what’s happening here. I’ll see how we can get this fixed!

[Jayden_Navarro]

@shida I just had this happen again and I’m positive that the collapse event (on mobile) happened after the diff version stored by the web client, so it’s a different issue than the one reproed above (that earlier one only applies if from the server side it doesn’t think there were any collapse events).

I have two theories:

1. The client collapse event is somehow not bumping the collapse version number. Maybe from the client’s perspective the item collapse state hasn’t changed since the last sync (the item deletion and addition happened from other clients and the API, so before syncing down it’s “collapsed”), so when I collapse the item it doesn’t increment the collapse document version even though it does sync the collapse state back to the server (verified by seeing the item collapsed in a new client). Is this possible? When it decides to increment the version number does it diff to the last sync state and only increment if they differ, or is it a simple “did any collapse events happen”? If it’s the former, this could be the issue. More generally, is incrementing the version done client side or server side? I think this only makes sense if it’s done client side.

2. There’s a race on stale clients update. Is something like the following possible?
   a. Client sees update that removes item, making it uncollapsed.
   b. Client thinks it’s now up to date since it processed an update and pulls down collapse diffs which includes the collapse. Since item has no children in the client it’s a no-op. The document version number on the client is updated, so this collapse diff is essentially lost.
   c. Client sees update that adds item. Since the collapse state was already applied (and ignored) the item is now improperly uncollapsed.

Before the client pulls down the collapse diffs, does it ensure it’s fully synced to the latest version and not just processed partial updates? If so, then this second theory doesn’t make sense.

[Jayden_Navarro]

I was able to repro theory #2 (update race)!

Here’s what I see in the network tab (some contents removed for brevity/privacy):

(1) bundle_binary

"user_version":1931
"added":[]
"removed":["T8FQfQlQbCi06qUBDmIkzRLd"]
{"id":"T8FQfQlQbCi06qUBDmIkzRLd","parent_old":"y8JtkASdlDsmcq-4Odz0n-UJ","index_old":0,"index_new":0}

(2) bundle_binary

"user_version":1933
"collapse_changes":[{"node_id":"y8JtkASdlDsmcq-4Odz0n-UJ","collapsed":true}]

(3) bundle_binary

"user_version":1933
"added":["gFtYTJrJ2Zx7USbnzLcCYbvM"]
"removed":[]
{"id":"gFtYTJrJ2Zx7USbnzLcCYbvM","parent_old":"","parent_new":"y8JtkASdlDsmcq-4Odz0n-UJ","index_old":0,"index_new":1}

So it seems that we aren’t guaranteed to be fully synced before we fetch the collapse changes.

From the server’s perspective the collapse event must occur on or after the new item is added, so this just appears to be a client syncing issue (eagerly fetching collapse changes before all changes synced down).

I’m curious if the same fix can be used for both of these bugs. If whenever an item is added it includes the collapse state of its parent in the meta that may work (unless add/remove updates can be processed out of order). A fix that only ensures we are fully synced before grabbing collapse changes will not fix the earlier bug.

[Jayden_Navarro]

I’ve actually now reproed theory #1 as well (client doesn’t properly bump version number). Although I think this is essentially the same problem as the very first repo.

Repro steps:

1. Open Dynalist in a tab (A).
2. Open Dynalist in a new tab (B). Create an item with a child and collapse it. Ensure this is fully synced to tab A (including being collapsed).
3. In tab B, verify the current document version number by inspecting user_version for the bundle_binary that is pulled down after hitting Ctrl-s in the network tab.
4. In tab B, zoom into the collapsed item and delete the child, then immediately create a new child item and zoom out and collapse the item.
5. Don’t Ctrl-s, just wait for the page to naturally sync.
6. Open Dynalist in a new tab ( C ) and ensure that the item is collapsed, meaning that the server thinks the item is collapsed.
7. In tab A, wait for it to naturally sync. If the item is uncollapsed, the issue reproed. Otherwise repeat from the top until this occurs.
8. Inspect the user_version in tab A (or any tab for that matter). It is likely the same as in step 3, meaning the version wasn’t bumped even after the item was collapsed.

Based on this, I assume the version bump is done on the server side, and since all it saw was that the item was collapsed before and after the client synced it didn’t think there was a difference.