Hi @Shida, when trying to download the contents of all documents of a Quick Dynalist user, the document rate limit becomes quickly a problem. I have a user with 500 documents, but the rate limit is at 50.
Would it be possible to increase the rate limit or create a new request type to query multiple documents at once?
Thanks!
To further reduce the load, it would also help to query all changes since a specific point in time.
Hmm that does sound problematic. We’ll discuss about it and come up with a plan for around Monday.
So what we’re thinking of right now is that 500 documents sound like quite an abnormal use case.
We could look into increasing the limit to something more reasonable like 200, but I still feel like 500 documents is already at the point where it has degraded performance for various things like tag counter, global search, etc.
Does this strengthen the case for an Archive area for documents which, may have a degraded performance, but it’s intentional? The 500 documents use case could be abnormal but my guess is they are accessing only about 5% of them on a regular basis.
I was actually just thinking about that. We did come up with some preliminary thoughts for it:
Does this sound reasonable? We could queue it up after our current list of tasks.
Personally, I would be in favour of this. Day to day performance outweighs occasional inconvenience for me. It would be good to hear the views of other users though.
I think we are discussing two different issues here though.
Only because the web browser does not support such a large amount of documents, that that may not hold true for an application that accesses the API. E.g. What if I wanted to query the archived documents that you are suggesting?
I love this. I have a lot of old documents that I occasionally use for reference, but would prefer not to have them loaded and searchable most of the time.
Did you consider raising the rate limit or introducing a query for all changes since a specific point in time?
@Shida querying all modified data since a given timestamp could also heavily reduce the load on your servers and the waiting time for quick dynalist users.
That sounds reasonable as well. We internally use a version number to account for changes so maybe that could be used instead.
We’ll start working on API improvements after we finish this batch of bugs caused by iOS 12.
A version number would work equally well. So you would query all changes since a specific version number and would get the current version number plus changes back?
That’s the plan.
Have you increased the rate limit in the meanwhile?
Furthermore, are the api changes work in progress?
Thanks!
Apologies for the late reply! The rate increase is already in place, but hasn’t been deployed to the server yet, which will happen sometime this weekend.
As for the API changes, it’s still backlogged by a rare sync issue we’re currently investigating (disappearing documents, documents that can’t be deleted, etc), but I’ve been going around to clean things off the backlog from time to time so tickets don’t go stale for too long. Sorry about that!
An update has been posted which hopefully helps this issue a lot:
@Shida can I pay you more $ for more requests?
Woah, we’re flattered!
Really curious what you’re trying to build there 