API rate limit problematic


#1

Hi @Shida, when trying to download the contents of all documents of a Quick Dynalist user, the document rate limit becomes quickly a problem. I have a user with 500 documents, but the rate limit is at 50.
Would it be possible to increase the rate limit or create a new request type to query multiple documents at once?

Thanks!


#2

To further reduce the load, it would also help to query all changes since a specific point in time.


#3

Hmm that does sound problematic. We’ll discuss about it and come up with a plan for around Monday.


#4

So what we’re thinking of right now is that 500 documents sound like quite an abnormal use case.
We could look into increasing the limit to something more reasonable like 200, but I still feel like 500 documents is already at the point where it has degraded performance for various things like tag counter, global search, etc.


#5

Does this strengthen the case for an Archive area for documents which, may have a degraded performance, but it’s intentional? The 500 documents use case could be abnormal but my guess is they are accessing only about 5% of them on a regular basis.


#6

I was actually just thinking about that. We did come up with some preliminary thoughts for it:

  • Archived documents won’t be loaded, so smaller memory footprint, faster startup time.
  • Won’t be included in Global Search, Tag Pane, Move item anywhere, Go to any item, etc.
  • Will load on-demand when you open it, and unload when you switch away.

Does this sound reasonable? We could queue it up after our current list of tasks.


#7

Personally, I would be in favour of this. Day to day performance outweighs occasional inconvenience for me. It would be good to hear the views of other users though.


#8

I think we are discussing two different issues here though.

  1. API
  2. web browser frontend

Only because the web browser does not support such a large amount of documents, that that may not hold true for an application that accesses the API. E.g. What if I wanted to query the archived documents that you are suggesting?


#9

I love this. I have a lot of old documents that I occasionally use for reference, but would prefer not to have them loaded and searchable most of the time.


#10

Did you consider raising the rate limit or introducing a query for all changes since a specific point in time?