On 01/03/2010 07:07 AM, Ralf Schlatterbeck wrote:
>
> Ralf Schlatterbeck<rsc@runtux.com>  added the comment:
>
> When iterating over *many* results from a database query, we should have
> an iterator interface instead of fetching (possibly thousands of) all
> whole nodes at once. I've experimented with queries that return
> thousands of nodes and simply enlarged roundups node cache. The result
> is only marginally faster due to two things
> - lots of sql queries
> - memory allocation for the cache
>
> only the first will be addressed by the proposed patch...
> Maybe we return an iterator from getnodes?

I agree that an iterator is a good idea. Though the bottleneck we 
observed (and which let me to propose a change in the API) is due to IPC 
overhead caused by many individual queries, instead of few fused ones.