Ticket #138 (closed enhancement: fixed)
Increase findbuf to speed file loading
| Reported by: | gyoung | Owned by: | gyoung |
|---|---|---|---|
| Priority: | major | Milestone: | Release_3.7 |
| Component: | fm/2 base | Version: | 3.07 |
| Keywords: | Cc: | ||
| Blocking: | Blocked By: |
Description
The findbuf had been limited to 128 files in many place. Increasing it appears to be a trade off between loading speed and memory use/temporary poor WPS response. Memory use should be trivial with modern computers and appropriate use of treading should protect responsiveness as well or better than limiting the findbuf. Load times are increased fro 7 min 20 sec to 36 sec by changing findbuf from 128 to 40960 for loading 33,000 files. I see know reason to limit the findbuf (other than expedience of programming) The problem is making end users understand how to optimize it for their usage patterns. I think a default around 10240 would probably be best the time performance per load is fast enough so that any WPS slowing should be trivial while it will load most directories in a single pass maximizing performance. The 36 sec load above was less than one third drive scanning so we need to work on optimizing the container loading code.
