README
¶
pwalk: parallel implementation of filepath.Walk
This is a wrapper for filepath.Walk which may speed it up by calling multiple callback functions (WalkFunc) in parallel, utilizing goroutines.
By default, it utilizes 2*runtime.NumCPU() goroutines for callbacks. This can be changed by using WalkN function which has the additional parameter, specifying the number of goroutines (concurrency).
Caveats
Please note the following limitations of this code:
-
Unlike filepath.Walk, the order of calls is non-deterministic;
-
Only primitive error handling is supported:
-
filepath.SkipDir is not supported;
-
no errors are ever passed to WalkFunc;
-
once any error is returned from any WalkFunc instance, no more new calls to WalkFunc are made, and the error is returned to the caller of Walk;
-
if more than one walkFunc instance will return an error, only one of such errors will be propagated and returned by Walk, others will be silently discarded.
-
Documentation
For the official documentation, see https://pkg.go.dev/github.com/opencontainers/selinux/pkg/pwalk?tab=doc
Benchmarks
For a WalkFunc that consists solely of the return statement, this implementation is about 10% slower than the standard library's filepath.Walk.
Otherwise (if a WalkFunc is doing something) this is usually faster, except when the WalkN(..., 1) is used.
Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Walk ¶
Walk is a wrapper for filepath.Walk which can call multiple walkFn in parallel, allowing to handle each item concurrently. A maximum of twice the runtime.NumCPU() walkFn will be called at any one time. If you want to change the maximum, use WalkN instead.
The order of calls is non-deterministic.
Note that this implementation only supports primitive error handling:
- no errors are ever passed to WalkFn;
- once a walkFn returns any error, all further processing stops and the error is returned to the caller of Walk;
- filepath.SkipDir is not supported;
- if more than one walkFn instance will return an error, only one of such errors will be propagated and returned by Walk, others will be silently discarded.
Source Files