go的并发任务如何优雅的实现错误终止

news/2024/10/22 14:27:17/

errgroup使用案例

在Go语言中，并发任务通常通过goroutine来实现，而错误处理和任务终止的优雅性则依赖于适当的同步机制和错误传播策略。

场景:

管理一个任务的一组子任务，每个子任务一个协程
每个子任务必须保证都成功，一个出现失败应当立马停止所有子任务
想知道子任务失败的原因

具体案例:

场景:计算一个目录下所有文件的 MD5 值，任何一个文件都需要正确计算，一旦一个任务出现错误立即返回。

分析:多个任务可以并发执行，可以使用WaitGroup管理，但是需要返回错误，可以使用errgroup。

示例代码:

package mainimport ("context""crypto/md5""fmt""log""os""path/filepath""golang.org/x/sync/errgroup"
)func main() {m, err := MD5All(context.Background(), ".")if err != nil {log.Fatal(err)}for k, sum := range m {fmt.Printf("%s:\t%x\n", k, sum)}
}type result struct {path stringsum  [md5.Size]byte
}// MD5All reads all the files in the file tree rooted at root and returns a map
// from file path to the MD5 sum of the file's contents. If the directory walk
// fails or any read operation fails, MD5All returns an error.
func MD5All(ctx context.Context, root string) (map[string][md5.Size]byte, error) {// ctx is canceled when g.Wait() returns. When this version of MD5All returns// - even in case of error! - we know that all of the goroutines have finished// and the memory they were using can be garbage-collected.g, ctx := errgroup.WithContext(ctx)paths := make(chan string)g.Go(func() error {defer close(paths)// Walk函数第二个参数是一个回调函数return filepath.Walk(root, func(path string, info os.FileInfo, err error) error {if err != nil {return err}if !info.Mode().IsRegular() {return nil}select {case paths <- path:case <-ctx.Done():return ctx.Err()}return nil})})// Start a fixed number of goroutines to read and digest files.c := make(chan result)const numDigesters = 20for i := 0; i < numDigesters; i++ {g.Go(func() error {for path := range paths {data, err := os.ReadFile(path)if err != nil {return err}select {case c <- result{path, md5.Sum(data)}:case <-ctx.Done():return ctx.Err()}}return nil})}// g所有任务执行完毕关闭cgo func() {g.Wait()close(c)}()m := make(map[string][md5.Size]byte)for r := range c {m[r.path] = r.sum}// Check whether any of the goroutines failed. Since g is accumulating the// errors, we don't need to send them (or check for them) in the individual// results sent on the channel.if err := g.Wait(); err != nil {return nil, err}return m, nil
}