Repository navigation
proposal: spec: reduce error handling boilerplate using ? #71203
Description
Proposal Details
Background
As discussed in the introduction to an earlier, declined, proposal, Go programs have a lot of error checking code. In surveys error handling is listed as the biggest specific challenge people face using Go today.
There have been many proposals to address this, summarized in a meta-proposal.
This is yet another such proposal. This proposal has some similarities to onerr return, add "or err: statement" after function calls for error handling, and use ? simplify handling of multiple-return-values. It is in some ways a reworking of simplify error handling with || err suffix. There are probably a number of other proposals that fed into this one even if I can't remember them now.
The goal of this proposal is to introduce a new syntax that reduces the amount of code required to check errors in the normal case, without obscuring flow of control.
New syntax
This section is an informal description of the proposal, with examples. A more precise description appears below.
I propose permitting statements of the form
r, err := SomeFunction()
if err != nil {
return fmt.Errorf("something failed: %v", err)
}to be written as
r := SomeFunction() ? {
return fmt.Errorf("something failed: %v", err)
}The ? absorbs the error result of the function. It introduces a new block, which is executed if the error result is not nil. Within the new block, the identifier err refers to the absorbed error result.
Similarly, statements of the form
if err := SomeFunction2(); err != nil {
return fmt.Errorf("something else failed: %v", err)
}may be written as
SomeFunction2() ? {
return fmt.Errorf("something else failed: %v", err)
}Further, I propose that the block following the ? is optional. If the block is omitted, it acts as though there were a block that simply returns the error from the function. For example, code like
if err := SomeFunction2(); err != nil {
return err
}may in many cases be written as
SomeFunction2() ?Formal proposal
This section presents the formal proposal.
An assignment or expression statement may be followed by a question mark (?). The question mark is a new syntactic element, the first permitted use of ? in Go outside of string and character constants. The ? causes conditional execution similar to an if statement. A ? at the end of a line causes a semicolon to be automatically inserted after it.
A ? uses a value as described below, referred to here as the qvalue.
For a ? after an assignment statement, the qvalue is the last of the values produced by the right hand side of the assignment. The number of variables on the left hand side of the assignment must be one less than the number of values produced by the right hand side (the right hand side values may come from a function call as usual). It is not valid to use a ? if there is only one value on the right hand side of the assignment.
For a ? after an expression statement the qvalue is the last of the values of the expression. It is not valid to use a ? after an expression statement that has no values.
The qvalue must be of interface type and must implement the predeclared type error; that is, it must have the method Error() string. In most cases it will simply be of type error.
A ? is optionally followed by a block. The block may be omitted if the statement using ? appears in the body of a function, and the enclosing function has at least one result, and the qvalue is assignable to the last result (this means that the type of the last result must implement the predeclared type error, and will often simply be error).
Execution of the ? depends on the qvalue. If the qvalue is nil, execution proceeds as normal, skipping over the block if there is one.
If the ? is not followed by a block, and the qvalue is not nil, then the function returns immediately. The qvalue is assigned to the final result. If the other results (if any) are named, they retain their current values. If they are not named, they are set to the zero value of their type. The results are then returned. Deferred functions are executed as usual.
If the ? is followed by a block, and the qvalue is not nil, then the block is executed. Within the block a new variable err is implicitly declared, possibly shadowing other variables named err. The value and type of this err variable will be those of the qvalue.
That completes the proposal.
Examples
func Run() error {
Start() ? // returns error from Start if not nil
Wait() ? // returns error from Wait if not nil
return nil
}func CopyFile(src, dst string) error {
r := os.Open(src) ? {
return fmt.Errorf("copy %s %s: %v", src, dst, err)
}
defer r.Close()
w := os.Create(dst) ? {
return fmt.Errorf("copy %s %s: %v", src, dst, err)
}
io.Copy(w, r) ? {
w.Close()
os.Remove(dst)
return fmt.Errorf("copy %s %s: %v", src, dst, err)
}
w.Close() ? {
os.Remove(dst)
return fmt.Errorf("copy %s %s: %v", src, dst, err)
}
}func MustOpen(n string) *os.File {
f := os.Open(n) ? {
panic(err)
}
return f
}func TestFileData(t *testing.T) {
f := os.Open("testfile") ? {
t.Fatal(err)
}
...
}func CreateIfNotExist(name string) error {
f, err := os.OpenFile(name, os.O_EXCL|os.O_WRONLY, 0o666)
if errors.Is(err, fs.ErrExist) {
return nil
}
err ? // returns err if it is not nil
// write to f ...
}Discussion
This new syntax is partly inspired by Rust's question mark operator, though Rust permits ? to appear in the middle of an expression and does not support the optional block. Also, I am suggesting that gofmt will enforce a space before the ?, which doesn't seem to be how Rust is normally written.
Absorbing the error returned by a function, and optionally returning automatically if the error is not nil, is similar to the earlier try proposal. However, it differs in that
?is an explicit syntactic element, not a call to a predeclared function, and?may only appear at the end of the statement, not in the middle of an expression.
Declaring the err variable
As discussed above, when a block follows the ? it implicitly declares a new err variable. There are no other cases in Go where we implicitly declare a new variable in a scope. Despite that fact, I believe this is the right compromise to maintain readability while reducing boilerplate.
A common suggestion among early readers of this proposal is to declare the variable explicitly, for example by writing
r := SomeFunction() ? err {
return fmt.Errorf("something failed: %v", err)
}In practice, though, the variable would essentially always be simply err. This would just become additional boilerplate. Since the main goal of this proposal is to reduce boilerplate, I believe that we should try our best to do just that, and introduce err in the scope rather than requiring people to declare it explicitly.
If the implicit declaration of err seems too problematic, another approach would be to introduce a new predeclared name. The name err would not be appropriate here, as that would be too often shadowed in existing code. However, a name like errval or erv would work. Within a ? optional block, this name would evaluate to the qvalue. Outside of a ? optional block, referring to the name would be a compilation error. This would have some similarities to the predeclared name iota, which is only valid within a const declaration.
A third approach would be for errval or erv to be a predeclared function that returns the qvalue.
Supporting other types
As discussed above the qvalue must be an interface type that implements error. It would be possible to support other interface types. However, the ? operator, and especially the implicitly declared err variable, is specifically for error handling. Supporting other types confuses that focus. Using ? with non-error types would also be confusing for the reader. Keeping a focus on just handling errors seems best.
It would also be possible to support non-interface types that implement error, such as the standard library type *os.SyscallError. However, returning a value of that type from a function that returns error would mean that the function always returns a non-nil error value, as discussed in the FAQ. Using different rules for ? would make an already-confusing case even more confusing.
Effects on standard library
I applied a simple rewriter to the standard library to introduce uses of ? where feasible. Here are some examples of new code:
archive/tar/common.go:
h.Gname = iface.Gname() ?
h.Uname = iface.Uname() ?archive/tar/writer_test.go:
// Test that we can get a long name back out of the archive.
reader := NewReader(&buf)
hdr = reader.Next() ? {
t.Fatal(err)
}
if hdr.Name != longName {
...archive/zip/reader.go:
var buf [directoryHeaderLen]byte
io.ReadFull(r, buf[:]) ?archive/zip/reader.go:
p, err := findDirectory64End(r, directoryEndOffset)
if err == nil && p >= 0 {
directoryEndOffset = p
err = readDirectory64End(r, p, d)
}
err ?archive/zip/reader_test.go:
b := hex.DecodeString(s) ? {
panic(err)
}cmd/cgo/godefs.go:
func gofmt(n interface{}) string {
gofmtBuf.Reset()
printer.Fprint(&gofmtBuf, fset, n) ? {
return "<" + err.Error() + ">"
}
return gofmtBuf.String()
}cmd/cgo/out.go:
fexp := creat(*exportHeader)
fgcch := os.Open(*objDir + "_cgo_export.h") ? {
fatalf("%s", err)
}
defer fgcch.Close()
io.Copy(fexp, fgcch) ? {
fatalf("%s", err)
}
fexp.Close() ? {
fatalf("%s", err)
}os/exec/exec.go:
func (c *Cmd) Run() error {
c.Start() ?
return c.Wait()
}The conversion tool found 544,294 statements in the standard library. It was able to convert 8820 of them to use ?. In all, 1.6% of all statements were changed. 1380 statements, or 0.25% of the total, were changed to use a ? with no optional block.
In other words, adopting this change across the ecosystem would touch an enormous number of lines of existing Go code. Of course, changing existing code could happen over time, or be skipped entirely, as current code would continue to work just fine.
Pros and cons
Pros
Advantage 1: Rewriting
r, err := SomeFunction()
if err != nil {
return fmt.Errorf("something failed: %v", err)
}to
r := SomeFunction() ? {
return fmt.Errorf("something failed: %v", err)
}reduces the error handling boilerplate from 9 tokens to 5, 24 non-whitespace characters to 12, and 3 boilerplate lines to 2.
Rewriting
r, err := SomeFunction()
if err != nil {
return err
}to
r := SomeFunction() ?reduces boilerplate from 9 tokens to 1, 24 non-whitespace characters to 1, and 3 boilerplate lines to 0.
Advantage 2: This change turns the main code flow into a straight line, with no intrusive if err != nil statements and no obscuring if v, err = F() { … } statements. All error handling either disappears or is indented into a block.
Advantage 3: That said, when a block is used the } remains on a line by itself, unindented, as a signal that something is happening. (I'm also listing this as a disadvantage, below.)
Advantage 4: Unlike the try proposal and some other error handling proposals, there is no hidden control flow. The control flow is called out by an explicit ? operator that can't be in the middle of an expression, though admittedly the operator is small and perhaps easy to miss at the end of the line. I hope the blank before it will make it more visible.
Advantage 5: To some extent this reduces a couple of common error handling patterns to just one, as there is no need to decide between
var (
v string
err error
)
if v, err = F(); err != nil {
...
}and
v, err := F()
if err != nil {
...
}Instead people can consistently write
v := F() ? {
...
}Cons
Disadvantage 1: This is unlike existing languages, which may make it harder for novices to understand. As noted above it is similar to the Rust ? operator, but still different. However, it may not be too bad: Todd Kulesza did a user study and discovered that people unfamiliar with the syntax were able to see that the code had to do with error handling.
Disadvantage 2: The shadowing of any existing err variable may be confusing. Here is an example from the standard library where the ? operator can not be easily used:
fmt/scan.go:
for n = 1; !utf8.FullRune(r.buf[:n]); n++ {
r.buf[n], err = r.readByte()
if err != nil {
if err == io.EOF {
err = nil // must change outer err
break
}
return
}
}
// code that later returns errIn this example the assignment err = nil has to change the err variable that exists outside of the for loop. Using the ? operator would introduce a new err variable shadowing the outer one. (In this example using the ? operator would cause a compiler error, because the assignment err = nil would set a variable that is never used.)
Disadvantage 3: When using a block, the } remains on a line itself, taking up space as pure boilerplate. (I'm also listing this as an advantage, above.)
Disadvantage 4: No other block in Go is optional. The semicolon insertion rule, and the fact that a block is permitted where a statement is permitted, means that inserting or removing a newline can convert one valid Go program into another. As far as I know, that is not true today.
For example, these two functions would both be valid and have different meanings, although the only difference is whitespace.
func F1() error {
err := G1()
log.Print(err)
G2() ?
{
log.Print(err)
}
return nil
}
func F2() error {
err := G1()
log.Print(err)
G2() ? {
log.Print(err)
}
return nil
}Disadvantage 5: For an expression statement that just calls a function that returns an error, it's easy to accidentally forget the ? and write F() rather than F() ?. Of course it's already easy to forget to check the error result, but once people become accustomed to this proposal it may be easy to overlook the missing ? when reading code.
Disadvantage 6: This proposal has no support for chaining function calls, as in F().G().H(), where F and G also have an error result.
Disadvantage 7: This proposal makes it easier to simply return an error than to annotate the error, by using a plain ? with no block. This may encourage programmers to skip error annotations even when they are desirable.
Disadvantage 8: We really only get one chance to change error handling syntax in Go. We aren't going to make a second change that touches 1.5% of the lines of existing Go code. Is this proposal the best that we can do?
Disadvantage 9: We don't actually have to make any changes to error handling. Although it is a common complaint about Go, it's clear that Go is usable today. Perhaps no change is better than this change. Perhaps no change is better than any change.
Transition
If we adopt this proposal, we should provide tools that can be used to automatically rewrite existing Go code into the new syntax. Not everyone will want to run such a tool, but many people will. Using such a tool will encourage Go code to continue to look the same in different projects, rather than taking different approaches. This tool can't be gofmt, as correct handling requires type checking which gofmt does not do. It could be an updated version of go fix. See also modernizers.
We will have to update the go/ast package to support the use of ?, and we will have to update all packages that use go/ast to support the new syntax. That is a lot of packages.
We will also have to update the introductory documentation and the tour. And, of course, existing Go books will be out of date and will need updating by their authors. The change to the language and compiler is the easiest part of the work.
Possible extensions
These are some possible extensions to the above proposal. These are not part of this proposal, but suggest ways that the language could be developed if this proposal seems useful.
Permit return …, err
This proposal works well with proposal #21182 which permits return …, err to return all zero values other than a final error value. For example,
// OpenOrCreate opens a file, creating it if necessary, and returns the
// file's modification time.
func OpenOrCreate(s string) (*os.File, time.Time, error) {
f := os.OpenFile(s, os.O_CREATE|os.O_RDWR, 0o666) ?
fi := f.Stat() ? {
f.Close()
return ..., err // instead of "return nil, time.Time{}, err"
}
return f, fi.ModTime(), nil
}That said, if we know that err != nil, then writing err ? is the same as writing return …, err (ignoring the details of named result parameters). We could use err ? in the above example. It would require some analysis to see how often return …, err would be useful when we don't know whether or not err is nil.
Permit using ? with no block outside of a function
We could permit using ? with no block outside of a function, by having it implicitly call panic(err). For example:
// fileContents is a package-scope variable that will
// be set to the contents of the file.
// If os.ReadFile fails, the program will panic
// at initialization time.
var fileContents = os.ReadFile("data") ?I don't think this comes up often enough to be worth doing. Note that this proposal permits
var fileContents = os.ReadFile("data") ? {
panic(fmt.Sprintf("could not read data file: %v", err))
}Permit using ? with no block in a test
We could permit using ? with no block in a test function, by having it implicitly call t.Fatal(err). For example:
func TestSomething(t *testing.T) {
f := os.ReadFile("data") ?
}I'm not fond of this because it means that we have to somehow recognize test functions in the language. This proposal does already permit
func TestSomething(t *testing.T) {
f := os.ReadFile("data") ? {
t.Fatal(err)
}
}or, for that matter,
func TestSomething(t *testing.T) {
testSomething(t) ? {
t.Error(err)
}
}
func testSomething(t *testing.T) error {
f := os.ReadFile("data") ?
... // other statements that use ? to handle errors
}Let gofmt retain single line
We could let gofmt retain a ? block on a single line, as in
r := os.Open(src) ? { return fmt.Errorf("copy %s %s: %v", src, dst, erv) }This would reduce the error handling boilerplate by 2 newlines, and keep all error handling indented.
I am not in favor of this myself but I know that people will suggest it.
@gabyhelp's overview of this issue: #71203 (comment)
Activity
jimmyfrasche commentedon Jan 9, 2025
If, instead of a block, it took an optional value of type
func(error) errorthat would allow reusing handlers and with #21498 then inline handlers would just bewhich is still quite short
kortschak commentedon Jan 9, 2025
Would
OtherFunction(SomeFunction() ?)be accepted wherefunc SomeFunction() (T, error)andfunc OtherFunction(v T)? Alsofor _, v := range (SomeFunction() ?) { … }wherefunc SomeFunction() ([]T, error). (both in a returning context)dominikh commentedon Jan 9, 2025
ianlancetaylor commentedon Jan 9, 2025
@kortschak
No. The
?is only permitted at the end of an assignment statement or an expression statement.kortschak commentedon Jan 9, 2025
@ianlancetaylor Is there a rationale for that that I missed?
Merovius commentedon Jan 9, 2025
Personally, I don't feel like the first version (with the block) pays for itself. The visual difference between
x, err := f(); if err != nil { … }andx := f() ? { … }just seems pretty small to me, especially with all the indentation. The second version (without the block) on the other hand, is a significant saving and I see the value in that. So I would only add the second version, if it where up to me.Also, a nit: I find the formatting with the extra space between the rvalue and the
?(i.e.x := f() ?vs.x := f()?) pretty ugly, though I understand that it's probably intentional to make the?stand out more.ianlancetaylor commentedon Jan 9, 2025
@jimmyfrasche
I think that's a little confusing because that
returnstatement is presumably not returning from the short function literal, it's returning from the function that is calling that function literal.ianlancetaylor commentedon Jan 9, 2025
No, I didn't write anything about that topic. The rationale is that I don't think a return should occur in the middle of an expression. I think that is harder to understand, especially as we heard toward cases like
A(B() ?, C() ?). I think that was one of the difficulties with the try proposal.kortschak commentedon Jan 9, 2025
I think that's fair for the calls, but what about the
rangecase (or thinking about it now,switchwith.(type); there is no ambiguity there as it's essentially just sugar for the assignment and then loop.gabyhelp commentedon Jan 9, 2025
Related Issues
(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)
gophun commentedon Jan 9, 2025
Introducing a special syntax solely for the undecorated (in many cases discouraged) error return might make it appear disproportionately more appealing than the preferred, decorated method of handling errors.
231 remaining items
ianlancetaylor commentedon Jan 27, 2025
@eihigh
That is a good question and I don't have a definite answer. My inclination is to say that it's OK to not reference
err, because there is in a sense an implicit test oferr != niljust to get to this code. That is, we are not ignoring the error value even if we don't use it. But I could be convinced otherwise.ianlancetaylor commentedon Jan 27, 2025
@ConradIrwin
Noted. Interesting example. Thanks.
ianlancetaylor commentedon Jan 27, 2025
@FiloSottile
Thanks for the detailed look.
It's a tooling issue due to generics. The tool doesn't recognize that
nilis the zero value for the type parameter.In the current proposal that would discard the error returned by
barand returnerrors.New("1"). Interesting example.ianlancetaylor commentedon Jan 27, 2025
@ncruces
This proposal does not permit
The
deferstatement requires a function call, and whilef.Close()is a function call,f.Close() ?is not.EDIT: the rest of this was wrong, see #71203 (comment)
This proposal would permitdefer func() { f.Close() ? }()which would have an effect roughly similar to the following when used in a function with a namederrresult:defer func() {if err1 := f.Close(); err1 != nil {
err = err1
}
}()
DmitriyMV commentedon Jan 27, 2025
@ianlancetaylor having
?always diverge control flow would make it actually quite similar to the Rust.map_err(...)?which will ease the movement back and forth between two languages. It could be also explained to the new users that they can think of the block after?as an implicit lambda.What happens if the function result error is already assigned by the code in the function body?
ncruces commentedon Jan 27, 2025
@ianlancetaylor
I'm starting to get the feeling that I either don't understand the language or the proposal. From the proposal:
The deferred function, on which
?is used, has no result. So how can a blockless?be used there?Is the key here the word “enclosing”? If so, how does that square with multiple levels of nesting? Does the
?always immediatelyreturnfrom the outermost function call? If so that's a significant control flow change, as there is no other way in the language to do that; “enclosing” carries a lot of weight in that sentence.Also, as I pointed out, that rough desugaring is not what people typically want. We make spaghetti out of errors in defer because we only want the deferred error when the function is not already returning an error.
I would also again ask for clarification of the following case, because although
errseems to match the EBNF of “expression statements” it does not match the plain english description of them:err ? { … }An isolated
errcannot usually be used as a statement, even if the check that trips this is evaluated later by the compiler (not the parser, but some later stage that checks for statements with no effect).Sorry if I'm out of my depth.
mitsuhiko commentedon Jan 27, 2025
It modifies the "err" variable on the outer function in the same way as a defer could manipulate a named return:
jfgiorgi commentedon Jan 27, 2025
I don't like the "explicit err declaration" part. it's confusing and not natural (or "err" must be a token/reserved word of the langage like "self" in some OO language.)
I'd rather have a new token/operator used in the language to "decorate" functions to indicate they return an extra unamed Error value.
For instance the token ! (or whatever is easier to implement in the lexer/parser)
Then use that Error value in this "?" proposal.
(I haven't read all proposals and comments, so sorry if this has already been proposed or it's out of subject).
"!" can be used to access the error in the ? scope block:
more details
declaring:
function with no return value(s) and bubbling up error:
explicitly getting the error outside the ? scope:
multiple & nested errors:
ps: to avoid too many '!' in code, it can be omitted when calling a function (granted type checking is ok).
ncruces commentedon Jan 27, 2025
func() { i = 1; }()here is just an anonymous function. It doesn't return an error. If it did, the returned error would be ignored bydefer(which is the entire issue withdefer f.Close().Also, a return statement statement inside an anonymous function returns from the anonymous function. There's no way to return from
xin your example from inside an anonymous function (deferred or not).The specification for the proposal does not special case deferred functions at all. It doesn't even mention anonymous functions/literals. So I fail to see how a
?inside a (possibly deferred) anonymous function, with or without a block, can be used to return from the outer function.If there's a goal here to simplify error handling within deferred functions, that's great actually. Because that is a pain point, and something many (myself very much included) don't handle correctly in many, many (most?) cases, because every solution is a contrived mess. I put this at the level of fixing
forloops, not at theif err != nil {all the time is slightly annoying.ianlancetaylor commentedon Jan 27, 2025
@ncruces My apologies, you're quite right, and I was wrong.
The proposal does not permit either
or
It does permit
but that is useless—it's basically the same as
defer f.Close().ianlancetaylor commentedon Jan 27, 2025
Thanks for all the feedback so far. It's very helpful.
I've opened a discussion for this proposal: #71460 (I should probably have started this as a discussion). I've started some initial comment threads for emoji voting. Let's move any further comments over to that discussion. I'm going to close this issue.
sean9999 commentedon Feb 4, 2025