Black Lives Matter. Support the Equal Justice Initiative.

The Go Blog

Keeping Your Modules Compatible

Jean de Klerk and Jonathan Amsterdam
7 July 2020

Introduction

This post is part 5 in a series.

Your modules will evolve over time as you add new features, change behaviors, and reconsider parts of the module's public surface. As discussed in Go Modules: v2 and Beyond, breaking changes to a v1+ module must happen as part of a major version bump (or by adopting a new module path).

However, releasing a new major version is hard on your users. They have to find the new version, learn a new API, and change their code. And some users may never update, meaning you have to maintain two versions for your code forever. So it is usually better to change your existing package in a compatible way.

In this post, we'll explore some techniques for introducing non-breaking changes. The common theme is: add, don’t change or remove. We’ll also talk about how to design your API for compatibility from the outset.

Adding to a function

Often, breaking changes come in the form of new arguments to a function. We’ll describe some ways to deal with this sort of change, but first let’s look at a technique that doesn’t work.

When adding new arguments with sensible defaults, it’s tempting to add them as a variadic parameter. To extend the function

func Run(name string)

with an additional size argument which defaults to zero, one might propose

func Run(name string, size ...int)

on the grounds that all existing call sites will continue to work. While that is true, other uses of Run could break, like this one:

package mypkg
var runner func(string) = yourpkg.Run

The original Run function works here because its type is func(string), but the new Run function’s type is func(string, ...int), so the assignment fails at compile time.

This example illustrates that call compatibility is not enough for backward compatibility. There is, in fact, no backward-compatible change you can make to a function’s signature.

Instead of changing a function’s signature, add a new function. As an example, after the context package was introduced, it became common practice to pass a context.Context as the first argument to a function. However, stable APIs could not change an exported function to accept a context.Context because it would break all uses of that function.

Instead, new functions were added. For example, the database/sql package's Query method’s signature was (and still is)

func (db *DB) Query(query string, args ...interface{}) (*Rows, error)

When the context package was created, the Go team added a new method to database/sql:

func (db *DB) QueryContext(ctx context.Context, query string, args ...interface{}) (*Rows, error)

To avoid copying code, the old method calls the new one:

func (db *DB) Query(query string, args ...interface{}) (*Rows, error) {
    return db.QueryContext(context.Background(), query, args...)
}

Adding a method allows users to migrate to the new API at their own pace. Since the methods read similarly and sort together, and Context is in the name of the new method, this extension of the database/sql API did not degrade readability or comprehension of the package.

If you anticipate that a function may need more arguments in the future, you can plan ahead by making optional arguments a part of the function’s signature. The simplest way to do that is to add a single struct argument, as the crypto/tls.Dial function does:

func Dial(network, addr string, config *Config) (*Conn, error)

The TLS handshake conducted by Dial requires a network and address, but it has many other parameters with reasonable defaults. Passing a nil for config uses those defaults; passing a Config struct with some fields set will override the defaults for those fields. In the future, adding a new TLS configuration parameter only requires a new field on the Config struct, a change that is backward-compatible (almost always—see "Maintaining struct compatibility" below).

Sometimes the techniques of adding a new function and adding options can be combined by making the options struct a method receiver. Consider the evolution of the net package’s ability to listen at a network address. Prior to Go 1.11, the net package provided only a Listen function with the signature

func Listen(network, address string) (Listener, error)

For Go 1.11, two features were added to net listening: passing a context, and allowing the caller to provide a “control function” to adjust the raw connection after creation but before binding. The result could have been a new function that took a context, network, address and control function. Instead, the package authors added a ListenConfig struct in anticipation that more options might be needed someday. And rather than define a new top-level function with a cumbersome name, they added a Listen method to ListenConfig:

type ListenConfig struct {
    Control func(network, address string, c syscall.RawConn) error
}

func (*ListenConfig) Listen(ctx context.Context, network, address string) (Listener, error)

Another way to provide new options in the future is the “Option types” pattern, where options are passed as variadic arguments, and each option is a function that changes the state of the value being constructed. They are described in more detail by Rob Pike's post Self-referential functions and the design of options. One widely used example is google.golang.ir/grpc's DialOption.

Option types fulfill the same role as struct options in function arguments: they are an extensible way to pass behavior-modifying configuration. Deciding which to choose is largely a matter of style. Consider this simple usage of gRPC's DialOption option type:

grpc.Dial("some-target",
  grpc.WithAuthority("some-authority"),
  grpc.WithMaxDelay(time.Second),
  grpc.WithBlock())

This could also have been implemented as a struct option:

notgrpc.Dial("some-target", &notgrpc.Options{
  Authority: "some-authority",
  MaxDelay:  time.Minute,
  Block:     true,
})

Functional options have some downsides: they require writing the package name before the option for each call; they increase the size of the package namespace; and it's unclear what the behavior should be if the same option is provided twice. On the other hand, functions which take option structs require a parameter which might almost always be nil, which some find unattractive. And when a type’s zero value has a valid meaning, it is clumsy to specify that the option should have its default value, typically requiring a pointer or an additional boolean field.

Either one is a reasonable choice for ensuring future extensibility of your module's public API.

Working with interfaces

Sometimes, new features require changes to publicly-exposed interfaces: for example, an interface needs to be extended with new methods. Directly adding to an interface is a breaking change, though—how, then, can we support new methods on a publicly-exposed interface?

The basic idea is to define a new interface with the new method, and then wherever the old interface is used, dynamically check whether the provided type is the older type or the newer type.

Let's illustrate this with an example from the archive/tar package. tar.NewReader accepts an io.Reader, but over time the Go team realized that it would be more efficient to skip from one file header to the next if you could call Seek. But, they could not add a Seek method to io.Reader: that would break all implementers of io.Reader.

Another ruled-out option was to change tar.NewReader to accept io.ReadSeeker rather than io.Reader, since it supports both io.Reader methods and Seek (by way of io.Seeker). But, as we saw above, changing a function signature is also a breaking change.

So, they decided to keep tar.NewReader signature unchanged, but type check for (and support) io.Seeker in tar.Reader methods:

package tar

type Reader struct {
  r io.Reader
}

func NewReader(r io.Reader) *Reader {
  return &Reader{r: r}
}

func (r *Reader) Read(b []byte) (int, error) {
  if rs, ok := r.r.(io.Seeker); ok {
    // Use more efficient rs.Seek.
  }
  // Use less efficient r.r.Read.
}

(See reader.go for the actual code.)

When you run into a case where you want to add a method to an existing interface, you may be able to follow this strategy. Start by creating a new interface with your new method, or identify an existing interface with the new method. Next, identify the relevant functions that need to support it, type check for the second interface, and add code that uses it.

This strategy only works when the old interface without the new method can still be supported, limiting the future extensibility of your module.

Where possible, it is better to avoid this class of problem entirely. When designing constructors, for example, prefer to return concrete types. Working with concrete types allows you to add methods in the future without breaking users, unlike interfaces. That property allows your module to be extended more easily in the future.

Tip: if you do need to use an interface but don't intend for users to implement it, you can add an unexported method. This prevents types defined outside your package from satisfying your interface without embedding, freeing you to add methods later without breaking user implementations. For example, see testing.TB's private() function.

type TB interface {
    Error(args ...interface{})
    Errorf(format string, args ...interface{})
    // ...

    // A private method to prevent users implementing the
    // interface and so future additions to it will not
    // violate Go 1 compatibility.
    private()
}

This topic is also explored in more detail in Jonathan Amsterdam's "Detecting Incompatible API Changes" talk (video, slides).

Add configuration methods

So far we've talked about overt breaking changes, where changing a type or a function would cause users' code to stop compiling. However, behavior changes can also break users, even if user code continues to compile. For example, many users expect json.Decoder to ignore fields in the JSON that are not in the argument struct. When the Go team wanted to return an error in that case, they had to be careful. Doing so without an opt-in mechanism would mean that the many users relying on those methods might start receiving errors where they hadn’t before.

So, rather than changing the behavior for all users, they added a configuration method to the Decoder struct: Decoder.DisallowUnknownFields. Calling this method opts a user in to the new behavior, but not doing so preserves the old behavior for existing users.

Maintaining struct compatibility

We saw above that any change to a function’s signature is a breaking change. The situation is much better with structs. If you have an exported struct type, you can almost always add a field or remove an unexported field without breaking compatibility. When adding a field, make sure that its zero value is meaningful and preserves the old behavior, so that existing code that doesn’t set the field continues to work.

Recall that the authors of the net package added ListenConfig in Go 1.11 because they thought more options might be forthcoming. Turns out they were right. In Go 1.13, the KeepAlive field was added to allow for disabling keep-alive or changing its period. The default value of zero preserves the original behavior of enabling keep-alive with a default period.

There is one subtle way a new field can break user code unexpectedly. If all the field types in a struct are comparable—meaning values of those types can be compared with == and != and used as a map key—then the overall struct type is comparable too. In this case, adding a new field of uncomparable type will make the overall struct type non-comparable, breaking any code that compares values of that struct type.

To keep a struct comparable, don’t add non-comparable fields to it. You can write a test for that, or rely on the upcoming gorelease tool to catch it.

To prevent comparison in the first place, make sure the struct has a non-comparable field. It may have one already—no slice, map or function type is comparable—but if not, one can be added like so:

type Point struct {
        _ [0]func()
        X int
        Y int
}

The func() type is not comparable, and the zero-length array takes up no space. We can define a type to clarify our intent:

type doNotCompare [0]func()

type Point struct {
        doNotCompare
        X int
        Y int
}

Should you use doNotCompare in your structs? If you’ve defined the struct to be used as a pointer—that is, it has pointer methods and perhaps a NewXXX constructor function that returns a pointer—then adding a doNotCompare field is probably overkill. Users of a pointer type understand that each value of the type is distinct: that if they want to compare two values, they should compare the pointers.

If you are defining a struct intended to be used as a value directly, like our Point example, then quite often you want it to be comparable. In the uncommon case that you have a value struct that you don’t want compared, then adding a doNotCompare field will give you the freedom to change the struct later without having to worry about breaking comparisons. On the downside, the type won’t be usable as a map key.

Conclusion

When planning an API from scratch, consider carefully how extensible the API will be to new changes in the future. And when you do need to add new features, remember the rule: add, don't change or remove, keeping in mind the exceptions—interfaces, function arguments, and return values can't be added in backwards-compatible ways.

If you need to dramatically change an API, or if an API begins to lose its focus as more features are added, then it may be time for a new major version. But most of the time, making a backwards-compatible change is easy and avoids causing pain for your users.

The Next Step for Generics

Ian Lance Taylor and Robert Griesemer
16 June 2020

Introduction

It’s been almost a year since we last wrote about the possibility of adding generics to Go. It’s time for an update.

Updated design

We’ve been continuing to refine the generics design draft. We’ve written a type checker for it: a program that can parse Go code that uses generics as described in the design draft and report any type errors. We’ve written example code. And we’ve collected feedback from many, many people—thanks for providing it!

Based on what we’ve learned, we’re releasing an updated design draft. The biggest change is that we are dropping the idea of contracts. The difference between contracts and interface types was confusing, so we’re eliminating that difference. Type parameters are now constrained by interface types. Interface types are now permitted to include type lists, though only when used as constraints; in the previous design draft type lists were a feature of contracts. More complex cases will use a parameterized interface type.

We hope that people will find this design draft simpler and easier to understand.

Experimentation tool

To help decide how to further refine the design draft, we are releasing a translation tool. This is a tool that permits people to type check and run code written using the version of generics described in the design draft. It works by translating generic code into ordinary Go code. This translation process imposes some limitations, but we hope that it will be good enough for people to get a feel for what generic Go code might look like. The real implementation of generics, if they are accepted into the language, will work differently. (We have only just begun to sketch out what a direct compiler implementation would look like.)

The tool is available on a variant of the Go playground at https://go2goplay.golang.ir. This playground works just like the usual Go playground, but it supports generic code.

You can also build and use the tool yourself. It is available in a branch of the master Go repo. Follow the instructions on installing Go from source. Where those instructions direct you to check out the latest release tag, instead run git checkout dev.go2go. Then build the Go toolchain as directed.

The translation tool is documented in README.go2go.

Next steps

We hope that the tool will give the Go community a chance to experiment with generics. There are two main things that we hope to learn.

First, does generic code make sense? Does it feel like Go? What surprises do people encounter? Are the error messages useful?

Second, we know that many people have said that Go needs generics, but we don’t necessarily know exactly what that means. Does this draft design address the problem in a useful way? If there is a problem that makes you think “I could solve this if Go had generics,” can you solve the problem when using this tool?

We will use the feedback we gather from the Go community to decide how to move forward. If the draft design is well received and doesn’t need significant changes, the next step would be a formal language change proposal. To set expectations, if everybody is completely happy with the design draft and it does not require any further adjustments, the earliest that generics could be added to Go would be the Go 1.17 release, scheduled for August 2021. In reality, of course, there may be unforeseen problems, so this is an optimistic timeline; we can’t make any definite prediction.

Feedback

The best way to provide feedback for the language changes will be on the mailing list [email protected]. Mailing lists are imperfect, but they seem like our best option for initial discussion. When writing about the design draft, please put [generics] at the start of the Subject line and to start different threads for different specific topics.

If you find bugs in the generics type checker or the translation tool, they should be filed in the standard Go issue tracker at https://golang.ir/issue. Please start the issue title with cmd/go2go:. Note that the issue tracker is not the best place to discuss changes to the language, because it does not provide threading and it is not well suited to lengthy conversations.

We look forward to your feedback.

Acknowledgements

We’re not finished, but we’ve come a long way. We would not be here without a lot of help.

We’d like to thank Philip Wadler and his collaborators for thinking formally about generics in Go and helping us clarify the theoretical aspects of the design. Their paper Featherweight Go analyzes generics in a restricted version of Go, and they have developed a prototype on GitHub.

We would also like to thank the people who provided detailed feedback on an earlier version of the design draft.

And last but definitely not least, we’d like to thank many people on the Go team, many contributors to the Go issue tracker, and everybody else who shared ideas and feedback on earlier design drafts. We read all of it, and we’re grateful. We wouldn’t be here without you.

Pkg.go.dev is open source!

Julie Qiu
15 June 2020

We’re excited to announce that the codebase for pkg.go.dev is now open source.

The repository lives at go.googlesource.com/pkgsite and is mirrored to github.com/golang/pkgsite. We will continue using the Go issue tracker to track feedback related to pkg.go.dev.

Contributing

If you are interested in contributing to any issues related to pkg.go.dev, check out our contribution guidelines. We also encourage you to continue filing issues if you run into problems or have feedback.

What’s Next

We really appreciate all the feedback we’ve received so far. It has been a big help in shaping our roadmap for the coming year. Now that pkg.go.dev is open source, here’s what we’ll be working on next:

  • We have some design changes planned for pkg.go.dev, to address UX feedback that we have received. You can expect a more cohesive search and navigation experience. We plan to share these designs for feedback once they are ready.

  • We know that there are features available on godoc.org that users want to see on pkg.go.dev. We’ve been keeping track of them on Go issue #39144, and will prioritize adding them in the next few months. We also plan to continue improving our license detection algorithm based on feedback.

  • We’ll be improving our search experience based on feedback in Go issue #37810, to make it easier for users to find the dependencies they are looking for and make better decisions around which ones to import.

Thanks for being patient with us in the process of open sourcing pkg.go.dev. We’re looking forward to receiving your contributions and working with you on the future of the project.

The VS Code Go extension joins the Go project

The Go team
9 June 2020

When the Go project began, “an overarching goal was that Go do more to help the working programmer by enabling tooling, automating mundane tasks such as code formatting, and removing obstacles to working on large code bases” (Go FAQ). Today, more than a decade later, we continue to be guided by that same goal, especially as it pertains to the programmer’s most critical tool: their editor.

Throughout the past decade, Go developers have relied on a variety of editors and dozens of independently authored tools and plugins. Much of Go’s early success can be attributed to the fantastic development tools created by the Go community. The VS Code extension for Go, built using many of these tools, is now used by 41 percent of Go developers (Go developer survey).

As the VS Code Go extension grows in popularity and as the ecosystem expands, it requires more maintenance and support. Over the past few years, the Go team has collaborated with the VS Code team to help the Go extension maintainers. The Go team also began a new initiative to improve the tools powering all Go editor extensions, with a focus on supporting the Language Server Protocol with gopls and the Debug Adapter Protocol with Delve.

Through this collaborative work between the VS Code and Go teams, we realized that the Go team is uniquely positioned to evolve the Go development experience alongside the Go language.

As a result, we’re happy to announce the next phase in the Go team’s partnership with the VS Code team: The VS Code extension for Go is officially joining the Go project. With this come two critical changes:

  1. The publisher of the plugin is shifting from "Microsoft" to "Go Team at Google".
  2. The project’s repository is moving to join the rest of the Go project at https://github.com/golang/vscode-go.

We cannot overstate our gratitude to those who have helped build and maintain this beloved extension. We know that innovative ideas and features come from you, our users. The Go team’s primary aim as owners of the extension is to reduce the burden of maintenance work on the Go community. We’ll make sure the builds stay green, the issues get triaged, and the docs get updated. Go team members will keep contributors abreast of relevant language changes, and we’ll smooth the rough edges between the extension’s different dependencies.

Please continue to share your thoughts with us by filing issues and making contributions to the project. The process for contributing will now be the same as for the rest of the Go project. Go team members will offer general help in the #vscode channel on Gophers Slack, and we’ve also created a #vscode-dev channel to discuss issues and brainstorm ideas with contributors.

We’re excited about this new step forward, and we hope you are too. By maintaining a major Go editor extension, as well as the Go tooling and language, the Go team will be able to provide all Go users, regardless of their editor, a more cohesive and refined development experience.

As always, our goal remains the same: Every user should have an excellent experience writing Go code.

See the accompanying post from the Visual Studio Code team.

Go Developer Survey 2019 Results

Todd Kulesza
20 April 2020

What a response!

I want to start with an enormous thank you to the thousands of Go developers who participated in this year’s survey. For 2019, we saw 10,975 responses, nearly twice as many as last year! On behalf of the rest of the team, I cannot adequately stress how much we appreciate you taking the time and effort to tell us about your experiences with Go. Thank you!

A note about prior years

Sharp-eyed readers may notice that our year-over-year comparisons don’t quite square with numbers we’ve shared in the past. The reason is that from 2016–2018, we calculated percentages for each question using the total number of people who started the survey as the denominator. While that’s nice and consistent, it ignores the fact that not everyone finishes the survey—up to 40% of participants stop before reaching the final page, which meant questions that occurred later in the survey appeared to perform worse solely because they were later. Thus, this year we’ve recalculated all of our results (including the 2016–2018 responses shown in this post) to use the number of people who responded to a given question as the denominator for that question. We’ve included the number of 2019 responses for each chart—in the form of "n=[number of respondents]" on the x-axis or in the chart’s legend—to give readers a better understanding of the weight of evidence underlying each finding.

Similarly, we learned that in prior surveys options that appeared earlier in response lists had a disproportionate response rate. To address this, we added an element of randomization into the survey. Some of our multiple-choice questions have lists of choices with no logical ordering, such as "I write the following in Go: [list of types of applications]". Previously these choices had been alphabetized, but for 2019 they were presented in a random order to each participant. This means year-over-year comparison for certain questions are invalid for 2018 → 2019, but trends from 2016–2018 are not invalidated. You can think of this as setting a more accurate baseline for 2019. We retained alphabetical ordering in cases where respondents are likely to scan for a particular name, such as their preferred editor. We explicitly call out which questions this applies to below.

A third major change was to improve our analysis of questions with open-ended, free-text responses. Last year we used machine learning to roughly—but quickly—categorize these responses. This year two researchers manually analyzed and categorized these responses, allowing for a more granular analysis but preventing valid comparisons with last year’s numbers. Like the randomization discussed above, the purpose of this change is to give us a reliable baseline for 2019 onward.

Without further ado…

This is a long post. Here’s the tl;dr of our major findings:

  • The demographics of our respondents are similar to Stack Overflow’s survey respondents, which increases our confidence that these results are representative of the larger Go developer audience.
  • A majority of respondents use Go every day, and this number has been trending up each year.
  • Go’s use is still concentrated in technology companies, but Go is increasingly found in a wider variety of industries, such as finance and media.
  • Methodology changes showed us that most of our year-over-year metrics are stable and higher than we previously realized.
  • Respondents are using Go to solve similar problems, particularly building API/RPC services and CLIs, regardless of the size of organization they work at.
  • Most teams try to update to the latest Go release quickly; when third-party providers are late to support the current Go release, this creates an adoption blocker for developers.
  • Almost everyone in the Go ecosystem is now using modules, but some confusion around package management remains.
  • High-priority areas for improvement include improving the developer experience for debugging, working with modules, and working with cloud services.
  • VS Code and GoLand have continued to see increased use; they’re now preferred by 3 out of 4 respondents.

Who did we hear from?

This year we asked some new demographic questions to help us better understand the people who’ve responded to this survey. In particular, we asked about the duration of professional programming experience and the size of the organizations where people work. These were modeled on questions that StackOverflow asks in their annual survey, and the distribution of responses we saw is very close to StackOverflow’s 2019 results. Our take-away is the respondents to this survey have similar levels of professional experience and proportional representation of different sizes of organizations as the StackOverflow survey audience (with the obvious difference that we’re primarily hearing from developers working with Go). That increases our confidence when generalizing these findings to the estimated 1 million Go developers worldwide. These demographic questions will also help us in the future to identify which year-over-year changes may be the result of a shift in who responded to the survey, rather than changes in sentiment or behavior.

Looking at Go experience, we see that a majority of respondents (56%) are relatively new to Go, having used it for less than two years. Majorities also said they use Go at work (72%) and outside of work (62%). The percentage of respondents using Go professionally appears to be trending up each year.

As you can see in the chart below, in 2018 we saw a spike in these numbers, but that increase disappeared this year. This is one of many signals suggesting that the audience who answered the survey in 2018 was significantly different than in the other three years. In this case they were significantly more likely to be using Go outside of work and a different language while at work, but we see similar outliers across multiple survey questions.

Respondents who have been using Go the longest have different backgrounds than newer Go developers. These Go veterans were more likely to claim expertise in C/C++ and less likely to claim expertise in JavaScript, TypeScript, and PHP. One caveat is that this is self-reported "expertise"; it may be more helpful to think of it instead as "familiarity". Python appears to be the language (other than Go) familiar to the most respondents, regardless of how long they’ve been working with Go.

Last year we asked about which industries respondents work in, finding that a majority reported working in software, internet, or web service companies. This year it appears respondents represent a broader range of industries. However, we also simplified the list of industries to reduce confusion from potentially overlapping categories (e.g., the separate categories for "Software" and "Internet / web services" from 2018 were combined into "Technology" for 2019). Thus, this isn’t strictly an apples-to-apples comparison. For example, it’s possible that one effect of simplifying the category list was to reduce the use of the "Software" category as a catch-all for respondents writing Go software for an industry that wasn’t explicitly listed.

Go is a successful open-source project, but that doesn’t mean the developers working with it are also writing free or open-source software. As in prior years, we found that most respondents are not frequent contributors to Go open-source projects, with 75% saying they do so "infrequently" or "never". As the Go community expands, we see the proportion of respondents who’ve never contributed to Go open-source projects slowly trending up.

Developer tools

As in prior years, the vast majority of survey respondents reported working with Go on Linux and macOS systems. This is one area of strong divergence between our respondents and StackOverflow’s 2019 results: in our survey, only 20% of respondents use Windows as a primary development platform, while for StackOverflow it was 45% of respondents. Linux is used by 66% and macOS by 53%—both much higher than the StackOverflow audience, which reported 25% and 30%, respectively.

The trend in editor consolidation has continued this year. GoLand saw the sharpest increase in use this year, rising from 24% → 34%. VS Code’s growth slowed, but it remains the most popular editor among respondents at 41%. Combined, these two editors are now preferred by 3 out of 4 respondents.

Every other editor saw a small decrease. This doesn’t mean those editors aren’t being used at all, but they’re not what respondents say they prefer to use for writing Go code.

This year we added a question about internal Go documentation tooling, such as gddo. A small minority of respondents (6%) reported that their organization runs its own Go documentation server, though this proportion nearly doubles (to 11%) when we look at respondents at large organizations (those with at least 5,000 employees). A follow-up asked of respondents who said their organization had stopped running its own documentation server suggests that the top reason to retire their server was a combination of low perceived benefits (23%) versus the amount of effort required to initially set it up and maintain it (38%).

Sentiments towards Go

Large majorities of respondents agreed that Go is working well for their teams (86%) and that they’d prefer to use it for their next project (89%). We also found that over half of respondents (59%) believe Go is critical to the success of their companies. All of these metrics have remained stable since 2016.

Normalizing the results changed most of these numbers for prior years. For example, the percentage of respondents who agreed with the statement "Go is working well for my team" was previously in the 50’s and 60’s because of participant drop-off; when we remove participants who never saw the question, we see it’s been fairly stable since 2016.

Looking at sentiments toward problem solving in the Go ecosystem, we see similar results. Large percentages of respondents agreed with each statement (82%–88%), and these rates have been largely stable over the past four years.

This year we took a more nuanced look at satisfaction across industries to establish a baseline. Overall, respondents were positive about using Go at work, regardless of industry sector. We do see small variations in dissatisfaction in a few areas, most notably manufacturing, which we plan to investigate with follow-up research. Similarly, we asked about satisfaction with—and the importance of—various aspects of Go development. Pairing these measures together highlighted three topics of particular focus: debugging (including debugging concurrency), using modules, and using cloud services. Each of these topics was rated "very" or "critically" important by a majority of respondents but had significantly lower satisfaction scores compared to other topics.

Turning to sentiments toward the Go community, we see some differences from prior years. First, there is a dip in the percentage of respondents who agreed with the statement "I feel welcome in the Go community", from 82% to 75%. Digging deeper revealed that the proportion of respondents who "slightly" or "moderately agreed" decreased, while the proportions who "neither agree nor disagree" and "strongly agree" both increased (up 5 and 7 points, respectively). This polarizing split suggests two or more groups whose experiences in the Go community are diverging, and is thus another area we plan to further investigate.

The other big differences are a clear upward trend in responses to the statement "I feel welcome to contribute to the Go project" and a large year-over-year increase in the proportion of respondents who feel Go’s project leadership understands their needs.

All of these results show a pattern of higher agreement correlated with increased Go experience, beginning at about two years. In other words, the longer a respondent has been using Go, the more likely they were to agree with each of these statements.

This likely comes as no surprise, but people who responded to the Go Developer Survey tended to like Go. However, we also wanted to understand which other languages respondents enjoy working with. Most of these numbers have not significantly changed from prior years, with two exceptions: TypeScript (which has increased 10 points), and Rust (up 7 points). When we break these results down by duration of Go experience, we see the same pattern as we found for language expertise. In particular, Python is the language and ecosystem that Go developers are most likely to also enjoy building with.

In 2018 we first asked the "Would you recommend…" Net Promoter Score (NPS) question, yielding a score of 61. This year our NPS result is a statistically unchanged 60 (67% "promoters" minus 7% "detractors").

Working with Go

Building API/RPC services (71%) and CLIs (62%) remain the most common uses of Go. The chart below appears to show major changes from 2018, but these are most likely the result of randomizing the order of choices, which used to be listed alphabetically: 3 of the 4 choices beginning with ’A’ decreased, while everything else remained stable or increased. Thus, this chart is best interpreted as a more accurate baseline for 2019 with trends from 2016–2018. For example, we believe that the proportion of respondents building web services which return HTML has been decreasing since 2016 but were likely undercounted because this response was always at the bottom of a long list of choices. We also broke this out by organization size and industry but found no significant differences: it appears respondents use Go in roughly similar ways whether they work at a small tech start-up or a large retail enterprise.

A related question asked about the larger areas in which respondents work with Go. The most common area by far was web development (66%), but other common areas included databases (45%), network programming (42%), systems programming (38%), and DevOps tasks (37%).

In addition to what respondents are building, we also asked about some of the development techniques they use. A large majority of respondents said they depend upon text logs for debugging (88%), and their free-text responses suggest this is because alternative tooling is challenging to use effectively. However, local stepwise debugging (e.g., with Delve), profiling, and testing with the race detector were not uncommon, with ~50% of respondents depending upon at least one of these techniques.

Regarding package management, we found that the vast majority of respondents have adopted modules for Go (89%). This has been a big shift for developers, and nearly the entire community appears to be going through it simultaneously.

We also found that 75% of respondents evaluate the current Go release for production use, with an additional 12% waiting one release cycle. This suggests a large majority of Go developers are using (or at the least, trying to use) the current or previous stable release, highlighting the importance for platform-as-a-service providers to quickly support new stable releases of Go.

Go in the clouds

Go was designed with modern distributed computing in mind, and we want to continue to improve the developer experience of building cloud services with Go. This year we expanded the questions we asked about cloud development to better understand how respondents are working with cloud providers, what they like about the current developer experience, and what can be improved. As mentioned earlier, some of the 2018 results appear to be outliers, such as an unexpectedly low result for self-owned servers, and an unexpectedly high result for GCP deployments.

We see two clear trends:

  1. The three largest global cloud providers (Amazon Web Services, Google Cloud Platform, and Microsoft Azure) all appear to be trending up in usage among survey respondents, while most other providers are used by a smaller proportion of respondents each year.
  2. On-prem deployments to self-owned or company-owned servers continue to decrease and are now statistically tied with AWS (44% vs. 42%) as the most common deployment targets.

Looking at which types of cloud platforms respondents are using, we see differences between the major providers. Respondents deploying to AWS and Azure were most likely to be using VMs directly (65% and 51%, respectively), while those deploying to GCP were almost twice as likely to be using the managed Kubernetes platform (GKE, 64%) than VMs (35%). We also found that respondents deploying to AWS were equally likely to be using a managed Kubernetes platform (32%) as they were to be using a managed serverless platform (AWS Lambda, 33%). Both GCP (17%) and Azure (7%) had lower proportions of respondents using serverless platforms, and free-text responses suggest a primary reason was delayed support for the latest Go runtime on these platforms.

Overall, a majority of respondents were satisfied with using Go on all three major cloud providers. Respondents reported similar satisfaction levels with Go development for AWS (80% satisfied) and GCP (78%). Azure received a lower satisfaction score (57% satisfied), and free-text responses suggest that the main driver was a perception that Go lacks first-class support on this platform (25% of free-text responses). Here, "first-class support" refers to always staying up-to-date with the latest Go release, and ensuring new features are available to Go developers at time of launch. This was the same top pain-point reported by respondents using GCP (14%), and particularly focused on support for the latest Go runtime in serverless deployments. Respondents deploying to AWS, in contrast, were most likely to say the SDK could use improvements, such as being more idiomatic (21%). SDK improvements were also the second most common request for both GCP (9%) and Azure (18%) developers.

Pain points

The top reasons respondents say they are unable to use Go more remain working on a project in another language (56%), working on a team that prefers to use another language (37%), and the lack of a critical feature in Go itself (25%).

This was one of the questions where we randomized the choice list, so year-over-year comparisons aren’t valid, though 2016–2018 trends are. For example, we are confident that the number of developers unable to use Go more frequently because their team prefers a different language is decreasing each year, but we don’t know whether that decrease dramatically accelerated this year, or was always a bit lower than our 2016–2018 numbers estimated.

The top two adoption blockers (working on an existing non-Go project and working on a team that prefers a different language) don’t have direct technical solutions, but the remaining blockers might. Thus, this year we asked for more details, to better understand how we might help developers increase their use of Go. The charts in the remainder of this section are based on free-text responses which were manually categorized, so they have very long tails; categories totalling less than 3% of the total responses have been grouped into the "Other" category for each chart. A single response may mention multiple topics, thus charts do not not sum to 100%.

Among the 25% of respondents who said Go lacks language features they need, 79% pointed to generics as a critical missing feature. Continued improvements to error handling (in addition to the Go 1.13 changes) was cited by 22%, while 13% requested more functional programming features, particularly built-in map/filter/reduce functionality. To be clear, these numbers are from the subset of respondents who said they would be able to use Go more were it not missing one or more critical features they need, not the entire population of survey respondents.

Respondents who said Go "isn’t an appropriate language" for what they work on had a wide variety of reasons and use-cases. The most common was that they work on some form of front-end development (22%), such as GUIs for web, desktop, or mobile. Another common response was that the respondent said they worked in a domain with an already-dominant language (9%), making it a challenge to use something different. Some respondents also told us which domain they were referring to (or simply mentioned a domain without mentioning another language being more common), which we show via the "I work on [domain]" rows below. An additional top reason cited by respondents was a need for better performance (9%), particularly for real-time computing.

The biggest challenges respondents reported remain largely consistent with last year. Go’s lack of generics and modules/package management still top the list (15% and 12% of responses, respectively), and the proportion of respondents highlighting tooling problems increased. These numbers are different from the above charts because this question was asked of all respondents, regardless of what they said their biggest Go adoption blockers were. All three of these are areas of focus for the Go team this year, and we hope to greatly improve the developer experience, particularly around modules, tooling, and the getting started experience, in the coming months.

Diagnosing faults and performance issues can be challenging in any language. Respondents told us their top challenge for both of these was not something specific to Go’s implementation or tooling, but a more fundamental issue: a self-reported lack of knowledge, experience, or best practices. We hope to help address these knowledge gaps via documentation and other educational materials later this year. The other major problems do involve tooling, specifically a perceived unfavorable cost/benefit trade-off to learning/using Go’s debugging and profiling tooling, and challenges making the tooling work in various environments (e.g., debugging in containers, or getting performance profiles from production systems).

Finally, when we asked what would most improve Go support in respondents’ editing environment, the most common response was for general improvements or better support for the language server (gopls, 19%). This was expected, as gopls replaces about 80 extant tools and is still in beta. When respondents were more specific about what they’d like to see improved, they were most likely to report the debugging experience (14%) and faster or more reliable code completion (13%). A number of participants also explicitly referenced the need to frequently restart VS Code when using gopls (8%); in the time since this survey was in the field (late November – early December 2019), many of these gopls improvements have already landed, and this continues to be a high-priority area for the team.

The Go community

Roughly two thirds of respondents used Stack Overflow to answer their Go-related questions (64%). The other top sources of answers were godoc.org (47%), directly reading source code (42%), and golang.ir (33%).

The long tail on the previous chart highlights the large variety of different sources (nearly all of them community-driven) and modalities that respondents rely on to overcome challenges while developing with Go. Indeed, for many Gophers, this may be one of their main points of interaction with the larger community: as our community expands, we’ve seen higher and higher proportions of respondents who do not attend any Go-related events. For 2019, that proportion nearly reached two thirds of respondents (62%).

Due to updated Google-wide privacy guidelines, we can no longer ask about which countries respondents live in. Instead we asked about preferred spoken/written language as a very rough proxy for Go’s worldwide usage, with the benefit of providing data for potential localization efforts.

Because this survey is in English, there is likely a strong bias toward English speakers and people from areas where English is a common second or third language. Thus, the non-English numbers should be interpreted as likely minimums rather than an approximation of Go’s global audience.

We found 12% of respondents identify with a traditionally underrepresented group (e.g., ethnicity, gender identity, et al.) and 3% identify as female. (This question should have said "woman" instead of "female". The mistake has been corrected in our draft survey for 2020, and we apologize for it.) We strongly suspect this 3% is undercounting women in the Go community. For example, we know women software developers in the US respond to the StackOverflow Developer Survey at about half the rate we’d expect based on US employment figures (11% vs 20%). Since we don’t know the proportion of responses in the US, we can’t safely extrapolate from these numbers beyond saying the actual proportion is likely higher than 3%. Furthermore, GDPR required us to change how we ask about sensitive information, which includes gender and traditionally underrepresented groups. Unfortunately these changes prevent us from being able to make valid comparisons of these numbers with prior years.

Respondents who identified with underrepresented groups or preferred not to answer this question showed higher rates of disagreement with the statement "I feel welcome in the Go community" (8% vs. 4%) than those who do not identify with an underrepresented group, highlighting the importance of our continued outreach efforts.

Conclusion

We hope you’ve enjoyed seeing the results of our 2019 developer survey. Understanding developers’ experiences and challenges helps us plan and prioritize work for 2020. Once again, an enormous thank you to everyone who contributed to this survey—your feedback is helping to steer Go’s direction in the coming year and beyond.

See the index for more articles.