Subscribed unsubscribe Subscribe Subscribe

【golang】Don't use filtering by build tags to vendoring

日本語版:【golang】vendoring時はビルドタグでのフィルタリングは使わない方がいい

TL;DR

  • govendor can specify build tags to ignore when vendoring
  • But, files necessary for build may not be copied under the vendor directory
  • Problems such as failure to cross-compile may occur by not copying necessary files
  • So, the size of the vendor directory may increase, but it is simpler not to use filtering by build tags

Introduction

Currently, golang has many package management tools as following:

Many other tools are also introduced on this page.

PackageManagementTools · golang/go Wiki · GitHub

Among them, I often use govendor.

github.com

govendor is very easy to use, I use it in many projects.
However, there was a problem depending on the setting of govendor.

What is the problem?

Last month , I made a following PullRequest to Terraform.

github.com

This PR is for adding Arukas provider.
This new provider was already released separately from Terraform as the following third-party-plugin.

github.com

This PR was immediately merged and released in Terraform v0.8.3.

But, at that time,,,

f:id:febc_yamamoto:20170115213957p:plain

Build on Windows broke!!!

What's!? What's happened!?!?

The Arukas provider release as a third-party-plugin included the Windows version, and it was able to build without any problems.
From that time on, the source code has not changed at all, why has the build broken?

Who broke the build on Windows?

I looked for gopkg.in/alecthomas/kingpin.v2 which was in the message from @jbardin.

gopkg.in/alecthomas/kingpin.v2 is a library imported by the Arukas provider.
This library has been used since Arukas provider was released as a third-party-plugin, so I know that it is compatible with Windows.

And, both Terraform and the Arukas provider as a third-party-plugin use govendor for vendoring.
So I decided to compare the files of gopkg.in/alecthomas/kingpin.v2 library copied under vendor directory(that was copied by govendor).

Compare gopkg.in/alecthomas/kingpin.v2 files under vendor directory

The following screen shot shows the comparison result. f:id:febc_yamamoto:20170115233145p:plain

It seems that guesswidth.go has not been copied for some reason.

Why??

Answer: govendor was setted to ignore some build tags

The govendor configuration file vendor/vendor.json has the following setting:

https://github.com/hashicorp/terraform/blob/v0.8.4/vendor/vendor.json#L3

   "ignore": "appengine test",

govendor compares the value set in ignore in vendor/vendor.json with the file name suffix or build tags to determine if it is the target file to ignore.
In this case test and appengine are specified as ignore targets.

And, the build tag of the missing file guesswidth.go is as follows:

https://github.com/alecthomas/kingpin/blob/v2.2.3/guesswidth.go#L1

// +build appengine !linux,!freebsd,!darwin,!dragonfly,!netbsd,!openbsd

In case of *nix platform, guesswidth_unix.go is used instead of this file.
It has the following build tags:

https://github.com/alecthomas/kingpin/blob/v2.2.3/guesswidth_unix.go#L1

// +build !appengine,linux freebsd darwin dragonfly netbsd openbsd

With these build tags, the guesswidth.go file is used when building on Windows.
However, appengine is specified in ignore in vendor.json, so govendor ignores guesswidth.go!!!

In this way, you can build with *nix but not build on Windows,,,

So what should we do?

Conclusion

Filtering by build tag has the advantage of being able to reduce the size of the vendor directory, but problems such as being unable to build in some platforms may occur.
In addition, troublesome work such as checking the build tag of individual files to be imported will occur to solve the problem.

For these reason, filtering by build tag should not be used when vendoring.
(Note: I think filtering works well only for "test" and "ignore" tags)

What is a use case that requires filtering when vendoring? I don't have it.
What do you think?
If you have a nice way please comment to this post!