Go Study Notes (10) - Catastrophic Record of Migrating Old Projects to Go Modules
Last updated: 2 years ago
Recently, I have been working on revamping an early project that involves replacing the original dependency management using “Vendor” with “Go Modules”. However, the process has been quite tumultuous. Here, I will summarize the issues encountered during this “Go Modules” revamp, as well as the solutions.
Background
go version：
1
2$ go version
go version go1.16.5 darwin/amd64Here is a simplified demo, it’s “simple” enough that we just need to output “hello world”.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27package main
import (
"github.com/coreos/etcd/pkg/transport"
"github.com/google/certificate-transparency-go/tls"
"github.com/qiniu/api.v7/auth/qbox"
"go.etcd.io/etcd/clientv3"
"google.golang.org/grpc"
"qiniupkg.com/x/log.v7"
)
func main() {
_ = transport.TLSInfo{}
_ = clientv3.WatchResponse{}
_, _ = clientv3.New(clientv3.Config{})
_ = qbox.NewMac("", "")
_ = tls.DigitallySigned{}
_ = grpc.ClientConn{}
log.Info("hello world")
}
Practical experience
Initialize directly and tidy.
1 | |
Oh boy, there’s an error. Let’s take a look at the first two lines:
qiniupkg.com/[email protected]doesn’t haveqiniupkg.com/x/log.v7;github.com/qiniu/[email protected]doesn’t havegithub.com/qiniu/x/bytes.v7/seekable;
This seems to be an issue. qiniupkg.com/x and github.com/qiniu/x should be the same package on different mirrors. So I went to Github and checked the code for version @latest, and indeed there’s no bytes.v7 package. After some manual searching, we found the bytes.v7 package in version v1.7.8.
Therefore, we can specify a version.
1 | |
Keep reading, the following questions are related to etcd.
It means that go.etcd.io/etcd/clientv3 imports github.com/coreos/etcd/Godeps/_workspace/src/golang.org/x/net/context, while github.com/coreos/[email protected] provides github.com/coreos/etcd/Godeps/_workspace/src/golang.org/x/net/context. However, we need github.com/coreos/[email protected] here, and this version does not provide github.com/coreos/etcd/Godeps/_workspace/src/golang.org/x/net/context.
Let’s try updating etcd to v3.3.10 directly.
1 | |
Let’s run go mod tidy again.
1 | |
This error is consistent with the one described in the article Etcd使用go module的灾难 . The package names go.etcd.io/bbolt and github.com/coreos/bbolt are inconsistent, so we need to replace them.
1 | |
Continue and run go mod tidy.
1 | |
Wow, it’s etcd again. Upon closer inspection, we have imported two versions of etcd: github.com/coreos/etcd and go.etcd.io/etcd. We only replaced one of them earlier. Now we’ll replace the other one as well.
1 | |
After running go mod tidy again, the previous error disappeared, but there was still an error with grpc. I continued to investigate and discovered that version v1.39.0 of google.golang.org/grpc did not have the google.golang.org/grpc/naming package. I went to the Github repository and searched through previous versions, and found that the package did exist in version v1.29.1, so I proceeded to replace it with that version.
1 | |
Finally, go mod tidy passed and I can happily output hello world. However,
1 | |
Surprise, surprise! It turns out that the etcd package depends on the resolver package from grpc, but the v1.29.1 version of grpc that I imported doesn’t have this package. I went through the versions in the grpc repository (https://github.com/grpc/grpc-go/blob/v1.26.0/resolver/resolver.go) one by one and found that only version v1.26.0 declares type BuildOption. So, we’ll have to replace it again.
1 | |
Run tidy again! Finally, you will see the long-lost hello world!
1 | |
Summary
Project Standards
Now let’s take a look back at this demo project, which actually has a lot of issues.
1 | |
It is possible to unify the packages of etcd and qiniupkg and import only one of them! Also, we later discovered that the log.v7 package was accidentally imported…
This was a problem we encountered when refactoring some of our older projects. We didn’t notice these issues when using vendor and go get before, so this needs to be standardized in advance.
Understanding go.mod
Let’s take a look at the go.mod file we obtained after going through various difficulties.
1 | |
Let’s take a look at some common directives in this documentation:
moduledefines the path of the main module;gospecifies the Go version used to write themodfile;requiredeclares the minimum required version of a given module dependency;replacemanually specifies a dependency module (can replace all versions, specific versions, local versions, etc.).
There is also the +incompatible after v3.3.20, which indicates a compatible version, meaning that the dependency library version is v2 or higher, but the go.mod and dependency library paths are not named according to the official specifications, hence the tag is added.
v0.0.0-00010101000000-000000000000 is a pseudo-version, which is used when an incompatible module or a tagged version is not available.
// indirect indicates that these are not dependencies that we directly reference.
In addition, the following directives are also worth noting.
1 | |
For more detailed information, please refer to the official documentation. Thank you for reading.
Reference
Unless otherwise stated, all articles on this blog are written in CC BY-SA 4.0 , Please indicate the source when reproducing!