internal/filetypes: replace encoding/gocode with go:embed

This package uses types.cue to describe the file types that we support,
with logic such as what extensions they match and how they behave.

We used encoding/gocode to generate a types.go file to build
a cue.Value which can then be used elsewhere in the package.
gocode is pretty advanced in terms of encoding a gzip-compressed blob
containing all the CUE files making up an instance.

Fortunately, here we have a single file, so we can use go:embed
alongside cue.Context.CompileString to build a cue.Value.
Note that we no longer compress types.cue, but that seems perfectly fine
given that it weighs 7.3KiB and is unlikely to grow much larger.

Besides no longer having to maintain gen.go, one benefit is that
the package init cost, as measured by benchinit, goes down slightly:

│ old │ new │
│ sec/op │ sec/op vs base │
CuelangOrgGoInternalFiletypes 2.350m ± 1% 2.312m ± 0% -1.60% (p=0.000 n=8)

│ old │ new │
│ B/op │ B/op vs base │
CuelangOrgGoInternalFiletypes 1.234Mi ± 0% 1.203Mi ± 0% -2.54% (p=0.000 n=8)

│ old │ new │
│ allocs/op │ allocs/op vs base │
CuelangOrgGoInternalFiletypes 17.83k ± 0% 18.34k ± 0% +2.87% (p=0.000 n=8)

It's unclear why the number of allocations goes up slightly,
but what matters is the wall clock time, as that affects how long
cmd/cue spends initializing packages before it starts doing useful work.

Another, and perhaps more relevant, advantage is a drop in the size
of cmd/cue, as compress/gzip and encoding/gob are no longer bundled.
A vanilla "go build" binary drops by 200KiB from 26.2MiB to 26.0MiB.

This also means that nowhere in the cue repo do we use encoding/gocode
alongside the cue.Value.Marshal and cue.Value.Unmarshal methods anymore.
They are still part of the public API and remain in place for now.

Finally, we rename cuegenValue as there is no longer code generation.

Note that internal/filetypes using CompileString rather than Unmarshal
means that its evaluator work now counted towards $CUE_STATS,
which we don't want to occur as it can be confusing for the end user.
Add a function to reset the global stats and call it from cmd/cue/cmd
once all init work has finished, akin to testing.B.ResetTimer.

Signed-off-by: Daniel Martí <mvdan@mvdan.cc>
Change-Id: Ic7ff13d445b46190b4809dee470dba07da8cadc9
Reviewed-on: https://review.gerrithub.io/c/cue-lang/cue/+/1190095
TryBot-Result: CUEcueckoo <cueckoo@cuelang.org>
Unity-Result: CUE porcuepine <cue.porcuepine@gmail.com>
Reviewed-by: Marcel van Lohuizen <mpvl@gmail.com>

Daniel Martí 2 years ago 3a9199f4 1b9b0799

+36 -96

6 changed files

expand all

cmd

cue

cmd

root.go

internal

core

adt

prof.go

filetypes

filetypes.go

gen.go

types.cue

types.go

cmd/cue/cmd/root.go

··· 92 92 if err := cuedebug.Init(); err != nil { 93 93 return err 94 94 } 95 + // Some init work, such as in internal/filetypes, evaluates CUE by design. 96 + // We don't want that work to count towards $CUE_STATS. 97 + adt.ResetStats() 95 98 96 99 err := f(c, args) 97 100

internal/core/adt/prof.go

··· 28 28 countsMu sync.Mutex 29 29 ) 30 30 31 + // ResetStats sets the global stats counters to zero. 32 + func ResetStats() { 33 + countsMu.Lock() 34 + counts = stats.Counts{} 35 + countsMu.Unlock() 36 + } 37 + 31 38 // AddStats adds the stats of the given OpContext to the global 32 39 // counters. 33 40 func AddStats(ctx *OpContext) {

+2 -4

internal/filetypes/filetypes.go

··· 12 12 // See the License for the specific language governing permissions and 13 13 // limitations under the License. 14 14 15 - //go:generate go run gen.go 16 - 17 15 package filetypes 18 16 19 17 import ( ··· 96 94 }, nil 97 95 } 98 96 99 - i, errs := update(nil, cuegenValue, cuegenValue, "modes", mode.String()) 97 + i, errs := update(nil, typesValue, typesValue, "modes", mode.String()) 100 98 v := i.LookupDef("FileInfo") 101 99 v = v.Fill(b) 102 100 ··· 293 291 } 294 292 295 293 func parseType(s string, mode Mode) (inst, val cue.Value, err error) { 296 - i := cuegenValue 294 + i := typesValue 297 295 i = i.Unify(i.Lookup("modes", mode.String())) 298 296 v := i.LookupDef("File") 299 297

-52

internal/filetypes/gen.go

··· 1 - // Copyright 2020 CUE Authors 2 - // 3 - // Licensed under the Apache License, Version 2.0 (the "License"); 4 - // you may not use this file except in compliance with the License. 5 - // You may obtain a copy of the License at 6 - // 7 - // http://www.apache.org/licenses/LICENSE-2.0 8 - // 9 - // Unless required by applicable law or agreed to in writing, software 10 - // distributed under the License is distributed on an "AS IS" BASIS, 11 - // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 - // See the License for the specific language governing permissions and 13 - // limitations under the License. 14 - 15 - //go:build ignore 16 - 17 - package main 18 - 19 - import ( 20 - "log" 21 - "os" 22 - 23 - "cuelang.org/go/cue/cuecontext" 24 - "cuelang.org/go/cue/load" 25 - "cuelang.org/go/encoding/gocode" 26 - ) 27 - 28 - func main() { 29 - cwd, err := os.Getwd() 30 - if err != nil { 31 - log.Fatal(err) 32 - } 33 - 34 - ctx := cuecontext.New() 35 - insts, err := ctx.BuildInstances(load.Instances([]string{"types.cue"}, &load.Config{ 36 - Dir: cwd, 37 - ModuleRoot: cwd, 38 - Module: "cuelang.org/go/cue/build", 39 - })) 40 - if err != nil { 41 - log.Fatal(err) 42 - } 43 - 44 - b, err := gocode.Generate(".", insts[0], &gocode.Config{}) 45 - if err != nil { 46 - log.Fatal(err) 47 - } 48 - 49 - if err := os.WriteFile("types.go", b, 0644); err != nil { 50 - log.Fatal(err) 51 - } 52 - }

+1 -3

internal/filetypes/types.cue

··· 17 17 // This file describes how various cross-cutting modes influence default 18 18 // settings. 19 19 // 20 - // It is used by gen.go to compile the instance into Go data, which is then 20 + // It is used by types.go to compile a cue.Value, which is then 21 21 // used by the rest of the package to determine settings. 22 - // 23 - // There 24 22 25 23 // A File corresponds to a Go build.File. 26 24 #File: {

+23 -37

internal/filetypes/types.go

··· 1 - // Code generated by gocode.Generate; DO NOT EDIT. 1 + // Copyright 2020 CUE Authors 2 + // 3 + // Licensed under the Apache License, Version 2.0 (the "License"); 4 + // you may not use this file except in compliance with the License. 5 + // You may obtain a copy of the License at 6 + // 7 + // http://www.apache.org/licenses/LICENSE-2.0 8 + // 9 + // Unless required by applicable law or agreed to in writing, software 10 + // distributed under the License is distributed on an "AS IS" BASIS, 11 + // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 + // See the License for the specific language governing permissions and 13 + // limitations under the License. 2 14 3 15 package filetypes 4 16 5 17 import ( 6 - "fmt" 18 + _ "embed" 7 19 8 20 "cuelang.org/go/cue" 9 - "cuelang.org/go/encoding/gocode/gocodec" 10 - _ "cuelang.org/go/pkg" 21 + "cuelang.org/go/cue/cuecontext" 11 22 ) 12 23 13 - var cuegenCodec, cuegenInstance_, cuegenValue = func() (*gocodec.Codec, *cue.Instance, cue.Value) { 14 - var r *cue.Runtime 15 - r = &cue.Runtime{} 16 - instances, err := r.Unmarshal(cuegenInstanceData) 17 - if err != nil { 24 + //go:embed types.cue 25 + var typesCUE string 26 + 27 + var typesValue = func() cue.Value { 28 + ctx := cuecontext.New() 29 + val := ctx.CompileString(typesCUE, cue.Filename("types.cue")) 30 + if err := val.Err(); err != nil { 18 31 panic(err) 19 32 } 20 - if len(instances) != 1 { 21 - panic("expected encoding of exactly one instance") 22 - } 23 - return gocodec.New(r, nil), instances[0], instances[0].Value() 33 + return val 24 34 }() 25 - 26 - // Deprecated: cue.Instance is deprecated. Use cuegenValue instead. 27 - var cuegenInstance = cuegenInstance_ 28 - 29 - // cuegenMake is called in the init phase to initialize CUE values for 30 - // validation functions. 31 - func cuegenMake(name string, x interface{}) cue.Value { 32 - f, err := cuegenValue.FieldByName(name, true) 33 - if err != nil { 34 - panic(fmt.Errorf("could not find type %q in instance", name)) 35 - } 36 - v := f.Value 37 - if x != nil { 38 - w, err := cuegenCodec.ExtractType(x) 39 - if err != nil { 40 - panic(err) 41 - } 42 - v = v.Unify(w) 43 - } 44 - return v 45 - } 46 - 47 - // Data size: 1703 bytes. 48 - var cuegenInstanceData = []byte("\x01\x1f\x8b\b\x00\x00\x00\x00\x00\x00\xff\xc4X\u074b\xe4\xc6\x11\x97\xf6.\x105N\x1e\ry\b\xd4\xe9\xc08\xcbE\x8b?\xc8\xc3\xc0r\x84\xdc]\xb8\x978\x04\xe7\u0258\xa1G*\xcdt,u+\xdd-{\x17\xef\x90\xc4q\xf2g{Cu\xb7\u0512F\xbb{\v\x0e\u0797\x9d\xa9_WuUu}\xce/n\xffs\x96\x9e\xdd\xfe7Io\xff\x99$\xbf\xfb\u01d34}OHc\xb9,\xf1\x15\xb7\x9c\xc8\xe9\x93\xf4\xe9_\x94\xb2\xe9Y\x92>\xfd3\xb7\x87\xf4\xbd$\xfd\xd9\x1b\u0460Io\xbfO\x92\xe4\u05f7\xff>K\xd3_~\xf1e\xd9cQ\x8b&p~\x9f\xa4\xb7\xdf%\u0247\xb7\xffz\x92\xa6?\x8f\xf4\xef\x92\xf4,}\xfa'\xde\"\tz\xea\x88,I\x92\x1f\xde\xff\x15)\x92\xa6gi\x9a\xd9\xeb\x0eMQ\xf6\x98\xfe\xf0~\xd2\xf1\xf2+\xbeG\xd8\xf5\xa2\xa9\x18\xbb\xb8\x80\xdf\x03\xdd\x0f\xa5\xd2\x1aM\xa7de\xc0*\xe0\xf0G\xe5\x0f\x15\x04\x17\xec9\xfd\xdb\xc0\xb7,\xa3\xeb%oq\x03\xe1\xcfX-\xe4\x9ee(KU\t\xb9\x1f\x81\xe7\xaf\x03\x85eBZ\u051dF\u02edP\xf2\xe5\x06\x9e\xbf\x9dQXV+\u077e\x1cY\x89\xfb\x8d\xd2-\xcb,\u07db\x97\xee\xe2\xec\v\x7f\u04d7\x9b\xf1\xca#;:#^a\xcd\xfb\u01820`\x0f\b\xa4\"\xf4\x06+\xa8\x95\x06c+!\x81\u02ca>\xa9\xde\x16\xf0\xf9\x01\xc1\xa0\xb5B\xee\rT\u0621\xacH\x8a\x92\x91\xbbU\x15Y\x1d\x04o\xc0\xd9\x0f\x1f\xcc\x1dp\x9e\xff6\x87\x9bA\x9b\xe3\u011foe\xad\xa0\xc2ZH4pP\xdf\x00\xf7b\x85\x01\xe7&\xac\x9cB\xa3[\xb0\n.&Fg\xad\xfb\u01b2\x8a[\x1e\xbdrnu\x8fp\x035o\f\xb2Lc\x8d\x1ae\x89fs\n\x96\xd7e\xe3\x81\x15N\xa7\x9a \xcf\u04c9\x9dR\r\xcbTG\xdfy\xe3Y<\xadT\xd2X\u0345\xb4\xf1\xdcW\x88]\xf0\x8b\xd9\x04\x9a\x90\xa5j\xbb\x06\xad\v\x8b@k;\xa5\xed\xa0\x81\xa7\x19\xab\x91\xb7\x83R\x9eV\xa9\xd2D\x13=\x8d[\xab\u016e\xb7\xde\x00G\xf3\xee\xa5w1\xf4x\xf4p^\a\xf7\u0215\xa8\x9d/,\xa8\x0e5\xf7\x96\xf8\xd3\x05\xbb\xb8 \xd6\xcf\x0fh\x10,\xb6]\xc3-\x1a\xe0\x1a\xdd\x03Hz\r\xab`\x87\xd0KQ\v\xa4w\x01n]0h\xa5,\xa8\x1a\xecA\x18\x12R*Y\x8b}\xefo(\x98\xbb\xc0\xbd\x97\x90]o}\x9c6h\xe1\n.\xdd\xe7\x99u\x8bG\xc8ff.\xc1#\u02f2\x18\x7fNV\u0330\xf3\xbc\xec\x91boK\xf4\xa2(\x06\x86\x18CW,2\x98 \xa0\xec)j)\xd5La\xca\x03\xb6<\x88 ^\xbc\xb2(\x8d\x0f\tw:/\xfef\x94\xcc\u00f7E\x0e\x93\x0e\xbc\xb7jT\xe2\xe8Y\xaey\xdb<\x96\xe5q\x1cG\xca\xfb\f\xaf(\xba&\x0e\xdf~\xb4\xe6\xf2\xe0\xd4\xf3U\x97/\xc1\a\\\xee\xbcq\xbf\u03f7\x1f=\xe0u\xca\xe7\xe8\xf3#\xcbT\xdf\xd9Y\xe0l?\xfeq\xec\x98j\xf5\xf1c\xb5\u00af\xa9\x0eD\x9d>\xf9\x7f\xfb\xf6\xe1p\xde~\xf2\x80\x11\xb5\xa0\x94\x9fZQa=5\xe2\u04df>'\xb7\x9f>2+\x87\x0e\xf7zHNhyg|3\x89\tK\xe5+\x94C\x0fu\x9a\u02a0\x15T\xfd\x16y\x9d\xe7\xd3.\xbbeYN\xc3\xc1H\xa4~K\x04\x16\xd3?\u04890\x00M@F\xa0!\xa4\xa9\"\xd3\x1c\x91w\"\xa1dDiD`caX\x01\uc55d\x03\x16\xaf,\x01{\x15\xads\xc0^\x11\xb9\xd3\u02aa\xa9\xbe\x8e\xe0$\xe1\x95\x1d\xd0Q\xd2\x1c\xddMt\x8e(\u02e8\xa5|\xf6\xea\xb3\r\x90!\x06\xff\xfe\u0091\xf2b`\x18\x99vBv;\xb8\xb8\x80\x9d\x90\\_w\xbbqT\x18\x06$\x10\xb2\x12\xa5\xefJ\xfe\x01)\x1a\xb8u\xadMc\xa7\u0460\xa4q\x058=\xed^\xf3\xb6`\xe3x\xb5\x81g\x97y\xeeEJ\x98\x0fVP\xa1E\xddN\xe6\x90\x12\xb5\xe5B\x0er\xc0\x1cT\xdfT\xd4\xfdf\xd3\xc8\xc5\x05\xbcQ\x1a\x86\x11\xf6\x05\xb8\x1a\xd1\xf2\xeb\xc5I\xe0\u0509M\xa9\xc5\xce\xeb\xe7#\xf8\x05|s\x10\xe5\x01\x845\xd8\u052esrI\xac\xa5\x92_\xa3\xb6\xbe\xe5r\xf8\xc3__\a\x8e\x82-f\xc2q\xccs\x93\xe04h\x03\xbdv#\xe9\x98^\xcb\xe9,\xaf\x95\xf2!\xec\xa7K/!\xf7\xb7\xe5\xe1\r\xe8\x81|J\x95\xaami&k\x84DO\xb6\xea4\x99\bpi\xe4\xc5\xf8\f\xf6\xd2G\u0254\xb7{\u037b\xc3\fu\x14\x0fV|?\x83*\xbe\x1f\x00\xcb\x17\x88\r\x02]\x91\xf8\x96M\v\x8e\xab7\x0e$+O\xd0`z\x80\x9bU\xbc\xf1\a(\xafNp\x97\x96\x0ev\x11\x7f\x82\xfb\xb4q\a\u01b489\x14\xf3\x8b\x0e\xba\f\xe9v4\x06\xbb\xf1\x1c\x85=\xa0&G\x0f\t\x10r\x04\x06\x11/@\xcdp\x96u\xbb\r\x9c\xcfo\xf1\x7f\xf9\x90^9;\x9d#r\xba\x1fn`\x8d\xf1\xd9\xe5\xfd\xac\x8e\x1c\xac\\50\x1f\x1f\xcc\xe9\x11\x1f\u034b=\xe1\xf1\xe4;\xb9\xf6'n\f\x06\xd2\xe2p\x97q\xd9\x18\x99Y\xd6pw\u035e\x9c\x1eZ!\xb1\xfe(R\x87\xd5+\u0225\xe9\xcc\xe3'\xecnp[\xb9p6H\x85\xe8\x9cf\u04c9\xa0x\xe0]\u0129\x0e%\xef\xc4\x1d\xb2\x02\xfa\x0e\x82|}p]y\xdc\xe4Bw\xa6\xaa\u031b\u0183\x05\xbc\xb5P)4 \x95\x05!\u02e6\xaf\xd0/\x92J\xb7\xf0\xf6U\xc1\xdc9\xa7\x90[cia\xbf\x1cw\u0671~9\xed\xa9;o\u05ea\v\x8cZ\x06W\xc0\r\xe4n\xe4q\x9f\x86\xea\xb2\u0630\x96S\xd8|O[\x8e7\xf3\xadp\x89\xce\xf7\xc3\x0fg\xf0o\xe0\x83%\x85e\x8b\xedq)o\xbeG.\xd1\xf9\xf6\xb8@\x8fT\xe7\xe50\xa2N'\xa7\x13\x7f\x05\x1f\x9d\u0737nU\x94\x7fR\xc0\xe3\x03x_\x93\u05e9p\xfb\xff.w\x17\xdb:\xe9|\xe2\xf3u_\u07eb\xcd\u008f\xeb\xfe[\xf7[\xb4g\xd6sL\xe1l\x98\xd8\xf6\xec2\x86\xd0\xf0\xcb\xc1\x94y\u0697h_\xd8/\xfd\xf2\xec2\xb4\xb1\xb9\xb6\x83Z\xb3\x9f*F\xbb\xa6?Q\xac\x1a\xb0\xea\x97Q\xaf#\x9b\x8f\xd2c\x8f\x1c\x92 Z\x10;d\xdcx\x16\xd9\xe2\x93\x04n\x86w\x9bn\t\x83\x1e\xd3\xe5 \n\x8f\xeds\xee\u0719\x1a\x94\x86^\xf2\xbc#\xaf\xea3\x1e\x8c=g\xf5\\\xd4a\xdaj\x1e8jU{\xdf\xdd\xf1\u0925/r\xec\x9d\u0180\x99\xf4;f\x82\xc9\v,m\xa1F\x7f\x9f\x98i\xcf^\x93\x12[\xdeB\xf9\x13C\x8fl\xde'\x1eQ\xab\xdd\x06\xe5\x9b\xe0\xfc\x96eW\xbb\u04c1\xf7\xf6\xafw\xe6Zu\xd62\x9a\x8e,I\xfe\x17\x00\x00\xff\xffj\xd7\xe3\u00e8\x16\x00\x00")

Configure Feed

Configure Feed