internal/core/adt: new data structure for reqSets
The typo checking algorithm associates a defID
with conjuncts of nodes that keep track from which
definitions or embeddings conjuncts originate.
Previously, each node kept track of each relevant
set of definitions and embeddings by keeping
track of all allowed ids within each set. This
results in polynomial space usage.
The new algorithm, instead, keep track of the
global parent relationship. To check if a
conjuncts is part of the set is than a matter
of repeatedly following the parent of the
conjunct until the ID matches the that of the
group or until the top is reached (not a member).
In addition, the algorithm needs to handle
"equivalences", tracked by replaceIDs. If the
same Vertex is added more than once, it is only
added once. It may originate, however, from
different closedness contexts. The redirects
keep track of this equivalence.
Note that the parent relationsip is now tracked
with a single allocation per defID. So there
is only a constant overhead per defID to track
the sets (apart from redirects), to track set
inclusion, instead of polynomial space overhead.
The time complexity for a lookup of one conjunct
is O(H), where H is the height of the closedness
inclusion tree.
The set of redirects per node is typically
small and should not significantly contribute
to memory usage. It is tracked by MaxRedirect.
This change can result in significant speedup.
For the Issue that prompted this change total
running time saw a reduction of 50%.
This change uncovered some bugs of the previous
algorithm: if conjuncts were added from Vertices
that were created from other OpContexts, the
defIDs were included as is. Of course they were
meaningless and determining whether they were
part of the group would result in more or less
random results.
In the new algorithm, the defID is used as an
index. This could result in a panic if the
defID was larger than the largest defID in the
current context.
Generally speaking, defIDs are overwritten by
the defID of the referrer upon first reference
of a conjunct. But if a Vertex is included as
data, this was not the case. This also uncovered
some bugs where a new OpContext was created
during evalution.
To make the code more robust, we now track the
opID of a Conjunct and ignore the defID if it
does not originate from the current conjunct.
This is strictly better than having random
behavior and generally should not matter as
a Verex for which conjunct info is copied
should be a data vertex, in which case the
defIDs are irrelevant.
That said, we should investigate the
Misaligned* counters.
Issue #3981
Signed-off-by: Marcel van Lohuizen <mpvl@gmail.com>
Change-Id: Ic10490f2f793213a1b8c266911a6b214ffe3db9a
Reviewed-on: https://review.gerrithub.io/c/cue-lang/cue/+/1218697
TryBot-Result: CUEcueckoo <cueckoo@cuelang.org>
Unity-Result: CUE porcuepine <cue.porcuepine@gmail.com>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>