Stitch any CI into Tangled
151
fork

Configure Feed

Select the types of activity you want to include in your feed.

provider/tekton: when pruning is configured and old builds are pruned, log lookup doesn't work #4

open opened by xeiaso.net

I think this is working as expected. Tack is not meant to be a meaningfully stateful service (beyond what is necessary to proxy live runs and cache basic associations that are required for jetstream). I don't think log storage is something we really want to take on, it dramatically increases the potential footprint of the sqlite database which at this point is never going to be large except under artificial adversarial cases that require repo access.

Would uploading the logs to S3/object storage be an option here?

So in upstream tekton, there is tektoncd/results that does kind-of support this already : it stores objects metadata and can also forward logs from some places.. so technically, the provider could query the results API for the logs. It would work once the PipelineRun or TaskRun is done (even if not gone yet), but it doesn't work "live" though.. It would also add quite some complexity I guess (at least with the current grpc API) though.

Looking at this a bit more, I think tack could support a fallback to the tektoncd/results API when the K8s resources have been pruned. The idea:

  1. Live/recent runs: unchanged - stream pod logs directly from K8s (works today)
  2. Pruned runs: when the PipelineRun is NotFound in K8s, query the Results API for stored logs

The Results API exposes REST endpoints (HTTPโ†”gRPC transcoding) so tack wouldn't need gRPC deps โ€” just plain HTTP against the in-cluster Results API server, using the same SA token it already has.

The lookup would be:

  • GET /apis/results.tekton.dev/v1alpha2/parents/{ns}/results/-/records?filter=data.metadata.labels['tekton.dev/pipelineRun']=='<name>' to find TaskRun records
  • GET .../logs/{log} to stream the actual log content
  • Wrap the raw log bytes into LogLine frames (same shape as live logs)

This would be opt-in - a new tekton.results_api config field pointing at the Results API server URL. If unset, current behavior is preserved. Only requirement on the user's side is having Results installed with log forwarding enabled, and a tekton-results-readonly RoleBinding for tack's SA.

Happy to put together a PR if this direction seems reasonable.

This sounds reasonable to me!

sign up or login to add to the discussion
Labels

None yet.

assignee

None yet.

Participants 3
AT URI
at://did:plc:e5nncb3dr5thdkjir5cfaqfe/sh.tangled.repo.issue/3ml52ugniqa22