A container registry that uses the AT Protocol for manifest storage and S3 for blob storage. atcr.io

Fork 2

Images Pulled From atcr.io Don't Work When Pushed to DockerHub and Deployed to Railway #16

Closed

evan.jarrett.net 6mo ago

Closed opened by

evan.jarrett.net 6mo ago

The details are here: https://station.railway.com/questions/unable-to-deploy-service-ef7d6343#z40k

Apparently there is some difference in the image that I pull from atcr.io and the Docker image that I get if I just add a useless RUN echo 'hi' layer to a docker image.

I'm not 100% sure this is an atcr.io issue, but there is some difference between atcr.io and the image that Docker creates it seems.

I'm not exactly sure how to debug the difference, but Railway claims that the workdir does not exist on the filesystem. While Docker doesn't have trouble it, Railway seems pickier, and they aren't using Docker, they have their own system.

evan.jarrett.net 6mo ago

atcr.io doesn't modify what gets pushed to it. OCI images are content-addressed: every layer and the manifest are identified by sha256 of their own bytes, and clients verify on receive. If the registry altered anything, clients would reject it. General OCI image problems aren't a registry problem.

I see that the Railway ticket is closed, but I don't feel like you got a good enough answer:

The original image was built with Buildah. Buildah's WORKDIR /app updates the image config's WorkingDir field but doesn't create /app in any layer. That's why Brody said "working directory /app does not exist on container", he was reading the rootfs literally.

When you rebuild with Docker (BuildKit), the builder actually creates the directory. So the rebuilt image has a real /app, and the deploy works. The "useless RUN echo layer" isn't doing the fixing; the switch from Buildah to BuildKit is.

At runtime, (at least testing with podman and docker) They both will create a new /app directory on the fly.

Railway doesn't run an OCI runtime; they assemble their own rootfs from the layers. So their solution seems to be reject images that do not have a proper working directory created in the image. Which means Buildah images will continue to fail.

Hopefully that explains it a little better, but sorry it's not something I can fix with atcr.io

zicklag.dev 6mo ago

Ah, OK, thanks for the clarification! I really wasn't that satisfied with the Railway response either, but their swamped right now and probably stressed out so I'll cut them a little slack. 😃

Anyway, that really clears things up.

I was even more confused by the fact that creating a docker image with RUN mkdir /app instead of RUN echo 'hi' actually complained that the /app dir already existed!

But that must be because of the previous WORKDIR command which caused buildkit to automatically create the folder before running my mkdir layer.

Thanks a bunch for the clarification!