Running container with writable setting

[sudughonge]

I work with docker containers which I run using docker run --env-file <envfile> <docker image name>. The docker container thus instantiation is thus setup to then launch from an entrypoint of /workdir and launch certain python script that's already inside that /workdir inside the docker file system.

I'm trying to replicate the behavior using that same docker image only with singularity (version 3.6.0). Now, I understand that that isn't going to work exactly the same way since docker has its own self contained read-write file system whereas singularity's is read only by default and the writing is usually done by a binding a directory in the host OS with an internal one.

Firstly, I made an .sif image and tried running singularity run --env-file <envfile> <sif image>. When I did that, it doesn't enter into the default /workdir like it does in the docker container and instead starts from the current directory in the host OS which is bound by default. So it's unable to launch the default python script and tells me as much.

That's okay, so I tried doing singularity shell --env-file <envfile> <sif image> and then cd'ed into /workdir and run python <the default launch script>. It gave me an error saying the file system is read only. I tried doing singularity shell -w --env-file <envfile> <sif image> and it said

FATAL:   no SIF writable overlay partition found in /tmp/onscale-reflex.sif

I saw past issues and it said I need to build the image in sandbox and store it a set of layers instead of an .sif file and then open it in shell using the --writable option . But that didn't seem to work either.

Next, I tried mounting the /workdir to the current directory in the host OS but when I open singularity shell again and went to /workdir all the files that are usually there weren't there (which is obvious since I guess the meaning of /workdir got overwritten).

My question is, is there a scenario where I can get this to work? Searching for things like
singularity docker run writable or singularity bind internal directory without losing contents did not show up useful results.

[vsoch]

Hey @sudughonge ! This repository (and actually organization) isn't maintained anymore - so for future help you can post to the Singularity Google Group (singularity@lbl.gov) or the software repository at https://github.com/hpcng/singularity/issues/.

For a quick answer, I would try using --pwd to set the present working directory for the container. It sounds like your python script is needing to write files, so I would next figure out where that is, and see if you can bind a folder from the host there. You could also try using a persistent overlay https://sylabs.io/guides/3.5/user-guide/persistent_overlays.html.

I hope that helps! Please go ahead and post to the issues board for hpcng or the user's list if you need more help.