sh: can't open 'run.dev.sh': No such file or directory when I mount pwd in volumes in docker-compose.yml?

[deadcoder0904]

I face this exact error but Idk how to solve it. Idk if my windows firewall is blocking it & if it is, then how would i solve it. for context, i'm not using docker desktop. just docker with wsl2 on windows 11.

If I remove - ./:/app from volumes in docker-compose.yml then everything works fine. i need that .sh script to run pnpm db:migrate:dev to migrate my development database.

I'm dockerizing a remix app in node.js.

docker-compose.yml

version: "3.8"

services:
  redis:
    image: redis:7.2.4-alpine
    container_name: aff_redis_dev
    env_file:
      - ../../.env.development
    restart: always
    ports:
      - 6379:6379
    command: redis-server --save 60 1 --loglevel warning --requirepass 123456abc
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 5s
      timeout: 30s
      retries: 50
    volumes:
      - redis_data:/data
  web:
    image: affiliates:0.0.1
    build:
      context: ../../
      dockerfile: docker/development/Dockerfile
    container_name: aff_dev
    env_file:
      - ../../.env.development
    environment:
      - WATCHPACK_POLLING=true
      - CHOKIDAR_USEPOLLING=true
    ports:
      - 3001:3000
      - 8002:8002
    volumes:
      - ./:/app
      - ../../data:/data
    restart: unless-stopped
    depends_on:
      redis:
        condition: service_healthy
volumes:
  redis_data:
    driver: local

Dockerfile

FROM node:20-alpine AS base

# mostly inspired from https://github.com/BretFisher/node-docker-good-defaults/blob/main/Dockerfile & https://github.com/remix-run/example-trellix/blob/main/Dockerfile

# Check https://github.com/nodejs/docker-node/tree/b4117f9333da4138b03a546ec926ef50a31506c3#nodealpine to understand why libc6-compat might be needed.
RUN apk add --no-cache libc6-compat
RUN corepack enable && corepack prepare pnpm@8.15.4 --activate 
# set the store dir to a folder that is not in the project
RUN pnpm config set store-dir ~/.pnpm-store
RUN pnpm fetch

# 1. Install all dependencies including dev dependencies
FROM base AS deps
# Root user is implicit so you don't have to actually specify it. From https://stackoverflow.com/a/45553149/6141587
# USER root
USER node
# WORKDIR now sets correct permissions if you set USER first so `USER node` has permissions on `/app` directory
WORKDIR /app

# Install dependencies based on the preferred package manager
COPY --chown=node:node package.json pnpm-lock.yaml* ./
COPY --chown=node:node /app/drizzle/migrations ./migrations

RUN pnpm install --frozen-lockfile --prefer-offline

# 2. Start the development server
# Inspired by https://github.com/vercel/next.js/discussions/36935
RUN mkdir -p /app/.cache && chown -R node:node /app/.cache
# Persist the remix cache in a volume
VOLUME ["/app/.cache"]
# COPY --from=deps --chown=node:node /app/node_modules ./node_modules
COPY --chown=node:node . .

# This will do the trick, use the corresponding env file for each environment.
COPY --chown=node:node .env.development .env.development

# Copied from https://stackoverflow.com/a/69867550/6141587
USER root
# Give /data directory correct permissions otherwise WAL mode won't work. It means you can't have 2 users writing to the database at the same time without this line as *.sqlite-wal & *.sqlite-shm are automatically created & deleted when *.sqlite is busy.
RUN mkdir -p /data && chown -R node:node /data

USER node
EXPOSE 3000

ENV PORT 3000
ENV HOSTNAME 0.0.0.0
ENV NODE_ENV development

COPY --chown=node:node public ./public

# Move the drizzle directory to the runtime image
COPY --chown=node:node /app/drizzle/migrations ./migrations
COPY --chown=node:node scripts/drizzle-migrate.mjs ./scripts/drizzle-migrate.mjs
COPY --chown=node:node scripts/package.json ./scripts/package.json
COPY --chown=node:node scripts/pnpm-lock.yaml ./scripts/pnpm-lock.yaml
COPY --chown=node:node scripts/run.dev.sh ./run.dev.sh
RUN chmod +x run.dev.sh

CMD ["sh", "run.dev.sh"]

run.dev.sh

#!/bin/bash
set -e

# Set the directory of the database in a variable
DB_PATH=/data/affiliates.dev.sqlite

# Set to true only when you want to run the migrate script, i.e, when changing database schema
# Automate it by only setting it to true by tracking `entries.idx` in `migrations/meta/_journal.json`
MIGRATE_DATABASE=false

# Inspired by https://pranavmalvawala.com/run-script-only-on-first-start-up & https://serverfault.com/a/1134812/1078165
FIRST_TIME_MIGRATION_DEV="FIRST_TIME_MIGRATION_DEV"
if [[ ! -e /data/$FIRST_TIME_MIGRATION_DEV ]] || [[ $MIGRATE_DATABASE = true ]]; then
	# Place your script that you only want to run on first startup.
  echo 'Initialize database first time only'
	touch /data/$FIRST_TIME_MIGRATION_DEV

	echo "Migrating database '/data/affiliates.dev.sqlite'"
	pnpm db:migrate:dev & PID=$!
	# Wait for migration to finish
	wait $PID
fi

echo "Starting development server..."
pnpm dev

if i remove the volumes mapping - ./:/app from docker-compose.yml, then everything works fine.

but if i add it, i get this error:

sh: can't open 'run.dev.sh': No such file or directory

as said in the other answer, i tried this command:

docker run -it --rm -v "$(pwd):/app" busybox ls -l /app

i got this:

Unable to find image 'busybox:latest' locally
latest: Pulling from library/busybox
7b2699543f22: Pull complete
Digest: sha256:650fd573e056b679a5110a70aabeb01e26b76e545ec4b9c70a9523f2dfaf18c6
Status: Downloaded newer image for busybox:latest
total 293
-rwxrwxrwx    1 1000     1000          1432 Mar 10 10:53 Makefile
-rwxrwxrwx    1 1000     1000            13 Jan 21 06:43 README.md
drwxrwxrwx    1 1000     1000           512 Jan 24 05:42 app
drwxrwxrwx    1 1000     1000           512 Jan 25 16:23 build
drwxrwxrwx    1 1000     1000           512 Mar 10 09:30 data
drwxrwxrwx    1 1000     1000           512 Mar  9 13:11 docker
-rwxrwxrwx    1 1000     1000           265 Jan 25 06:22 drizzle.config.ts
-rwxrwxrwx    1 1000     1000            80 Mar 10 14:17 env.d.ts
-rwxrwxrwx    1 1000     1000           324 Feb 28 07:21 litestream.yml
drwxrwxrwx    1 1000     1000           512 Mar 11 07:27 node_modules
-rwxrwxrwx    1 1000     1000          2574 Mar 11 07:27 package.json
-rwxrwxrwx    1 1000     1000        289002 Mar 11 07:27 pnpm-lock.yaml
-rwxrwxrwx    1 1000     1000            80 Mar 11 07:26 postcss.config.mjs
-rwxrwxrwx    1 1000     1000           247 Mar 10 10:36 prettier.config.cjs
drwxrwxrwx    1 1000     1000           512 Jan 25 16:23 public
-rwxrwxrwx    1 1000     1000           156 Mar 10 11:56 remix.config.cjs
drwxrwxrwx    1 1000     1000           512 Mar 10 12:10 scripts
-rwxrwxrwx    1 1000     1000           171 Jan 19 09:18 tailwind.config.ts
-rwxrwxrwx    1 1000     1000           585 Mar 10 14:18 tsconfig.json
-rwxrwxrwx    1 1000     1000           450 Mar 11 07:20 vite.config.ts

i understood that the error meant that it didn't find run.dev.sh in the top-most directory on my host side but i don't know how to solve the error?

i tried $pwd, $(pwd), $(PWD), ${PWD} & many other variations. finally, settled on ./:/app as it is the provided solution here that works cross-platform.

how do i solve this?

the reason i'm mapping the current directory is because i want hmr/live-reloading to work with remix.

but i have to use docker as i need redis which isn't directly available in windows. of course, i can just start a redis server but i'd like to use docker to control it all.

[luciangabor]

Do you expect that when you mount . inside the container at /app the "old" content of the container's /app is still "there"?

[deadcoder0904]

@luciangabor i'm not mounting node_modules so i don't think windows & linux would matter in this case even though i'm using a native module better-sqlite3. bdw, i simplified the code to use 1 dockerfile, 1 run.sh, & 3 docker-compose.yml. yet, i still get sh: can't open './scripts/run.sh': No such file or directory.

i used this blog as inspiration which kinda does the same thing except it doesn't use sh.

mind you, if i exec into sh & use MODE=development sh ./scripts/run.sh, then it runs fine but sh ./scripts/run.sh gives error as $MODE is used in run.sh. i used command in docker-compose.yml instead of CMD in Dockerfile.

any ideas? code is at https://github.com/deadcoder0904/easypanel-nextjs-sqlite

[luciangabor]

I'd simplify:

1. modify the shebang from the run.sh script - currently it's #!/bin/bash but you don't have bash installed in the image

• I'd settle for /bin/sh (#!/usr/bin/env sh being the more "formal" alternative)

2. I'd chmod a+x the script and add it like that to git - git is perfectly capable of handing file permissions
3. With an executable script, I'd put it directly as entrypoint, it's like this [ "/app/run.sh" ] in Dockerfile (even " matters if I'm not wrong ...), see below for the equiv in compose - this avoids understanding this: https://docs.docker.com/reference/dockerfile/#understand-how-cmd-and-entrypoint-interact
4. I'd set dev-specific environment variables as ... compose environment variables:

services:
  service:
    image: image
    entrypoint:
      - /app/run.sh
    environment:
      MODE: dev

[deadcoder0904]

1. i did that but it was working as is.
2. i checked permissions using ls -al & got -rwxrwxrwx 1 node node 2014 Mar 13 10:40 run.sh. i actually had chmod + x in dockerfile previously in the last from runner but since i put it in docker-compose, i removed it. but i'm not sure i'm understand you here. what does git have to do with this? its already added to my private git repo.
3. i did that using /scripts/run.sh & ./scripts/run.sh and got Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "./scripts/run.sh": stat ./scripts/run.sh: no such file or directory: unknown for both.
4. that is clean but i'm wondering that if it sends that environment variable to sh script. i'll check.

but this didn't solve it.

[deadcoder0904]

so i did command: sh -c "ls -al; sh ./scripts/run.sh;" & got just this:

total 8
drwxrwxrwx    1 node     node           512 Mar 13 05:58 .
drwxr-xr-x    1 root     root          4096 Mar 13 11:57 ..
drwxrwxrwx    1 node     node           512 Mar 12 06:36 .next
-rwxrwxrwx    1 node     node           953 Mar 13 11:57 docker-compose.yml

i wonder how & where this happened. no wonder it doesn't find ./scripts/run.sh i guess.

same with entrypoint: sh -c "ls -al; sh ./scripts/run.sh;"

[luciangabor]

I'm sorry I can't help, as I said before it's pretty hard to follow all the operations and paths ... maybe if you take a look at what's possible now with docker's buildkit you might change your approach - https://youtu.be/saSJa9YVroA - and invest more time in learning about what's new with compose's development support - I haven't used it, but it looks promising and I do think it requires a buildkit-friendly Dockerfile and of that I'm a satisfied user. Here's a another link that might be useful https://docs.docker.com/compose/file-watch/

[deadcoder0904]

fuck it, i couldn't solve it although i've seen many solutions that work for nestjs & nextjs but don't work for me as i use .sh script as i wanted to run pnpm db:migrate:dev before pnpm dev.

i'll revisit it later.

i went with using redis to start the docker container & for everything else, i just start it without docker using pnpm dev for hmr.

working repo -> https://github.com/deadcoder0904/easypanel-nextjs-sqlite