Create an individual MD5 file for each folder in a main directory

[JonBerry]

The specs for a file delivery I'm making require an individual MD5 file per folder, with the contents of that folder being inside the MD5 file. Such as:

DAY_01 (MAIN FOLDER)
A01
-(files 1-1000)
-md5 for files 1-1000
A02
-files 1001-2000
-md5 for files 1001-2000

And so on. Is there a script or program that can make those MD5 files automatically or do I hace to go and do each one individually? Thanks

[mfram]

You didn't specify what kind of interface you are looking for. Are you comfortable with the shell in Terminal?

Create the following text file in your home directory.

#!/bin/bash

function do_sum
{
	local $ans;

	for f in $@ ;
	do
		if [ -f $f ];
		then
			ans=`openssl md5 < $f`
			echo "$ans $f" >> md5sums.out
		fi
	done
}

s=$(pwd)

cd $1
if [ -f md5sums.out ]; then 
	rm -f md5sums.out
fi
do_sum *
cd $s

If you call this file "do_sums.sh". Then:

chmod +x do_sums.sh
find MAIN_DIR -type d -exec ~/do_sums.sh {} \;

... where MAIN_DIR is the top level directory in question. This script will create a file "md5sums.out" in each directory.

[JonBerry]

I tried that, created the do_sums.sh and put it in the home folder as requested. Ran the terminal command though and got and error like this:

/Tusers/jberry/do_sums.sh: line 1: {rtf1ansiansicpg1252cocoartf1038cocoasubrtf360: command not found
/Tusers/jberry/do_sums.sh: line 2: syntax error near unexpected token `}'
/Tusers/jberry/do_sums.sh: line 2: `{\fonttbl\f0\fswiss\fcharset0 Helvetica;}'

I'm a complete novice when deciphering terminal codes so any help would appreciated. Thanks!

[mfram]

That's RTF, not a text file. And the script takes a parameter, the name of the directory where you want to generate the file.

[JonBerry]

Fixed the rtf issue. Here's what I've been doing, and let me know how completely far off I am:

-copy do_sums.sh (renamed from a text file) into the home directory (under my user)
-open terminal and cd into the main directory above the folder I want to make MD5's of. So:

FOLDER_1
-MAIN FOLDER <---CD into this via terminal)
-Shot1
-Shot2
-etc.

type in the commands you listed with /VOLUMES/V1/MAIN_FOLDER in place of MAIN_DIR

Still get an issues of No such file or directory with regards to the do_sums.sh:

find: /Tusers/jberry/do_sums.sh: No such file or directory.

Any thoughts? Am I in the right place?

[mfram]

Maybe you have spaces in the filenames? My initial script wouldn't have handled that well. Also, be careful of case. Although the filesystem may not be case-sensitive, the shell is often case sensitive. Finally, be sure to double-check the name of your script. Here's another try:

File ~/do_sums.sh

#!/bin/bash

s=$(pwd)

cd "$@"
for i in *; do
	if [ -f "$i" ]; then
		ans=$(openssl md5 < "$i")
		echo "$ans $i" >> md5sums.out
	fi
done
cd "$s"

Make sure to "chmod +x do_sums.sh" the script after it is created.

The lines starting with "$" are the commands entered. The rest are the output. "ls" means list files
"cat" means print the contents of a file.
"find" means to find files and do some operation on them.

$ ls -l ~/do_sums.sh
-rwxr-xr-x  1 user  staff  145 Nov 12 22:39 do_sums.sh*

Make sure the "x" shows up as above.

Here's an example. I'm using an external Flash drive with a FAT filesystem.

$ ls -l -R "/Volumes/NO NAME/test"
total 16
drwxrwxrwx  1 user  staff  4096 Nov 12 22:45 Dir 1/
drwxrwxrwx  1 user  staff  4096 Nov 12 22:45 Dir 2/

/Volumes/NO NAME/test/Dir 1:
total 24
-rwxrwxrwx  1 user  staff  12 Nov 12 22:21 1 - File.txt*
-rwxrwxrwx  1 user  staff  11 Nov 12 22:21 2 - File.txt*
-rwxrwxrwx  1 user  staff  15 Nov 12 22:21 3 - File.txt*

/Volumes/NO NAME/test/Dir 2:
total 16
-rwxrwxrwx  1 user  staff  16 Nov 12 22:22 01 - FF.txt*
-rwxrwxrwx  1 user  staff  14 Nov 12 22:22 02 - FF.txt*

$ find "/Volumes/NO NAME/test" -type d -exec ~/do_sums.sh {} \;

$ ls -l -R "/Volumes/NO NAME/test"
total 16
drwxrwxrwx  1 user  staff  4096 Nov 12 22:46 Dir 1/
drwxrwxrwx  1 user  staff  4096 Nov 12 22:46 Dir 2/

/Volumes/NO NAME/test/Dir 1:
total 32
-rwxrwxrwx  1 user  staff   12 Nov 12 22:21 1 - File.txt*
-rwxrwxrwx  1 user  staff   11 Nov 12 22:21 2 - File.txt*
-rwxrwxrwx  1 user  staff   15 Nov 12 22:21 3 - File.txt*
-rwxrwxrwx  1 user  staff  138 Nov 12 22:46 md5sums.out*

/Volumes/NO NAME/test/Dir 2:
total 24
-rwxrwxrwx  1 user  staff  16 Nov 12 22:22 01 - FF.txt*
-rwxrwxrwx  1 user  staff  14 Nov 12 22:22 02 - FF.txt*
-rwxrwxrwx  1 user  staff  90 Nov 12 22:46 md5sums.out*

$ find "/Volumes/NO NAME/test" -name 'md5*' -print -exec cat {} \;
/Volumes/NO NAME/test/Dir 1/md5sums.out
c6ad45b534cb0b5d94915f70d06805fb 1 - File.txt
ae73b736bf34cbd74f9b1926b16d9815 2 - File.txt
6a48187d882925248e6dd0fe8c907d8e 3 - File.txt
/Volumes/NO NAME/test/Dir 2/md5sums.out
3ad65dd4430b55270c2829d1380c3794 01 - FF.txt
d6e6e2ae2539151395860a3943298806 02 - FF.txt

[JonBerry]

I follow you completely up until this line of code:

$ find "/Volumes/NO NAME/test" -type d -exec ~/do_sums.sh {} \;

I do that and I get this repeated over and over again:

find: /Tusers/jberry/do_sums.sh: No such file or directory

I've def put the do_sums.sh file in the home directory and it matches what you've had me put in there, so I'm just not sure why it won't recognize that the file is in there.

[kryten2]

find: /Tusers/jberry/do_sums.sh: No such file or directory

Unless you created it there shouldn't be a Tusers folder. Your home directory should be in the /Users folder. Open Terminal and post the output of pwd and echo $HOME and ls commands eg :

pwd
echo $HOME
ls -l ~/do_sums.sh

[mfram]

Then take out the "~/" and specify the path directly.

[JonBerry]

I was logged in as a network user. I logged in as an Admin on the system, put the do_sums.sh folder in the home directory and still for the same issue. Here's the output of the code you gave me

-rwxrwxrwx@ 1 tadmin staff 145 Nov 15 09:49 /Users/tadmin/do_sums.sh

Still same error as before, except it's /Users/tadmin. The file is def. where it's supposed to be but terminal just isn't seeing it

Here's the order of what I've been doing:

1. Create the do_sums.sh file, put it in the home directory
2. Run CODE: chmod +x do_sums.sh
3. Navigate to /Volumes/TRAID/PROJECT/DPX
4. Run CODE: ls -l -R "/Volumes/TRAID/Project/DAY_01"
5. Run CODE: find "/Volumes/TRAID/Project/DAY_01" -type d -exec ~/do_sums.sh {} \;

That's what brings up the error:

find: /Users/tadmin/do_sums.sh: No such file or directory

THoughts?ee

----------

Also the output of that reply you had me do was -rwxrwxrwx @ 1 tadmin staff 145 Nov 15 09:49 /Users/tadmin/do_sums.sh"

----------

Tried the specific path and that gives me the same error

----------

The reason there's a Tuser folder is because we're using a domain server log in system, so it's technically a network user. The OS creates it automatically if you log in using the connection to the domain server. I logged in using a user local to the system, but the same issues were there

[mfram]

Either the script is not marked as executable or the drive it is on does not allow things to be executed on it. It doesn't matter where the script is as long as it is marked executable and can be executed. Is there a local directory on the computer where you can put the script and mark it executable with 'chmod'? If so, move it there and try again.

[JonBerry]

I've checked that the file is executable using ls -l do_sums.sh and I get back:
-rwxr-xr-x@ 1 tadmin staff 145 Nov 15 13:42 do_sums.sh

So it's not the files executable status (at least I don't think so) Tried the file on a few different drives, but still nothing.

If the file isn't executable somehow, or just not being read properly, is there another way to do the md5 writing without using that script? Something I can just put into terminal? A command maybe?

[mfram]

Sure, go to the directory with all the files and type "openssl md5 *". You'll get the output. That's what the script is doing. The script is just filtering out all the directories it may encounter. The "find" command is iterating the script over all the directories it sees.

The only other thing I can suggest is make sure the do_sums.sh file has the "#!/bin/bash" line as the first line and there's nothing before the "#!".

[chown33]

Originally Posted by JonBerry

So it's not the files executable status (at least I don't think so) Tried the file on a few different drives, but still nothing.

We'll need more details than that, for anyone here to have a chance of diagnosing it.

Please post the complete actual command-lines you used in Terminal, along with the complete error messages that were output. It's possible you entered the wrong commands, or did something else wrong, which might fail for a completely different reason than the reason for the shell script failing.

There's no way of evaluating the correctness of what you did, without knowing exactly what you did. If you don't have the commands any more, then there's no way to know exactly what was done.


Discovering why the shell script won't run is going to take some digging and diagnostics.

First, we need to see if there's a restriction on your networked login directory that prevents execution.

Enter the following command (copy and paste it into a Terminal window):

mount

Then, copy and paste the complete output into a reply post.

For readability, enclose all posted output in CODE tags, as described here.


Second, post the output of this command:

ls -le ~

Third, run a test as follows. Copy and paste these exact command-lines into a Terminal window:

cd ~
echo '#!/bin/bash' >test.sh
echo "echo '-- Example' \$\$" >>test.sh
chmod a+x test.sh
./test.sh; ~/test.sh

The above commands should make a file named "test.sh" in your home folder. It will then try and run it as a shell script. If it runs, it will output two lines containing "-- Example" followed by a number.

Whether the script runs or not, copy and paste the entire output in a reply post, enclosed in CODE tags as noted above.