OS X Terminal File Names Order and Reference

[jamesadrian]

In writing a bash script in the terminal, it would be helpful if I could refer to either the first file name in a given directory or the last so that I can send that file through a process written in C and then delete it. I notice that ls shows the file names of a directory in the same order every time. Is there a name or a code for the file names in the directory? I cannot know the names of the files in advance and I would like avoid renaming them by hand before the script is used.

Thank you for your help.

James Adrian
jim@futurebeacon.com

[robvas]

Have you looked into xargs?

[sero]

man ls
man head
man tail

first_file=$(ls|head -n1)

[subsonix]

Originally Posted by jamesadrian

In writing a bash script in the terminal, it would be helpful if I could refer to either the first file name in a given directory or the last so that I can send that file through a process written in C and then delete it. I notice that ls shows the file names of a directory in the same order every time.

What order do you want to view them in? Default is alphabetic order, if you look up the manual for ls you will see that there are other options, I use -t quite often which sorts by creation time, newest first for example.

[chown33]

Originally Posted by subsonix

... I use -t quite often which sorts by creation time, newest first for example.

Actually, -t sorts by last-modified time, not creation time.

[jamesadrian]

Originally Posted by jamesadrian

In writing a bash script in the terminal, it would be helpful if I could refer to either the first file name in a given directory or the last so that I can send that file through a process written in C and then delete it. I notice that ls shows the file names of a directory in the same order every time. Is there a name or a code for the file names in the directory? I cannot know the names of the files in advance and I would like avoid renaming them by hand before the script is used.

Thank you for your help.

Maybe it will help if I gave some examples of what I would like to do.

1. move any file from directory A to directory B without know the name of any file in directory A.

2. Copy the first file in directory A to directory B.

3. ./function1 last-file-in-A MYFILENAME-in-B
where A is the current directory and B is some other directory and function1 is a C program requiring two file names - a source and a destination and MYFILENAME is a name I make up myself.

Thank you for your help.

James Adrian
jim@futurebeacon.com

[subsonix]

Originally Posted by chown33

Actually, -t sorts by last-modified time, not creation time.

Of the top of my head.

[chown33]

Originally Posted by jamesadrian

2. Copy the first file in directory A to directory B.

3. ./function1 last-file-in-A MYFILENAME-in-B
where A is the current directory and B is some other directory and function1 is a C program requiring two file names - a source and a destination and MYFILENAME is a name I make up myself.

You need to tell us exactly what you mean by "first" and "last". A directory doesn't have an intrinsic order in any useful sense, so there's no such thing as a "first" or "last" file.

There can be orderings (sorts) imposed on the files in a directory, such as ordered by name, ordered by last-modified date, etc. Given an order, one can determine a first and last item. Without such an imposed order, no first or last can be determined.

Some criteria can't be used to impose an order. For example, there is no easily accessible list of files in the order they were added to a directory (copied, moved, or created there).

1. move any file from directory A to directory B without know the name of any file in directory A.

This is confusing. Do you mean "without knowing beforehand the names of the files in directory A"? Unless the program is allowed to get a listing of the files in a directory, and thus know the names of the files to work on, it's impossible to operate on a file without knowing its name.

[jamesadrian]

Originally Posted by chown33

You need to tell us exactly what you mean by "first" and "last". A directory doesn't have an intrinsic order in any useful sense, so there's no such thing as a "first" or "last" file.

There can be orderings (sorts) imposed on the files in a directory, such as ordered by name, ordered by last-modified date, etc. Given an order, one can determine a first and last item. Without such an imposed order, no first or last can be determined.

Some criteria can't be used to impose an order. For example, there is no easily accessible list of files in the order they were added to a directory (copied, moved, or created there).

This is confusing. Do you mean "without knowing beforehand the names of the files in directory A"? Unless the program is allowed to get a listing of the files in a directory, and thus know the names of the files to work on, it's impossible to operate on a file without knowing its name.

chown33,

If a bash script can be written to discover the first file name in directory A and then use that name in subsequent script commands without me or any person consciously knowing or writing that name, I have a way to proceed. I just can't think of a way to discover the first or last file name in directory A. I imagined that there might be a code name for the first name in a directory that could be used to move that file, but apparently there is no such structure. Any script strategy that allows the moving of one file at a time out of directory A without doing anything by hand is fine with me. I hope that explains my problem. I will be glad to elaborate or answer any questions.

Thank you for thinking about this problem.

James Adrian
jim@futurebeacon.com

[chown33]

Originally Posted by jamesadrian

If a bash script can be written to discover the first file name in directory A and then use that name in subsequent script commands without me or any person consciously knowing or writing that name, I have a way to proceed. I just can't think of a way to discover the first or last file name in directory A.

Already answered in post #3. This is the bash-script code:

first_file=$(ls|head -n1)

It uses the 'ls' and 'head' commands to get the first file name in alphabetical order and assign it to the shell variable first_file. The 'ls' produces a list in alphabetical order. The piped 'head -n1' outputs only the first line. In post #3, he also tells you how to read the man pages for the ls, head, and tail commands. Some of those options might be useful when writing your shell script.

If you move the file out of the directory, then it will no longer appear when 'ls' is run on that directory. So logically, a shell-script loop (see 'man bash') with the above line in it would eventually list every file, move it, and end up with an empty string in first_file.

[milbournosphere]

Originally Posted by jamesadrian

chown33,

If a bash script can be written to discover the first file name in directory A and then use that name in subsequent script commands without me or any person consciously knowing or writing that name, I have a way to proceed.

Use a combo of ls and sed:

ls -1 | sed q

That'll get you just the first line of output from the ls command, which will be the 'first' file name. Using ls options, you should be able to sort the files in the way you need.

Conversly, using some ls magic with some grep goodness will get you the last file:

 ls -1p | grep -v '/$' | tail -1

This appends a control character to the directories, passes the list to grep, which cuts them out (leaving you only the files), then passes to tail, with gives you the last line, which holds the name 'last' file.

You can either store those temporarily, write to file, or redirect to your heart's desire.

man ls, grep, tail, and sed for more help.

[jamesadrian]

Originally Posted by milbournosphere

Use a combo of ls and sed:

ls -1 | sed q

That'll get you just the first line of output from the ls command, which will be the 'first' file name. Using ls options, you should be able to sort the files in the way you need.

Conversly, using some ls magic with some grep goodness will get you the last file:

 ls -1p | grep -v '/$' | tail -1

This appends a control character to the directories, passes the list to grep, which cuts them out (leaving you only the files), then passes to tail, with gives you the last line, which holds the name 'last' file.

You can either store those temporarily, write to file, or redirect to your heart's desire.

man ls, grep, tail, and sed for more help.

milbournosphere,

Thank you for this detailed and very informative reply.

I have been looking into grep. I am in OS X on an iMac. The man grep page does not seem to explain the -v. Also, I don't yet see where I would put my directory names A and B. Directory A has three files in it. Direcotyr B is initially empty. I'm sorry to be so dense, but could you use

 ls -1p | grep -v '/$' | tail -1

with the directory names as an example?

Thank you so very much.

James Adrian
jim@futurebeacon.com

[milbournosphere]

Originally Posted by jamesadrian

The man grep page does not seem to explain the -v.

'-v' flag inverts the logic. '/$' would normally return only the lines ending in '/', but now it will pull all lines EXCEPT those ending in '/' (which is the control character that the '-p' flag on the ls command adds).

Also, I don't yet see where I would put my directory names A and B.

ls -1p /path/to/dir/A | grep -v '/$' | xargs cp /path/to/dir/B

Or something like that...it might need some massaging for weird file names. It uses xargs to parse the file list for cp to use. Of course, this will copy ALL the files in directory A, not just the 'first'. I will leave you to experiment with the hints dropped regarding sed(), head(), and tail() to figure out how to get your first and last files only.

[jamesadrian]

I want to thank all of you for your help on this. The whole thing is making sense.

Thank you for your help.

James Adrian
jim@futurebeacon.com

[peoplevoice]

Take a look at the attached file with the code.

[jamesadrian]

Originally Posted by peoplevoice

Take a look at the attached file with the code.

peoplevoice,

Thank you for this file. It is educating me a lot. I have been using search engines to discover the various meanings of $

I understand things like
x=12
echo "the value of x is $x"

in this case, the value of x is $x and is printed as 12.

I also ran across $1 and $2 which can mean the first and second argument on the command line of a program with two arguments. I hope I have that right.

"if test $# -ne 2" is another matter. I suspect that "test" is a reserved word in this context. I do not know what $# means. Likewise, I do not know what $* means.

Your script contains $i which seams to mean "the value of i."

I would be a long way ahead if you would comment on these things for me.

Thank you for your help.

James Adrian
jim@futurebeacon.com

[chown33]

Originally Posted by jamesadrian

I would be a long way ahead if you would comment on these things for me.

All your questions can be answered by reading the bash man page:
https://developer.apple.com/library/...n1/bash.1.html

Near the top of that page is a link to a Shell Scripting Primer, which you should probably start with. Quoting from its introduction:

You should read this document if you are interested in learning the basics of shell scripting. This document assumes that you already have some basic understanding of at least one procedural programming language such as C. It does not assume that you have very much knowledge of commands executed from the terminal, though, and thus should be readable even if you have never run the Terminal application before.

There are also books on bash, and extensive online references:
http://www.gnu.org/software/bash/manual/

Google search terms: bash reference


The word test is not a reserved word. It's a command, with its own man page (man test). There are other forms that can also be used, such as [ and [[ (refer to the bash man page).

[jamesadrian]

Originally Posted by chown33

All your questions can be answered by reading the bash man page:
https://developer.apple.com/library/...n1/bash.1.html

Near the top of that page is a link to a Shell Scripting Primer, which you should probably start with. Quoting from its introduction:

You should read this document if you are interested in learning the basics of shell scripting. This document assumes that you already have some basic understanding of at least one procedural programming language such as C. It does not assume that you have very much knowledge of commands executed from the terminal, though, and thus should be readable even if you have never run the Terminal application before.

There are also books on bash, and extensive online references:
http://www.gnu.org/software/bash/manual/

Google search terms: bash reference


The word test is not a reserved word. It's a command, with its own man page (man test). There are other forms that can also be used, such as [ and [[ (refer to the bash man page).

chown33,

Thank you for your straightforward answer on the question of the test program.

I have used these links many times and purchased books on bash. I have not found the meaning of $# or $*

With all the usages of $, I would expect a chapter on the subject, but the subject is not organized that way. Can you confirm that $1 and $2 mean the first and second argument on the command line of a program with two arguments?

James Adrian
jim@futurebeacon.com

[chown33]

Originally Posted by jamesadrian

I have used these links many times and purchased books on bash. I have not found the meaning of $# or $*

Exactly what did you try to find their meaning?

If I go to the previously linked man page, and search the page for $1 using the normal Find feature of any browser, the first occurrence is under the sub-heading Special Parameters.

Directly above that sub-heading is one named Positional Parameters.

The higher-level hierarchical heading is PARAMETERS, which begins by defining the terms parameter, variable, etc.

If you're trying to learn shell scripting simply by searching and skimming, I think you're setting yourself up for failure. You should go through the entire bash man page at least once, and note things that are worth returning to.

I also recommend trying things to see what happens (experimentation and observation). Write a shell script. Have it echo $1, $2, $#, etc. Note what happens.

[peoplevoice]

$1 first input parameter or argument
$2 2nd input parameter or argument

$# Number of input parameters or arguments

$* List of all input parameters or arguments

[jamesadrian]

Originally Posted by peoplevoice

$1 first input parameter or argument
$2 2nd input parameter or argument
$# Number of input parameters or arguments
$* List of all input parameters or arguments

peoplevoice,

This is very helpful. Thank you.

I have been under the impression that $1 always refers to the first argument in a command line or script command, but I am starting to get the impression that it could also be the variable name for the first file name in a directory.

Under Special Parameters in https://developer.apple.com/library/...n1/bash.1.html $ is defined as that which "Expands to the process ID of the shell. In a () subshell, it expands to the process ID of the current shell, not the subshell." This will require an extensive investigation.

Thank you for your help.

James Adrian
jim@futurebeacon.com

[chown33]

Originally Posted by jamesadrian

Under Special Parameters in https://developer.apple.com/library/...n1/bash.1.html $ is defined as that which "Expands to the process ID of the shell. In a () subshell, it expands to the process ID of the current shell, not the subshell." This will require an extensive investigation.

You're misinterpreting that part of the documentation. Each one of those characters ($, #, ?, *, @) is preceded by a $ in actual use. So the process ID of the shell is $$, the count of positional parameters is $#, etc.

I again refer you to the Shell Scripting Primer link near the top of the bash man page. It shows examples with explanations.

[peoplevoice]

Adrian,

Are you aware that your C program doesn't need to know anything about filenames and directories? It just needs to read from standard input and write to standard output.

[jamesadrian]

Originally Posted by peoplevoice

Adrian,

Are you aware that your C program doesn't need to know anything about filenames and directories? It just needs to read from standard input and write to standard output.

peoplevoice,

I don't see how to move, remove or copy the first file in a directory (let's call it X) without knowing its file name. I'm really sorry, but I don't see how to use X as a file name in a command or instruction that invokes a C program without knowing the name of X.

James Adrian

[peoplevoice]

Adrian,

The below statement is giving you access to all and each one of the files in the input directory ($1), except directories. Not only the first file as you have been thinking.
--------------------------------------------------------------
for inputFile in `ls -l $1 | grep "^-" | awk '{print $(NF)}'`
do
---------------------------------------------------------------


Your C program redirects the standard input and output through < and > symbols. So inside your C program you don't need to open and close files. This is the reason I say the program doesn't need to know anything about filenames.
---------------------------------------------------------------------------
if yourCProgram < $1/$inputFile > $2/$inputFile
---------------------------------------------------------------------------


If the execution of your C program is okay the file in progress is removed.
----------------------------------------------------------------------------------
then rm $1/$inputFile
----------------------------------------------------------------------------------



Putting everything together

for inputFile in `ls -l $1 | grep "^-" | awk '{print $(NF)}'`
do
if yourCProgram < $1/$inputFile > $2/$inputFile
then rm $1/$inputFile
fi
done