ATTN: Techies...need Unix questions answered!

Discussion in 'Mac Help/Tips' started by eyelikeart, Jun 22, 2002.

  1. eyelikeart Moderator emeritus

    eyelikeart

    Joined:
    Jan 2, 2001
    Location:
    Metairie, LA
    #1
    Ok, first off, this isn't for me but a friend. He called me last night asking me if I knew anything about Unix, and when we came to the conclusion I couldn't help him, I thought about u guys who can. Here are the questions needing to be answered:

    1) What unix utility would you use to remove all trailing white space from the
    end of every line in a file.

    2) To extract a specified segment of data froma a coma-separated file I.E.
    entry 2 and 3, not knowing the data contained, what unix command would you
    use?

    3) What authentication mechanisms are used with the Berkeley r-commands?(I.E. rlogin)


    thanks ahead for the help... ;)
     
  2. mc68k macrumors 68000

    mc68k

    Joined:
    Apr 16, 2002
    #2
    Re: ATTN: Techies...need Unix questions answered!

    For now w/o research I can answer question 1. I would use the vi editor and its powerful substitution commands for this operation.

    Provided this is a text file that is readable by a text editor like vi, you would type the following:

    vi [filename]

    The next step depends on how much whitespace is at the end of the line. If it's just one character of whitespace type:

    :%s/ $//g
    That's one space after the first slash
    If it's two characters of whitespace type:

    :%s/ $//g
    That's 2 spaces after the first slash
    If 3 or more character of whitespace type:

    :%s/ *$//g
    That's three spaces before the *

    The %s stands for substitue, $ is a metacharacter that means end-of-line, and the g means global. So the net effect of these commands is replacing what is after the first slash with what is after the second slash, throughout the whole document. Take for example the first command-- you replace the space that comes before the $ (end-of-line) with what is after the second slash, nothing. The g at the end just means to do this throughout the whole document.

    The problem might come if he has a dynamic amount of whitespace at the end of each line. If there is other whitespace in the document then you have to be careful not to substitute this as well. Like if it's a text document and you end every sentence with two spaces after the period then a :%s/ *$//g command would put all your sentences right next to each other. If this is a document of a more techincal nature and there are many spaces to seperate things then :%s/ *$// would replace all instances with double spaces with nothing.

    I chose to give you those three commands in the way that I did so that you could choose the best one for the situation. Most documents will not have three spaces in a row, so the last command is pretty fitting to remove any amount of whitespace dynamically from the end of the line.

    :%s
    /[replace this]
    /[with this]
    /g

    Some of the special metacharacters are:

    $ end-of-line
    ^ beginning of line
    * anything # of characters from 0 to ∞
    . any 1 character

    So use the substitute command in conjunction with the metacharacters to suit your needs. Exit out of the vi session by typing:

    esc, ZZ <- to save
    -or-
    :q! <-to exit w/o saving changes

    If any of this is unclear, then ask me to clarify. :) Unix is not easy, and I may have made it harder than it has to be. I know that Taft and szark have helped me in the past w/Unix and there are probably others here as well. I will try to find answers to the other questions. Question 2 is unclear to me.
     
  3. mc68k macrumors 68000

    mc68k

    Joined:
    Apr 16, 2002
    #3
    Re: ATTN: Techies...need Unix questions answered!

    Well, I don't know about the other r commands, but rlogin is as bad as telnet for security. There is no authentication.

    Telnet and rlogin use the same protocol. The only difference is that it doesn't accept an optional connection port and it automatically fills in your user ID on the remote system based on your local system user ID.

    For the sake of security you should use secure-shell software like slogin, scp, sftp, ssh, etc. These have better authentification protocols.
     
  4. Taft macrumors 65816

    Taft

    Joined:
    Jan 31, 2002
    Location:
    Chicago
    #4
    Answers to #1 and #2.

    Use awk to parse through lines of a text file. This can be used to trim the end off of lines as well as break up the line into pieces and get only the ones you want.

    The basic way to interface a file with awk is this:
    Code:
    awk 'pattern {action}' filename
    
    For instance, by leaving pattern blank and specifying an action to print out the line, awk will go through each line and print it out. Like this:
    Code:
    awk '{print $0}' testfile
    
    To put the output of this command into a new file, just redirect it with the > character. Like this:
    Code:
    awk '{print $0}' testfile > resultfile
    
    The previous command essentially copies each line of testfile into resultfile performing a filesystem copy.


    ---------------------------------------------------
    For #1:

    Just print out each line, minus the last character. Like this:
    Code:
    awk '{print substr($0, 1, length-1)}' testfile > resultfile
    
    resultfile will contain the results of taking the last character off of each line. Use length-2 to take the last two characters off of eachline.

    You can also test to see if the last character in the line is a space, so it wouldn't take a character off the end of a line ending in 'g' for example. Use this for your action:
    Code:
    '{if(substr($0,length,length)==" ") print substr($0,1,length-1); else print $0;}'
    
    This will only strip the last character of a line if the last character is a space.

    ---------------------------------------------------
    For #2

    awk also breaks lines into pieces. By default, it breaks up the line by spaces. In the line 'This rocks!', awk would break it into two pieces.

    To get the different pieces you want, use $1 where 1 is the piece number you want to access. $0 means the whole line (we used that in earlier examples). For an enxample:
    Code:
    awk '{print $1, $3}' testfile
    
    This would print the first and third word of every line.

    To specify what character to use as the delimiter, use the -F option. Like this:
    Code:
    awk -F ',' '{print $1, substr($3, 1, 3)}' testfile
    
    This example would print the first element and the first three characters of the third element in each line where the elements are separated by a comma.

    ---------------------------------------------
    Final note:

    You can use the output of other commands on the commandline as the input to awk. For example you can pipe the output of a process listing to awk like this:
    Code:
    ps -ef | awk '{print $8}'
    
    This example prints off the name of every process currently running.


    ---------------------------------------------
    Quiz!!!

    Can anyone come up with a way to kill any current more process using a combination of kill, ps, and awk??

    To test out your answer, open a terminal window and cd to directory with text file in it. Type 'more textfile'. Now open another terminal window and run your kill, ps, awk script. Does it work??



    I hope this quick guide to awk helps. Now I can say I was somewhat productive this Saturday.

    Taft
     
  5. szark macrumors 68030

    szark

    Joined:
    May 14, 2002
    Location:
    Arid-Zone-A
    #5
    Better late than never...

    Sorry I didn't respond sooner, but I'm on vacation. :D

    1) What unix utility would you use to remove all trailing white space from the
    end of every line in a file.


    You could also use sed:

    Code:
    sed -e 's/ *$//g' input_file > output_file
    Note that this will only remove spaces, not other whitespace characters such as tabs.

    2) To extract a specified segment of data froma a coma-separated file I.E.
    entry 2 and 3, not knowing the data contained, what unix command would you
    use?


    Yet another method of doing this is to use the cut command:

    Code:
    cut -f 2,3 -d',' input_file > output_file
    -f specifies the field numbers and -d specifies the field separation character.

    3) What authentication mechanisms are used with the Berkeley r-commands?(I.E. rlogin)

    I agree that rlogin is not usually a secure solution, but it appears (reading the man page) that the OS X version supports some form of Kerberos authentication. Otherwise it just uses a lookup file (.rhosts) to determine which remote users/machines are allowed to connect.
     
  6. Taft macrumors 65816

    Taft

    Joined:
    Jan 31, 2002
    Location:
    Chicago
    #6
    Re: Better late than never...

    I think that the kerberos authentication would only occur if you were using kerberos for authentication against the local network. At least thats the impression I got from the man page.

    In other words, it would only work if you were logging into a machine on your local network where the computers were on a domain authenticated by kerberos. Thats a pretty unlikely situation with OS X.

    I hadn't seen the 'cut' command before. Interestink.

    Taft
     
  7. mc68k macrumors 68000

    mc68k

    Joined:
    Apr 16, 2002
    #7
    Funny how I said you guys would show up— and you did.

    Count on the *nix nerd squad. :)
     
  8. eyelikeart thread starter Moderator emeritus

    eyelikeart

    Joined:
    Jan 2, 2001
    Location:
    Metairie, LA
    #8
    update:

    I just wanted to thank all of u who contributed answers here. I forwarded this to my buddy and it helped him out a lot. He was interviewing for a new job, and needed to know answers to some Unix questions...well he got the job! :D

    thanks again!! ;)
     
  9. sparkleytone macrumors 68020

    sparkleytone

    Joined:
    Oct 28, 2001
    Location:
    Greensboro, NC
    #9
    tell your friend that the internet is his greatest ally. you can find ANYTHING about *NIX just with a simple search for whatever you are looking for at google plus the word HOWTO. its great.
     
  10. eyelikeart thread starter Moderator emeritus

    eyelikeart

    Joined:
    Jan 2, 2001
    Location:
    Metairie, LA
    #10
    he he he...that's exactly what I told him when he called me up asking me about it...

    I told him that I had some great resources on the web for answers...MacRumors! ;)
     
  11. Taft macrumors 65816

    Taft

    Joined:
    Jan 31, 2002
    Location:
    Chicago
    #11
    Re: update:

    Good to hear!!

    I'm just glad that OS X is giving me the opportunity to show off some of the Unix knowledge I gained (quite painfully, I might add) in college. I love showing off my Unix geekiness! Heh.

    Taft
     

Share This Page