Filling free space with single file ..

Discussion in 'Mac Apps and Mac App Store' started by Shrek-Moscow, Dec 15, 2009.

  1. Shrek-Moscow macrumors member

    Joined:
    Apr 11, 2008
    #1
    Hello guys,
    I'd like to find a program, a script, a command line or whatsoever any other solution that will allow me to write a single file that will set all available free bits to "I".

    It seem that this will be a good compromise for restoring decent performance on SSD that has already been filled up by sequential writings. Sadly the option present in disk utility "delete free space" seem to be ineffective (and is setting everithing to "0").

    I'd like to boot from a USB pen drive and than run this task on main disk setting all free bite to "I" until is totally out of space, than to delete the new file.

    Any ideas about how to?

    :confused:
     
  2. mstrze macrumors 68000

    Joined:
    Nov 6, 2009
    #2
    And this will help you how?

    How would this be any different from setting it all to "0"? If the drive had some 'bad sectors' as it were, they won't be affected by writing '1' anymore than '0'....or is the assumption that somehow you are turning them ON instead of switching them OFF?

    (And I believe you are seeking something to 'write' "1", the number one, not the letter 'I' FWIW)
     
  3. GGJstudios macrumors Westmere

    GGJstudios

    Joined:
    May 16, 2008
    #3
    That would have absolutely zero effect on your SSD performance. Mac OS X does a much better job of managing your drive's space than you could do.
     
  4. Shrek-Moscow thread starter macrumors member

    Joined:
    Apr 11, 2008
    #4
    Yes, I'd like to write a file containing all "1", or to set all free bits to "1", than to cancel this file. This will help a lot, as it appear from a day of internet research, but don't ask me a detailed explanation..

    However it seem that SSD write "0" when requested to write "1" and vice versa. When a bit is physically marked 0 and is not part of a file (once the file is deleted) it looks empty and don't need to be erased before re-written, this will help to restore a good writing performance on an SSD that has already been filled up from previous writings.

    There are already some win-based programs to perform this task and they are effective on several kind of SSDs,
     
  5. angelwatt Moderator emeritus

    angelwatt

    Joined:
    Aug 16, 2005
    Location:
    USA
    #5
    I think you should just move on as you'll likely just corrupt things or cause a system halt as you don't appear to know what you're doing. No offense, just trying to save you some grief.
     
  6. GGJstudios macrumors Westmere

    GGJstudios

    Joined:
    May 16, 2008
    #6
    That's because Windows doesn't manage hard drive space as effectively as Mac OS X. What you're attempting to do is unnecessary and will have no effect on your drive's performance under Mac OS X.
     
  7. CylonGlitch macrumors 68030

    CylonGlitch

    Joined:
    Jul 7, 2009
    Location:
    SoCal
    #7
    On a write an SSD device has to do a read modify write operation if the write does not fill the entire block of RAM. This takes time, but has to be performed regardless of the data in the RAM. This is because the SSD memory operates in blocks, all accesses either read a block, or write a block of memory. If you're writing to a block of memory but are not filling the block, the controller has to read the entire block, replace the components that you are writing, and then write the entire, modified, block back to the memory. This is normal operation and the controllers have this functionality built into them; thus to the end user, and even the OS, they don't see this operation. But it does mean that writes are slower.

    What the OP is saying is that by filling the drive with 0xFF bytes (all 1's) it will reset the controller in some way so that it doesn't have to read the block before doing the write because it will know that block is empty. Well, sounds all good, but how does the controller know this? There MIGHT be come blocks of reserved memory used to indicate which blocks have been used and which ones haven't; but that doesn't mean that filling the memory with 1's will reset it. If they are using something like this, it may be in another NVRAM location or a reserved section that you can't write to (don't want some file filling that location).

    As to writing 1's or 0's, that's depends on the technology; right now most drivers are using NAND and thus writing 1's does effectively write 0's to the RAM. BUT the storage of 0's is more power hungry then storing 1's (just the way the technology works). Thus filling the drive with 1's (inverted to 0's) will draw more power then if it was filled with 0's (inverted to 1's) -- I know, sounds backwards but it isn't. Drawing more power into the chip puts more stress on the junctions and the other units on the die and could bring about premature failure. Reading and writing inverting patterns will draw the most power during the operations, but having the "drive" filled with 0's will draw the most steady state power.

    I did a quick google search for this and found nothing that indicated this would help your write cycles; could you please link the websites that you've been reading because I don't think it will truly help. Possibly writing directly to the controller and resetting itself that way might work; but I can't see the data on the "drive" being an issue.
     
  8. Shrek-Moscow thread starter macrumors member

    Joined:
    Apr 11, 2008
    #8
    Do you know something about the very specific SSD problem of decreasing performance after they had been used for a while or you are just talking about general HDD related problems (like fragmentation and co)?

    This performance drop is not fully recovered even re-initializing the disk. The only way is to phisically empty all unused bits so that SSD controller will not have to erase them before re-writing new files.

    All people using SSDs, even on this forum, are experiencing some drop in performance after some time of usage depending on the amount of free space left on their SSD. The original performance cannot be restored even after deleting unused files to free SSD space since only index are deleted but not bit content (and then bit need to be deleted before being rewritten, a kind of process incredibly slower than simple writing, especially on ML technology).

    Reconditioning "used" SSDs mean setting all unused bits to a physical empty status, that on SSDs seem to be a logical 1 (instead of logical 0 of HDDs).

    there are tons of informations about on Internet, just have a look about.
    By the way, I'm using TimeMachine since ever and I'll be able to restore my disk in case I'll mess the OS.

    Please, if you have any idea about how to reply my initial request, just do it.

    :)

    CylonGlitch:

    what you say is correct, however the big drop in performance is related to what I just wrote in my previous post. Now I must really leave, can't do anything since tomorrow!
     
  9. CylonGlitch macrumors 68030

    CylonGlitch

    Joined:
    Jul 7, 2009
    Location:
    SoCal
    #9
    What I wrote was only relevant to SSD devices, not HDD. I did do a quick search for this and found nothing; as I mentioned. Please link the websites you are referring to. I have no performance lose on my SSD devices. The drive still needs to perform a read modify write every time you do a write unless the block is completely written. Some technologies, or some controllers may write zeros to the block before writing the new block back. But in this case I wouldn't think the controller would try to detect if the block is blank first because this operation inside the controller would actually take quite a while to perform, checking every bit to verify it is unset (either 1 or 0 depending on the technology of the storage) and then skipping the write of a blank block back.

    Let's say that the tech requires a blank block when writing. In the SSD's that I worked with this isn't the case; but let's assume it is here. From the controller side, you get a command to write X bytes to a block; here is what the flow of operations would look like.

    1) Read Block
    2) Merge Block with Writing block (however many bytes are being written)
    3) If block Read is all 0's, go to Step #4, else Write a block of 0's (in this case to initialize the block)
    4) Write the new block from step #2.

    What you are claiming is that step 3 can be skipped if the block is already 0'ed. (NOTE I am saying 0's for sake of understanding).

    What I claim is that step #2 and #3 can happen in parallel. Step #2 is the long pole here, merging X bytes together takes quite a bit of time where as one full block write is actually quite quick. Thus as a developer you would code the controller to do both operations at the same time and thus increase throughput to the chip. Internally in the controller you just have a preset block hard coded and dump that to any block that needs it -- it's quite easy to do. Thus in this case, the checking if all bytes in the block that was just read, which in itself takes a good amount of time, can be skipped all the time if you just assume that you ALWAYS have to write a fixed block back. Now, with the technology being so fast and small, it is possible to OR all of the bits of a block together and find out in one clock cycle if the write needs to go through, then that could be useful but it would require significant hardware to do this.

    Again, please point me to the articles you're reading, I would like to see this.
     
  10. CTechKid macrumors regular

    Joined:
    Sep 9, 2008
  11. CylonGlitch macrumors 68030

    CylonGlitch

    Joined:
    Jul 7, 2009
    Location:
    SoCal
    #11
    Interesting article, although it talks about using a specific drive with a specific chipset. After reading the article and thinking about it, it does seem that there is more going on then what they understand. The internal fragmentation of the device is very different then the OS system fragmentation this internal fragmentation is called the wear level. Different types of controllers handle this differently and it seems that the reviewer has found that there really is an issue with the way the Intel controller handles huge amounts of reads and writes over time. What the wear level is used for is that each block of a SSD has a wear level associated with it. As data is written to the device the controller remaps the drive internally so that, although it looks like you are writing to the same area over and over again, you really are not, it is spread out across the SSD so that you don't experience failures as quickly or as often.

    It looks like the Intel controller has something wrong with it and over time as you perform significant write, erase, write cycles the controller gets lost or does something odd. Now Intel is not the only controller and after looking at what I see, this is really specifically targeting the Intel controller. I wouldn't be surprised if other controllers have this problem; or something similar to it. In fact I could see how this happens. As the wear level gets worse over the writes, and blocks start to fail the device has to compensate and thus things are really spread out all over the drive. Interblock reads and writes are typically fast, but blocks all over the place can take longer to write.

    What this does, it looks like, is to cause the controller to reset it's internal wear level and thus "increasing" performance. At the same time though, by doing this, you're resetting he wear level, specifically for the high write blocks and could cause the chips to fail sooner.

    Thanks for the info, interesting read. I would like to see how other controllers work.
     
  12. Shrek-Moscow thread starter macrumors member

    Joined:
    Apr 11, 2008
    #12
    Ok, but that program is not free ware and, after all, I just need the "write one file function", and is better if I can decide which content to insert in the file (like all 0 or 1).

    I don't need graphs or statistics or other functions. I could pay something for a stupid program with a clean GUI who can:
    1 - recondition a non-boot disk deleting everything.
    2 - recondition free space on a boot or non boot disk.

    And that's designed to do just this. I don't want to pay for a "Swiss knife" just because I need his plastic toothpick..
    More complicate programs are welcome just if free ware or if available in 30 days trial version (I want to test them well and however I'll need to recondition my disk every sometimes)
     
  13. Shrek-Moscow thread starter macrumors member

    Joined:
    Apr 11, 2008
    #13
  14. Detektiv-Pinky macrumors 6502a

    Detektiv-Pinky

    Joined:
    Feb 25, 2006
    Location:
    Berlin, Germany
    #14
    Use the command line to quickly generate large files:

    dd if =/dev/zero of=zero.bin bs=1k
    quickly fills up your harddisk with a file containing only zeros.

    You probably want to limit the size of the file somehow. You can use count to determine how many 1k blocks should be generated.
    dd if =/dev/zero of=zero.bin bs=1k count=10

    If you need to write other bit patterns to the disk you could try to substitute characters in the data stream:

    tr '\000' '\3ff' < /dev/zero | dd of=zero.bin bs=1k count=10

    However this somehow not quite produces the 'ff' character (or All-Ones-Bitpattern) on my Mac - it works on Linux through.
     
  15. CTechKid macrumors regular

    Joined:
    Sep 9, 2008
    #15

    In your haste to respond, you forgot to fully read the link I posted; its clear you are not understanding what this suite of tools accomplishes. I encourage you to return to the page and reread its entire function set.

    If all you're after is free tools, regardless of their use and function, then disregard this entirely.
     
  16. Shrek-Moscow thread starter macrumors member

    Joined:
    Apr 11, 2008
    #16
    Thanks a lot! :)

    A few more questions:

    • Changing "bs=1k" to "bs=4k" will write 4k blocks instead of 1k blocks? (it seem that on SSD standard pages are made of 4k, so probably there's no reason to write 1k blocks since the minimum writeable is 4k)
    • Can you tell me more about tr '\000' '\3ff' syntax? why 000 to 3FF? is it related to the block size of 1kb?
    • Writing 4k blocks should I use '\000' '\FFF'?
    • Can I reduce this command to '\00' '\FF' even maintaining 1 or 4k blocks?

    Now I can't check because my Mac is far.. later I'll make a few initial tests writing just a small number of blocks..

    :)


    Thanks again!
     
  17. Shrek-Moscow thread starter macrumors member

    Joined:
    Apr 11, 2008
    #17
    I read it and I decided that I personally want and need JUST the ability of filling my SSD free space of "1" (physical "0"). I don't care of other thinks or functionalities. I did read also this page and I found nothing interesting for me except the ability of filling the disk with a huge single file.

    Now it happen that I can do this simple function with the standard Mac OS X command line.

    And, just to be clear, all my SW is original, I do buy applications when I need them! And when I'm in doubt, like this time, a free trial can eventually help to make a final decision..
     
  18. Detektiv-Pinky macrumors 6502a

    Detektiv-Pinky

    Joined:
    Feb 25, 2006
    Location:
    Berlin, Germany
    #18
    Yes, change the blocksize to whatever is sensible. I think you have to experiment with the settings.

    /dev/zero is a character device that gives you a never-ending stream of zeroes. If you want to write anything other into your very large file, you need to convert these zeros to some other bitpattern. That is what tr does. It sees a Zero '/000' in octal notation and converts it in the character that has all the bits set to 1 '/3ff' (again in octal).

    You can look at the bitpattern in your binary files with od (od -h zero.bin).
    Just a note, on my Mac tr did not set all the character bits to One as exected, but maybe this is good enough for what you are trying to do.
     

Share This Page