Resolved Files or Folders with Non-U.S. filename characters are not shown under OS X

[Thom75]

Hello Everyone,

This is my first post so I would like to say hello before I continue.

I'm trying for longer period of time copy files from my Windows 7 disk to OS X disk with no success. Google giving me no answer at all, so I've decided to ask here.

Whether disk is formatted with NTFS or extFAT file system problem remain the same, files are not seen on OS X site but on Windows 7/Bootcamp are. Under Windows I can access them copy, delete, rename as normal but under OS X there are not visible. I've found that those files contain Non-U.S. characters in names, most of them are with Polish characters but I've found German and French too.

When I trying copy them using Midnight Commander (installed from Home Brew) I can see those files but they file name is displayed in red and size is 0 bytes. Other files are visible normally.

I've tried copy those files over the SMB network from Windows 7 PC but I getting information “file not found” or “file not exist” - I can't remember right now.

I've tried on both Mavericks & Yosemite as well as NTFS read only build in driver and third party RW driver installed.

Setting OS X to PL language not helping at all so even I've tried to rename those files but I can't find any working tool. One I've found replacing only first found letter besides this is not any good solution.

Does Anyone have idea how to solve this problem?

Kind Regards
Thom

 

[Thom75]

I've found workaround how to copy files

I've found workaround how to copy files but it takes twice or more time than copy files directly and required a lot of manual work.
Problem comes with file name encoding witch would be UTF-8.

I've discovered accidentally trying to compress files using Total Commander's build in zip compression that it's asking whether I would like to store files with UTF-8 encoding as files in a folder got not standard characters or other way. Rest files/folders where Total Commander didn't ask for encoding was copied as normal. That works! I got all files copied to OS X site.

BTW. Zip compression level I've used was set to 0. I've tried tar but it reports a lot of read files problems witch zip doesn't.

I hope someone else will find that method useful.

Thom

 

[grahamperrin]

NFD: normalization=formD (normalisation form D)

I'm glad that you found the cause of the problem.

Also worth noting: normalisation.

With ZFS on OS X, for example:

sh-3.2$ zfs get utf8only,normalization gjp22
NAME   PROPERTY       VALUE          SOURCE
gjp22  utf8only       on             -
gjp22  normalization  formD          -
sh-3.2$

From an archive of the ZEVO support forum, originally posted in October 2012:

http://www.ustream.tv/recorded/25862520 around 00:10:33 on the timeline:

… subtle bugs that, I feel like no-one else would appreciate my pain. Like in the Unicode space there's actually two different ways to store, several characters – like an
é on the Mac traditionally is stored as an
e and an ´ character. When it's rendered they composite them.

On any other platform … store composite characters … one click, one character.

So on the Mac, without intervention, you can get into some nasty problems because the Finder stores it one way, Terminal chose a different way. So you can actually go into the Finder and create a directory – café – then go into the Terminal and

touch café

then you have two objects – you have a directory and a file with exactly the same name, which is, it leads to all kinds of … (!) … it looks the same but unlike … where you have differentiator, there's nothing, it's like, and in the Finder, depending on the Finder view you get different experiences. Sometimes you see two folders, sometimes you see a folder and a file, sometimes you see one folder. It's like, it's bizarre. So unfortunately …

… there's a formD-explicit setting so, on the Mac we highly recommend and in fact that's the default, you should use formD so then that problem, you can't do that – when you do the touch it'll actually map it back to the correct way.

You pay a little bit of an overhead but you can keep your sanity. It's crazy to have different stacks using different variants of the encoding.

– Don Brady, amongst panel members at the 2012 Illumos ZFS Day.​

From a July 2013 topic:

Compatibility with other ZFS systems: NFD

… are there any precautions to using that form of normalisation with ZFS with systems such as FreeBSD® or PC-BSD™?

I asked in irc://chat.freenode.net/#zfsonlinux

>> Is any Linux distro (or other ZFS-cabable OS) likely to have difficulty with ZFS file systems that use normalization=formD (normalisation form D)?

Richard Yao responded:

> I use normalization=formD on Gentoo (and include it in my zfs-install notes). I do not have any issues and no one has reported issues.

https://github.com/ryao/zfs-overlay/blob/master/zfs-install​