Resolved XCode "invalid character in source file"

Discussion in 'Mac Programming' started by maculateConception, May 28, 2017.

  1. maculateConception macrumors 6502

    maculateConception

    Joined:
    May 28, 2017
    Location:
    Die Bundesstaat Kalifornien
    #1
    I've been learning to program in Swift within the XCode IDE. I'm still really new to it (2 weeks).

    I've noticed a consistent and extremely annoying behavior of XCode that I am not able to understand and/or predictably reproduce and/or resolve.

    At (seemingly) random times, XCode complains that there is an invalid character in the source file. And, to "fix it", it suggests inserting a single whitespace character. I tried, just now, to type some code in XCode in an attempt to reproduce the error, but was unsuccessful. I have not been able to detect a pattern in the occurrences so as to be able to reproduce it. The offending character seems to appear near parenthesis or braces, but not always. Sometimes, it is right in the middle of a word.

    I looked online. I found only a small handful of articles that talk about this issue. They suggest that some keyboard shortcut is entering the offending character ... some shortcut involving "Alt". But, the thing is, I don't use any shortcuts in XCode, because I don't even know any !!! So, how could I be entering these offending characters ?

    Please help ! This is driving me bonkers, to the point of considering writing Swift code in Eclipse (they're writing a plugin for Swift code). Thank you !
     
  2. chown33 macrumors 604

    Joined:
    Aug 9, 2009
    #2
    I don't know a way of discovering the problem without looking at the file data. So the next time it happens, save the file, zip it, and upload it as an attachment to a post here.

    If you're copying and pasting code from PDFs, online articles, etc. then you can run into illegal characters in the text you're copying from. One that's pretty common in PDFs and online articles is curly quotes. For example, see these:
    https://forums.macrumors.com/threads/why-wont-this-code-run-total-beginner.967986/#post-10540547
    https://forums.macrumors.com/threads/keychain-password-removal-command.2017004/#post-23968019
    https://forums.macrumors.com/thread...lipse-compile-properly.1961462/#post-22674628
    https://forums.macrumors.com/threads/problem-setting-variables-using-echo-sed.1882465/#post-21324378

    The problem is that you sometimes can't easily see the difference in the Xcode editor. A large font size in Xcode, even temporarily, may be enough to show a visible difference, but not always. Then you have to use a different approach, such as the command-line tool 'hexdump -C'.

    In addition to curly quotes (both single and double), I've seen:
    - a non-breaking space (typical keystroke: option spacebar)
    - a zero-width space (no keystroke I know of)
    - centered dot (resembles a period/dot but above the baseline)
    - the ≠ sign (typical keystroke: option = )
     
  3. maculateConception thread starter macrumors 6502

    maculateConception

    Joined:
    May 28, 2017
    Location:
    Die Bundesstaat Kalifornien
    #3
    chown33 - Thank you very much for the response and great info.

    I will check out the links you shared and try to reproduce the issue using the possible illegal characters you listed and upload the code as a zip when the problem shows up again.

    Thanks !
     
  4. Senor Cuete macrumors regular

    Joined:
    Nov 9, 2011
    #4
    Edit menu->Select All. Edit menu->Copy. Open a new window in a source code editor that will show all characters, like BBEdit. Edit->paste. Select show invisible characters in the preferences pane. Look for non ASCII characters.
     
  5. maculateConception thread starter macrumors 6502

    maculateConception

    Joined:
    May 28, 2017
    Location:
    Die Bundesstaat Kalifornien
    #5
    Gracias, Senor. I did exactly what you suggested, and I've found out what that character is. Please see the attached images (3 images) and main.swift file (renamed as SwiftCode.txt, and containing the offending character).
     

    Attached Files:

  6. chown33 macrumors 604

    Joined:
    Aug 9, 2009
    #6
    Thanks for posting all that. You misinterpreted a few things, so I'll post my analysis first, then go into those mistakes.

    First, here's a hex dump of the file you uploaded.
    Code:
    hexdump -C /Volumes/Browser/Browser-downloads/SwiftCode.txt
    
    00000000  2f 2f 0a 2f 2f 20 20 6d  61 69 6e 2e 73 77 69 66  |//.//  main.swif|
    00000010  74 0a 2f 2f 20 20 44 72  61 77 65 72 0a 2f 2f 0a  |t.//  Drawer.//.|
    00000020  2f 2f 20 20 43 72 65 61  74 65 64 20 62 79 20 20  |//  Created by  |
    00000030  35 2f 32 37 2f 31 37 2e  0a 2f 2f 20 20 43 6f 70  |5/27/17..//  Cop|
    00000040  79 72 69 67 68 74 20 c2  a9 20 32 30 31 37 20 20  |yright .. 2017  |
    00000050  72 69 67 68 74 73 20 72  65 73 65 72 76 65 64 2e  |rights reserved.|
    00000060  0a 2f 2f 0a 0a 69 6d 70  6f 72 74 20 46 6f 75 6e  |.//..import Foun|
    00000070  64 61 74 69 6f 6e 0a 0a  66 75 6e 63 20 74 65 73  |dation..func tes|
    00000080  74 28 29 20 7b 0a 20 20  20 20 0a 20 20 20 20 6c  |t() {.    .    l|
    00000090  65 74 20 61 72 72 3a 20  5b 49 6e 74 5d 20 3d 20  |et arr: [Int] = |
    000000a0  5b 31 32 2c 20 34 2c 20  36 2c 20 31 37 2c 20 32  |[12, 4, 6, 17, 2|
    000000b0  32 2c 20 33 2c 20 35 2c  20 37 39 5d 0a 20 20 20  |2, 3, 5, 79].   |
    000000c0  20 70 72 69 6e 74 28 61  72 72 29 0a 20 20 20 20  | print(arr).    |
    000000d0  73 6f 72 74 28 61 72 72  29 0a 20 20 20 20 70 72  |sort(arr).    pr|
    000000e0  69 6e 74 28 61 72 72 29  0a 20 20 20 20 6c 65 74  |int(arr).    let|
    000000f0  20 69 6e 64 65 78 20 3d  20 62 69 6e 61 72 79 53  | index = binaryS|
    00000100  65 61 72 63 68 28 61 72  72 2c 20 35 29 0a 20 20  |earch(arr, 5).  |
    00000110  20 20 70 72 69 6e 74 28  22 49 6e 64 65 78 3d 22  |  print("Index="|
    00000120  20 2b 20 53 74 72 69 6e  67 28 69 6e 64 65 78 29  | + String(index)|
    00000130  29 0a 7d 0a 0a 66 75 6e  63 20 73 6f 72 74 28 61  |).}..func sort(a|
    00000140  72 72 3a 20 5b 49 6e 74  5d 29 20 7b 0a 20 20 20  |rr: [Int]) {.   |
    00000150  20 0a 7d 0a 0a 66 75 6e  63 20 62 69 6e 61 72 79  | .}..func binary|
    00000160  53 65 61 72 63 68 28 61  72 72 3a 20 5b 49 6e 74  |Search(arr: [Int|
    00000170  5d 2c 20 74 61 72 67 65  74 3a 20 49 6e 74 1b 29  |], target: Int.)|
    00000180  20 2d 3e 20 49 6e 74 20  7b 0a 20 20 20 20 72 65  | -> Int {.    re|
    00000190  74 75 72 6e 20 2d 31 0a  7d                       |turn -1.}|
    00000199
    
    The first line shown is the Terminal command line. The pathname refers to the file in its location on my machine. Yours will be in a different place on your machine.

    You can replicate this hex dump by typing hexdump -C followed by a space, then dragging the file you want dumped over the Terminal window and dropping it. Terminal will fill in the path name. Then press RETURN in the Terminal window. Voila.


    The line where the Xcode error occurs is the one that starts 00000170. The character between Int and ) is the offending character. It happens to be 0x1b, not the "inverted question mark" you noted.

    0x1b is an ESC character, which isn't something one would usually type by accident. However, if you're using the Touch Bar Macbook Pro, maybe you're accidentally touching the ESC icon.

    I can't even type ESC in my Xcode here, because it's bound to a symbol-matching popup action using Xcode's key bindings. You should probably check whether you've got a key binding for ESC, and if not, at least bind it to some action so it doesn't get inserted into the text.


    I completely forgot that Xcode is capable of showing space characters. It should be one of the menu items under its Editor menu. Look for something like "Show Spaces" or "Show Whitespace", although it may be called "Show Invisibles".

    You haven't told us which Xcode or OS version you're using, so knowing that may help us give you better info about how to do things in Xcode.


    Your interpretation of what the offending character is went wrong in two ways.

    First, BBEdit doesn't actually show characters that have no displayable glyph. If it did that, then the centered dot wouldn't be shown for SPACE, and ¬ wouldn't be shown for NEWLINE. Instead, it shows proxies for invisible characters, and draws them in gray so you can distinguish an invisible from a literal visible character You incorrectly concluded that just because BBEdit shows a ¿ for an invisible character, that the code in the file is for that character.

    Second, you used the wrong character encoding table for deciphering what the code for ¿ should be. BBEdit tells us that the file is being interpreted as UTF-8 Unicode (see its status bar at bottom), so the "extended ASCII" table simply isn't relevant.

    You can even confirm that BBEdit is telling us the truth about UTF-8 by looking at this line in the hex dump:
    Code:
    00000040  79 72 69 67 68 74 20 c2  a9 20 32 30 31 37 20 20  |yright .. 2017  |
    
    The sequence c2 a9 is the © symbol represented as two UTF-8 bytes. You can manually write out the bits and form them into a full Unicode code-point, and confirm it's the © symbol.


    If I were still teaching programming classes, I'd give an assignment at this point to write a program that scans an input file for any bytes not in the range of 0x20-0x7E, and displays them in a unique and visible way. Control characters 0x00-0x1F should appear as ^ followed by the non-control code (e.g. 0x01 would show "^A"). Newlines produce both the "^J" form and a newline. Bytes 0x7F-0xFF should appear as "\xXY" where the "\x" is literal, and XY is the hex representation of the byte. All other characters appear as themselves on output. The program should read from stdin and write to stdout. This assignment is worth 10 points.
     
  7. maculateConception thread starter macrumors 6502

    maculateConception

    Joined:
    May 28, 2017
    Location:
    Die Bundesstaat Kalifornien
    #7
    Wow, thank you so much, chown ! You really went above and beyond, with your analysis. Much appreciated !

    I've been programming for many years (Java), but never understood character encoding all that well, except that ASCII is a single byte representation and Unicode formats use multiple bytes. Thanks for the detailed info.

    I'm running XCode 7.0 on OS X 10.10.5 (Yosemite).

    I do type ESC intentionally in XCode, sometimes, usually to escape out of its code completion / content assist, as I find it to be a bit intrusive. I can't think of any other times when I type ESC.

    I will try to reproduce the issue with ESC (content assist), and let y'all know if I'm able to circumvent the problem.

    Thanks again !
     
  8. chown33 macrumors 604

    Joined:
    Aug 9, 2009
    #8
    You should be able to turn it off completely. Or try using ⌘. instead of ESC.

    I'm not using Xcode 7, so I can't give you specific details, but I've always been able to turn off all of Xcode's automatic code completion, even if I have to hunt through menus or preferences, or even just google some keywords (like "disable xcode 7 code completion").
     
  9. maculateConception thread starter macrumors 6502

    maculateConception

    Joined:
    May 28, 2017
    Location:
    Die Bundesstaat Kalifornien
    #9
    Thanks, but I was hoping to be able to still use code completion, as I'm relatively new to Swift and the assist definitely does help sometimes. I've programmed for many many years with Eclipse, and never ran into an issue like this, which is why I was so dumbfounded about this.

    I mean ... I could disable code comp. but I shouldn't have to do that, right ? I should still be able to leverage the code completion feature, without it screwing up my source file. I.E. this seems like an XCode bug, not something I'm doing wrong.

    I guess I'll just have to play around with XCode to get the behavior I want.

    Thanks again.
     
  10. chown33 macrumors 604

    Joined:
    Aug 9, 2009
    #10
    It doesn't seem like an Xcode bug to me. It seems like you're sometimes accidentally hitting ESC after a popup has already been dismissed, and that's then being inserted into your text file.

    ESC is a perfectly valid ASCII / Unicode character. If you haven't bound the keystroke to some command function, then it would be wrong of Xcode to not put it in your file, when that's what you type in.

    The ⌘. I suggested above is NOT a character that can be inserted into a text file, because it has the ⌘ modifier pressed. If Xcode hasn't bound that keystroke to a command function, then it will probably beep at you.

    I strongly recommend either switching to ⌘. to cancel code-completion popups (confirm it works in your Xcode version), or using Xcode's key bindings to bind the ESC key to a command function, so it won't get inserted into text.

    I also recommend looking under Xcode's Editor menu for what it can do about showing invisible or hidden characters. Older Xcode versions had a "Show Control-Characters" option that showed control-characters 0x00-0x1F as ^ followed by a visible letter. ESC would show as ^[, basically what I described in the "assigned program worth 10 points" in my earlier post.
     
  11. maculateConception, May 31, 2017
    Last edited: May 31, 2017

    maculateConception thread starter macrumors 6502

    maculateConception

    Joined:
    May 28, 2017
    Location:
    Die Bundesstaat Kalifornien
    #11
    Let's look at the problem from the standpoint of a user. I have used numerous text/HTML/hex editors / IDEs over the past 17 years of my life, pressing ESC a billion times during that time period. And, none of those editors has ever inserted a single foul character in my files.

    So, why then, do we have to make an excuse for this ****** XCode behavior ?

    If it is not a bug, at the very least, it is a horrible user experience. When I press the ON button on a TV, I expect it to turn ON. When I press ESC on a keyboard, I expect it to behave like ESC normally behaves in any app. I.E. get me out of whatever menu is currently in focus, and if nothing is in focus, don't do anything.

    Sorry, I think I have to disagree with you about this ... this is at best a crappy XCode user experience, and at worst, a bug.
     
  12. chown33 macrumors 604

    Joined:
    Aug 9, 2009
    #12
    You've pressed ESC a billion times because the UI allows it. If that UI was on Windows, then it differs from the Mac, in ways not always immediately apparent. You may well expect it to work that way based on your experience, but depending on what that experience was based on, it might not be a justified expectation on the Mac (or in a programming editor on the Mac).

    I almost never use ESC to mean "Cancel". I almost always use ⌘. , and not just because the ESC key is up and away from home keys while ⌘. is low and two-handed I do this based on my experience of how things work on the Mac. Our experience differs, so it should be unsurprising if our expectations also differ.

    I've already mentioned that Xcode supports key bindings, so you can look into that.

    I've also mentioned about looking into Editor viewing preferences, which might be able to show you unexpected control characters present in your file.

    You might be able to setup a key binding using System Preferences (Keyboard Shortcuts), so Xcode always treats ESC the way you want.

    The choice of a programming editor can be deeply personal and its expressiveness deeply ingrained, so I fully understand the desire to make it work to meet your needs, rather than changing your work habits to meet its needs.

    At this point, I have no further suggestions about how to fix the problem. Xcode is what it is, and I've summarized the customization its editor offers. You can file a bug-report on it if you wish. I see no value whatsoever in arguing over whether its behavior with ESC is a bug or not.

    If you dislike Xcode sufficiently, and can't customize it to your liking with the menus, dialogs, key bindings, view options, etc. it offers, then you may be better served with a different editor. BBEdit is quite nice, and is a lot more readily customizable than Xcode's editor. YMMV, of course.
     
  13. maculateConception thread starter macrumors 6502

    maculateConception

    Joined:
    May 28, 2017
    Location:
    Die Bundesstaat Kalifornien
    #13
    chown - I agree that it's pointless to argue. Since you refuted my claim that it's a bug, I just wanted to mention where I'm coming from and my reasons.

    Yes, thanks again, for your help. I will look into customizing XCode to work around the problem, as it is a great tool otherwise.
     
  14. cqexbesd macrumors regular

    Joined:
    Jun 4, 2009
    #14
    FWIW I can't easily enter a literal escape into Xcode's editor (8.2.1) no matter what order of keypresses I do before and after the ESC key.

    Have you managed to get a reproducible test case to insert ESC?
     
  15. maculateConception thread starter macrumors 6502

    maculateConception

    Joined:
    May 28, 2017
    Location:
    Die Bundesstaat Kalifornien
    #15
    Thanks. That is further proof that this behavior is abnormal. Unfortunately, I'm running OS X Yosemite, and cant' run anything higher than XCode 7.2.

    I haven't tried to reproduce the issue with ESC yet. Will post updates. Thanks.
     
  16. maculateConception thread starter macrumors 6502

    maculateConception

    Joined:
    May 28, 2017
    Location:
    Die Bundesstaat Kalifornien
    #16
    This issue has been resolved.

    How ? I'm no longer using XCode 7. I'm using XCode 8.3 :D
     

Share This Page