How can I find which software dumping log.txt files in my Documents folder?

[NJRonbo]

I have a lot of software installed on my computer so troubleshooting this issue has been cumbersome.

SOMETHING is dumping log.txt files in my DOCUMENTS folder to the tune of 1 every few minutes. I have thousands of them that I have to clean out every week.

Is there a way to find the source of these log files? I did right-click -> INFO on the file but it doesn't tell me its source.

Thanks

 

[iStorm]

Is there no content in the files to give you clues?

Otherwise, you can try the fs_usage terminal command.

How to determine which process is creating a file?

Two files have suddenly appeared in my home directory, called "aa" and "err". They are empty. I'm wondering how they got there. I deleted the files and they are created again after some seconds. Is

apple.stackexchange.com

 

[NJRonbo]

Is there no content in the files to give you clues?

I'm sorry, I'll include it below:

2023-05-15 15:11:21 +0000 : Internet Connected
2023-05-15 15:11:30 +0000 : Refresh Stauts
2023-05-15 15:11:30 +0000 : Draw Menu
2023-05-15 15:11:30 +0000 : Sorted Interfaces : (
en1,
en0
)
2023-05-15 15:11:30 +0000 : Internet Connected
2023-05-15 15:11:35 +0000 : Refresh Stauts
2023-05-15 15:11:35 +0000 : Draw Menu
2023-05-15 15:11:35 +0000 : Sorted Interfaces : (
en1,


(and it goes on and on)

 

[chown33]

Judging by the "Draw Menu" line, my guess is that it's a menubar extension or status control of some kind. The fact that it's concerned with network interfaces suggests something related to the network or internet. This includes things like Little Snitch, or possibly even VPNs.


If we assume that whatever process is writing to the file is keeping it open the whole time, then the following command line pasted into a Terminal window might show it:

lsof | grep /Documents/

The 'lsof' command lists every open file descriptor, and what file it refers to. That output is then run through 'grep' which outputs only the lines that contain the string "/Documents/". That is, it outputs only the lines which refer to files in a Documents folder.

If you have something like Preview or TextEdit open, it may well list files it has open. Quit as many apps as possible to limit the amount of irrelevance.

Please copy and paste the output into a post here.

The 'grep' removes the heading from 'lsof' that labels the columns. Here's how to get that on a line by itself:

lsof | head -1

You can drag-select to select text in a Terminal window, then copy it to the clipboard. Paste into a text file along with the prior output and you should have column labels on the output.


It's possible the output from the first command lists nothing.

To expand the output from 'lsof' to include every process, not just those owned by the current user, try this command:

sudo lsof | grep /Documents/

It will ask for the password of your account, and it WILL NOT echo anything as you type it in. Simply type it in and press the Return key.

Because the 'lsof' output is much larger, it may take noticeably longer to run than before. A few seconds isn't unusual, compared to a fraction of a second before.


If you get an error message from any of the commands, please post the complete exact text of the message.

 

[NJRonbo]

First, I want to thank you for taking the time to write all that and offer your assistance.

I feel bad because I'm a bit of a novice at this, though the terminal commands were easy to execute with no errors.

Here is the output from the command:


sudo lsof | grep /Documents/

Wherever you see NAME below is a substitution for my real name which I changed for security purposes.

Not sure any of the information is going to help as I don't see that log file mentioned.

Thank you again




filecoord 552 root 29r DIR 1,13 288 23145499 /Users/NAME/Documents/iCollections/Collection 1
filecoord 552 root 30r DIR 1,13 96 23145498 /Users/NAME/Documents/iCollections
AMPArtwor 2542 NAME txt REG 1,13 32768 1511981 /Users/NAME/Library/Containers/com.apple.AMPArtworkAgent/Data/Documents/artworkd.sqlite-shm
AMPArtwor 2542 NAME 3u REG 1,13 9371648 1511976 /Users/NAME/Library/Containers/com.apple.AMPArtworkAgent/Data/Documents/artworkd.sqlite
AMPArtwor 2542 NAME 4u REG 1,13 1866392 1511980 /Users/NAME/Library/Containers/com.apple.AMPArtworkAgent/Data/Documents/artworkd.sqlite-wal
AMPArtwor 2542 NAME 5u REG 1,13 32768 1511981 /Users/NAME/Library/Containers/com.apple.AMPArtworkAgent/Data/Documents/artworkd.sqlite-shm
StocksWid 17288 NAME 12r DIR 1,13 96 45657 /Users/NAME/Library/Group Containers/group.co

 

[chown33]

I don't see anything in that output that looks like it might be a log file. I thought there was a possibility the process might have an open directory, but if that's all the output, then it seems not.

It looks like the process is opening and closing the file each time it logs to it, which means you'd have to catch a very narrow window of time when the file is open. It might be worthwhile repeating the cmd-line a few times, to see if anything new comes up. The timestamps you posted look like roughly every 5 secs, so a bit of timely keyboarding might get lucky and reveal something.

Nothing else comes to mind as a way to pinpoint what process is writing the file. You may have to resort to turning off your various menubar or status controls to figure it out. If you do that, I suggest a binary search instead of one-by-one, unless the list of network-related items is small. A binary search means you disable half of them and see if the problem goes away. If it does, you know the problematic item is in the set you disabled. If not, disable the other half and see if the problem goes away. Again, if it does, you know it's in the disable set. Once you know which half the item is in, you split that set in half and disable one half. Repeat until you narrow it down to a set of 2 items.

If disabling all menubar or status controls doesn't stop the problem, then it's not in that set of items at all. You'll have to expand your search.


I did a web search on the distinctlvely misspelled "Refresh Stauts" but I didn't see anything useful, just a bunch of other people who'd apparently misspelled the word "status". You could try your own searches using the specific text to see if anything rings a bell for you.

 

[NJRonbo]

I don't see anything in that output that looks like it might be a log file. I thought there was a possibility the process might have an open directory, but if that's all the output, then it seems not.

It looks like the process is opening and closing the file each time it logs to it, which means you'd have to catch a very narrow window of time when the file is open. It might be worthwhile repeating the cmd-line a few times, to see if anything new comes up. The timestamps you posted look like roughly every 5 secs, so a bit of timely keyboarding might get lucky and reveal something.

Nothing else comes to mind as a way to pinpoint what process is writing the file. You may have to resort to turning off your various menubar or status controls to figure it out. If you do that, I suggest a binary search instead of one-by-one, unless the list of network-related items is small. A binary search means you disable half of them and see if the problem goes away. If it does, you know the problematic item is in the set you disabled. If not, disable the other half and see if the problem goes away. Again, if it does, you know it's in the disable set. Once you know which half the item is in, you split that set in half and disable one half. Repeat until you narrow it down to a set of 2 items.

If disabling all menubar or status controls doesn't stop the problem, then it's not in that set of items at all. You'll have to expand your search.


I did a web search on the distinctlvely misspelled "Refresh Stauts" but I didn't see anything useful, just a bunch of other people who'd apparently misspelled the word "status". You could try your own searches using the specific text to see if anything rings a bell for you.

Thank you again for taking the time to look at this.

Perhaps the recommendation to repeat the terminal command often enough to catch it when the file is open and creating the log may be the answer.

The good news is, whatever it was, it has stopped writing logs for the past hour. All that remains in my DOCUMENTS folder is a single log file. If I delete it, another one shortly appears. However, at least, it's not writing a log file every few minutes to the point where I am deleting thousands every week.

 

[chown33]

If it continually writes to the same log file, you might try another trick. This may not do anything useful, but it couldn't hurt to see what happens.

The trick is to disable writing on the existing log file. Then when the process tries writing, it will fail.

You can prevent writing by doing Get Info on the file in Finder. Then find the Locked checkbox in that window. Enable that, and everyone will be unable to write, even your normal account.

Lock a document on Mac

In the Finder on your Mac, lock documents to protect them from being accidentally changed or deleted.

support.apple.com

One thing that might happen is that the log-writing process will start writing to a new log file of a different name. If that happens, then you may was well unlock the earlier log, or try locking the new log file just to see what happens.


If it really does keep writing to the same log file, there's a way to make a filename redirect to the "bit bucket" (null device) to prevent the file from growing interminably. This requires telling us the exact name of the file it writes.

 

[f54da]

Leave fs_usage | grep /path/to/document running in the background, when something writes or accesses the document you should see it.

 

[NJRonbo]

Leave fs_usage | grep /path/to/document running in the background, when something writes or accesses the document you should see it.

Can you explain this further? How do I leave this running? Thanks.

 

[HDFan]

How do I leave this running?

It runs until you stop it with control-c.

 

[NJRonbo]

It runs until you stop it with control-c.

Thank you. Is that a terminal command in the top post or a file path?

Just trying to clarify the steps in finding what is writing to the documents file. It has stopped, for the moment, but continues writing log files when *something* is running.

 

[chown33]

In order to give you a correct Terminal command, we need to know exactly what the names of the log files look like.

In your first post you wrote:

SOMETHING is dumping log.txt files in my DOCUMENTS folder to the tune of 1 every few minutes. I have thousands of them that I have to clean out every week.

If there are thousands of them (or really, more than 2) then they can't all be named "log.txt". So apparently there's some part of a filename before "log.txt", which you haven't told us yet. We need to see some examples of the actual names, so we can tell you what the correct Terminal command should be.

Here's a command that will list the "long info" for every file in the "Documents" folder whose name ends with "log.txt":

ls -l ~/Documents/*log.txt

Please post a few lines of the output from that command. If there isn't any output, then it's because there are no files whose name ends with "log.txt".

 

[NJRonbo]

In order to give you a correct Terminal command, we need to know exactly what the names of the log files look like.

In your first post you wrote:

If there are thousands of them (or really, more than 2) then they can't all be named "log.txt". So apparently there's some part of a filename before "log.txt", which you haven't told us yet. We need to see some examples of the actual names, so we can tell you what the correct Terminal command should be.

Here's a command that will list the "long info" for every file in the "Documents" folder whose name ends with "log.txt":

ls -l ~/Documents/*log.txt

Please post a few lines of the output from that command. If there isn't any output, then it's because there are no files whose name ends with "log.txt".

That command found the single log.txt that currently exists in my Downloads folder

Today is a good day. Nothing is currently writing to that folder.

However, when it was writing, it would number the log.txt files as: log.txt, log2.tx, log3.txt, log4.txt....

I would have hundreds of them in my shared iCloud Documents folder.

Been quiet for a few days. So, I still don't know which software was responsible.

Thank you for the help!

 

[chown33]

Thanks for providing the log-file naming pattern. Here's a command that will output creation of such log files to a Terminal window:

sudo fs_usage | grep 'Documents/log.*txt'

Type in your password when prompted, then press the RETURN key. The password will NOT echo, which is normal.

If you leave the Terminal window open, the command will continue to run. It will initially show no output. You can activate the window occasionally to see if it has anything new. It's likely that apps like Finder and mdworker will be listed in the output.

You can test whether it's working by creating a file or folder in "Documents" that has the correct name. You can do this in an app like TextEdit.app, or you can open another Terminal window and enter a command like:

touch ~/Documents/logtest.txt

 

[NJRonbo]

Thanks for providing the log-file naming pattern. Here's a command that will output creation of such log files to a Terminal window:

sudo fs_usage | grep 'Documents/log.*txt'

Type in your password when prompted, then press the RETURN key. The password will NOT echo, which is normal.

If you leave the Terminal window open, the command will continue to run. It will initially show no output. You can activate the window occasionally to see if it has anything new. It's likely that apps like Finder and mdworker will be listed in the output.

You can test whether it's working by creating a file or folder in "Documents" that has the correct name. You can do this in an app like TextEdit.app, or you can open another Terminal window and enter a command like:

touch ~/Documents/logtest.txt

Will start working on this, but meanwhile, this is what my documents folder looks like...

 

[NJRonbo]

Here is a screenshot I took of the terminal command in Post #15. Not sure if this is helpful or not

 

[chown33]

Will start working on this, but meanwhile, this is what my documents folder looks like...

View attachment 2207868

It would have been easier to read the output from an ls -l ~/Documents/log* command, or from a List view in Finder.

It's difficult to see the details, even zoomed in, but I think this screenshot shows the filenames actually have a space between the word 'log' and the number. The examples you gave earlier lack such a space. The pattern I posted doesn't rely on a space, or even a number.

Here is a screenshot I took of the terminal command in Post #15. Not sure if this is helpful or not

View attachment 2207872

The names of the programs accessing that "log.txt" file are:

bird -- at least 6 times​

Finder --3 times​

mdworker_sha --1 time​

mds --1 time​

I don't know what 'bird' is. It might be the offending program. I can't tell whether 'bird' wrote to the file, because the earlier parts of the output are missing.

If an earlier part of the output says 'write' in the column where you see stat64, getattrlist, etc. in the screenshot, then that's evidence of who wrote to the file. If you drag-select text in the Terminal window, you can then copy the hilited text to the clipboard, and paste it into a post here.

Finder should be obvious. mdworker_sha is a utility that reads metadata from files for maintaining the Spotlight database. mds is the metadata server, i.e. a faceless utility that provides Spotlight data to other apps.

 

[bogdanw]

I don't know what 'bird' is.

From the manual
"bird – Documents in the Cloud
bird is one of the system daemons backing the Documents in the Cloud feature."

Related https://forums.macrumors.com/threads/icloud-bird-process-gobbling-up-memory.2373524/

 

[f54da]

add a -v to grep to exclude finder and mds

 

[Mr.Heisenberg]

I'm sorry, I'll include it below:

2023-05-15 15:11:21 +0000 : Internet Connected
2023-05-15 15:11:30 +0000 : Refresh Stauts
2023-05-15 15:11:30 +0000 : Draw Menu
2023-05-15 15:11:30 +0000 : Sorted Interfaces : (
en1,
en0
)
2023-05-15 15:11:30 +0000 : Internet Connected
2023-05-15 15:11:35 +0000 : Refresh Stauts
2023-05-15 15:11:35 +0000 : Draw Menu
2023-05-15 15:11:35 +0000 : Sorted Interfaces : (
en1,


(and it goes on and on)

@NJRonbo
I find myself with the same problem... after a couple of hours thinking about it, it occurred to me to perform a sudo fs_usage | grep "path and file file" and what I could perceive that was repeated simultaneously was always writing when turning off and on the wifi or changing internet connections.

therefore I discovered that closing a program called "Ethernet Status.app" that I have installed no longer happened. Therefore whoever is writing this log file is said program.

I don't know if you have it too, you can check it.

The other way to stop having more activity is to open the log.txt file, delete all its content, save it and then right click "get information" and block option. So no one can write on it.

 

[NJRonbo]

@NJRonbo
I find myself with the same problem... after a couple of hours thinking about it, it occurred to me to perform a sudo fs_usage | grep "path and file file" and what I could perceive that was repeated simultaneously was always writing when turning off and on the wifi or changing internet connections.

therefore I discovered that closing a program called "Ethernet Status.app" that I have installed no longer happened. Therefore whoever is writing this log file is said program.

I don't know if you have it too, you can check it.

The other way to stop having more activity is to open the log.txt file, delete all its content, save it and then right click "get information" and block option. So no one can write on it.

Wow!

I absolutely am running that app!

So that has been the culprit, eh?

I will open the log.txt file, delete everything in it, and then set it to block so hopefully it can't write more of them.

THANK YOU for looking at this and discovering the issue. I thought I'd never find the solution.

 

[Septercius]

I think I'd be contacting the developer to complain about the log files (and the terrible spelling). Why's it not logging to a file in "~/Library/Application Support"?

https://ethernetstatus.com/ - is that it?

If you want to hide the file in Finder (it will still exist, but it won't be visible), you can type a Terminal command to do so:

chflags hidden ~/Documents/log.txt

(Change the file name as required.)

 

[NJRonbo]

I think I'd be contacting the developer to complain about the log files (and the terrible spelling). Why's it not logging to a file in "~/Library/Application Support"?

https://ethernetstatus.com/ - is that it?

If you want to hide the file in Finder (it will still exist, but it won't be visible), you can type a Terminal command to do so:

chflags hidden ~/Documents/log.txt

(Change the file name as required.)

I already put in a support ticket to the developer this morning.

Thank you for the instructions hiding the log files. Would rather just find a way to prevent the app from writing them and perhaps an answer from the developer will help. Will keep you updated if I hear anything.

 

[NJRonbo]

First response from the developer:

I'll get this checked with the technical team and get back to you. Please give me some time.