[mac] Ignore .DS_Store file in pphtml

[tangledhelix]

Discussed here.

PPhtml (and potentially other checks) should ignore the presence of a .DS_Store file on Mac. The system creates them any time the folder is opened in Finder (the file explorer). So it's a bit of a waste of PPer effort to repeatedly delete them just to make a validator happy (IMO).

[windymilla]

I agree that deleting them to make the validator happy is a waste of time. However, as detailed here, when the PPer is preparing their zip for upload to PPV (or WW if direct uploading) they are asked to

Please check there are no extra images or files (including system/hidden OS files such as .DS_Store) other than the images used in the books. It's best (though less important) to remove the thumbs.db file too.

So, PPhtml is informing them that they have files they should remove before uploading. That instruction has been around a long time, and I've no idea why it's less important to remove thumbs.db (a Windows system file). I'm guessing the idea is that we don't want .DS_Store to end up being posted at PG, though I expect the WWers already check for it.

[rtonsing]

Please leave the check. As a WWer, I use GG2 because it conveniently provides multiple tools for validation. I would probably see the extra file anyway, but I would prefer for it to be reported, especially if it helps prevent it being uploaded.

[srjfoo]

Ah ... I failed to understand/remember the purpose of the check, so I agree that it seems to not be appropriate to delete it, if the purpose is to warn the user that it's there and should not be when it's time to upload to PPV and/or PG.

As far as thumbs.db -- I thought we quit having trouble with those a long time ago: I remember seeing them in SR files when I was doing the SR coordinator job, but hadn't noticed any mention of them, or seen them in CP files since. I thought that Windows had reorganized things so that they don't appear in individual folders anymore, but some internet searching indicates otherwise. Perhaps it would be good to, at least temporarily, add that check?

@rtonsing -- I think all of you WWers use Windows? Are thumbs.db files automatically removed by PG?

[windymilla]

As far as thumbs.db -- I thought we quit having trouble with those a long time ago: I remember seeing them in SR files when I was doing the SR coordinator job, but hadn't noticed any mention of them, or seen them in CP files since. I thought that Windows had reorganized things so that they don't appear in individual folders anymore, but some internet searching indicates otherwise. Perhaps it would be good to, at least temporarily, add that check?

Another possibility (I haven't checked) is that default zipping on Windows doesn't include the thumbs.db file in the zip, hence it's never an issue in uploaded files?

[srjfoo]

As far as thumbs.db -- I thought we quit having trouble with those a long time ago: I remember seeing them in SR files when I was doing the SR coordinator job, but hadn't noticed any mention of them, or seen them in CP files since. I thought that Windows had reorganized things so that they don't appear in individual folders anymore, but some internet searching indicates otherwise. Perhaps it would be good to, at least temporarily, add that check?

Another possibility (I haven't checked) is that default zipping on Windows doesn't include the thumbs.db file in the zip, hence it's never an issue in uploaded files?

Possibly, but I did see mention of them in zip files done by the system. Dunno.

Apple apparently refuses to provide the ability to exclude invisible system files from the native zip utility.

[windymilla]

After further discussion...

If PG's upload were changed to delete .DS_Store (and thumbs.db?) from uploaded zips, then there would be no need to be concerned over it, and we could make PPhtml ignore it. PPVers can just ignore it if it happens to get into a zip. (Note that it is in the PPV report under the "Strongly Recommended" heading, so would probably need to be removed from there too).

[rtonsing]

Yes, all Windows, I believe. There is an issue on Workflow, and some discussion back in November, but nothing done yet.

It is tricky, maybe we should block files on submission, but there might be exceptions, and prompting the user there would be difficult, as well as removing them from the zip.

Workflow takes what the WWer uploads, it would be good to warn of any possible problems. I don't think any automatic stripping, though, since the WWer included the files. Except for 'output.txt', as of the version in review. (What if the user submitted an actual source named 'output.txt'? Hadn't thought about that, but it would cause problems in any case, old code or new.)

Haven't seen a thumbs.db in a while, the only ones on my system are in old archives.

[windymilla]

Haven't seen a thumbs.db in a while, the only ones on my system are in old archives.

According to Wikipedia, Windows stopped putting thumbs.db in folders with Windows Vista (2006)

[rtonsing]

Actually, looking at my proposed list of extensions, they should fail submission anyway. Executables, .bak files, etc.

[tangledhelix]

After further discussion...

If PG's upload were changed to delete .DS_Store

Some reasons to consider doing this:

• The PPer may have no idea the file is there, as they are hidden by the system. The Finder does not display them; the ls command does not display them without specifying -a to show them specifically, etc.
• You can do basically every step of PP without going into the terminal (assuming you don't use ppgen), but there's no way to remove these files without going to the terminal (since, again, Finder hides them)
• They exist if the user has made some view change option in Finder (such as sorting the files differently than default, or changing which view that folder displays...), so that if we make the user delete the file, they've also lost their local settings for how that folder is supposed to display, which can be quite annoying.

Just my 2¢. I have my stuff scripted with Makefile to automatically delete them before creating my zip for upload, but most people don't have that kind of stuff in place, nor would I expect them to.

[windymilla]

Good arguments - thanks Dan

[tangledhelix]

I just did a quick test and the "Compress folder" function in finder is also creating these __MACOSX/ copies, probably for compatibility with Windows? I'm not sure why it does it.

❯ unzip -l catholichistory.zip |grep DS_Store
    10244  02-02-2026 00:27   catholichistory/.DS_Store
      120  02-02-2026 00:27   __MACOSX/catholichistory/._.DS_Store

[tangledhelix]

Spent a few minutes reading; the __MACOSX variant seems to be a way the system exports things that are only supported in APFS (filesystem attrs, resource forks, icons, etc). They can't be excluded from zips created by the "Compress" command in the finder. You can create zips without them if you zip from the terminal, with zip -X.

Or I gather third-party compression programs probably exist. I have no idea about those or what their behaviors might be.

[windymilla]

As usual, cross-platform "standards" are a mess. :)

[tangledhelix]

https://xkcd.com/927/