February 19, 2004

Perforce SCM on OS X

by peterb

At the behest of one of the commentators here, I decided to try to download and install Perforce on my OS X laptop and see how the experience compares to installing it on FreeBSD or Windows. It worked fine for me, but I can understand how one would find the experience confusing, particularly if you're not familiar with the unix side of the world. Here's some notes on the experience.

I have a few advantages over your novice user, in that I've been using Perforce for about four years, and so already understand the model it uses, and why it is better than CVS -- how the repository stores data, what the relationship between a client, a branch, and the repository is -- so I deliberately tried to forget as much as I could and approach this with a fresh mind. I tried to follow the instructions as written, rather than filling in the blanks with things I've learned over the years.

Finding the packages themselves was no problem; Perforce has a page for Mac downloads, and there is client, server, and the visual client software available, along with some extra Macintosh-specific installation notes.

Installing the server was fairly easy; per the install notes I used the Darwin p4d software installation. I then became root via 'sudo', copied the p4d to /usr/local/bin, and did a "chmod a+x /usr/local/bin/p4d". /usr/local/bin is already in my path, so I didn't have to worry about that. But to make a long story short, I agree with reader David Trevas about this point -- it really wouldn't have killed Perforce to have provided a 5 line shell script to do this busywork for me.

Likewise, I installed the client and set up my P4PORT environment variable correctly, but then I already knew how to do that. The installation notes are indeed a bit scatterbrained about this. Hey, Perforce, you're already printing a 3-line error message when my P4PORT is set wrong -- instead of just saying "check $P4PORT", why not spend the extra 25 characters or so to give an example of how to set it to something well formed? Or point to a URL that explains it?

In the release notes, I saw a disclaimer that, at first, I simply didn't believe: "IMPORTANT: The Perforce server on Darwin is case-insensitive, unlike Perforce servers on other UNIX platforms." Ouch. "That can't be right, I thought." I was all ready to be angry at Perforce for being stupid, when I decided to do a little experimenting, and learned Today's Fun Fact: HFS+ is a case insensitive filesystem. This is so mind-numbingly stupid that I'm actually kind of mad at Apple about it, although I guess it's a well known bug^H^H^H"feature" of HFS+. Apparently the bug is fixed in 10.3 Server, which provides an option to make the filesystem not suck so much. So my formal recommendation would have to be "Don't run p4d on 10.2 or earlier, and if you run it on 10.3, turn on the 'make the filesystem not suck' bit (you can't actually just twiddle a bit; you have to newfs the filesystem as case-sensitive HFS. That, in turn, will probably cause a bunch of your other applications to break).

This is actually going to present a problem to anyone using the p4 client -- or, in fact, any other version control system -- which has files in a directory with filenames that are identical apart from case. In my quick tests, the behavior can be described as "last sync wins" -- if you have two files, "test.txt" and "TEST.TXT", and you sync, whichever is synced last overwrites the other. If you sync a deletion of TEST.TXT, test.txt gets deleted. Sloppy, Apple, really sloppy. OK, I know the filesystem is just doing what it's specced to do -- but what it is specced to do is dumb. For what it's worth, Windows/NTFS seems to have the same behavior. I think it's dumb there, too.

Apart from the case problems, which will afflict any SCM system on this platform, the p4 client worked as expected -- I was able to connect to both the p4d I just started up on my Powerbook, and also to the Perforce server at work. I tried their visual client "p4v", but didn't love it overmuch -- right off the bat it feels like it's missing important functionality. One major point of a visual client is to make things easier for newcomers, it seems to me, and since p4v won't even let you create a client workspace from the login screen, it seems kind of pointless to me. Perhaps it gets "better" once you're used to it, but I quickly reverted to just using the command line client. In p4v's favor, it comes in a .sit file and is a clickable OS X app, suitable for installation in your Applications folder. Hopefully future revisions will make it a bit more fully-featured

In summary: Perforce++ for having a readily available OS X client. Perforce++ for providing a powerful and reliable SCM system. And Perforce-- for not making their installation procedures as easy as they should be.

For another perspective from someone actually using Perforce on the Mac for Real Work, read Chris Hanson's journal (the 3:19 pm entry).

I use Perforce day in and day out, but in a nearly pure Unix environment. I'd be interested in feedback from those of you who run p4 clients on MacOS regularly, and I'm very interested to know if there are any of you out there using MacOS as a p4 server. Please share your experience in the comments section below, if you would.

Posted by peterb at February 19, 2004 03:39 PM | Bookmark This

It's funny, I used to really think a case insensitive file system was a stupid idea, I mean, Text.txt and text.txt are two different file names after all, and refering to Text.txt as text.txt shows a lack of care and discipline (the sort of thing you see in a case insensitive programming language).

But then I discovered that the filesystem code does more work to make Text.txt and text.txt collide at the file name level - and I thought, why did the engineers do that?

In thinking about it I realised that while I don't want anyone to be able to reference Text.txt as text.txt (because that is sloppy), I also don't want to actually have files that differ only in name case - that is also sloppy and confusing - I mean do I want to access Text.txt, tEXt.tXt or teXT.Txt this time?

Off the top of my head I can't think of any good reason to need two files with name case as the only filename difference, so I'm happy to let the filesystem do the extra work to dictate that.

I'm actually starting to lean towards the idea that the error is more in having software that requires a case sensitive file system, how hard is it to give a different file name to different files?

Posted by Michael Baltaks at February 19, 2004 06:03 PM

Hmm. There IS a MAJOR DRAWBACK using Perforce: it doesn't work with .NIB files. At least I couldn't figure it out and also remember that Apple did say that in their XCode release notes...

This is a real show stopper for those of us doing Cocoa or Carbon development.

Also Perforce is to blame: I needed hours of getting the thing running. Read this, read that, try copying the executable here and there - at the end wasted time because Perforce or XCode can't use .Nib files. If Perforce want to sell something to Mac users: why can't they make a nice Installer package? It would take them just an hour, but for the end user its a big difference.

BTW: its ashame that Apple didn't provide a version control plugin architecture like CodeWarrior. The best VC I've ever used is VOODOO Server...

Posted by Michael Keller at February 19, 2004 06:51 PM

Michael: That's a fair question. In fact, I sat here for a minute, and said "Well, gee, when _would_ I want to have two files with the same name," and couldn't come up with an example for a minute or so. Then I picked up my copy of "The Design and Implementation of the 4.4BSD Operating System" and inhaled its intoxicating scent deeply and came back to my senses.

I agree with you that case-insensitivity can be a useful behavior. What offends me about the implementation here is that the behavior is being implemented at the wrong layer. If you don't want the user to give the "same name" to two files, then your shell (and in a MacOS context we can interpret "shell" as meaning the AppKit / dialogue boxes, etc) is the right place to enforce those semantics. The reasons to not put that enforcement at the filesystem layer include:

- It is English (or at least roman-charset) centric. Does Japanese MacOS prevent me from naming a file "kana" in katakana if I already have a file named "kana" in hiragana? So case-sensitivity involves doing some tricky locale work, which is problematic for reasons I'll present momentarily.

- You're making the filesystem do more work than it needs to, which may have performance implications (ok, that one's a stretch, I admit it.)

- Most importantly, it takes a set of assumptions about how the user interacts with the system and _burns them into the filesystem_ layer. Suppose I have a program that collects data and spits it out into small files in the filesystem. It creates the names in some mechanical fashion; the names don't have to be human readable. So it goes merrily along, creating "aaa" then "bbb" ... "zzz" and moves around to "AAA", and now I get an error back from open() telling me that that file already exists. The issue here is not that HFS+ will hose programs that use stupid naming schemes like the one I describe, but that it's not _just_ making the filesystem do more work, but now all applications that generate their own filenames -- and here you'll have to take my word for it that this is a huge percentage of filesystem "users" -- have to do _the same_ work to make sure they enforce the semantics the filesystem wants. Semantics that the application might not necessarily want. In a POSIX-compliant case-sensitive world, there's just one rule: "Don't use '/'." That's a heck of a lot easier to implement.

So my point is that while case insensitivity may be a valuable behavior for users, it is a mistake for it to be implemented at the filesystem layer. Every additional rule the filesystem imposes on filenames is going to translate directly into additional lines of code at the application layer to conform with those rules.

I found this thread ( over at the Omnigroup forums to be interesting in regards to this topic. Kragen Sitaker (http://lists.canonical.org/pipermail/kragen-tol/2002-March/000698.html) agrees with me (and he's right!). An opposite viewpoint is presented by Brian Tiemann (http://www.xahlee.org/UnixResource_dir/_/fileCaseSens.html), who loves case-insensitivity (but he's wrong!)

Posted by peterb at February 19, 2004 07:07 PM

the reason .nib files do not work is because they are not files, they are "bundles" or "packages", which means that they are directories that various user interface layers present to the user as single files, but in the file system they are actually folders.

this is another case of the wrong layer doing the packaging. the convention that .nib "files", .pbproj "files" and so on be treated as single atomic entities is completely constructed in the Finder and other tools that manipulate these things. but, because of this, every application that has to deal with the file system in arbitrary ways needs to magically know to treat these things as a special case.

in any case, it's easy enough to deal with nib files, just add them as directories to the p4 repo from the command line and deal with them as such. since that's what they are anyway.

Posted by Pete at February 19, 2004 07:13 PM

@Pete: Of course you're right. .Nibs/Bundles are just folders.

BUT: as long XCode can't add them from its SCM GUI and the other Perforce GUIs just so terrible - Perforce is no option for me...

The only reason to learn command line stuff would be, if Perforce is using something like Voodoos "modules" - a project could consists of some "modules" which could be used in other projects as well. In Perforce I may archive this via mapping depots to views (??), but there is no easy and fool proof way to do this... . *

I would say that Perforce COULD be an ideal solution for Mac developers. But to archive this the guys at Perforce should not look at the Mac as another Unix machine...

* As you may have guessed I'm also not interested in reading tons of PDFs to get the thing just working. It should be - like any good Mac software - self explaining.

Posted by Michael Keller at February 19, 2004 07:51 PM

Yup, the case-sensitive vs. case-insensitive thing is a messy issue.

Michael said: "I'm actually starting to lean towards the idea that the error is more in having software that requires a case sensitive file system, how hard is it to give a different file name to different files?"

If the lowest level of the system treats these two strings as identical by default, there's something nasty going on. Why? Well, Pete pointed out that it's Roman-character centric. But to be honest, it's worse than that. In Japanese, hiragana and katakana are in one way "just variations" on the same characters. But they're used in significantly different ways, changing the semantics. So you can't "case-fold" them away (and indeed, Unicode does not mark these as being upper/lowercase variations.)

The worse thing is that not even all languages that use Roman characters handle case in the same way. There are two really easy examples. One is that in certain locales of the French language, a lowercase becomes E when it is written in uppercase. (Forgive me if I did the accent the wrong way or anything. :) That means that if you have two files, 1 and e1, and try to write to E1, you don't know which of the two files it should clobber. A nastier example is Turkish. In Turkish, the capital form of i is a dotted version of I, and the lower case form of I is an un-dotted version of i. (In other words: there's a difference between dotted Ii and undotted Ii.)

In summary, case folding is an operation that you can *only* do if you know what the language *and* the specific locale of that language is. If you want to be extra-good about it, you should probably keep track of that information on a per file basis--and just thinking about the massive confusion that would cause should make you pause for a moment and consider what a horrible horrible can of worms you've opened up. It's true that most reasonable people will never make identifiers or file names that are identical except for case--but you have to let them make that choice, because the system can't tell that they're the same except for case (no matter how much the system's programmers want to believe that it is.)

And that's why I'm pretty dead-set on the idea that anything low-level needs to treat each character as unique. A user-interface (shell, graphical shell, whatever) might want to take the locale into account in order to make things more pleasant for a given user--but even there you will get into a tangle because of the mis-match between the underlying layer and the user. (And the possibility of a three-way collision like E vs. vs. e.) Still, at the level of user interaction you have a chance to ask the user what to do.

The whole argument also applies to programming language identifiers (which is the context where I originally heard it.)

Big. Huge. Bag of worms. Just say no to case-insensitivity in low-level file systems and in programming languages. :)

Posted by John Prevost at February 19, 2004 07:55 PM

Well, the reality for me is that, in porting many applications from Solaris and HP-UX to MacOS X, I have never encountered an issue.

I have a conceptual issue with any possible file naming conflicts at the file system level. If it is the same name in any language or any case, abstract it and maintain a reference elsewhere.

Look at the havoc it can cause with webservices or when going from EBCDIC to ASCII.


Posted by Karl Gretton at February 19, 2004 11:53 PM

I used to use Perforce and loved using it on Windows 2000. When I switched to Mac OS X and had to choose version control for my private projects that don't make loads of money, I couldn't justify the cost of Perforce. I tried to use CVS again it made me change my mind and go back to Perforce. But then I started looking at the GUI client that Perforce provides for the Mac and thought yet again. Finally, my savior was Subversion. It is free like CVS, yet has a clean command-line syntax and handles renaming files and operations on folders. Like Perforce it has atomic check-ins and good branching. Haven't quite found a good GUI for Subversion, but other than that, I have no reason now to fork out all the $ for Perforce.

Posted by Winston at February 20, 2004 12:47 AM

Case sensitive file systems are a form of bizarre brain damage foisted on the world by UNIX. They lead to bizarre conversations about which version of the file "readme" is the one to read, and more user errors than you can shake a stick at. But, for those who are beyond help, you can use the case-sensitive version of HFS+ on 10.3 client as well as server.

(Actually, IIRC, the real issue was that making HFS+ case-sensitive at the start would instantly break lots of Mac apps which assumed the same sorts of file system semantics as OS9 had. It would also likely cause all sorts of trouble for average users who don't give a care for UNIX.)

Posted by Faisal at February 23, 2004 01:20 AM

Faisal, not all file systems are intended to be used directly by users. A case-insensitive file system makes writing applications for those filesystems harder.

Of course, this is a major reason that MacOS never penetrated into the enterprise market. It's also a major reason Apple implicitly admitted their mistake and offered case-sensitive HFS+.

Again, it's clear that "make things seem case-insensitive" is a useful and valuable behavior for users, but what's brain-damaged is implementing that restriction at the filesystem layer rather than at the user interface layer.

Another way to phrase this is: "Don't tell Oracle how to name their files, stupid."

Posted by peterb at February 23, 2004 09:30 AM

On some level, you have to do this at the FS layer, or apps will behave badly with other apps (yes, perforce overwriting File with file is bad, but so is one app maintaing File and file and then another app just blowing up entirely because it can't figure out which one to use).

I agree that having the case-sensitive option is a good thing, largely for compatibility reasons, but it's the wrong default choice for a consumer OS. I also don't buy that case sensitivity is an issue for the enterprise market, or Windows would be nowhere.

Posted by Faisal N. Jawdat at February 23, 2004 11:06 PM

FWIW, Fred Sanchez wrote a paper a few years ago about issues Apple ran into while integrating UNIX and the Mac: http://www.wsanchez.net/papers/USENIX_2000/

Posted by Faisal N. Jawdat at February 25, 2004 02:55 AM

Case-insensitive case-preserving filesystems are not 'sucky' or stupid. They are logical. Considering trying to explain to a non-technical user why the files 'MyLetter', 'myletter' and 'MYLETTER" are not all the same files. There is no reason in this day and age to have different files with the same name, separated only by case. That is stupid.

Witness how LWP::Perl's /usr/bin/HEAD collides with /usr/bin/head. That could've been avoided if the LWP::Perl developers had had the actual sense to THINK OF ANOTHER USEFUL NAME rather than going "oh well, we'll just uppercase it". F-ing ridiculous and it's that standard braindead carry-over from UNIX filesystems and academia that is ruining it.

Wonder why Windows and Mac use case-insensitivity? Because it makes sense to the non-nerds.

Posted by Frederick Abbott at March 9, 2004 07:54 AM


You're mistaken. If you read the comments, you'd see that we hashed this out quite a bit.

Case-insensitive/case-preserving is not stupid as a desired behavior for end users. You're right about that.

It is, however, stupid as a 'feature' to be built into the filesystem. The presentation layer should be enforcing this behavior, not the filesystem.

I said it before, and I'll say it again: not all "users" are people. Some users are machines. Some machines have to serve other machines that may have different filesystem semantics. By building case-insensitivity in to your filesystem you are hanging a big sign on it that says "I am not serious about competing in the enterprise" (and yes, Faisal, I disagree with your belief that because a few banks have installed NTFS that that belies my statement: NTFS supports case-sensitive filenames.)

The fact that Apple rolled out a case-sensitive option after they introduced the Xserve and nobody bought it is evidence of this.

Posted by peterb at March 9, 2004 08:08 AM

Ok, there's some very good discussion here on file systems and case sensitivity. I think that the two best arguments are 1) simplicity of design and implementation (ie. don't use "/"), and 2) locale dependance.

To be honest, I hadn't even considered the second point, even though I had puzzled over the problem Mac OS X has, where it shows certain user folders in the Finder according to the user's locale. So, ~/Music will be translated into the user's language, even though it is still ~/Music at the filesystem level. This feature can cause much confusion.

This leads me to think that we really need both a posix style filesystem, with very simple rules, and there's probably a very good case (excuse pun) for a higher level of filesystem that supports locale dependant features - a filesystem designed for actual people, implemented on top of a posix filesystem, but well documented so there is not confusion about what a file's name actually is (and of course then the name would be free of having to store other meta data for the file, the name would simply be a name).

Posted by Michael Baltaks at May 11, 2004 06:24 PM

> So, ~/Music will be translated into the user's
> language, even though it is still ~/Music at the
> filesystem level. This feature can cause much
> confusion.


Great Ghoulessarian. Please, please, please tell me you're kidding.

That's one of the craziest things I've ever heard. Wow.

Posted by peterb at August 28, 2004 12:53 PM

Please help support Tea Leaves by visiting our sponsors.

November October September August July June May April March February January

December November October September August July June May April March February January

December November October September August July June May April March February January